vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #465
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13lb # Disk image does not exist, creating the virtualisation disk image...14lb: QEMU running (pid 49)15worker2 # Disk image does not exist, creating the virtualisation disk image...16client: QEMU running (pid 50)17worker1: QEMU running (pid 51)18worker1 # Formatting '/build/vm-state-worker1/tmp.YLpvwU6C8k', fmt=raw size=107374182419client # Disk image does not exist, creating the virtualisation disk image...20lb # Formatting '/build/vm-state-lb/tmp.1lmkEyTzZS', fmt=raw size=107374182421worker2: QEMU running (pid 52)22worker2 # Formatting '/build/vm-state-worker2/tmp.lKNr6oDX3m', fmt=raw size=107374182423lb # mke2fs 1.47.4 (6-Mar-2025)24client # Formatting '/build/vm-state-client/tmp.xxKtQNXkIu', fmt=raw size=107374182425worker1 # mke2fs 1.47.4 (6-Mar-2025)26worker2 # mke2fs 1.47.4 (6-Mar-2025)27lb # Discarding device blocks: 0/262144 done28worker1 # Discarding device blocks: 0/262144 done29worker2 # Discarding device blocks: 0/262144 done30lb # Creating filesystem with 262144 4k blocks and 65536 inodes31worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes32worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes33lb # Filesystem UUID: 3ce0c967-ba8e-44be-822a-4cd33d5382f134worker1 # Filesystem UUID: d8719de8-9303-447a-92f1-443ba59ec9e635worker2 # Filesystem UUID: 4679f22f-08ba-435b-9f4a-daa35ddbe0e536lb # Superblock backups stored on blocks:37worker1 # Superblock backups stored on blocks:38worker2 # Superblock backups stored on blocks:39lb # 32768, 98304, 163840, 22937640worker1 # 32768, 98304, 163840, 22937641worker2 # 32768, 98304, 163840, 22937642lb # 43worker1 # 44worker2 # 45lb # Allocating group tables: 0/8 done46worker1 # Allocating group tables: 0/8 done47worker2 # Allocating group tables: 0/8 done48lb # Writing inode tables: 0/8 done49worker1 # Writing inode tables: 0/8 done50worker2 # Writing inode tables: 0/8 done51lb # Creating journal (8192 blocks): done52worker1 # Creating journal (8192 blocks): done53worker2 # Creating journal (8192 blocks): done54lb # Writing superblocks and filesystem accounting information: 0/8 done55worker1 # Writing superblocks and filesystem accounting information: 0/8 done56worker2 # Writing superblocks and filesystem accounting information: 0/8 done57lb # 58worker1 # 59worker2 # 60lb # Virtualisation disk image created.61worker1 # Virtualisation disk image created.62worker2 # Virtualisation disk image created.63(finished: start all VMs, in 0.47 seconds)64client # mke2fs 1.47.4 (6-Mar-2025)65client: waiting for unit network-addresses-eth1.service66client: waiting for the VM to finish booting67client # Discarding device blocks: 0/262144 done68client # Creating filesystem with 262144 4k blocks and 65536 inodes69client # Filesystem UUID: fa70b7c7-7f78-4bc7-8d21-bd9d2ca7957e70client # Superblock backups stored on blocks:71client # 32768, 98304, 163840, 22937672client # 73client # Allocating group tables: 0/8 done74client # Writing inode tables: 0/8 done75client # Creating journal (8192 blocks): done76client # Writing superblocks and filesystem accounting information: 0/8 done77client # 78client # Virtualisation disk image created.79worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681worker2 # [ 0.000000] KASLR enabled82worker2 # [ 0.000000] random: crng init done83lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]84worker2 # [ 0.000000] Machine model: linux,dummy-virt85worker2 # [ 0.000000] efi: UEFI not found.86worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT87lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202688lb # [ 0.000000] KASLR enabled89worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]90lb # [ 0.000000] random: crng init done91lb # [ 0.000000] Machine model: linux,dummy-virt92worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]93lb # [ 0.000000] efi: UEFI not found.94worker2 # [ 0.000000] Zone ranges:95worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]96worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]97lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT98worker2 # [ 0.000000] DMA32 empty99worker2 # [ 0.000000] Normal empty100worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026101lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]102worker2 # [ 0.000000] Device empty103worker1 # [ 0.000000] KASLR enabled104worker2 # [ 0.000000] Movable zone start for each node105worker1 # [ 0.000000] random: crng init done106lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]107lb # [ 0.000000] Zone ranges:108worker2 # [ 0.000000] Early memory node ranges109worker1 # [ 0.000000] Machine model: linux,dummy-virt110worker1 # [ 0.000000] efi: UEFI not found.111worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]112lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]113lb # [ 0.000000] DMA32 empty114worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT115lb # [ 0.000000] Normal empty116worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]117lb # [ 0.000000] Device empty118worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000119worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]120lb # [ 0.000000] Movable zone start for each node121lb # [ 0.000000] Early memory node ranges122worker2 # [ 0.000000] psci: probing for conduit method from DT.123worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]124worker1 # [ 0.000000] Zone ranges:125worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.126lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]127worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]128worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs129worker1 # [ 0.000000] DMA32 empty130lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]131worker2 # [ 0.000000] psci: Trusted OS migration not required132worker1 # [ 0.000000] Normal empty133lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000134worker1 # [ 0.000000] Device empty135worker2 # [ 0.000000] psci: SMC Calling Convention v1.1136worker1 # [ 0.000000] Movable zone start for each node137lb # [ 0.000000] psci: probing for conduit method from DT.138worker1 # [ 0.000000] Early memory node ranges139worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)140lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.141client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]142lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs143worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]144worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296145lb # [ 0.000000] psci: Trusted OS migration not required146worker2 # [ 0.000000] Detected PIPT I-cache on CPU0147worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]148lb # [ 0.000000] psci: SMC Calling Convention v1.1149client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026150worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000151client # [ 0.000000] KASLR enabled152worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)153worker1 # [ 0.000000] psci: probing for conduit method from DT.154client # [ 0.000000] random: crng init done155lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface157client # [ 0.000000] Machine model: linux,dummy-virt158worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.159worker2 # [ 0.000000] CPU features: detected: Spectre-v4160client # [ 0.000000] efi: UEFI not found.161lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296162worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs163worker2 # [ 0.000000] CPU features: detected: Spectre-BHB164lb # [ 0.000000] Detected PIPT I-cache on CPU0165worker1 # [ 0.000000] psci: Trusted OS migration not required166client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT167worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38168worker1 # [ 0.000000] psci: SMC Calling Convention v1.1169lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)170client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]171worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23172lb # [ 0.000000] CPU features: detected: GICv3 CPU interface173client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]174worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)175worker2 # [ 0.000000] alternatives: applying boot alternatives176client # [ 0.000000] Zone ranges:177lb # [ 0.000000] CPU features: detected: Spectre-v4178worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296179lb # [ 0.000000] CPU features: detected: Spectre-BHB180client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]181worker1 # [ 0.000000] Detected PIPT I-cache on CPU0182client # [ 0.000000] DMA32 empty183lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38184client # [ 0.000000] Normal empty185client # [ 0.000000] Device empty186worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)187lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23188client # [ 0.000000] Movable zone start for each node189worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface190client # [ 0.000000] Early memory node ranges191lb # [ 0.000000] alternatives: applying boot alternatives192worker1 # [ 0.000000] CPU features: detected: Spectre-v4193worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/w0djksilfhz0x4sfhpdpmkf3iy7y39rh-nixos-system-worker2-test/init regInfo=/nix/store/nqjf3pq8l58q3rgaldbcq08a2kda91gi-closure-info/registration console=ttyAMA0,115200n8 console=tty0194client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]195worker1 # [ 0.000000] CPU features: detected: Spectre-BHB196client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]197worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38198client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000199worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/nqjf3pq8l58q3rgaldbcq08a2kda91gi-closure-info/registration", will be passed to user space.200worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23201client # [ 0.000000] psci: probing for conduit method from DT.202worker1 # [ 0.000000] alternatives: applying boot alternatives203worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes204client # [ 0.000000] psci: PSCIv1.3 detected in firmware.205lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/sy3pchz5vg22ylc3whvkp4s8qngi8wmd-nixos-system-lb-test/init regInfo=/nix/store/4q693lxjjy3z9gghvdrv4s7qcr63j8a1-closure-info/registration console=ttyAMA0,115200n8 console=tty0206client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs207worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)208client # [ 0.000000] psci: Trusted OS migration not required209client # [ 0.000000] psci: SMC Calling Convention v1.1210worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)211worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB212lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4q693lxjjy3z9gghvdrv4s7qcr63j8a1-closure-info/registration", will be passed to user space.213client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)214worker2 # [ 0.000000] software IO TLB: area num 1.215worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/v0lyrrn8c44dmsycpizv4zk1h70zydxf-nixos-system-worker1-test/init regInfo=/nix/store/xi2sa5r0rxp25jn6mwpjlq1r7pww37n0-closure-info/registration console=ttyAMA0,115200n8 console=tty0216client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296217lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes218worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB219client # [ 0.000000] Detected PIPT I-cache on CPU0220lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)221worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)222client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)223worker2 # [ 0.000000] Fallback order for Node 0: 0224worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/xi2sa5r0rxp25jn6mwpjlq1r7pww37n0-closure-info/registration", will be passed to user space.225lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)226client # [ 0.000000] CPU features: detected: GICv3 CPU interface227worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216228worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes229client # [ 0.000000] CPU features: detected: Spectre-v4230lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB231lb # [ 0.000000] software IO TLB: area num 1.232client # [ 0.000000] CPU features: detected: Spectre-BHB233worker2 # [ 0.000000] Policy zone: DMA234worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)235lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB236client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38237worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off238worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)239client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23240lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)241worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1242client # [ 0.000000] alternatives: applying boot alternatives243lb # [ 0.000000] Fallback order for Node 0: 0244worker2 # [ 0.000000] allocated 3145728 bytes of page_ext245worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB246lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216247worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages248worker1 # [ 0.000000] software IO TLB: area num 1.249lb # [ 0.000000] Policy zone: DMA250worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups251worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB252lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off253worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.254worker2 # [ 0.000000] rcu: RCU event tracing is enabled.255worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)256lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1257worker1 # [ 0.000000] Fallback order for Node 0: 0258lb # [ 0.000000] allocated 3145728 bytes of page_ext259worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.260client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/1riwj2yl1fyad56y7izqz2z8iyxylapg-nixos-system-client-test/init regInfo=/nix/store/qj3vdfrkilk0aca7lsjqss7zd6k5k971-closure-info/registration console=ttyAMA0,115200n8 console=tty0261worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.262lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages263worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216264worker1 # [ 0.000000] Policy zone: DMA265worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.266lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups267worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off268worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.269lb # [ 0.000000] rcu: Hierarchical RCU implementation.270client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/qj3vdfrkilk0aca7lsjqss7zd6k5k971-closure-info/registration", will be passed to user space.271lb # [ 0.000000] rcu: RCU event tracing is enabled.272worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1273worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.274client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes275worker1 # [ 0.000000] allocated 3145728 bytes of page_ext276lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.277worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1278worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages279lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.280client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)281lb # [ 0.000000] Rude variant of Tasks RCU enabled.282worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups283worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.284client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)285lb # [ 0.000000] Tracing variant of Tasks RCU enabled.286worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.287worker1 # [ 0.000000] rcu: RCU event tracing is enabled.288worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.289client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB290lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.291client # [ 0.000000] software IO TLB: area num 1.292worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.293worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.294lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1295worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.296client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB297worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0298worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.299lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.300client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)301worker2 # [ 0.000000] GICv3: 256 SPIs implemented302worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.303client # [ 0.000000] Fallback order for Node 0: 0304worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented305lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.306worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq307worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.308client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216309worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI310client # [ 0.000000] Policy zone: DMA311worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1312lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.313worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0314client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off315lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0316worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.317worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000318lb # [ 0.000000] GICv3: 256 SPIs implemented319client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1320worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]321client # [ 0.000000] allocated 3145728 bytes of page_ext322lb # [ 0.000000] GICv3: 0 Extended SPIs implemented323worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.324lb # [ 0.000000] Root IRQ handler: gic_handle_irq325client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages326worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)327lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI328client # [ 0.000000] ftrace: allocated 294 pages with 4 groups329worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.330lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0331client # [ 0.000000] rcu: Hierarchical RCU implementation.332worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0333worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)334client # [ 0.000000] rcu: RCU event tracing is enabled.335worker1 # [ 0.000000] GICv3: 256 SPIs implemented336lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000337worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000338lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]339worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented340client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.341worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq342worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000343client # [ 0.000000] Trampoline variant of Tasks RCU enabled.344worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI345lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)346client # [ 0.000000] Rude variant of Tasks RCU enabled.347worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.348worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0349client # [ 0.000000] Tracing variant of Tasks RCU enabled.350lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)351worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000352worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns353client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.354worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]355lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000356worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).357client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1358lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000359worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)360client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.361worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns362lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.363worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)364client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.365worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns366lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns367worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000368worker2 # [ 0.000030] arm-pv: using stolen time PV369lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).370worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000371client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.372worker2 # [ 0.000439] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)373client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0374worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.375worker2 # [ 0.000614] Console: colour dummy device 80x25376client # [ 0.000000] GICv3: 256 SPIs implemented377lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns378client # [ 0.000000] GICv3: 0 Extended SPIs implemented379worker2 # [ 0.000623] printk: legacy console [tty0] enabled380worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns381client # [ 0.000000] Root IRQ handler: gic_handle_irq382lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns383client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI384worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).385lb # [ 0.000031] arm-pv: using stolen time PV386worker2 # [ 0.000825] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)387client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0388worker2 # [ 0.000833] pid_max: default: 32768 minimum: 301389lb # [ 0.000456] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)390client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000391worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns392worker2 # [ 0.000927] LSM: initializing lsm=capability,landlock,yama,bpf,ima393lb # [ 0.000608] Console: colour dummy device 80x25394client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]395worker2 # [ 0.001065] landlock: Up and running.396worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns397worker2 # [ 0.001068] Yama: becoming mindful.398lb # [ 0.000616] printk: legacy console [tty0] enabled399worker1 # [ 0.000030] arm-pv: using stolen time PV400worker2 # [ 0.001555] LSM support for eBPF active401client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)402lb # [ 0.000808] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)403worker2 # [ 0.001727] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)404worker1 # [ 0.000436] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)405lb # [ 0.000815] pid_max: default: 32768 minimum: 301406client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)407worker1 # [ 0.000582] Console: colour dummy device 80x25408worker2 # [ 0.001764] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)409lb # [ 0.000884] LSM: initializing lsm=capability,landlock,yama,bpf,ima410client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000411worker1 # [ 0.000590] printk: legacy console [tty0] enabled412lb # [ 0.001034] landlock: Up and running.413worker2 # [ 0.002896] cacheinfo: Unable to detect cache hierarchy for CPU 0414client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000415lb # [ 0.001037] Yama: becoming mindful.416worker2 # [ 0.003653] rcu: Hierarchical SRCU implementation.417lb # [ 0.001535] LSM support for eBPF active418worker1 # [ 0.000787] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)419worker2 # [ 0.003658] rcu: Max phase no-delay instances is 1000.420client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.421worker1 # [ 0.000793] pid_max: default: 32768 minimum: 301422lb # [ 0.001673] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)423worker2 # [ 0.004886] fsl-mc MSI: its@8080000 domain created424worker1 # [ 0.000869] LSM: initializing lsm=capability,landlock,yama,bpf,ima425client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns426worker2 # [ 0.004979] EFI services will not be available.427lb # [ 0.001707] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)428worker1 # [ 0.001038] landlock: Up and running.429worker2 # [ 0.005090] smp: Bringing up secondary CPUs ...430worker1 # [ 0.001041] Yama: becoming mindful.431client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).432worker2 # [ 0.005098] smp: Brought up 1 node, 1 CPU433lb # [ 0.002828] cacheinfo: Unable to detect cache hierarchy for CPU 0434worker1 # [ 0.001454] LSM support for eBPF active435worker2 # [ 0.005101] SMP: Total of 1 processors activated.436lb # [ 0.003551] rcu: Hierarchical SRCU implementation.437client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns438worker2 # [ 0.005107] CPU: All CPU(s) started at EL1439worker1 # [ 0.001592] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)440lb # [ 0.003555] rcu: Max phase no-delay instances is 1000.441worker2 # [ 0.005118] CPU features: detected: Branch Target Identification442lb # [ 0.004758] fsl-mc MSI: its@8080000 domain created443worker1 # [ 0.001624] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)444client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns445lb # [ 0.004856] EFI services will not be available.446client # [ 0.000028] arm-pv: using stolen time PV447worker2 # [ 0.005123] CPU features: detected: ARMv8.4 Translation Table Level448worker1 # [ 0.002735] cacheinfo: Unable to detect cache hierarchy for CPU 0449lb # [ 0.004968] smp: Bringing up secondary CPUs ...450worker1 # [ 0.003449] rcu: Hierarchical SRCU implementation.451lb # [ 0.004977] smp: Brought up 1 node, 1 CPU452worker2 # [ 0.005127] CPU features: detected: Instruction cache invalidation not required for I/D coherence453client # [ 0.000386] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)454worker1 # [ 0.003454] rcu: Max phase no-delay instances is 1000.455lb # [ 0.004980] SMP: Total of 1 processors activated.456client # [ 0.000550] Console: colour dummy device 80x25457worker1 # [ 0.004662] fsl-mc MSI: its@8080000 domain created458lb # [ 0.004985] CPU: All CPU(s) started at EL1459worker2 # [ 0.005130] CPU features: detected: Data cache clean to the PoU not required for I/D coherence460client # [ 0.000557] printk: legacy console [tty0] enabled461worker1 # [ 0.004752] EFI services will not be available.462lb # [ 0.004996] CPU features: detected: Branch Target Identification463worker2 # [ 0.005134] CPU features: detected: Common not Private translations464worker1 # [ 0.004816] smp: Bringing up secondary CPUs ...465worker2 # [ 0.005137] CPU features: detected: CRC32 instructions466client # [ 0.000773] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)467lb # [ 0.005001] CPU features: detected: ARMv8.4 Translation Table Level468worker1 # [ 0.004824] smp: Brought up 1 node, 1 CPU469worker1 # [ 0.004827] SMP: Total of 1 processors activated.470client # [ 0.000780] pid_max: default: 32768 minimum: 301471worker2 # [ 0.005140] CPU features: detected: Data cache clean to Point of Deep Persistence472lb # [ 0.005004] CPU features: detected: Instruction cache invalidation not required for I/D coherence473worker1 # [ 0.004833] CPU: All CPU(s) started at EL1474client # [ 0.000858] LSM: initializing lsm=capability,landlock,yama,bpf,ima475worker2 # [ 0.005144] CPU features: detected: Data cache clean to Point of Persistence476client # [ 0.001032] landlock: Up and running.477worker1 # [ 0.004843] CPU features: detected: Branch Target Identification478lb # [ 0.005007] CPU features: detected: Data cache clean to the PoU not required for I/D coherence479client # [ 0.001035] Yama: becoming mindful.480worker2 # [ 0.005147] CPU features: detected: Data independent timing control (DIT)481worker1 # [ 0.004848] CPU features: detected: ARMv8.4 Translation Table Level482client # [ 0.001507] LSM support for eBPF active483lb # [ 0.005011] CPU features: detected: Common not Private translations484worker2 # [ 0.005150] CPU features: detected: E0PD485lb # [ 0.005014] CPU features: detected: CRC32 instructions486client # [ 0.001636] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)487worker1 # [ 0.004851] CPU features: detected: Instruction cache invalidation not required for I/D coherence488worker2 # [ 0.005152] CPU features: detected: Enhanced Counter Virtualization489lb # [ 0.005017] CPU features: detected: Data cache clean to Point of Deep Persistence490client # [ 0.001667] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)491worker2 # [ 0.005155] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)492worker1 # [ 0.004855] CPU features: detected: Data cache clean to the PoU not required for I/D coherence493client # [ 0.002779] cacheinfo: Unable to detect cache hierarchy for CPU 0494worker2 # [ 0.005158] CPU features: detected: Enhanced Virtualization Traps495lb # [ 0.005021] CPU features: detected: Data cache clean to Point of Persistence496worker1 # [ 0.004858] CPU features: detected: Common not Private translations497worker2 # [ 0.005162] CPU features: detected: Fine Grained Traps498client # [ 0.003478] rcu: Hierarchical SRCU implementation.499worker1 # [ 0.004861] CPU features: detected: CRC32 instructions500lb # [ 0.005024] CPU features: detected: Data independent timing control (DIT)501client # [ 0.003482] rcu: Max phase no-delay instances is 1000.502lb # [ 0.005027] CPU features: detected: E0PD503worker2 # [ 0.005165] CPU features: detected: Generic authentication (architected QARMA5 algorithm)504worker1 # [ 0.004864] CPU features: detected: Data cache clean to Point of Deep Persistence505client # [ 0.004671] fsl-mc MSI: its@8080000 domain created506lb # [ 0.005030] CPU features: detected: Enhanced Counter Virtualization507worker2 # [ 0.005170] CPU features: detected: RCpc load-acquire (LDAPR)508client # [ 0.004761] EFI services will not be available.509worker1 # [ 0.004868] CPU features: detected: Data cache clean to Point of Persistence510client # [ 0.004840] smp: Bringing up secondary CPUs ...511worker2 # [ 0.005173] CPU features: detected: LSE atomic instructions512lb # [ 0.005033] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)513client # [ 0.004848] smp: Brought up 1 node, 1 CPU514worker1 # [ 0.004871] CPU features: detected: Data independent timing control (DIT)515worker2 # [ 0.005176] CPU features: detected: Privileged Access Never516client # [ 0.004851] SMP: Total of 1 processors activated.517lb # [ 0.005036] CPU features: detected: Enhanced Virtualization Traps518worker1 # [ 0.004874] CPU features: detected: E0PD519worker2 # [ 0.005178] CPU features: detected: PMUv3520client # [ 0.004856] CPU: All CPU(s) started at EL1521lb # [ 0.005039] CPU features: detected: Fine Grained Traps522worker1 # [ 0.004876] CPU features: detected: Enhanced Counter Virtualization523worker2 # [ 0.005181] CPU features: detected: RAS Extension Support524client # [ 0.004867] CPU features: detected: Branch Target Identification525lb # [ 0.005042] CPU features: detected: Generic authentication (architected QARMA5 algorithm)526worker1 # [ 0.004879] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)527worker2 # [ 0.005184] CPU features: detected: RASv1p1 Extension Support528client # [ 0.004872] CPU features: detected: ARMv8.4 Translation Table Level529worker1 # [ 0.004883] CPU features: detected: Enhanced Virtualization Traps530lb # [ 0.005047] CPU features: detected: RCpc load-acquire (LDAPR)531worker2 # [ 0.005187] CPU features: detected: Random Number Generator532worker1 # [ 0.004886] CPU features: detected: Fine Grained Traps533client # [ 0.004875] CPU features: detected: Instruction cache invalidation not required for I/D coherence534lb # [ 0.005050] CPU features: detected: LSE atomic instructions535worker2 # [ 0.005189] CPU features: detected: Speculation barrier (SB)536lb # [ 0.005053] CPU features: detected: Privileged Access Never537worker2 # [ 0.005192] CPU features: detected: Stage-2 Force Write-Back538worker1 # [ 0.004889] CPU features: detected: Generic authentication (architected QARMA5 algorithm)539client # [ 0.004879] CPU features: detected: Data cache clean to the PoU not required for I/D coherence540lb # [ 0.005056] CPU features: detected: PMUv3541worker2 # [ 0.005195] CPU features: detected: TLB range maintenance instructions542client # [ 0.004883] CPU features: detected: Common not Private translations543worker1 # [ 0.004893] CPU features: detected: RCpc load-acquire (LDAPR)544lb # [ 0.005059] CPU features: detected: RAS Extension Support545worker1 # [ 0.004896] CPU features: detected: LSE atomic instructions546client # [ 0.004886] CPU features: detected: CRC32 instructions547lb # [ 0.005061] CPU features: detected: RASv1p1 Extension Support548worker2 # [ 0.005199] CPU features: detected: Speculative Store Bypassing Safe (SSBS)549worker2 # [ 0.005239] alternatives: applying system-wide alternatives550lb # [ 0.005064] CPU features: detected: Random Number Generator551worker1 # [ 0.004899] CPU features: detected: Privileged Access Never552client # [ 0.004888] CPU features: detected: Data cache clean to Point of Deep Persistence553worker1 # [ 0.004902] CPU features: detected: PMUv3554lb # [ 0.005066] CPU features: detected: Speculation barrier (SB)555client # [ 0.004892] CPU features: detected: Data cache clean to Point of Persistence556worker2 # [ 0.008197] CPU features: detected: BBM Level 2 without TLB conflict abort557worker1 # [ 0.004904] CPU features: detected: RAS Extension Support558lb # [ 0.005069] CPU features: detected: Stage-2 Force Write-Back559client # [ 0.004895] CPU features: detected: Data independent timing control (DIT)560worker1 # [ 0.004907] CPU features: detected: RASv1p1 Extension Support561client # [ 0.004898] CPU features: detected: E0PD562lb # [ 0.005072] CPU features: detected: TLB range maintenance instructions563worker1 # [ 0.004909] CPU features: detected: Random Number Generator564worker2 # [ 0.008387] Memory: 1406564K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)565worker2 # [ 0.008774] devtmpfs: initialized566client # [ 0.004901] CPU features: detected: Enhanced Counter Virtualization567worker1 # [ 0.004912] CPU features: detected: Speculation barrier (SB)568lb # [ 0.005077] CPU features: detected: Speculative Store Bypassing Safe (SSBS)569worker1 # [ 0.004915] CPU features: detected: Stage-2 Force Write-Back570worker2 # [ 0.010542] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)571lb # [ 0.005115] alternatives: applying system-wide alternatives572client # [ 0.004904] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)573worker1 # [ 0.004917] CPU features: detected: TLB range maintenance instructions574client # [ 0.004907] CPU features: detected: Enhanced Virtualization Traps575lb # [ 0.008159] CPU features: detected: BBM Level 2 without TLB conflict abort576client # [ 0.004910] CPU features: detected: Fine Grained Traps577worker1 # [ 0.004921] CPU features: detected: Speculative Store Bypassing Safe (SSBS)578worker1 # [ 0.004957] alternatives: applying system-wide alternatives579client # [ 0.004914] CPU features: detected: Generic authentication (architected QARMA5 algorithm)580lb # [ 0.008408] Memory: 1406524K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)581lb # [ 0.008772] devtmpfs: initialized582worker1 # [ 0.007939] CPU features: detected: BBM Level 2 without TLB conflict abort583client # [ 0.004920] CPU features: detected: RCpc load-acquire (LDAPR)584client # [ 0.004923] CPU features: detected: LSE atomic instructions585lb # [ 0.010666] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)586client # [ 0.004926] CPU features: detected: Privileged Access Never587worker1 # [ 0.008144] Memory: 1406512K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)588client # [ 0.004929] CPU features: detected: PMUv3589lb # [ 0.010687] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).590worker1 # [ 0.008502] devtmpfs: initialized591client # [ 0.004931] CPU features: detected: RAS Extension Support592lb # [ 0.010871] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL593worker1 # [ 0.010308] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)594client # [ 0.004934] CPU features: detected: RASv1p1 Extension Support595client # [ 0.004936] CPU features: detected: Random Number Generator596worker1 # [ 0.010330] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).597client # [ 0.004939] CPU features: detected: Speculation barrier (SB)598worker1 # [ 0.010528] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL599client # [ 0.004942] CPU features: detected: Stage-2 Force Write-Back600worker1 # [ 0.010533] 0 pages in range for non-PLT usage601worker1 # [ 0.010534] 508288 pages in range for PLT usage602client # [ 0.004945] CPU features: detected: TLB range maintenance instructions603worker2 # [ 0.010563] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).604worker1 # [ 0.010647] pinctrl core: initialized pinctrl subsystem605client # [ 0.004950] CPU features: detected: Speculative Store Bypassing Safe (SSBS)606worker2 # [ 0.010747] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL607worker1 # [ 0.011449] DMI not present or invalid.608worker2 # [ 0.010752] 0 pages in range for non-PLT usage609client # [ 0.004985] alternatives: applying system-wide alternatives610worker1 # [ 0.014594] NET: Registered PF_NETLINK/PF_ROUTE protocol family611worker2 # [ 0.010753] 508288 pages in range for PLT usage612client # [ 0.007862] CPU features: detected: BBM Level 2 without TLB conflict abort613worker1 # [ 0.016765] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations614worker2 # [ 0.010854] pinctrl core: initialized pinctrl subsystem615worker2 # [ 0.011642] DMI not present or invalid.616worker2 # [ 0.014702] NET: Registered PF_NETLINK/PF_ROUTE protocol family617client # [ 0.007997] Memory: 1406504K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)618client # [ 0.008384] devtmpfs: initialized619worker2 # [ 0.016939] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations620lb # [ 0.010876] 0 pages in range for non-PLT usage621client # [ 0.010178] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)622worker2 # [ 0.017141] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations623lb # [ 0.010876] 508288 pages in range for PLT usage624client # [ 0.010202] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).625worker2 # [ 0.017348] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations626lb # [ 0.010970] pinctrl core: initialized pinctrl subsystem627lb # [ 0.011706] DMI not present or invalid.628worker2 # [ 0.017372] audit: initializing netlink subsys (disabled)629client # [ 0.010378] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL630lb # [ 0.015072] NET: Registered PF_NETLINK/PF_ROUTE protocol family631client # [ 0.010382] 0 pages in range for non-PLT usage632worker2 # [ 0.017977] thermal_sys: Registered thermal governor 'fair_share'633client # [ 0.010383] 508288 pages in range for PLT usage634lb # [ 0.017266] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations635worker2 # [ 0.017979] thermal_sys: Registered thermal governor 'bang_bang'636client # [ 0.010487] pinctrl core: initialized pinctrl subsystem637client # [ 0.011223] DMI not present or invalid.638worker2 # [ 0.017982] thermal_sys: Registered thermal governor 'step_wise'639lb # [ 0.017455] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations640client # [ 0.014359] NET: Registered PF_NETLINK/PF_ROUTE protocol family641worker2 # [ 0.017987] thermal_sys: Registered thermal governor 'user_space'642lb # [ 0.017655] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations643worker2 # [ 0.017990] thermal_sys: Registered thermal governor 'power_allocator'644client # [ 0.016556] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations645lb # [ 0.017676] audit: initializing netlink subsys (disabled)646worker1 # [ 0.016955] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations647worker2 # [ 0.018014] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1648lb # [ 0.018246] thermal_sys: Registered thermal governor 'fair_share'649client # [ 0.016757] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations650worker2 # [ 0.018023] cpuidle: using governor ladder651lb # [ 0.018248] thermal_sys: Registered thermal governor 'bang_bang'652worker1 # [ 0.017151] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations653worker2 # [ 0.018029] cpuidle: using governor menu654lb # [ 0.018252] thermal_sys: Registered thermal governor 'step_wise'655worker1 # [ 0.017172] audit: initializing netlink subsys (disabled)656client # [ 0.016950] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations657worker2 # [ 0.018214] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.658lb # [ 0.018257] thermal_sys: Registered thermal governor 'user_space'659worker1 # [ 0.017707] thermal_sys: Registered thermal governor 'fair_share'660client # [ 0.016971] audit: initializing netlink subsys (disabled)661worker2 # [ 0.018229] ASID allocator initialised with 65536 entries662worker1 # [ 0.017709] thermal_sys: Registered thermal governor 'bang_bang'663lb # [ 0.018260] thermal_sys: Registered thermal governor 'power_allocator'664worker2 # [ 0.019472] Serial: AMBA PL011 UART driver665client # [ 0.017527] thermal_sys: Registered thermal governor 'fair_share'666worker1 # [ 0.017713] thermal_sys: Registered thermal governor 'step_wise'667client # [ 0.017529] thermal_sys: Registered thermal governor 'bang_bang'668lb # [ 0.018292] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1669worker2 # [ 0.024863] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1670lb # [ 0.018301] cpuidle: using governor ladder671worker1 # [ 0.017718] thermal_sys: Registered thermal governor 'user_space'672worker2 # [ 0.025027] printk: console [ttyAMA0] enabled673client # [ 0.017533] thermal_sys: Registered thermal governor 'step_wise'674lb # [ 0.018307] cpuidle: using governor menu675worker1 # [ 0.017721] thermal_sys: Registered thermal governor 'power_allocator'676client # [ 0.017541] thermal_sys: Registered thermal governor 'user_space'677lb # [ 0.018502] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.678worker2 # [ 0.144748] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages679worker1 # [ 0.017742] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1680client # [ 0.017543] thermal_sys: Registered thermal governor 'power_allocator'681lb # [ 0.018518] ASID allocator initialised with 65536 entries682worker1 # [ 0.017752] cpuidle: using governor ladder683worker2 # [ 0.144769] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page684lb # [ 0.019714] Serial: AMBA PL011 UART driver685client # [ 0.017567] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1686worker1 # [ 0.017757] cpuidle: using governor menu687client # [ 0.017577] cpuidle: using governor ladder688worker2 # [ 0.144774] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages689lb # [ 0.024941] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1690worker1 # [ 0.017953] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.691client # [ 0.017583] cpuidle: using governor menu692worker2 # [ 0.144778] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page693lb # [ 0.025086] printk: console [ttyAMA0] enabled694worker1 # [ 0.017967] ASID allocator initialised with 65536 entries695client # [ 0.017764] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.696worker1 # [ 0.019179] Serial: AMBA PL011 UART driver697worker2 # [ 0.144788] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages698client # [ 0.017779] ASID allocator initialised with 65536 entries699lb # [ 0.145892] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages700worker2 # [ 0.144793] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page701client # [ 0.018925] Serial: AMBA PL011 UART driver702worker1 # [ 0.024370] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1703worker1 # [ 0.024489] printk: console [ttyAMA0] enabled704lb # [ 0.145910] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page705worker2 # [ 0.144797] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages706client # [ 0.023999] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1707worker2 # [ 0.144801] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page708lb # [ 0.145915] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages709client # [ 0.024129] printk: console [ttyAMA0] enabled710worker1 # [ 0.143933] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages711lb # [ 0.145920] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page712worker1 # [ 0.143953] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page713lb # [ 0.145924] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages714worker1 # [ 0.143958] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages715lb # [ 0.145928] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page716worker2 # [ 0.152015] fbcon: Taking over console717worker1 # [ 0.143963] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page718worker2 # [ 0.152030] ACPI: Interpreter disabled.719lb # [ 0.145933] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages720worker2 # [ 0.153883] iommu: Default domain type: Translated721worker1 # [ 0.143967] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages722lb # [ 0.145937] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page723worker2 # [ 0.153893] iommu: DMA domain TLB invalidation policy: strict mode724worker1 # [ 0.143972] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page725worker2 # [ 0.155573] SCSI subsystem initialized726worker1 # [ 0.143976] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages727worker1 # [ 0.143980] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page728lb # [ 0.153234] fbcon: Taking over console729lb # [ 0.153247] ACPI: Interpreter disabled.730client # [ 0.140715] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages731lb # [ 0.155117] iommu: Default domain type: Translated732client # [ 0.140731] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page733lb # [ 0.155127] iommu: DMA domain TLB invalidation policy: strict mode734worker1 # [ 0.151348] fbcon: Taking over console735client # [ 0.140737] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages736worker1 # [ 0.151363] ACPI: Interpreter disabled.737lb # [ 0.156849] SCSI subsystem initialized738client # [ 0.140741] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page739worker1 # [ 0.153206] iommu: Default domain type: Translated740client # [ 0.140745] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages741worker1 # [ 0.153216] iommu: DMA domain TLB invalidation policy: strict mode742client # [ 0.140750] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page743worker1 # [ 0.154947] SCSI subsystem initialized744client # [ 0.140754] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages745client # [ 0.140758] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page746worker2 # [ 0.163233] usbcore: registered new interface driver usbfs747worker2 # [ 0.163280] usbcore: registered new interface driver hub748client # [ 0.148215] fbcon: Taking over console749worker2 # [ 0.163300] usbcore: registered new device driver usb750client # [ 0.148230] ACPI: Interpreter disabled.751worker2 # [ 0.163573] pps_core: LinuxPPS API ver. 1 registered752worker2 # [ 0.163579] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>753worker2 # [ 0.163590] PTP clock support registered754lb # [ 0.164301] usbcore: registered new interface driver usbfs755worker2 # [ 0.163630] EDAC MC: Ver: 3.0.0756lb # [ 0.164332] usbcore: registered new interface driver hub757worker2 # [ 0.168279] scmi_core: SCMI protocol bus registered758lb # [ 0.164351] usbcore: registered new device driver usb759lb # [ 0.164642] pps_core: LinuxPPS API ver. 1 registered760worker1 # [ 0.162358] usbcore: registered new interface driver usbfs761worker2 # [ 0.169267] FPGA manager framework762worker2 # [ 0.170281] vgaarb: loaded763worker1 # [ 0.162391] usbcore: registered new interface driver hub764lb # [ 0.164648] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>765worker1 # [ 0.162409] usbcore: registered new device driver usb766lb # [ 0.164658] PTP clock support registered767lb # [ 0.164708] EDAC MC: Ver: 3.0.0768worker1 # [ 0.162686] pps_core: LinuxPPS API ver. 1 registered769worker2 # [ 0.170918] clocksource: Switched to clocksource arch_sys_counter770lb # [ 0.169356] scmi_core: SCMI protocol bus registered771worker2 # [ 0.172278] VFS: Disk quotas dquot_6.6.0772worker1 # [ 0.162692] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>773worker1 # [ 0.162702] PTP clock support registered774worker2 # [ 0.172309] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)775worker1 # [ 0.162743] EDAC MC: Ver: 3.0.0776lb # [ 0.170350] FPGA manager framework777lb # [ 0.171313] vgaarb: loaded778client # [ 0.157262] iommu: Default domain type: Translated779worker2 # [ 0.173901] netfs: FS-Cache loaded780worker1 # [ 0.167340] scmi_core: SCMI protocol bus registered781worker2 # [ 0.174028] pnp: PnP ACPI: disabled782client # [ 0.157272] iommu: DMA domain TLB invalidation policy: strict mode783client # [ 0.157612] SCSI subsystem initialized784worker1 # [ 0.168285] FPGA manager framework785lb # [ 0.171956] clocksource: Switched to clocksource arch_sys_counter786worker1 # [ 0.169281] vgaarb: loaded787lb # [ 0.172548] VFS: Disk quotas dquot_6.6.0788client # [ 0.159571] usbcore: registered new interface driver usbfs789lb # [ 0.172575] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)790client # [ 0.159602] usbcore: registered new interface driver hub791worker1 # [ 0.169914] clocksource: Switched to clocksource arch_sys_counter792client # [ 0.159620] usbcore: registered new device driver usb793lb # [ 0.174976] netfs: FS-Cache loaded794worker1 # [ 0.170560] VFS: Disk quotas dquot_6.6.0795lb # [ 0.175112] pnp: PnP ACPI: disabled796client # [ 0.159885] pps_core: LinuxPPS API ver. 1 registered797worker1 # [ 0.170587] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)798client # [ 0.159891] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>799worker1 # [ 0.172906] netfs: FS-Cache loaded800client # [ 0.159901] PTP clock support registered801worker1 # [ 0.173017] pnp: PnP ACPI: disabled802client # [ 0.159949] EDAC MC: Ver: 3.0.0803client # [ 0.164568] scmi_core: SCMI protocol bus registered804client # [ 0.165560] FPGA manager framework805client # [ 0.166492] vgaarb: loaded806worker2 # [ 0.180101] NET: Registered PF_INET protocol family807client # [ 0.167113] clocksource: Switched to clocksource arch_sys_counter808worker2 # [ 0.180377] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)809lb # [ 0.180959] NET: Registered PF_INET protocol family810client # [ 0.171059] VFS: Disk quotas dquot_6.6.0811lb # [ 0.181234] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)812client # [ 0.171093] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)813worker1 # [ 0.178876] NET: Registered PF_INET protocol family814worker1 # [ 0.179150] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)815client # [ 0.174681] netfs: FS-Cache loaded816client # [ 0.174808] pnp: PnP ACPI: disabled817client # [ 0.178668] NET: Registered PF_INET protocol family818client # [ 0.178945] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)819worker2 # [ 0.209337] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)820worker2 # [ 0.209385] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)821worker2 # [ 0.209411] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)822worker2 # [ 0.209485] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)823worker2 # [ 0.209629] TCP: Hash tables configured (established 16384 bind 16384)824lb # [ 0.210492] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)825worker2 # [ 0.209713] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)826lb # [ 0.210544] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)827worker2 # [ 0.209748] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)828lb # [ 0.210573] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)829worker1 # [ 0.208296] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)830worker2 # [ 0.209784] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)831lb # [ 0.210650] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)832worker1 # [ 0.208343] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)833worker2 # [ 0.209881] NET: Registered PF_UNIX/PF_LOCAL protocol family834lb # [ 0.210793] TCP: Hash tables configured (established 16384 bind 16384)835worker2 # [ 0.209904] NET: Registered PF_XDP protocol family836worker1 # [ 0.208372] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)837worker2 # [ 0.209923] PCI: CLS 0 bytes, default 64838lb # [ 0.210881] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)839worker1 # [ 0.208444] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)840worker2 # [ 0.210160] Trying to unpack rootfs image as initramfs...841lb # [ 0.210917] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)842worker1 # [ 0.208589] TCP: Hash tables configured (established 16384 bind 16384)843lb # [ 0.210967] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)844worker1 # [ 0.208674] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)845lb # [ 0.211057] NET: Registered PF_UNIX/PF_LOCAL protocol family846worker1 # [ 0.208712] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)847lb # [ 0.211081] NET: Registered PF_XDP protocol family848worker1 # [ 0.208747] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)849lb # [ 0.211099] PCI: CLS 0 bytes, default 64850worker1 # [ 0.208835] NET: Registered PF_UNIX/PF_LOCAL protocol family851lb # [ 0.211350] Trying to unpack rootfs image as initramfs...852worker1 # [ 0.208861] NET: Registered PF_XDP protocol family853worker1 # [ 0.208880] PCI: CLS 0 bytes, default 64854worker1 # [ 0.209118] Trying to unpack rootfs image as initramfs...855worker2 # [ 0.224554] kvm [1]: HYP mode not available856client # [ 0.207649] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)857client # [ 0.207696] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)858client # [ 0.207726] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)859client # [ 0.207800] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)860lb # [ 0.225751] kvm [1]: HYP mode not available861client # [ 0.207943] TCP: Hash tables configured (established 16384 bind 16384)862client # [ 0.208025] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)863worker1 # [ 0.223355] kvm [1]: HYP mode not available864client # [ 0.208060] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)865client # [ 0.208096] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)866client # [ 0.208215] NET: Registered PF_UNIX/PF_LOCAL protocol family867client # [ 0.208238] NET: Registered PF_XDP protocol family868client # [ 0.208257] PCI: CLS 0 bytes, default 64869client # [ 0.208507] Trying to unpack rootfs image as initramfs...870client # [ 0.224747] kvm [1]: HYP mode not available871worker2 # [ 0.315495] Initialise system trusted keyrings872worker2 # [ 0.316241] workingset: timestamp_bits=42 max_order=19 bucket_order=0873worker2 # [ 0.317500] squashfs: version 4.0 (2009/01/31) Phillip Lougher874worker2 # [ 0.318266] 9p: Installing v9fs 9p2000 file system support875lb # [ 0.320545] Initialise system trusted keyrings876lb # [ 0.321294] workingset: timestamp_bits=42 max_order=19 bucket_order=0877worker1 # [ 0.318582] Initialise system trusted keyrings878lb # [ 0.322595] squashfs: version 4.0 (2009/01/31) Phillip Lougher879worker1 # [ 0.319338] workingset: timestamp_bits=42 max_order=19 bucket_order=0880lb # [ 0.323371] 9p: Installing v9fs 9p2000 file system support881worker1 # [ 0.320656] squashfs: version 4.0 (2009/01/31) Phillip Lougher882worker1 # [ 0.321419] 9p: Installing v9fs 9p2000 file system support883client # [ 0.315995] Initialise system trusted keyrings884client # [ 0.316752] workingset: timestamp_bits=42 max_order=19 bucket_order=0885client # [ 0.317994] squashfs: version 4.0 (2009/01/31) Phillip Lougher886client # [ 0.318757] 9p: Installing v9fs 9p2000 file system support887worker2 # [ 0.338874] Key type asymmetric registered888worker2 # [ 0.338901] Asymmetric key parser 'x509' registered889worker2 # [ 0.347016] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)890worker2 # [ 0.348092] io scheduler mq-deadline registered891worker2 # [ 0.348103] io scheduler kyber registered892lb # [ 0.352041] Key type asymmetric registered893lb # [ 0.352073] Asymmetric key parser 'x509' registered894lb # [ 0.352159] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)895worker1 # [ 0.350134] Key type asymmetric registered896lb # [ 0.354344] io scheduler mq-deadline registered897worker1 # [ 0.350164] Asymmetric key parser 'x509' registered898lb # [ 0.354354] io scheduler kyber registered899worker1 # [ 0.350254] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)900worker1 # [ 0.352375] io scheduler mq-deadline registered901worker1 # [ 0.352385] io scheduler kyber registered902worker2 # [ 0.359056] pl061_gpio 9030000.pl061: PL061 GPIO chip registered903worker2 # [ 0.360455] ledtrig-cpu: registered to indicate activity on CPUs904client # [ 0.347395] Key type asymmetric registered905worker2 # [ 0.360991] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:906client # [ 0.347424] Asymmetric key parser 'x509' registered907worker2 # [ 0.361012] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000908client # [ 0.347491] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)909client # [ 0.349645] io scheduler mq-deadline registered910worker2 # [ 0.361025] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000911client # [ 0.349656] io scheduler kyber registered912worker2 # [ 0.361033] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000913lb # [ 0.364094] pl061_gpio 9030000.pl061: PL061 GPIO chip registered914worker2 # [ 0.361058] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits915worker2 # [ 0.361083] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]916worker1 # [ 0.362059] pl061_gpio 9030000.pl061: PL061 GPIO chip registered917worker2 # [ 0.361163] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00918lb # [ 0.365490] ledtrig-cpu: registered to indicate activity on CPUs919worker2 # [ 0.361173] pci_bus 0000:00: root bus resource [bus 00-ff]920lb # [ 0.365901] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:921worker2 # [ 0.361179] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]922worker2 # [ 0.361185] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]923worker1 # [ 0.363472] ledtrig-cpu: registered to indicate activity on CPUs924lb # [ 0.365918] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000925worker2 # [ 0.361190] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]926worker1 # [ 0.363863] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:927lb # [ 0.365931] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000928worker2 # [ 0.361246] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint929worker1 # [ 0.363880] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000930lb # [ 0.365939] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000931worker2 # [ 0.361668] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint932worker1 # [ 0.363906] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000933lb # [ 0.365966] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits934worker2 # [ 0.361855] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]935client # [ 0.359249] pl061_gpio 9030000.pl061: PL061 GPIO chip registered936worker2 # [ 0.361872] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]937worker1 # [ 0.363915] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000938lb # [ 0.365989] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]939worker2 # [ 0.361901] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]940worker1 # [ 0.363934] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits941lb # [ 0.366064] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00942worker2 # [ 0.361917] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]943lb # [ 0.366074] pci_bus 0000:00: root bus resource [bus 00-ff]944worker1 # [ 0.363957] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]945worker2 # [ 0.362372] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint946lb # [ 0.366080] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]947client # [ 0.360562] ledtrig-cpu: registered to indicate activity on CPUs948worker1 # [ 0.364049] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00949worker2 # [ 0.362557] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]950lb # [ 0.366085] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]951worker1 # [ 0.364058] pci_bus 0000:00: root bus resource [bus 00-ff]952client # [ 0.360964] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:953worker2 # [ 0.362575] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]954lb # [ 0.366090] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]955worker1 # [ 0.364064] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]956worker2 # [ 0.362604] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]957client # [ 0.360981] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000958worker1 # [ 0.364069] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]959lb # [ 0.366147] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint960client # [ 0.360993] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000961worker1 # [ 0.364074] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]962lb # [ 0.366601] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint963worker2 # [ 0.381908] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint964lb # [ 0.366798] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]965client # [ 0.361001] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000966worker1 # [ 0.364162] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint967worker2 # [ 0.382096] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]968lb # [ 0.366814] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]969client # [ 0.361029] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits970worker1 # [ 0.364599] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint971worker2 # [ 0.382112] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]972lb # [ 0.366844] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]973worker1 # [ 0.364787] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]974worker2 # [ 0.382143] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]975client # [ 0.361052] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]976lb # [ 0.366860] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]977worker1 # [ 0.364803] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]978client # [ 0.361128] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00979worker2 # [ 0.382600] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint980worker1 # [ 0.364833] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]981lb # [ 0.367319] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint982client # [ 0.361137] pci_bus 0000:00: root bus resource [bus 00-ff]983worker2 # [ 0.382788] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]984lb # [ 0.367505] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]985worker1 # [ 0.364849] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]986client # [ 0.361144] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]987worker2 # [ 0.382804] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]988lb # [ 0.367521] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]989worker1 # [ 0.365308] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint990client # [ 0.361149] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]991worker2 # [ 0.382836] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]992lb # [ 0.367550] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]993worker1 # [ 0.365491] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]994client # [ 0.361154] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]995worker1 # [ 0.365507] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]996client # [ 0.361212] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint997worker1 # [ 0.365536] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]998lb # [ 0.386610] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint999client # [ 0.361647] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1000lb # [ 0.386801] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1001client # [ 0.361835] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1002lb # [ 0.386817] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1003client # [ 0.361852] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1004worker1 # [ 0.384389] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1005lb # [ 0.386849] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1006client # [ 0.361881] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1007worker1 # [ 0.384575] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1008lb # [ 0.387318] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1009client # [ 0.361897] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1010worker1 # [ 0.384591] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1011lb # [ 0.387505] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1012worker1 # [ 0.384621] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1013client # [ 0.362348] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1014lb # [ 0.387520] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1015client # [ 0.362535] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1016worker1 # [ 0.385302] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1017lb # [ 0.387553] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1018client # [ 0.362550] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1019worker1 # [ 0.385484] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1020client # [ 0.362580] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1021worker1 # [ 0.385500] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1022worker1 # [ 0.385532] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1023client # [ 0.363034] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1024client # [ 0.382760] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1025client # [ 0.382779] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1026client # [ 0.382808] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1027worker2 # [ 0.395384] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1028client # [ 0.385442] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1029worker2 # [ 0.395581] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1030client # [ 0.385625] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1031worker2 # [ 0.395597] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1032client # [ 0.385640] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1033worker2 # [ 0.395627] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1034client # [ 0.385671] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1035worker2 # [ 0.396087] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1036client # [ 0.386124] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1037worker2 # [ 0.396270] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1038client # [ 0.386306] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1039worker2 # [ 0.396285] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1040client # [ 0.386321] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1041worker2 # [ 0.396314] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1042client # [ 0.386349] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1043worker2 # [ 0.396761] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1044client # [ 0.386798] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1045worker2 # [ 0.396951] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1046client # [ 0.386981] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1047worker2 # [ 0.396967] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1048lb # [ 0.400052] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1049client # [ 0.386997] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1050lb # [ 0.400248] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1051worker2 # [ 0.396996] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1052client # [ 0.387025] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1053lb # [ 0.400264] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1054worker2 # [ 0.397012] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1055worker1 # [ 0.398036] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1056lb # [ 0.400294] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1057worker2 # [ 0.397461] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1058worker1 # [ 0.398239] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1059worker2 # [ 0.397646] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1060lb # [ 0.400778] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1061worker1 # [ 0.398255] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1062lb # [ 0.400964] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1063worker2 # [ 0.397675] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1064worker1 # [ 0.398285] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1065lb # [ 0.400980] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1066worker2 # [ 0.398121] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1067worker1 # [ 0.398759] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1068lb # [ 0.401009] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1069worker2 # [ 0.398308] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1070worker1 # [ 0.398941] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1071lb # [ 0.401474] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1072worker2 # [ 0.398337] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1073worker1 # [ 0.398957] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1074lb # [ 0.401664] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1075worker2 # [ 0.398729] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1076worker1 # [ 0.398986] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1077lb # [ 0.401680] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1078worker2 # [ 0.398909] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1079worker1 # [ 0.399445] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1080lb # [ 0.401712] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1081worker1 # [ 0.399628] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1082worker2 # [ 0.399171] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1083lb # [ 0.401728] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1084worker2 # [ 0.399460] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1085worker1 # [ 0.399643] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1086lb # [ 0.402191] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1087worker2 # [ 0.399477] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1088worker1 # [ 0.399672] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1089lb # [ 0.402390] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1090worker2 # [ 0.399506] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1091worker1 # [ 0.399688] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1092lb # [ 0.402420] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1093worker2 # [ 0.399962] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1094worker1 # [ 0.400180] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1095lb # [ 0.402876] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1096worker2 # [ 0.400141] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1097worker1 # [ 0.400366] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1098lb # [ 0.403066] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1099worker2 # [ 0.400157] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1100worker1 # [ 0.400396] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1101lb # [ 0.403096] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1102worker2 # [ 0.400186] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1103worker1 # [ 0.400850] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1104worker2 # [ 0.400752] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1105lb # [ 0.403485] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1106worker1 # [ 0.401035] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1107lb # [ 0.403668] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1108worker2 # [ 0.400763] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1109worker1 # [ 0.401065] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1110client # [ 0.403539] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1111lb # [ 0.403911] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1112worker2 # [ 0.400770] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1113client # [ 0.403737] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1114worker1 # [ 0.401454] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1115lb # [ 0.404207] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1116worker2 # [ 0.400820] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1117client # [ 0.403753] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1118worker1 # [ 0.401637] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1119lb # [ 0.404224] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1120client # [ 0.403782] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1121worker2 # [ 0.400867] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1122lb # [ 0.404253] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1123worker1 # [ 0.401883] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1124client # [ 0.403798] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1125worker1 # [ 0.402170] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1126worker2 # [ 0.400914] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1127lb # [ 0.404725] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1128worker1 # [ 0.402187] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1129client # [ 0.404261] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1130lb # [ 0.404909] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1131worker2 # [ 0.400960] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1132worker1 # [ 0.402217] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1133client # [ 0.404449] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1134lb # [ 0.404924] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1135worker2 # [ 0.401005] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1136client # [ 0.404479] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1137worker1 # [ 0.402695] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1138lb # [ 0.404953] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1139worker2 # [ 0.401051] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1140worker1 # [ 0.402878] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1141lb # [ 0.405542] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1142client # [ 0.404935] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1143worker1 # [ 0.402894] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1144worker2 # [ 0.401097] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1145client # [ 0.405123] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1146lb # [ 0.405553] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1147worker1 # [ 0.402923] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1148client # [ 0.405152] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1149worker2 # [ 0.401143] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1150lb # [ 0.405559] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1151worker1 # [ 0.403498] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1152worker2 # [ 0.401189] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1153client # [ 0.405536] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1154lb # [ 0.405604] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1155worker1 # [ 0.403509] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1156client # [ 0.405720] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1157worker2 # [ 0.401260] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1158lb # [ 0.405652] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1159worker1 # [ 0.403515] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1160worker2 # [ 0.401305] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1161client # [ 0.405966] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1162lb # [ 0.405700] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1163client # [ 0.406246] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1164worker1 # [ 0.403558] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1165worker2 # [ 0.401327] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1166client # [ 0.406263] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1167lb # [ 0.405747] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1168worker2 # [ 0.401348] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1169worker1 # [ 0.403605] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1170client # [ 0.406292] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1171worker2 # [ 0.401370] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1172lb # [ 0.405794] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1173worker1 # [ 0.403652] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1174client # [ 0.406755] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1175worker2 # [ 0.401394] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1176lb # [ 0.405840] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1177worker1 # [ 0.403698] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1178client # [ 0.406940] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1179worker2 # [ 0.401416] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1180lb # [ 0.405888] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1181client # [ 0.406956] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1182worker1 # [ 0.403743] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1183worker2 # [ 0.401437] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1184lb # [ 0.405934] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1185client # [ 0.406986] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1186worker1 # [ 0.403789] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1187worker2 # [ 0.401459] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1188client # [ 0.407576] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1189lb # [ 0.405981] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1190worker2 # [ 0.401481] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1191worker1 # [ 0.403835] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1192client # [ 0.407587] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1193lb # [ 0.406041] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1194worker2 # [ 0.401503] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1195worker1 # [ 0.403881] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1196client # [ 0.407593] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1197lb # [ 0.406087] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1198worker2 # [ 0.401525] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1199worker1 # [ 0.403927] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1200client # [ 0.407637] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1201lb # [ 0.406108] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1202worker2 # [ 0.401548] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1203worker1 # [ 0.403987] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1204worker2 # [ 0.401569] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1205lb # [ 0.406130] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1206client # [ 0.407684] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1207worker2 # [ 0.401591] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1208worker1 # [ 0.404033] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1209lb # [ 0.406151] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1210client # [ 0.407732] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1211worker2 # [ 0.401612] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1212worker1 # [ 0.404054] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1213lb # [ 0.406176] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1214worker2 # [ 0.401634] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1215client # [ 0.407779] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1216lb # [ 0.406198] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1217worker1 # [ 0.404076] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1218worker2 # [ 0.401655] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1219client # [ 0.407826] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1220worker1 # [ 0.404098] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1221lb # [ 0.406230] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1222worker2 # [ 0.401677] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1223worker1 # [ 0.404122] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1224lb # [ 0.406252] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1225client # [ 0.407873] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1226worker2 # [ 0.401698] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1227lb # [ 0.406274] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1228worker1 # [ 0.404143] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1229worker2 # [ 0.401720] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1230client # [ 0.407920] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1231worker1 # [ 0.404165] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1232lb # [ 0.406296] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1233worker2 # [ 0.401742] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1234client # [ 0.407966] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1235worker2 # [ 0.401769] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1236lb # [ 0.406318] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1237worker1 # [ 0.404187] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1238client # [ 0.408013] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1239worker2 # [ 0.401778] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1240worker1 # [ 0.404209] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1241lb # [ 0.406340] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1242client # [ 0.408074] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1243worker2 # [ 0.401784] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1244lb # [ 0.406362] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1245worker1 # [ 0.404234] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1246client # [ 0.408122] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1247lb # [ 0.406383] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1248worker1 # [ 0.404257] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1249client # [ 0.408151] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1250lb # [ 0.406404] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1251worker1 # [ 0.404282] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1252lb # [ 0.406425] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1253client # [ 0.408173] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1254worker1 # [ 0.404304] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1255lb # [ 0.406446] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1256client # [ 0.408194] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1257worker1 # [ 0.404326] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1258lb # [ 0.406468] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1259worker1 # [ 0.404349] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1260client # [ 0.408218] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1261lb # [ 0.406489] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1262worker1 # [ 0.404370] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1263client # [ 0.408240] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1264lb # [ 0.406511] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1265worker1 # [ 0.404392] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1266client # [ 0.408262] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1267worker2 # [ 0.458944] pci 0000:00:0a.0: enabling device (0000 -> 0002)1268lb # [ 0.406533] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1269worker1 # [ 0.404413] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1270client # [ 0.408284] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1271worker2 # [ 0.459482] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 55239 usecs1272lb # [ 0.406560] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1273worker1 # [ 0.404434] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1274client # [ 0.408306] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1275lb # [ 0.406569] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1276worker1 # [ 0.404456] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1277client # [ 0.408327] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1278lb # [ 0.406573] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1279worker1 # [ 0.404479] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1280client # [ 0.408350] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1281lb # [ 0.407737] pci 0000:00:0a.0: enabling device (0000 -> 0002)1282worker1 # [ 0.404504] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1283client # [ 0.408372] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1284worker1 # [ 0.404514] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1285client # [ 0.408393] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1286worker1 # [ 0.404519] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1287client # [ 0.408415] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1288worker1 # [ 0.405688] pci 0000:00:0a.0: enabling device (0000 -> 0002)1289client # [ 0.408436] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1290client # [ 0.408457] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1291client # [ 0.408479] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1292client # [ 0.408500] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1293client # [ 0.408522] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1294client # [ 0.408544] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1295client # [ 0.408565] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1296client # [ 0.408591] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1297lb # [ 0.468071] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 58925 usecs1298client # [ 0.408600] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1299client # [ 0.408605] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1300worker1 # [ 0.466035] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 58938 usecs1301client # [ 0.409764] pci 0000:00:0a.0: enabling device (0000 -> 0002)1302worker2 # [ 0.485774] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1303worker2 # [ 0.488711] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1304worker2 # [ 0.490796] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1305lb # [ 0.488531] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1306lb # [ 0.490725] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1307worker1 # [ 0.486374] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1308worker1 # [ 0.488555] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1309client # [ 0.481625] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1310worker2 # [ 0.500320] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1311worker2 # [ 0.502491] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1312lb # [ 0.501769] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1313lb # [ 0.503759] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1314worker1 # [ 0.498481] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1315worker1 # [ 0.500382] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1316client # [ 0.491734] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1317worker2 # [ 0.513115] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1318client # [ 0.495987] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1319client # [ 0.497894] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1320lb # [ 0.514804] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1321worker1 # [ 0.510084] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1322worker1 # [ 0.512031] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1323worker2 # [ 0.519123] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1324worker2 # [ 0.521259] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1325lb # [ 0.520966] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1326client # [ 0.507683] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1327client # [ 0.510593] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1328lb # [ 0.523263] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1329worker2 # [ 0.531818] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1330worker2 # [ 0.533688] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1331worker1 # [ 0.522131] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1332worker1 # [ 0.524276] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1333worker1 # [ 0.526317] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1334lb # [ 0.533489] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1335client # [ 0.522002] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1336lb # [ 0.535756] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1337worker1 # [ 0.528149] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1338client # [ 0.524955] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1339client # [ 0.527019] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1340worker2 # [ 0.544021] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1341worker1 # [ 0.538507] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1342worker2 # [ 0.549920] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1343lb # [ 0.546816] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1344lb # [ 0.550276] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1345client # [ 0.536584] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1346client # [ 0.539106] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1347worker2 # [ 0.556175] msm_serial: driver initialized1348worker2 # [ 0.556319] SuperH (H)SCI(F) driver initialized1349worker2 # [ 0.556373] STM32 USART driver initialized1350worker1 # [ 0.554595] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1351worker1 # [ 0.557236] msm_serial: driver initialized1352worker1 # [ 0.557372] SuperH (H)SCI(F) driver initialized1353worker1 # [ 0.557425] STM32 USART driver initialized1354lb # [ 0.563190] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1355lb # [ 0.565839] msm_serial: driver initialized1356lb # [ 0.565973] SuperH (H)SCI(F) driver initialized1357lb # [ 0.566028] STM32 USART driver initialized1358client # [ 0.560302] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1359client # [ 0.562868] msm_serial: driver initialized1360client # [ 0.563003] SuperH (H)SCI(F) driver initialized1361client # [ 0.563056] STM32 USART driver initialized1362worker2 # [ 0.587600] loop: module loaded1363worker2 # [ 0.587772] virtio_blk virtio5: 1/0/0 default/read/poll queues1364worker2 # [ 0.588459] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1365worker1 # [ 0.587193] loop: module loaded1366worker1 # [ 0.587377] virtio_blk virtio5: 1/0/0 default/read/poll queues1367worker1 # [ 0.588135] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1368lb # [ 0.598765] loop: module loaded1369lb # [ 0.598957] virtio_blk virtio5: 1/0/0 default/read/poll queues1370lb # [ 0.599680] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1371worker1 # [ 0.594591] megasas: 07.734.00.00-rc11372worker1 # [ 0.595316] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1373worker1 # [ 0.597502] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001374worker2 # [ 0.599580] megasas: 07.734.00.00-rc11375worker1 # [ 0.597526] Intel/Sharp Extended Query Table at 0x00311376worker2 # [ 0.600305] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1377worker2 # [ 0.602509] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001378worker2 # [ 0.602567] Intel/Sharp Extended Query Table at 0x00311379worker2 # [ 0.604225] Using buffer write method1380lb # [ 0.604522] megasas: 07.734.00.00-rc11381worker2 # [ 0.604291] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1382client # [ 0.593999] loop: module loaded1383lb # [ 0.605209] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1384client # [ 0.594171] virtio_blk virtio5: 1/0/0 default/read/poll queues1385client # [ 0.594877] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1386worker2 # [ 0.605922] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001387worker2 # [ 0.605945] Intel/Sharp Extended Query Table at 0x00311388worker1 # [ 0.606943] Using buffer write method1389worker1 # [ 0.607013] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1390worker1 # [ 0.609409] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001391worker1 # [ 0.609431] Intel/Sharp Extended Query Table at 0x00311392client # [ 0.599683] megasas: 07.734.00.00-rc11393client # [ 0.600412] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1394client # [ 0.602271] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001395client # [ 0.602296] Intel/Sharp Extended Query Table at 0x00311396lb # [ 0.615996] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001397lb # [ 0.616023] Intel/Sharp Extended Query Table at 0x00311398lb # [ 0.617708] Using buffer write method1399lb # [ 0.617770] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1400worker2 # [ 0.625404] Using buffer write method1401worker2 # [ 0.625441] Concatenating MTD devices:1402worker2 # [ 0.625446] (0): "0.flash"1403worker2 # [ 0.625450] (1): "0.flash"1404worker2 # [ 0.625454] into device "0.flash"1405worker1 # [ 0.623028] Using buffer write method1406worker1 # [ 0.623067] Concatenating MTD devices:1407worker1 # [ 0.623071] (0): "0.flash"1408worker1 # [ 0.623075] (1): "0.flash"1409worker1 # [ 0.623079] into device "0.flash"1410client # [ 0.611915] Using buffer write method1411client # [ 0.611981] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1412client # [ 0.613579] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001413lb # [ 0.628596] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001414client # [ 0.613607] Intel/Sharp Extended Query Table at 0x00311415lb # [ 0.628619] Intel/Sharp Extended Query Table at 0x00311416lb # [ 0.629945] Using buffer write method1417lb # [ 0.629970] Concatenating MTD devices:1418lb # [ 0.629974] (0): "0.flash"1419lb # [ 0.629978] (1): "0.flash"1420lb # [ 0.629982] into device "0.flash"1421client # [ 0.627159] Using buffer write method1422client # [ 0.627184] Concatenating MTD devices:1423client # [ 0.627188] (0): "0.flash"1424client # [ 0.627192] (1): "0.flash"1425client # [ 0.627195] into device "0.flash"1426worker2 # [ 0.856997] Freeing initrd memory: 26148K1427worker2 # [ 0.863124] tun: Universal TUN/TAP device driver, 1.61428worker2 # [ 0.866843] thunder_xcv, ver 1.01429worker1 # [ 0.858859] Freeing initrd memory: 26148K1430worker2 # [ 0.866889] thunder_bgx, ver 1.01431worker2 # [ 0.866914] nicpf, ver 1.01432client # [ 0.851525] Freeing initrd memory: 26160K1433worker1 # [ 0.864839] tun: Universal TUN/TAP device driver, 1.61434worker2 # [ 0.868657] e1000: Intel(R) PRO/1000 Network Driver1435worker2 # [ 0.868668] e1000: Copyright (c) 1999-2006 Intel Corporation.1436worker2 # [ 0.868692] e1000e: Intel(R) PRO/1000 Network Driver1437lb # [ 0.869550] Freeing initrd memory: 26140K1438client # [ 0.857506] tun: Universal TUN/TAP device driver, 1.61439worker2 # [ 0.868700] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1440worker2 # [ 0.868731] igb: Intel(R) Gigabit Ethernet Network Driver1441worker2 # [ 0.868737] igb: Copyright (c) 2007-2014 Intel Corporation.1442worker1 # [ 0.868627] thunder_xcv, ver 1.01443worker2 # [ 0.868761] igbvf: Intel(R) Gigabit Virtual Function Network Driver1444worker1 # [ 0.868671] thunder_bgx, ver 1.01445worker1 # [ 0.868694] nicpf, ver 1.01446worker2 # [ 0.868767] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1447worker2 # [ 0.868917] sky2: driver version 1.301448worker1 # [ 0.869245] e1000: Intel(R) PRO/1000 Network Driver1449worker2 # [ 0.870529] usbcore: registered new interface driver usb-storage1450worker1 # [ 0.869253] e1000: Copyright (c) 1999-2006 Intel Corporation.1451lb # [ 0.875622] tun: Universal TUN/TAP device driver, 1.61452worker1 # [ 0.869279] e1000e: Intel(R) PRO/1000 Network Driver1453client # [ 0.861226] thunder_xcv, ver 1.01454worker2 # [ 0.870587] usbcore: registered new interface driver usbserial_generic1455client # [ 0.861273] thunder_bgx, ver 1.01456worker1 # [ 0.869288] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1457client # [ 0.861295] nicpf, ver 1.01458worker2 # [ 0.870605] usbserial: USB Serial support registered for generic1459worker1 # [ 0.869320] igb: Intel(R) Gigabit Ethernet Network Driver1460client # [ 0.861846] e1000: Intel(R) PRO/1000 Network Driver1461worker2 # [ 0.871827] ehci-pci 0000:00:0a.0: EHCI Host Controller1462worker1 # [ 0.869326] igb: Copyright (c) 2007-2014 Intel Corporation.1463client # [ 0.861853] e1000: Copyright (c) 1999-2006 Intel Corporation.1464worker2 # [ 0.871859] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11465client # [ 0.861878] e1000e: Intel(R) PRO/1000 Network Driver1466worker1 # [ 0.869351] igbvf: Intel(R) Gigabit Virtual Function Network Driver1467worker2 # [ 0.872095] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001468worker1 # [ 0.869357] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1469client # [ 0.861886] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1470lb # [ 0.879453] thunder_xcv, ver 1.01471lb # [ 0.879498] thunder_bgx, ver 1.01472worker1 # [ 0.869488] sky2: driver version 1.301473client # [ 0.861916] igb: Intel(R) Gigabit Ethernet Network Driver1474lb # [ 0.879521] nicpf, ver 1.01475client # [ 0.861922] igb: Copyright (c) 2007-2014 Intel Corporation.1476worker2 # [ 0.883028] hv_vmbus: registering driver hyperv_keyboard1477client # [ 0.861944] igbvf: Intel(R) Gigabit Virtual Function Network Driver1478worker2 # [ 0.884353] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001479client # [ 0.861950] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1480worker2 # [ 0.885394] hub 1-0:1.0: USB hub found1481client # [ 0.862087] sky2: driver version 1.301482worker1 # [ 0.878679] ehci-pci 0000:00:0a.0: EHCI Host Controller1483worker2 # [ 0.885941] hub 1-0:1.0: 6 ports detected1484worker1 # [ 0.878708] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11485worker1 # [ 0.878973] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001486client # [ 0.870705] usbcore: registered new interface driver usb-storage1487lb # [ 0.881285] e1000: Intel(R) PRO/1000 Network Driver1488client # [ 0.870784] usbcore: registered new interface driver usbserial_generic1489worker2 # [ 0.887054] rtc-pl031 9010000.pl031: registered as rtc01490lb # [ 0.881294] e1000: Copyright (c) 1999-2006 Intel Corporation.1491worker1 # [ 0.881372] usbcore: registered new interface driver usb-storage1492client # [ 0.870798] usbserial: USB Serial support registered for generic1493lb # [ 0.881319] e1000e: Intel(R) PRO/1000 Network Driver1494worker2 # [ 0.887080] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:27:28 UTC (1789982848)1495worker1 # [ 0.881477] usbcore: registered new interface driver usbserial_generic1496lb # [ 0.881328] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1497worker2 # [ 0.887416] i2c_dev: i2c /dev entries driver1498worker1 # [ 0.881493] usbserial: USB Serial support registered for generic1499lb # [ 0.881356] igb: Intel(R) Gigabit Ethernet Network Driver1500client # [ 0.873911] ehci-pci 0000:00:0a.0: EHCI Host Controller1501lb # [ 0.881362] igb: Copyright (c) 2007-2014 Intel Corporation.1502client # [ 0.873938] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11503worker1 # [ 0.884304] hv_vmbus: registering driver hyperv_keyboard1504lb # [ 0.881384] igbvf: Intel(R) Gigabit Virtual Function Network Driver1505client # [ 0.874143] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001506lb # [ 0.881390] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1507worker1 # [ 0.885760] rtc-pl031 9010000.pl031: registered as rtc01508lb # [ 0.881530] sky2: driver version 1.301509lb # [ 0.883183] usbcore: registered new interface driver usb-storage1510worker1 # [ 0.885786] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:27:28 UTC (1789982848)1511client # [ 0.876786] hv_vmbus: registering driver hyperv_keyboard1512lb # [ 0.883257] usbcore: registered new interface driver usbserial_generic1513worker1 # [ 0.887825] i2c_dev: i2c /dev entries driver1514worker2 # [ 0.892285] sdhci: Secure Digital Host Controller Interface driver1515lb # [ 0.883272] usbserial: USB Serial support registered for generic1516client # [ 0.878311] rtc-pl031 9010000.pl031: registered as rtc01517worker2 # [ 0.892296] sdhci: Copyright(c) Pierre Ossman1518lb # [ 0.883864] hv_vmbus: registering driver hyperv_keyboard1519lb # [ 0.884674] ehci-pci 0000:00:0a.0: EHCI Host Controller1520worker2 # [ 0.892567] Synopsys Designware Multimedia Card Interface Driver1521client # [ 0.878340] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:27:28 UTC (1789982848)1522client # [ 0.878629] i2c_dev: i2c /dev entries driver1523worker2 # [ 0.892952] sdhci-pltfm: SDHCI platform and OF driver helper1524lb # [ 0.884701] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11525worker2 # [ 0.894585] hid: raw HID events driver (C) Jiri Kosina1526lb # [ 0.884925] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001527worker2 # [ 0.894840] usbcore: registered new interface driver usbhid1528worker2 # [ 0.894847] usbhid: USB HID core driver1529lb # [ 0.896513] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001530lb # [ 0.896845] hub 1-0:1.0: USB hub found1531worker1 # [ 0.891023] sdhci: Secure Digital Host Controller Interface driver1532lb # [ 0.896867] hub 1-0:1.0: 6 ports detected1533worker1 # [ 0.891032] sdhci: Copyright(c) Pierre Ossman1534worker1 # [ 0.891301] Synopsys Designware Multimedia Card Interface Driver1535lb # [ 0.899080] rtc-pl031 9010000.pl031: registered as rtc01536worker1 # [ 0.891683] sdhci-pltfm: SDHCI platform and OF driver helper1537worker2 # [ 0.900425] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1538worker1 # [ 0.893271] hid: raw HID events driver (C) Jiri Kosina1539client # [ 0.883481] sdhci: Secure Digital Host Controller Interface driver1540worker2 # [ 0.901958] drop_monitor: Initializing network drop monitor service1541lb # [ 0.899109] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:27:28 UTC (1789982848)1542worker1 # [ 0.893522] usbcore: registered new interface driver usbhid1543client # [ 0.883491] sdhci: Copyright(c) Pierre Ossman1544worker2 # [ 0.902133] NET: Registered PF_INET6 protocol family1545lb # [ 0.899424] i2c_dev: i2c /dev entries driver1546worker1 # [ 0.893529] usbhid: USB HID core driver1547client # [ 0.883749] Synopsys Designware Multimedia Card Interface Driver1548worker1 # [ 0.898370] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001549worker2 # [ 0.905130] Segment Routing with IPv61550client # [ 0.884113] sdhci-pltfm: SDHCI platform and OF driver helper1551worker2 # [ 0.905163] In-situ OAM (IOAM) with IPv61552worker1 # [ 0.899374] hub 1-0:1.0: USB hub found1553client # [ 0.885762] hid: raw HID events driver (C) Jiri Kosina1554worker2 # [ 0.905192] NET: Registered PF_PACKET protocol family1555worker1 # [ 0.899842] hub 1-0:1.0: 6 ports detected1556client # [ 0.886025] usbcore: registered new interface driver usbhid1557client # [ 0.886031] usbhid: USB HID core driver1558worker2 # [ 0.906787] 9pnet: Installing 9P2000 support1559client # [ 0.891446] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001560client # [ 0.892494] hub 1-0:1.0: USB hub found1561worker1 # [ 0.901387] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1562client # [ 0.892996] hub 1-0:1.0: 6 ports detected1563lb # [ 0.904307] sdhci: Secure Digital Host Controller Interface driver1564worker2 # [ 0.909644] Key type dns_resolver registered1565worker1 # [ 0.903999] drop_monitor: Initializing network drop monitor service1566lb # [ 0.904317] sdhci: Copyright(c) Pierre Ossman1567worker1 # [ 0.904131] NET: Registered PF_INET6 protocol family1568client # [ 0.893935] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1569lb # [ 0.904587] Synopsys Designware Multimedia Card Interface Driver1570worker1 # [ 0.906021] Segment Routing with IPv61571lb # [ 0.904956] sdhci-pltfm: SDHCI platform and OF driver helper1572worker1 # [ 0.906041] In-situ OAM (IOAM) with IPv61573client # [ 0.896522] drop_monitor: Initializing network drop monitor service1574lb # [ 0.906655] hid: raw HID events driver (C) Jiri Kosina1575worker1 # [ 0.906070] NET: Registered PF_PACKET protocol family1576client # [ 0.896669] NET: Registered PF_INET6 protocol family1577lb # [ 0.906892] usbcore: registered new interface driver usbhid1578lb # [ 0.906898] usbhid: USB HID core driver1579client # [ 0.898500] Segment Routing with IPv61580client # [ 0.898519] In-situ OAM (IOAM) with IPv61581client # [ 0.898548] NET: Registered PF_PACKET protocol family1582worker1 # [ 0.907731] 9pnet: Installing 9P2000 support1583worker1 # [ 0.909786] Key type dns_resolver registered1584lb # [ 0.912531] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1585lb # [ 0.914054] drop_monitor: Initializing network drop monitor service1586lb # [ 0.914233] NET: Registered PF_INET6 protocol family1587worker2 # [ 0.916035] registered taskstats version 11588client # [ 0.900216] 9pnet: Installing 9P2000 support1589worker2 # [ 0.916177] Loading compiled-in X.509 certificates1590client # [ 0.902325] Key type dns_resolver registered1591lb # [ 0.917244] Segment Routing with IPv61592lb # [ 0.917270] In-situ OAM (IOAM) with IPv61593lb # [ 0.917310] NET: Registered PF_PACKET protocol family1594lb # [ 0.918959] 9pnet: Installing 9P2000 support1595worker1 # [ 0.916616] registered taskstats version 11596worker1 # [ 0.916774] Loading compiled-in X.509 certificates1597worker2 # [ 0.924723] Demotion targets for Node 0: null1598worker2 # [ 0.924844] Key type .fscrypt registered1599worker2 # [ 0.924851] Key type fscrypt-provisioning registered1600lb # [ 0.921836] Key type dns_resolver registered1601worker2 # [ 0.924943] ima: No TPM chip found, activating TPM-bypass!1602client # [ 0.908977] registered taskstats version 11603worker2 # [ 0.924962] ima: Allocated hash algorithm: sha11604client # [ 0.909118] Loading compiled-in X.509 certificates1605worker2 # [ 0.924981] ima: No architecture policies found1606worker2 # [ 0.929015] input: gpio-keys as /devices/platform/gpio-keys/input/input01607worker1 # [ 0.925443] Demotion targets for Node 0: null1608worker1 # [ 0.925557] Key type .fscrypt registered1609worker1 # [ 0.925563] Key type fscrypt-provisioning registered1610worker1 # [ 0.925657] ima: No TPM chip found, activating TPM-bypass!1611client # [ 0.917749] Demotion targets for Node 0: null1612lb # [ 0.928280] registered taskstats version 11613client # [ 0.917857] Key type .fscrypt registered1614worker1 # [ 0.925676] ima: Allocated hash algorithm: sha11615lb # [ 0.928438] Loading compiled-in X.509 certificates1616worker1 # [ 0.925700] ima: No architecture policies found1617client # [ 0.917863] Key type fscrypt-provisioning registered1618client # [ 0.917958] ima: No TPM chip found, activating TPM-bypass!1619client # [ 0.917977] ima: Allocated hash algorithm: sha11620worker1 # [ 0.929826] input: gpio-keys as /devices/platform/gpio-keys/input/input01621client # [ 0.917996] ima: No architecture policies found1622client # [ 0.921917] input: gpio-keys as /devices/platform/gpio-keys/input/input01623lb # [ 0.937010] Demotion targets for Node 0: null1624lb # [ 0.937114] Key type .fscrypt registered1625lb # [ 0.937121] Key type fscrypt-provisioning registered1626lb # [ 0.937219] ima: No TPM chip found, activating TPM-bypass!1627lb # [ 0.937240] ima: Allocated hash algorithm: sha11628lb # [ 0.937261] ima: No architecture policies found1629lb # [ 0.941261] input: gpio-keys as /devices/platform/gpio-keys/input/input01630worker2 # [ 0.946567] clk: Disabling unused clocks1631worker2 # [ 0.946592] PM: genpd: Disabling unused power domains1632worker2 # [ 0.950751] Freeing unused kernel memory: 4736K1633worker2 # [ 0.951634] Run /init as init process1634worker1 # [ 0.947953] clk: Disabling unused clocks1635worker1 # [ 0.947983] PM: genpd: Disabling unused power domains1636client # [ 0.939291] clk: Disabling unused clocks1637client # [ 0.939317] PM: genpd: Disabling unused power domains1638worker1 # [ 0.952232] Freeing unused kernel memory: 4736K1639worker1 # [ 0.952438] Run /init as init process1640client # [ 0.943528] Freeing unused kernel memory: 4736K1641client # [ 0.943734] Run /init as init process1642lb # [ 0.959184] clk: Disabling unused clocks1643lb # [ 0.959220] PM: genpd: Disabling unused power domains1644worker2 # [ 0.965256] systemd[1]: Successfully made /usr/ read-only.1645lb # [ 0.963513] Freeing unused kernel memory: 4736K1646lb # [ 0.963872] Run /init as init process1647worker1 # [ 0.968256] systemd[1]: Successfully made /usr/ read-only.1648client # [ 0.961402] systemd[1]: Successfully made /usr/ read-only.1649lb # [ 0.978193] systemd[1]: Successfully made /usr/ read-only.1650worker2 # [ 1.131018] usb 1-1: new high-speed USB device number 2 using ehci-pci1651lb # [ 1.144043] usb 1-1: new high-speed USB device number 2 using ehci-pci1652worker1 # [ 1.146005] usb 1-1: new high-speed USB device number 2 using ehci-pci1653client # [ 1.139198] usb 1-1: new high-speed USB device number 2 using ehci-pci1654worker2 # [ 1.281015] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11655lb # [ 1.296265] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11656worker2 # [ 1.300018] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1657worker1 # [ 1.298295] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11658worker2 # [ 1.309799] systemd[1]: Detected virtualization qemu.1659worker2 # [ 1.311373] systemd[1]: Detected architecture arm64.1660client # [ 1.291613] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11661worker2 # [ 1.312915] systemd[1]: Running in initrd.1662worker2 # [ 1.314966] systemd[1]: Initializing machine ID from random generator.1663worker2 # [ 1.317119] systemd[1]: Hostname set to <worker2>.1664worker1 # [ 1.304463] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1665worker1 # [ 1.304528] systemd[1]: Detected virtualization qemu.1666client # [ 1.297701] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1667lb # [ 1.313013] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1668worker1 # [ 1.304613] systemd[1]: Detected architecture arm64.1669client # [ 1.297757] systemd[1]: Detected virtualization qemu.1670lb # [ 1.325338] systemd[1]: Detected virtualization qemu.1671worker1 # [ 1.304638] systemd[1]: Running in initrd.1672client # [ 1.297829] systemd[1]: Detected architecture arm64.1673lb # [ 1.327326] systemd[1]: Detected architecture arm64.1674worker1 # [ 1.305622] systemd[1]: Initializing machine ID from random generator.1675client # [ 1.297852] systemd[1]: Running in initrd.1676lb # [ 1.329315] systemd[1]: Running in initrd.1677worker1 # [ 1.305908] systemd[1]: Hostname set to <worker1>.1678client # [ 1.298758] systemd[1]: Initializing machine ID from random generator.1679lb # [ 1.331895] systemd[1]: Initializing machine ID from random generator.1680client # [ 1.299026] systemd[1]: Hostname set to <client>.1681lb # [ 1.334921] systemd[1]: Hostname set to <lb>.1682worker2 # [ 1.367219] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683lb # [ 1.384258] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01684client # [ 1.379388] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01685worker1 # [ 1.394174] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01686worker2 # [ 1.487017] usb 1-2: new high-speed USB device number 3 using ehci-pci1687lb # [ 1.504011] usb 1-2: new high-speed USB device number 3 using ehci-pci1688client # [ 1.499166] usb 1-2: new high-speed USB device number 3 using ehci-pci1689worker1 # [ 1.514015] usb 1-2: new high-speed USB device number 3 using ehci-pci1690worker2 # [ 1.593517] systemd[1]: bpf-restrict-fs: LSM BPF program attached1691worker1 # [ 1.632285] systemd[1]: bpf-restrict-fs: LSM BPF program attached1692worker2 # [ 1.641745] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21693worker2 # [ 1.643906] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01694client # [ 1.635442] systemd[1]: bpf-restrict-fs: LSM BPF program attached1695lb # [ 1.656227] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21696worker2 # [ 1.663423] systemd[1]: Queued start job for default target Initrd Default Target.1697lb # [ 1.662001] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01698worker2 # [ 1.673126] systemd[1]: Created slice Slice /system/modprobe.1699worker2 # [ 1.674180] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1700worker2 # [ 1.675389] systemd[1]: Expecting device /dev/disk/by-label/nixos...1701worker2 # [ 1.676316] systemd[1]: Reached target Path Units.1702worker2 # [ 1.677049] systemd[1]: Reached target Slice Units.1703worker2 # [ 1.677754] systemd[1]: Reached target Swaps.1704worker2 # [ 1.678468] systemd[1]: Reached target Timer Units.1705worker2 # [ 1.679417] systemd[1]: Listening on D-Bus System Message Bus Socket.1706client # [ 1.659679] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21707worker2 # [ 1.680468] systemd[1]: Listening on Journal Socket (/dev/log).1708worker2 # [ 1.681471] systemd[1]: Listening on Journal Sockets.1709worker2 # [ 1.682341] systemd[1]: Listening on udev Control Socket.1710worker1 # [ 1.673231] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21711worker2 # [ 1.683344] systemd[1]: Listening on udev Kernel Socket.1712client # [ 1.664517] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01713worker2 # [ 1.684123] systemd[1]: Reached target Socket Units.1714worker2 # [ 1.686257] systemd[1]: Starting Create List of Static Device Nodes...1715worker1 # [ 1.682258] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01716worker2 # [ 1.694578] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1717worker2 # [ 1.695680] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1718worker2 # [ 1.708215] systemd[1]: Mounting Kernel Configuration File System...1719lb # [ 1.721616] systemd[1]: bpf-restrict-fs: LSM BPF program attached1720worker2 # [ 1.727077] systemd[1]: Starting Journal Service...1721worker2 # [ 1.729167] systemd[1]: Starting Load Kernel Modules...1722worker2 # [ 1.729934] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1723worker2 # [ 1.742721] systemd[1]: Starting Coldplug All udev Devices...1724worker1 # [ 1.746539] systemd[1]: Queued start job for default target Initrd Default Target.1725worker2 # [ 1.759099] systemd[1]: Finished Create List of Static Device Nodes.1726worker2 # [ 1.759928] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1727worker1 # [ 1.756372] systemd[1]: Created slice Slice /system/modprobe.1728worker2 # [ 1.760216] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1729worker2 # [ 1.760462] systemd[1]: Mounted Kernel Configuration File System.1730worker1 # [ 1.757586] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1731worker1 # [ 1.759023] systemd[1]: Expecting device /dev/disk/by-label/nixos...1732worker1 # [ 1.760091] systemd[1]: Reached target Path Units.1733client # [ 1.750442] systemd[1]: Queued start job for default target Initrd Default Target.1734worker1 # [ 1.760915] systemd[1]: Reached target Slice Units.1735worker1 # [ 1.761745] systemd[1]: Reached target Swaps.1736worker1 # [ 1.762644] systemd[1]: Reached target Timer Units.1737worker1 # [ 1.763647] systemd[1]: Listening on D-Bus System Message Bus Socket.1738worker1 # [ 1.764879] systemd[1]: Listening on Journal Socket (/dev/log).1739worker1 # [ 1.766044] systemd[1]: Listening on Journal Sockets.1740worker1 # [ 1.767048] systemd[1]: Listening on udev Control Socket.1741worker1 # [ 1.768079] systemd[1]: Listening on udev Kernel Socket.1742worker1 # [ 1.769000] systemd[1]: Reached target Socket Units.1743client # [ 1.761501] systemd[1]: Created slice Slice /system/modprobe.1744worker1 # [ 1.771552] systemd[1]: Starting Create List of Static Device Nodes...1745client # [ 1.762702] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1746lb # [ 1.778094] systemd[1]: Queued start job for default target Initrd Default Target.1747client # [ 1.764130] systemd[1]: Expecting device /dev/disk/by-label/nixos...1748worker2 # [ 1.775398] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1749client # [ 1.764167] systemd[1]: Reached target Path Units.1750client # [ 1.764187] systemd[1]: Reached target Slice Units.1751client # [ 1.764208] systemd[1]: Reached target Swaps.1752client # [ 1.764228] systemd[1]: Reached target Timer Units.1753client # [ 1.764415] systemd[1]: Listening on D-Bus System Message Bus Socket.1754client # [ 1.764560] systemd[1]: Listening on Journal Socket (/dev/log).1755lb # [ 1.785818] systemd[1]: Created slice Slice /system/modprobe.1756client # [ 1.764714] systemd[1]: Listening on Journal Sockets.1757worker1 # [ 1.782091] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1758client # [ 1.764850] systemd[1]: Listening on udev Control Socket.1759lb # [ 1.786752] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1760client # [ 1.764969] systemd[1]: Listening on udev Kernel Socket.1761lb # [ 1.787735] systemd[1]: Expecting device /dev/disk/by-label/nixos...1762worker1 # [ 1.783243] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1763client # [ 1.764990] systemd[1]: Reached target Socket Units.1764lb # [ 1.788606] systemd[1]: Reached target Path Units.1765lb # [ 1.789295] systemd[1]: Reached target Slice Units.1766lb # [ 1.789925] systemd[1]: Reached target Swaps.1767client # [ 1.775888] systemd[1]: Starting Create List of Static Device Nodes...1768lb # [ 1.790521] systemd[1]: Reached target Timer Units.1769worker1 # [ 1.786603] systemd[1]: Mounting Kernel Configuration File System...1770lb # [ 1.791291] systemd[1]: Listening on D-Bus System Message Bus Socket.1771lb # [ 1.792262] systemd[1]: Listening on Journal Socket (/dev/log).1772lb # [ 1.793137] systemd[1]: Listening on Journal Sockets.1773lb # [ 1.793894] systemd[1]: Listening on udev Control Socket.1774lb # [ 1.794699] systemd[1]: Listening on udev Kernel Socket.1775lb # [ 1.795391] systemd[1]: Reached target Socket Units.1776worker2 # [ 1.798224] systemd-journald[73]: Collecting audit messages is disabled.1777client # [ 1.783277] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1778lb # [ 1.797342] systemd[1]: Starting Create List of Static Device Nodes...1779client # [ 1.784422] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1780worker1 # [ 1.798501] systemd[1]: Starting Journal Service...1781lb # [ 1.803836] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1782client # [ 1.791268] systemd[1]: Mounting Kernel Configuration File System...1783lb # [ 1.807418] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1784worker2 # [ 1.813267] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1785worker1 # [ 1.811432] systemd[1]: Starting Load Kernel Modules...1786worker2 # [ 1.819612] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1787worker1 # [ 1.811552] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1788worker2 # [ 1.823014] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1789worker2 # [ 1.825307] systemd[1]: Starting Create Static Device Nodes in /dev...1790lb # [ 1.821326] systemd[1]: Mounting Kernel Configuration File System...1791client # [ 1.810060] systemd[1]: Starting Journal Service...1792worker1 # [ 1.828114] systemd[1]: Starting Coldplug All udev Devices...1793client # [ 1.819743] systemd[1]: Starting Load Kernel Modules...1794client # [ 1.820615] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1795lb # [ 1.836178] systemd[1]: Starting Journal Service...1796lb # [ 1.838342] systemd[1]: Starting Load Kernel Modules...1797worker2 # [ 1.838699] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01798lb # [ 1.839126] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1799client # [ 1.827342] systemd[1]: Starting Coldplug All udev Devices...1800lb # [ 1.848161] systemd[1]: Starting Coldplug All udev Devices...1801worker2 # [ 1.850987] [drm] features: -virgl +edid -resource_blob -host_visible1802worker2 # [ 1.851004] [drm] features: -context_init1803worker2 # [ 1.851763] [drm] number of scanouts: 11804worker2 # [ 1.851783] [drm] number of cap sets: 01805worker1 # [ 1.846198] systemd[1]: Finished Create List of Static Device Nodes.1806worker1 # [ 1.847118] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1807worker1 # [ 1.847419] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1808worker1 # [ 1.847646] systemd[1]: Mounted Kernel Configuration File System.1809client # [ 1.847312] systemd[1]: Finished Create List of Static Device Nodes.1810lb # [ 1.862508] systemd[1]: Finished Create List of Static Device Nodes.1811client # [ 1.848317] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1812worker2 # [ 1.859741] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1813lb # [ 1.863328] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1814client # [ 1.848598] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1815worker2 # [ 1.859765] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01816worker1 # [ 1.858455] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1817lb # [ 1.872271] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1818lb # [ 1.872764] systemd[1]: Mounted Kernel Configuration File System.1819client # [ 1.859593] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1820client # [ 1.859849] systemd[1]: Mounted Kernel Configuration File System.1821lb # [ 1.880435] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1822worker2 # [ 1.883639] systemd[1]: Finished Create Static Device Nodes in /dev.1823worker2 # [ 1.884839] systemd[1]: Reached target Preparation for Local File Systems.1824worker2 # [ 1.885737] systemd[1]: Reached target Local File Systems.1825worker1 # [ 1.891145] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1826worker2 # [ 1.891185] Console: switching to colour frame buffer device 160x501827worker2 # [ 1.897786] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1828worker2 # [ 1.901946] systemd[1]: Starting Rule-based Manager for Device Events and Files...1829worker1 # [ 1.894348] systemd-journald[73]: Collecting audit messages is disabled.1830client # [ 1.890362] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1831lb # [ 1.907421] systemd-journald[73]: Collecting audit messages is disabled.1832worker1 # [ 1.904520] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1833worker1 # [ 1.907313] systemd[1]: Starting Create Static Device Nodes in /dev...1834lb # [ 1.916879] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1835client # [ 1.903798] systemd-journald[73]: Collecting audit messages is disabled.1836client # [ 1.908665] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1837client # [ 1.911081] systemd[1]: Starting Create Static Device Nodes in /dev...1838worker2 # [ 1.927165] systemd[1]: Finished Load Kernel Modules.1839worker1 # [ 1.917990] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1840worker1 # [ 1.922635] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01841worker2 # [ 1.930468] systemd[1]: Starting Apply Kernel Variables...1842worker1 # [ 1.922872] [drm] features: -virgl +edid -resource_blob -host_visible1843worker1 # [ 1.922882] [drm] features: -context_init1844worker1 # [ 1.923607] [drm] number of scanouts: 11845worker1 # [ 1.923625] [drm] number of cap sets: 01846lb # [ 1.932706] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1847client # [ 1.915267] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1848lb # [ 1.934973] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1849client # [ 1.920006] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01850client # [ 1.920246] [drm] features: -virgl +edid -resource_blob -host_visible1851client # [ 1.920256] [drm] features: -context_init1852client # [ 1.920988] [drm] number of scanouts: 11853client # [ 1.921011] [drm] number of cap sets: 01854lb # [ 1.937455] systemd[1]: Starting Create Static Device Nodes in /dev...1855worker1 # [ 1.938384] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1856lb # [ 1.944835] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01857lb # [ 1.945081] [drm] features: -virgl +edid -resource_blob -host_visible1858worker1 # [ 1.938407] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01859lb # [ 1.945091] [drm] features: -context_init1860lb # [ 1.945807] [drm] number of scanouts: 11861lb # [ 1.945825] [drm] number of cap sets: 01862client # [ 1.939578] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1863client # [ 1.939599] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01864lb # [ 1.960445] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1865lb # [ 1.960468] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01866worker1 # [ 1.966371] systemd[1]: Finished Create Static Device Nodes in /dev.1867worker1 # [ 1.966731] systemd[1]: Reached target Preparation for Local File Systems.1868worker1 # [ 1.966761] systemd[1]: Reached target Local File Systems.1869worker1 # [ 1.974390] systemd[1]: Starting Rule-based Manager for Device Events and Files...1870worker2 # [ 1.987674] systemd[1]: Finished Apply Kernel Variables.1871worker2 # [ 1.990666] systemd[1]: Started Journal Service.1872worker1 # [ 1.982177] Console: switching to colour frame buffer device 160x501873worker1 # [ 1.988649] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1874client # [ 1.981739] systemd[1]: Finished Create Static Device Nodes in /dev.1875worker2 # [ 1.984413] systemd-modules-load[74]: Inserted module 'dm_mod'1876client # [ 1.981934] systemd[1]: Reached target Preparation for Local File Systems.1877lb # [ 1.996633] systemd[1]: Finished Create Static Device Nodes in /dev.1878client # [ 1.981956] systemd[1]: Reached target Local File Systems.1879lb # [ 1.996831] systemd[1]: Reached target Preparation for Local File Systems.1880lb # [ 1.996859] systemd[1]: Reached target Local File Systems.1881worker2 # [ 1.988292] systemd-modules-load[74]: Module 'virtio_balloon' is built in1882client # [ 1.985857] systemd[1]: Starting Rule-based Manager for Device Events and Files...1883worker2 # [ 1.989387] systemd-modules-load[74]: Module 'virtio_console' is built in1884worker2 # [ 1.990454] systemd-modules-load[74]: Inserted module 'virtio_gpu'1885worker2 # [ 1.991422] systemd-modules-load[74]: Module 'virtio_rng' is built in1886lb # [ 2.000826] Console: switching to colour frame buffer device 160x501887lb # [ 2.007505] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1888lb # [ 2.010932] systemd[1]: Starting Rule-based Manager for Device Events and Files...1889client # [ 1.998851] Console: switching to colour frame buffer device 160x501890client # [ 2.001641] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1891worker1 # [ 2.014670] systemd[1]: Finished Load Kernel Modules.1892worker2 # [ 2.009593] systemd[1]: Starting Create System Files and Directories...1893worker2 # [ 2.010994] systemd-udevd[79]: Using default interface naming scheme 'v261'.1894worker1 # [ 2.022302] systemd[1]: Starting Apply Kernel Variables...1895client # [ 2.019402] systemd[1]: Finished Load Kernel Modules.1896client # [ 2.022513] systemd[1]: Starting Apply Kernel Variables...1897worker2 # [ 2.029350] systemd[1]: Finished Create System Files and Directories.1898lb # [ 2.044217] systemd[1]: Finished Load Kernel Modules.1899lb # [ 2.047585] systemd[1]: Starting Apply Kernel Variables...1900worker2 # [ 2.046716] systemd[1]: Started Rule-based Manager for Device Events and Files.1901worker1 # [ 2.077664] systemd[1]: Finished Apply Kernel Variables.1902worker1 # [ 2.080624] systemd[1]: Started Journal Service.1903worker1 # [ 2.068506] systemd-modules-load[74]: Inserted module 'dm_mod'1904client # [ 2.078543] systemd[1]: Finished Apply Kernel Variables.1905client # [ 2.065977] systemd-modules-load[74]: Inserted module 'dm_mod'1906client # [ 2.082138] systemd[1]: Started Journal Service.1907lb # [ 2.096633] systemd[1]: Finished Apply Kernel Variables.1908worker1 # [ 2.076462] systemd-modules-load[74]: Module 'virtio_balloon' is built in1909worker1 # [ 2.077587] systemd-modules-load[74]: Module 'virtio_console' is built in1910worker1 # [ 2.078647] systemd-modules-load[74]: Inserted module 'virtio_gpu'1911worker1 # [ 2.079678] systemd-modules-load[74]: Module 'virtio_rng' is built in1912client # [ 2.076123] systemd-modules-load[74]: Module 'virtio_balloon' is built in1913client # [ 2.080313] systemd-modules-load[74]: Module 'virtio_console' is built in1914client # [ 2.082402] systemd-modules-load[74]: Inserted module 'virtio_gpu'1915worker1 # [ 2.089909] systemd[1]: Starting Create System Files and Directories...1916worker1 # [ 2.093530] systemd-udevd[79]: Using default interface naming scheme 'v261'.1917client # [ 2.088323] systemd-modules-load[74]: Module 'virtio_rng' is built in1918lb # [ 2.117320] systemd[1]: Started Journal Service.1919client # [ 2.091442] systemd[1]: Starting Create System Files and Directories...1920worker2 # [ 2.108112] systemd[1]: Starting Virtual Console Setup...1921client # [ 2.094529] systemd-udevd[79]: Using default interface naming scheme 'v261'.1922lb # [ 2.108389] systemd-modules-load[74]: Inserted module 'dm_mod'1923lb # [ 2.112401] systemd-modules-load[74]: Module 'virtio_balloon' is built in1924lb # [ 2.113532] systemd-modules-load[74]: Module 'virtio_console' is built in1925lb # [ 2.114601] systemd-modules-load[74]: Inserted module 'virtio_gpu'1926lb # [ 2.115603] systemd-modules-load[74]: Module 'virtio_rng' is built in1927lb # [ 2.122694] systemd-udevd[80]: Using default interface naming scheme 'v261'.1928worker1 # [ 2.125230] systemd[1]: Finished Create System Files and Directories.1929lb # [ 2.133008] systemd[1]: Starting Create System Files and Directories...1930worker1 # [ 2.127521] systemd[1]: Started Rule-based Manager for Device Events and Files.1931client # [ 2.125648] systemd[1]: Finished Create System Files and Directories.1932client # [ 2.136459] systemd[1]: Started Rule-based Manager for Device Events and Files.1933worker2 # [ 2.156500] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1934lb # [ 2.153688] systemd[1]: Finished Create System Files and Directories.1935worker2 # [ 2.159951] systemd[1]: Finished Virtual Console Setup.1936lb # [ 2.164947] systemd[1]: Started Rule-based Manager for Device Events and Files.1937worker1 # [ 2.196238] systemd[1]: Starting Virtual Console Setup...1938client # [ 2.192120] systemd[1]: Starting Virtual Console Setup...1939lb # [ 2.217604] systemd[1]: Starting Virtual Console Setup...1940worker1 # [ 2.240503] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1941worker1 # [ 2.244033] systemd[1]: Finished Virtual Console Setup.1942client # [ 2.244496] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1943client # [ 2.247785] systemd[1]: Finished Virtual Console Setup.1944lb # [ 2.268479] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1945lb # [ 2.271761] systemd[1]: Finished Virtual Console Setup.1946worker2 # [ 2.768795] systemd[1]: Finished Coldplug All udev Devices.1947worker2 # [ 2.769743] systemd[1]: Reached target System Initialization.1948worker2 # [ 2.770575] systemd[1]: Reached target Basic System.1949client # [ 2.840346] systemd[1]: Finished Coldplug All udev Devices.1950client # [ 2.841579] systemd[1]: Reached target System Initialization.1951client # [ 2.844516] systemd[1]: Reached target Basic System.1952worker1 # [ 2.851597] systemd[1]: Finished Coldplug All udev Devices.1953worker1 # [ 2.852715] systemd[1]: Reached target System Initialization.1954worker1 # [ 2.853557] systemd[1]: Reached target Basic System.1955lb # [ 2.884658] systemd[1]: Finished Coldplug All udev Devices.1956lb # [ 2.886430] systemd[1]: Reached target System Initialization.1957lb # [ 2.888661] systemd[1]: Reached target Basic System.1958worker2 # [ 2.908225] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1959worker2 # [ 2.943822] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1960worker2 # [ 2.951593] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1961client # [ 2.979950] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1962worker2 # [ 3.000158] systemd[1]: Found device /dev/disk/by-label/nixos.1963worker2 # [ 3.002178] systemd[1]: Reached target Initrd Root Device.1964worker2 # [ 3.005038] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1965worker1 # [ 2.999906] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1966worker1 # [ 3.016420] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1967client # [ 3.020552] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1968worker1 # [ 3.029826] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1969lb # [ 3.039196] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1970client # [ 3.027688] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1971worker2 # [ 3.053068] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1972lb # [ 3.055310] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1973worker2 # [ 3.061121] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1974lb # [ 3.060527] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1975worker2 # [ 3.068308] systemd[1]: Mounting /sysroot...1976worker1 # [ 3.084919] systemd[1]: Found device /dev/disk/by-label/nixos.1977worker1 # [ 3.087459] systemd[1]: Reached target Initrd Root Device.1978client # [ 3.082382] systemd[1]: Found device /dev/disk/by-label/nixos.1979client # [ 3.085047] systemd[1]: Reached target Initrd Root Device.1980worker1 # [ 3.092279] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1981worker2 # [ 3.115292] EXT4-fs (vda): mounted filesystem 4679f22f-08ba-435b-9f4a-daa35ddbe0e5 r/w with ordered data mode. Quota mode: none.1982client # [ 3.088574] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1983worker2 # [ 3.106218] systemd[1]: Mounted /sysroot.1984worker2 # [ 3.108638] systemd[1]: Reached target Initrd Root File System.1985worker2 # [ 3.110931] systemd[1]: Starting Mountpoints Configured in the Real Root...1986lb # [ 3.120177] systemd[1]: Found device /dev/disk/by-label/nixos.1987lb # [ 3.122388] systemd[1]: Reached target Initrd Root Device.1988lb # [ 3.126107] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1989worker2 # [ 3.135770] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload.1990worker2 # [ 3.140440] systemd[1]: Reload requested from client PID 119 ('systemd-sysroot') (unit initrd-parse-etc.service)...1991worker2 # [ 3.144825] systemd[1]: Reloading...1992worker1 # [ 3.141839] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1993client # [ 3.141042] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1994worker1 # [ 3.148695] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1995client # [ 3.145786] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1996worker1 # [ 3.153500] systemd[1]: Mounting /sysroot...1997client # [ 3.151863] systemd[1]: Mounting /sysroot...1998lb # [ 3.176633] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1999lb # [ 3.182745] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.2000lb # [ 3.196207] systemd[1]: Mounting /sysroot...
loading 1,659 hidden lines…
3660worker1 # [ 23.088838] nix-grpc-daemon[775]: ts=2026-09-21T09:27:50Z level=info event=scheduler_connected addr=lb:500513661worker1 # [ 23.093902] nix-grpc-daemon[775]: ts=2026-09-21T09:27:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3662worker2 # [ 23.239639] nix-grpc-daemon[777]: ts=2026-09-21T09:27:50Z level=info event=scheduler_connected addr=lb:500513663worker2 # [ 23.244606] nix-grpc-daemon[777]: ts=2026-09-21T09:27:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3664worker1 # [ 23.296639] nix-grpc-daemon[775]: ts=2026-09-21T09:27:50Z level=info event=scheduler_connected addr=lb:500513665worker1 # [ 23.300956] nix-grpc-daemon[775]: ts=2026-09-21T09:27:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3666worker2 # [ 23.449242] nix-grpc-daemon[777]: ts=2026-09-21T09:27:51Z level=info event=scheduler_connected addr=lb:500513667worker2 # [ 23.453879] nix-grpc-daemon[777]: ts=2026-09-21T09:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3668worker1 # [ 23.498399] nix-grpc-daemon[775]: ts=2026-09-21T09:27:51Z level=info event=scheduler_connected addr=lb:500513669worker1 # [ 23.504556] nix-grpc-daemon[775]: ts=2026-09-21T09:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3670worker2 # [ 23.650155] nix-grpc-daemon[777]: ts=2026-09-21T09:27:51Z level=info event=scheduler_connected addr=lb:500513671worker2 # [ 23.655528] nix-grpc-daemon[777]: ts=2026-09-21T09:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3672worker1 # [ 23.705670] nix-grpc-daemon[775]: ts=2026-09-21T09:27:51Z level=info event=scheduler_connected addr=lb:500513673worker1 # [ 23.711234] nix-grpc-daemon[775]: ts=2026-09-21T09:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3674worker2 # [ 23.857167] nix-grpc-daemon[777]: ts=2026-09-21T09:27:51Z level=info event=scheduler_connected addr=lb:500513675worker2 # [ 23.862427] nix-grpc-daemon[777]: ts=2026-09-21T09:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3676worker1 # [ 23.912148] nix-grpc-daemon[775]: ts=2026-09-21T09:27:51Z level=info event=scheduler_connected addr=lb:500513677worker1 # [ 23.917739] nix-grpc-daemon[775]: ts=2026-09-21T09:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3678worker2 # [ 24.064602] nix-grpc-daemon[777]: ts=2026-09-21T09:27:51Z level=info event=scheduler_connected addr=lb:500513679worker2 # [ 24.070116] nix-grpc-daemon[777]: ts=2026-09-21T09:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3680worker1 # [ 24.118995] nix-grpc-daemon[775]: ts=2026-09-21T09:27:51Z level=info event=scheduler_connected addr=lb:500513681worker1 # [ 24.125038] nix-grpc-daemon[775]: ts=2026-09-21T09:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3682worker2 # [ 24.272060] nix-grpc-daemon[777]: ts=2026-09-21T09:27:51Z level=info event=scheduler_connected addr=lb:500513683worker2 # [ 24.277944] nix-grpc-daemon[777]: ts=2026-09-21T09:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3684worker1 # [ 24.329283] nix-grpc-daemon[775]: ts=2026-09-21T09:27:51Z level=info event=scheduler_connected addr=lb:500513685worker1 # [ 24.333669] nix-grpc-daemon[775]: ts=2026-09-21T09:27:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3686worker2 # [ 24.482456] nix-grpc-daemon[777]: ts=2026-09-21T09:27:52Z level=info event=scheduler_connected addr=lb:500513687worker2 # [ 24.487676] nix-grpc-daemon[777]: ts=2026-09-21T09:27:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3688worker1 # [ 24.530437] nix-grpc-daemon[775]: ts=2026-09-21T09:27:52Z level=info event=scheduler_connected addr=lb:500513689worker1 # [ 24.536123] nix-grpc-daemon[775]: ts=2026-09-21T09:27:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3690worker2 # [ 24.689751] nix-grpc-daemon[777]: ts=2026-09-21T09:27:52Z level=info event=scheduler_connected addr=lb:500513691worker2 # [ 24.695512] nix-grpc-daemon[777]: ts=2026-09-21T09:27:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3692worker1 # [ 24.737687] nix-grpc-daemon[775]: ts=2026-09-21T09:27:52Z level=info event=scheduler_connected addr=lb:500513693worker1 # [ 24.743303] nix-grpc-daemon[775]: ts=2026-09-21T09:27:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3694worker2 # [ 24.897949] nix-grpc-daemon[777]: ts=2026-09-21T09:27:52Z level=info event=scheduler_connected addr=lb:500513695worker2 # [ 24.903542] nix-grpc-daemon[777]: ts=2026-09-21T09:27:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3696worker1 # [ 24.944795] nix-grpc-daemon[775]: ts=2026-09-21T09:27:52Z level=info event=scheduler_connected addr=lb:500513697worker1 # [ 24.950487] nix-grpc-daemon[775]: ts=2026-09-21T09:27:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3698worker2 # [ 25.105828] nix-grpc-daemon[777]: ts=2026-09-21T09:27:52Z level=info event=scheduler_connected addr=lb:500513699worker2 # [ 25.111677] nix-grpc-daemon[777]: ts=2026-09-21T09:27:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3700worker1 # [ 25.151793] nix-grpc-daemon[775]: ts=2026-09-21T09:27:52Z level=info event=scheduler_connected addr=lb:500513701worker1 # [ 25.157427] nix-grpc-daemon[775]: ts=2026-09-21T09:27:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3702worker2 # [ 25.313879] nix-grpc-daemon[777]: ts=2026-09-21T09:27:52Z level=info event=scheduler_connected addr=lb:500513703worker2 # [ 25.319869] nix-grpc-daemon[777]: ts=2026-09-21T09:27:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3704worker1 # [ 25.361302] nix-grpc-daemon[775]: ts=2026-09-21T09:27:52Z level=info event=scheduler_connected addr=lb:500513705worker1 # [ 25.366084] nix-grpc-daemon[775]: ts=2026-09-21T09:27:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3706worker2 # [ 25.525463] nix-grpc-daemon[777]: ts=2026-09-21T09:27:53Z level=info event=scheduler_connected addr=lb:500513707worker2 # [ 25.530113] nix-grpc-daemon[777]: ts=2026-09-21T09:27:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3708worker1 # [ 25.561744] nix-grpc-daemon[775]: ts=2026-09-21T09:27:53Z level=info event=scheduler_connected addr=lb:500513709worker1 # [ 25.567429] nix-grpc-daemon[775]: ts=2026-09-21T09:27:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3710worker2 # [ 25.726401] nix-grpc-daemon[777]: ts=2026-09-21T09:27:53Z level=info event=scheduler_connected addr=lb:500513711worker2 # [ 25.730108] nix-grpc-daemon[777]: ts=2026-09-21T09:27:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3712worker1 # [ 25.768608] nix-grpc-daemon[775]: ts=2026-09-21T09:27:53Z level=info event=scheduler_connected addr=lb:500513713worker1 # [ 25.774710] nix-grpc-daemon[775]: ts=2026-09-21T09:27:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3714worker2 # [ 25.931964] nix-grpc-daemon[777]: ts=2026-09-21T09:27:53Z level=info event=scheduler_connected addr=lb:500513715worker2 # [ 25.938024] nix-grpc-daemon[777]: ts=2026-09-21T09:27:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3716worker1 # [ 25.975731] nix-grpc-daemon[775]: ts=2026-09-21T09:27:53Z level=info event=scheduler_connected addr=lb:500513717worker1 # [ 25.979110] nix-grpc-daemon[775]: ts=2026-09-21T09:27:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3718worker2 # [ 26.140443] nix-grpc-daemon[777]: ts=2026-09-21T09:27:53Z level=info event=scheduler_connected addr=lb:500513719worker2 # [ 26.146483] nix-grpc-daemon[777]: ts=2026-09-21T09:27:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3720worker1 # [ 26.180985] nix-grpc-daemon[775]: ts=2026-09-21T09:27:53Z level=info event=scheduler_connected addr=lb:500513721worker1 # [ 26.189015] nix-grpc-daemon[775]: ts=2026-09-21T09:27:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3722worker2 # [ 26.348509] nix-grpc-daemon[777]: ts=2026-09-21T09:27:53Z level=info event=scheduler_connected addr=lb:500513723worker2 # [ 26.354701] nix-grpc-daemon[777]: ts=2026-09-21T09:27:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3724worker1 # [ 26.391889] nix-grpc-daemon[775]: ts=2026-09-21T09:27:54Z level=info event=scheduler_connected addr=lb:500513725worker1 # [ 26.396428] nix-grpc-daemon[775]: ts=2026-09-21T09:27:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3726worker2 # [ 26.559320] nix-grpc-daemon[777]: ts=2026-09-21T09:27:54Z level=info event=scheduler_connected addr=lb:500513727worker2 # [ 26.564146] nix-grpc-daemon[777]: ts=2026-09-21T09:27:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3728worker1 # [ 26.593168] nix-grpc-daemon[775]: ts=2026-09-21T09:27:54Z level=info event=scheduler_connected addr=lb:500513729worker1 # [ 26.599040] nix-grpc-daemon[775]: ts=2026-09-21T09:27:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3730worker2 # [ 26.761422] nix-grpc-daemon[777]: ts=2026-09-21T09:27:54Z level=info event=scheduler_connected addr=lb:500513731worker2 # [ 26.767432] nix-grpc-daemon[777]: ts=2026-09-21T09:27:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3732worker1 # [ 26.800321] nix-grpc-daemon[775]: ts=2026-09-21T09:27:54Z level=info event=scheduler_connected addr=lb:500513733worker1 # [ 26.806167] nix-grpc-daemon[775]: ts=2026-09-21T09:27:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3734worker2 # [ 26.970787] nix-grpc-daemon[777]: ts=2026-09-21T09:27:54Z level=info event=scheduler_connected addr=lb:500513735worker2 # [ 26.976609] nix-grpc-daemon[777]: ts=2026-09-21T09:27:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3736worker1 # [ 27.007850] nix-grpc-daemon[775]: ts=2026-09-21T09:27:54Z level=info event=scheduler_connected addr=lb:500513737worker1 # [ 27.013797] nix-grpc-daemon[775]: ts=2026-09-21T09:27:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3738worker2 # [ 27.178431] nix-grpc-daemon[777]: ts=2026-09-21T09:27:54Z level=info event=scheduler_connected addr=lb:500513739worker2 # [ 27.184204] nix-grpc-daemon[777]: ts=2026-09-21T09:27:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3740worker1 # [ 27.215244] nix-grpc-daemon[775]: ts=2026-09-21T09:27:54Z level=info event=scheduler_connected addr=lb:500513741worker1 # [ 27.221045] nix-grpc-daemon[775]: ts=2026-09-21T09:27:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3742worker2 # [ 27.385902] nix-grpc-daemon[777]: ts=2026-09-21T09:27:55Z level=info event=scheduler_connected addr=lb:500513743worker2 # [ 27.391515] nix-grpc-daemon[777]: ts=2026-09-21T09:27:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3744worker1 # [ 27.420983] nix-grpc-daemon[775]: ts=2026-09-21T09:27:55Z level=info event=scheduler_connected addr=lb:500513745worker1 # [ 27.425314] nix-grpc-daemon[775]: ts=2026-09-21T09:27:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3746worker2 # [ 27.595922] nix-grpc-daemon[777]: ts=2026-09-21T09:27:55Z level=info event=scheduler_connected addr=lb:500513747worker2 # [ 27.600231] nix-grpc-daemon[777]: ts=2026-09-21T09:27:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3748worker1 # [ 27.621993] nix-grpc-daemon[775]: ts=2026-09-21T09:27:55Z level=info event=scheduler_connected addr=lb:500513749worker1 # [ 27.627968] nix-grpc-daemon[775]: ts=2026-09-21T09:27:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3750worker2 # [ 27.798005] nix-grpc-daemon[777]: ts=2026-09-21T09:27:55Z level=info event=scheduler_connected addr=lb:500513751worker2 # [ 27.803652] nix-grpc-daemon[777]: ts=2026-09-21T09:27:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3752worker1 # [ 27.829311] nix-grpc-daemon[775]: ts=2026-09-21T09:27:55Z level=info event=scheduler_connected addr=lb:500513753worker1 # [ 27.835261] nix-grpc-daemon[775]: ts=2026-09-21T09:27:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3754worker2 # [ 28.005475] nix-grpc-daemon[777]: ts=2026-09-21T09:27:55Z level=info event=scheduler_connected addr=lb:500513755worker2 # [ 28.011168] nix-grpc-daemon[777]: ts=2026-09-21T09:27:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3756worker1 # [ 28.036465] nix-grpc-daemon[775]: ts=2026-09-21T09:27:55Z level=info event=scheduler_connected addr=lb:500513757worker1 # [ 28.042332] nix-grpc-daemon[775]: ts=2026-09-21T09:27:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3758worker2 # [ 28.212978] nix-grpc-daemon[777]: ts=2026-09-21T09:27:55Z level=info event=scheduler_connected addr=lb:500513759worker2 # [ 28.218342] nix-grpc-daemon[777]: ts=2026-09-21T09:27:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3760worker1 # [ 28.242954] nix-grpc-daemon[775]: ts=2026-09-21T09:27:55Z level=info event=scheduler_connected addr=lb:500513761worker1 # [ 28.248792] nix-grpc-daemon[775]: ts=2026-09-21T09:27:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3762worker2 # [ 28.420608] nix-grpc-daemon[777]: ts=2026-09-21T09:27:56Z level=info event=scheduler_connected addr=lb:500513763worker2 # [ 28.426708] nix-grpc-daemon[777]: ts=2026-09-21T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3764worker1 # [ 28.453128] nix-grpc-daemon[775]: ts=2026-09-21T09:27:56Z level=info event=scheduler_connected addr=lb:500513765worker1 # [ 28.457608] nix-grpc-daemon[775]: ts=2026-09-21T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3766worker2 # [ 28.632312] nix-grpc-daemon[777]: ts=2026-09-21T09:27:56Z level=info event=scheduler_connected addr=lb:500513767worker2 # [ 28.636574] nix-grpc-daemon[777]: ts=2026-09-21T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3768worker1 # [ 28.653981] nix-grpc-daemon[775]: ts=2026-09-21T09:27:56Z level=info event=scheduler_connected addr=lb:500513769worker1 # [ 28.659559] nix-grpc-daemon[775]: ts=2026-09-21T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3770worker2 # [ 28.833141] nix-grpc-daemon[777]: ts=2026-09-21T09:27:56Z level=info event=scheduler_connected addr=lb:500513771worker2 # [ 28.838847] nix-grpc-daemon[777]: ts=2026-09-21T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3772worker1 # [ 28.861744] nix-grpc-daemon[775]: ts=2026-09-21T09:27:56Z level=info event=scheduler_connected addr=lb:500513773worker1 # [ 28.868134] nix-grpc-daemon[775]: ts=2026-09-21T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3774worker2 # [ 29.040156] nix-grpc-daemon[777]: ts=2026-09-21T09:27:56Z level=info event=scheduler_connected addr=lb:500513775worker2 # [ 29.044768] nix-grpc-daemon[777]: ts=2026-09-21T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3776worker1 # [ 29.069395] nix-grpc-daemon[775]: ts=2026-09-21T09:27:56Z level=info event=scheduler_connected addr=lb:500513777worker1 # [ 29.075076] nix-grpc-daemon[775]: ts=2026-09-21T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3778worker2 # [ 29.245350] nix-grpc-daemon[777]: ts=2026-09-21T09:27:56Z level=info event=scheduler_connected addr=lb:500513779worker2 # [ 29.250487] nix-grpc-daemon[777]: ts=2026-09-21T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3780worker1 # [ 29.275480] nix-grpc-daemon[775]: ts=2026-09-21T09:27:56Z level=info event=scheduler_connected addr=lb:500513781worker1 # [ 29.278133] nix-grpc-daemon[775]: ts=2026-09-21T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3782worker2 # [ 29.451577] nix-grpc-daemon[777]: ts=2026-09-21T09:27:57Z level=info event=scheduler_connected addr=lb:500513783worker2 # [ 29.457076] nix-grpc-daemon[777]: ts=2026-09-21T09:27:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3784worker1 # [ 29.481080] nix-grpc-daemon[775]: ts=2026-09-21T09:27:57Z level=info event=scheduler_connected addr=lb:500513785worker1 # [ 29.485159] nix-grpc-daemon[775]: ts=2026-09-21T09:27:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3786worker2 # [ 29.659966] nix-grpc-daemon[777]: ts=2026-09-21T09:27:57Z level=info event=scheduler_connected addr=lb:500513787worker2 # [ 29.663967] nix-grpc-daemon[777]: ts=2026-09-21T09:27:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3788worker1 # [ 29.681475] nix-grpc-daemon[775]: ts=2026-09-21T09:27:57Z level=info event=scheduler_connected addr=lb:500513789worker1 # [ 29.686733] nix-grpc-daemon[775]: ts=2026-09-21T09:27:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3790worker2 # [ 29.860805] nix-grpc-daemon[777]: ts=2026-09-21T09:27:57Z level=info event=scheduler_connected addr=lb:500513791worker2 # [ 29.865948] nix-grpc-daemon[777]: ts=2026-09-21T09:27:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3792worker1 # [ 29.887572] nix-grpc-daemon[775]: ts=2026-09-21T09:27:57Z level=info event=scheduler_connected addr=lb:500513793worker1 # [ 29.893172] nix-grpc-daemon[775]: ts=2026-09-21T09:27:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3794lb # [ 30.040086] rustfs-bucket-start[1013]: mb s3://farm3795lb # [ 30.048827] systemd[1]: Finished rustfs-bucket.service.3796lb # [ 30.064081] systemd[1]: Starting niks3 server...3797worker2 # [ 30.067122] nix-grpc-daemon[777]: ts=2026-09-21T09:27:57Z level=info event=scheduler_connected addr=lb:500513798worker2 # [ 30.072340] nix-grpc-daemon[777]: ts=2026-09-21T09:27:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3799worker1 # [ 30.094040] nix-grpc-daemon[775]: ts=2026-09-21T09:27:57Z level=info event=scheduler_connected addr=lb:500513800worker1 # [ 30.099095] nix-grpc-daemon[775]: ts=2026-09-21T09:27:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3801worker2 # [ 30.273572] nix-grpc-daemon[777]: ts=2026-09-21T09:27:57Z level=info event=scheduler_connected addr=lb:500513802worker2 # [ 30.278774] nix-grpc-daemon[777]: ts=2026-09-21T09:27:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3803lb # [ 30.286525] postgres[1029]: [1029] ERROR: relation "goose_db_version" does not exist at character 363804lb # [ 30.287844] postgres[1029]: [1029] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3805worker1 # [ 30.299793] nix-grpc-daemon[775]: ts=2026-09-21T09:27:57Z level=info event=scheduler_connected addr=lb:500513806worker1 # [ 30.305108] nix-grpc-daemon[775]: ts=2026-09-21T09:27:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3807lb # [ 30.325808] niks3-server[1023]: 2026/09/21 09:27:57 OK 20241026095416_initial_model.sql (21.68ms)3808lb # [ 30.329656] niks3-server[1023]: 2026/09/21 09:27:57 OK 20251210153512_drop_unused_gin_index.sql (4.27ms)3809lb # [ 30.335261] niks3-server[1023]: 2026/09/21 09:27:57 OK 20251218171726_add_pins.sql (5.55ms)3810lb # [ 30.339889] niks3-server[1023]: 2026/09/21 09:27:57 OK 20260628120000_add_object_size_and_stats.sql (4.56ms)3811lb # [ 30.345075] niks3-server[1023]: 2026/09/21 09:27:57 OK 20260905000000_add_claims.sql (5.19ms)3812lb # [ 30.349793] niks3-server[1023]: 2026/09/21 09:27:57 OK 20260920000000_drop_claims.sql (4.36ms)3813lb # [ 30.352234] niks3-server[1023]: 2026/09/21 09:27:57 goose: successfully migrated database to version: 202609200000003814lb # [ 30.358521] niks3-server[1023]: 2026/09/21 09:27:57 OK 1_commit_pending_closure.sql (8.94ms)3815lb # [ 30.361565] niks3-server[1023]: 2026/09/21 09:27:57 OK 2_object_stats_trigger.sql (2.93ms)3816lb # [ 30.363021] niks3-server[1023]: 2026/09/21 09:27:57 goose: up to current file version: 23817lb # [ 30.368497] niks3-server[1023]: 2026/09/21 09:27:57 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3818lb # [ 30.426614] niks3-server[1023]: 2026/09/21 09:27:58 INFO Read proxy enabled — serving cache objects from S33819lb # [ 30.428415] niks3-server[1023]: 2026/09/21 09:27:58 INFO Using socket-activated listener address=0.0.0.0:57513820lb # [ 30.431835] niks3-server[1023]: 2026/09/21 09:27:58 INFO systemd watchdog enabled interval=15s3821lb # [ 30.435705] niks3-server[1023]: 2026/09/21 09:27:58 INFO Starting HTTP server address=0.0.0.0:57513822lb # [ 30.437562] systemd[1]: Started niks3 server.3823lb # [ 30.440283] systemd[1]: Reached target Multi-User System.3824lb # [ 30.444117] niks3-server[1023]: 2026/09/21 09:27:58 INFO lead: acquired remote=192.168.1.4:568443825lb # [ 30.445291] systemd[1]: Startup finished in 952ms (kernel) + 5.455s (initrd) + 24.027s (userspace) = 30.434s.3826worker2 # [ 30.447721] nix-grpc-daemon[777]: ts=2026-09-21T09:27:58Z level=info event=scheduler_take_over3827worker2 # [ 30.479836] nix-grpc-daemon[777]: ts=2026-09-21T09:27:58Z level=info event=scheduler_connected addr=lb:500513828worker2 # [ 30.485098] nix-grpc-daemon[777]: ts=2026-09-21T09:27:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3829worker1 # [ 30.507677] nix-grpc-daemon[775]: ts=2026-09-21T09:27:58Z level=info event=scheduler_connected addr=lb:500513830worker1 # [ 30.511885] nix-grpc-daemon[775]: ts=2026-09-21T09:27:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3831worker2 # [ 30.688311] nix-grpc-daemon[777]: ts=2026-09-21T09:27:58Z level=info event=scheduler_connected addr=lb:500513832worker2 # [ 30.692345] nix-grpc-daemon[777]: ts=2026-09-21T09:27:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3833worker1 # [ 30.707840] nix-grpc-daemon[775]: ts=2026-09-21T09:27:58Z level=info event=scheduler_connected addr=lb:500513834worker1 # [ 30.713099] nix-grpc-daemon[775]: ts=2026-09-21T09:27:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3835worker2 # [ 30.889150] nix-grpc-daemon[777]: ts=2026-09-21T09:27:58Z level=info event=scheduler_connected addr=lb:500513836worker2 # [ 30.894549] nix-grpc-daemon[777]: ts=2026-09-21T09:27:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3837worker1 # [ 30.913794] nix-grpc-daemon[775]: ts=2026-09-21T09:27:58Z level=info event=scheduler_connected addr=lb:500513838worker1 # [ 30.918990] nix-grpc-daemon[775]: ts=2026-09-21T09:27:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3839worker2 # [ 31.095866] nix-grpc-daemon[777]: ts=2026-09-21T09:27:58Z level=info event=scheduler_connected addr=lb:500513840worker2 # [ 31.101207] nix-grpc-daemon[777]: ts=2026-09-21T09:27:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3841worker1 # [ 31.119785] nix-grpc-daemon[775]: ts=2026-09-21T09:27:58Z level=info event=scheduler_connected addr=lb:500513842worker1 # [ 31.125154] nix-grpc-daemon[775]: ts=2026-09-21T09:27:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3843worker2 # [ 31.302052] nix-grpc-daemon[777]: ts=2026-09-21T09:27:58Z level=info event=scheduler_connected addr=lb:500513844worker2 # [ 31.307450] nix-grpc-daemon[777]: ts=2026-09-21T09:27:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3845worker1 # [ 31.325772] nix-grpc-daemon[775]: ts=2026-09-21T09:27:58Z level=info event=scheduler_connected addr=lb:500513846worker1 # [ 31.330938] nix-grpc-daemon[775]: ts=2026-09-21T09:27:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3847worker2 # [ 31.508653] nix-grpc-daemon[777]: ts=2026-09-21T09:27:59Z level=info event=scheduler_connected addr=lb:500513848worker2 # [ 31.537982] nix-grpc-daemon[777]: ts=2026-09-21T09:27:59Z level=info event=oidc_jwks provider=mock keys=13849worker1 # [ 31.533033] nix-grpc-daemon[775]: ts=2026-09-21T09:27:59Z level=info event=scheduler_connected addr=lb:500513850worker2 # [ 31.544885] nix-grpc-daemon[777]: ts=2026-09-21T09:27:59Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:402863851lb: (finished: waiting for unit niks3.service, in 18.11 seconds)3852lb: waiting for TCP port 5751 on localhost3853worker2 # [ 31.549607] nix-grpc-daemon[777]: ts=2026-09-21T09:27:59Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03854worker2 # [ 31.554877] nix-grpc-daemon[777]: ts=2026-09-21T09:27:59Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:402863855worker2 # [ 31.560197] nix-grpc-daemon[777]: ts=2026-09-21T09:27:59Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03856lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3857lb: (finished: waiting for TCP port 5751 on localhost, in 0.09 seconds)3858lb: waiting for unit envoy.service3859lb: (finished: waiting for unit envoy.service, in 0.07 seconds)3860lb: waiting for TCP port 50051 on localhost3861lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3862lb # [ 31.750766] envoy[913]: conn peer=[::1]:39060 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=33863lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds)3864worker2 # [ 31.946606] systemd[1]: Starting scheduler-token.service...3865worker2 # [ 32.030839] systemd[1]: scheduler-token.service: Deactivated successfully.3866worker2 # [ 32.035836] systemd[1]: Finished scheduler-token.service.3867worker2 # [ 32.037443] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 87ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3868lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3869lb: must succeed: curl -sf localhost:9901/clusters3870lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)3871lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds)3872subtest: one node schedules and both builders hold a WorkerSession on it3873lb: must succeed: curl -sf localhost:9901/clusters3874lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3875worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3876worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)3877lb: must succeed: curl -sf localhost:9901/clusters3878lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3879lb: must succeed: curl -sf localhost:9901/clusters3880lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3881lb: must succeed: curl -sf localhost:9901/clusters3882lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3883lb: must succeed: curl -sf localhost:9901/clusters3884lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3885worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'3886worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds)3887worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3888worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)3889lb: must succeed: curl -sf localhost:9901/clusters3890lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3891lb: must succeed: curl -sf localhost:9901/clusters3892lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)3893worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3894worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)3895worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3896worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)3897worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3898worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3899worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3900worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)3901(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.90 seconds)3902subtest: balancer auth: client cert, bearer token, nothing, unknown CN3903worker1 # [ 33.417788] nix-grpc-daemon[775]: ts=2026-09-21T09:28:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376563904worker1 # [ 33.437483] systemd[1]: Started Nix Daemon.3905worker1 # [ 33.707702] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)3906worker1 # [ 33.734910] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2 ms (attempt 1/5)3907worker1 # [ 33.739921] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 2/5)3908worker1 # [ 33.883295] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 3/5)3909worker1 # [ 34.026613] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 299 ms (attempt 4/5)3910worker1 # [ 34.335065] nix-grpc-daemon[775]: disabling binary cache 'https://cache.nixos.org' for 60 seconds3911worker1 # [ 34.341166] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3912worker2 # [ 34.384848] nix-grpc-daemon[777]: ts=2026-09-21T09:28:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:402983913worker2 # [ 34.401934] systemd[1]: Started Nix Daemon.3914worker2 # [ 34.700139] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3915worker2 # [ 34.724436] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 1/5)3916worker2 # [ 34.821820] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 2/5)3917worker2 # [ 34.843005] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 361 ms (attempt 3/5)3918worker2 # [ 35.211416] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 543 ms (attempt 4/5)3919worker2 # [ 35.757709] nix-grpc-daemon[777]: disabling binary cache 'https://cache.nixos.org' for 60 seconds3920worker2 # [ 35.763406] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3921worker2 # [ 35.776140] nix-grpc-daemon[777]: ts=2026-09-21T09:28:03Z level=debug event=path_info_miss path=1riwj2yl1fyad56y7izqz2z8iyxylapg-nixos-system-client-test error="error: path '/nix/store/1riwj2yl1fyad56y7izqz2z8iyxylapg-nixos-system-client-test' is required, but there is no substituter that can build it"3922worker2 # [ 35.799993] nix-grpc-daemon[777]: ts=2026-09-21T09:28:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:402983923worker2 # [ 35.811924] nix-grpc-daemon[777]: ts=2026-09-21T09:28:03Z level=debug event=path_info_miss path=1riwj2yl1fyad56y7izqz2z8iyxylapg-nixos-system-client-test error="error: path '/nix/store/1riwj2yl1fyad56y7izqz2z8iyxylapg-nixos-system-client-test' is required, but there is no substituter that can build it"3924client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3925client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.10 seconds)3926client: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3927client: (finished: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)3928worker1 # [ 36.246018] nix-grpc-daemon[775]: ts=2026-09-21T09:28:03Z level=info event=oidc_jwks provider=mock keys=13929worker1 # [ 36.250761] nix-grpc-daemon[775]: ts=2026-09-21T09:28:03Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:376563930worker2 # [ 36.272672] nix-grpc-daemon[777]: ts=2026-09-21T09:28:03Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:402983931worker2 # [ 36.283578] nix-grpc-daemon[777]: ts=2026-09-21T09:28:03Z level=debug event=path_info_miss path=1riwj2yl1fyad56y7izqz2z8iyxylapg-nixos-system-client-test error="error: path '/nix/store/1riwj2yl1fyad56y7izqz2z8iyxylapg-nixos-system-client-test' is required, but there is no substituter that can build it"3932worker2 # [ 36.297196] nix-grpc-daemon[777]: ts=2026-09-21T09:28:03Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:402983933worker2 # [ 36.304800] nix-grpc-daemon[777]: ts=2026-09-21T09:28:03Z level=debug event=path_info_miss path=1riwj2yl1fyad56y7izqz2z8iyxylapg-nixos-system-client-test error="error: path '/nix/store/1riwj2yl1fyad56y7izqz2z8iyxylapg-nixos-system-client-test' is required, but there is no substituter that can build it"3934client: must succeed: rm -r /var/lib/nix-grpc-store3935client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.05 seconds)3936worker2 # [ 38.637324] nix-grpc-daemon[777]: ts=2026-09-21T09:28:06Z level=info event=denied method=QueryMissing cn=- role=none3937worker2 # [ 38.889487] nix-grpc-daemon[777]: ts=2026-09-21T09:28:06Z level=info event=denied method=QueryMissing cn=stranger role=none3938(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 5.89 seconds)3939subtest: DAG build is scheduled across workers and lands in the cache3940client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&23941client # don't know how to build these paths:3942client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3943worker1 # [ 39.159409] nix-grpc-daemon[775]: ts=2026-09-21T09:28:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376563944worker2 # [ 39.174720] nix-grpc-daemon[777]: ts=2026-09-21T09:28:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:402983945worker2 # [ 39.216944] nix-grpc-daemon[777]: ts=2026-09-21T09:28:06Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=03946worker2 # [ 39.222755] nix-grpc-daemon[777]: ts=2026-09-21T09:28:06Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=13947worker1 # [ 39.257893] nix-grpc-daemon[775]: ts=2026-09-21T09:28:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376563948worker1 # [ 39.274165] nix-grpc-daemon[775]: ts=2026-09-21T09:28:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376563949client # copying 1 paths to 'lb:50051'...3950worker1 # [ 39.284185] nix-grpc-daemon[775]: ts=2026-09-21T09:28:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376563951client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...3952worker1 # [ 39.311337] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)3953worker1 # [ 39.331531] nix-grpc-daemon[775]: ts=2026-09-21T09:28:06Z level=info event=niks3_push_started pid=8533954worker1 # [ 39.602763] nix-grpc-daemon[853]: time=2026-09-21T09:28:07.233Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""3955worker1 # [ 39.795448] nix-daemon[839]: accepted connection from pid 866, user nix-grpc-daemon (trusted)3956worker1 # [ 39.813762] nix-daemon[839]: reaped child process 869, status = succeeded3957worker1 # [ 39.986559] nix-daemon[839]: accepted connection from pid 871, user nix-grpc-daemon (trusted)3958worker1 # [ 40.002621] nix-daemon[839]: reaped child process 874, status = succeeded3959lb # [ 40.021067] niks3-server[1023]: 2026/09/21 09:28:07 INFO Received uploads request method=POST path=/api/pending_closures3960worker1 # [ 40.039857] nix-grpc-daemon[853]: time=2026-09-21T09:28:07.671Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3961worker1 # [ 40.043638] nix-grpc-daemon[853]: time=2026-09-21T09:28:07.673Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"3962lb # [ 40.092519] niks3-server[1023]: 2026/09/21 09:28:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign3963lb # [ 40.108508] niks3-server[1023]: 2026/09/21 09:28:07 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst3964lb # [ 40.145312] niks3-server[1023]: 2026/09/21 09:28:07 INFO Signed narinfos id=1 count=13965worker1 # [ 40.142778] nix-grpc-daemon[853]: time=2026-09-21T09:28:07.773Z level=INFO msg="Uploading 1 narinfos"3966lb # [ 40.156339] niks3-server[1023]: 2026/09/21 09:28:07 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls3967lb # [ 40.167447] niks3-server[1023]: 2026/09/21 09:28:07 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete3968lb # [ 40.181086] niks3-server[1023]: 2026/09/21 09:28:07 INFO Completed upload id=13969worker1 # [ 40.175721] nix-grpc-daemon[853]: time=2026-09-21T09:28:07.807Z level=INFO msg="Upload complete. (564ms)"3970lb # [ 40.192805] niks3-server[1023]: 2026/09/21 09:28:07 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo3971worker1 # [ 40.189426] nix-grpc-daemon[775]: ts=2026-09-21T09:28:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=0 paths=1 nar_bytes_in=5043972worker1 # [ 40.208062] nix-grpc-daemon[775]: ts=2026-09-21T09:28:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376563973worker1 # [ 40.217738] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)3974worker1 # [ 40.224193] nix-daemon[839]: reaped child process 851, status = succeeded3975client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...3976client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3977worker1 # [ 40.234449] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)3978lb # [ 40.853434] envoy[913]: conn peer=[2001:db8:1::1]:32868 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=24413979lb # [ 40.858534] envoy[913]: conn peer=[2001:db8:1::1]:32880 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=773980lb # [ 40.863775] envoy[913]: conn peer=[2001:db8:1::1]:32886 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=133981lb # [ 40.869459] envoy[913]: conn peer=[2001:db8:1::1]:32890 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=133982client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds3983client # farm-a> LOG-farm-a-t13984worker1 # [ 41.483680] nix-daemon[839]: reaped child process 878, status = succeeded3985worker1 # [ 41.672380] nix-daemon[839]: accepted connection from pid 894, user nix-grpc-daemon (trusted)3986worker1 # [ 41.689576] nix-daemon[839]: reaped child process 897, status = succeeded3987lb # [ 41.710101] niks3-server[1023]: 2026/09/21 09:28:09 INFO Received uploads request method=POST path=/api/pending_closures3988worker1 # [ 41.720844] nix-grpc-daemon[853]: time=2026-09-21T09:28:09.350Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3989worker1 # [ 41.723373] nix-grpc-daemon[853]: time=2026-09-21T09:28:09.351Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"3990lb # [ 41.770554] niks3-server[1023]: 2026/09/21 09:28:09 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3991worker1 # [ 41.778302] nix-grpc-daemon[853]: time=2026-09-21T09:28:09.410Z level=INFO msg="Uploading 1 narinfos"3992lb # [ 41.786262] niks3-server[1023]: 2026/09/21 09:28:09 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst3993lb # [ 41.794019] niks3-server[1023]: 2026/09/21 09:28:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign3994lb # [ 41.802262] niks3-server[1023]: 2026/09/21 09:28:09 INFO Signed narinfos id=2 count=13995lb # [ 41.806311] niks3-server[1023]: 2026/09/21 09:28:09 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls3996lb # [ 41.817589] niks3-server[1023]: 2026/09/21 09:28:09 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete3997lb # [ 41.823970] niks3-server[1023]: 2026/09/21 09:28:09 INFO Completed upload id=23998worker1 # [ 41.818422] nix-grpc-daemon[853]: time=2026-09-21T09:28:09.450Z level=INFO msg="Upload complete. (365ms)"3999lb # [ 41.827542] niks3-server[1023]: 2026/09/21 09:28:09 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4000worker1 # [ 41.831160] nix-grpc-daemon[775]: ts=2026-09-21T09:28:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14001worker2 # [ 41.853287] nix-grpc-daemon[777]: ts=2026-09-21T09:28:09Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04002worker1 # [ 41.850941] nix-daemon[839]: reaped child process 876, status = succeeded4003worker2 # [ 41.858962] nix-grpc-daemon[777]: ts=2026-09-21T09:28:09Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=14004worker1 # [ 41.890622] nix-grpc-daemon[775]: ts=2026-09-21T09:28:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376564005worker1 # [ 41.905658] nix-grpc-daemon[775]: ts=2026-09-21T09:28:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376564006client # copying 1 paths to 'lb:50051'...4007worker1 # [ 41.920315] nix-grpc-daemon[853]: time=2026-09-21T09:28:09.552Z level=INFO msg="All 1 paths already cached"4008worker1 # [ 41.926319] nix-grpc-daemon[775]: ts=2026-09-21T09:28:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376564009client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4010worker1 # [ 41.948491] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4011worker1 # [ 42.153247] nix-daemon[839]: accepted connection from pid 901, user nix-grpc-daemon (trusted)4012worker1 # [ 42.173363] nix-daemon[839]: reaped child process 904, status = succeeded4013worker1 # [ 42.347099] nix-daemon[839]: accepted connection from pid 906, user nix-grpc-daemon (trusted)4014worker1 # [ 42.363980] nix-daemon[839]: reaped child process 909, status = succeeded4015lb # [ 42.382061] niks3-server[1023]: 2026/09/21 09:28:09 INFO Received uploads request method=POST path=/api/pending_closures4016worker1 # [ 42.387843] nix-grpc-daemon[853]: time=2026-09-21T09:28:10.019Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4017worker1 # [ 42.390180] nix-grpc-daemon[853]: time=2026-09-21T09:28:10.021Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4018lb # [ 42.417122] niks3-server[1023]: 2026/09/21 09:28:10 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4019lb # [ 42.431986] niks3-server[1023]: 2026/09/21 09:28:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4020worker1 # [ 42.432732] nix-grpc-daemon[853]: time=2026-09-21T09:28:10.064Z level=INFO msg="Uploading 1 narinfos"4021lb # [ 42.444206] niks3-server[1023]: 2026/09/21 09:28:10 INFO Signed narinfos id=3 count=14022lb # [ 42.450348] niks3-server[1023]: 2026/09/21 09:28:10 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4023lb # [ 42.460058] niks3-server[1023]: 2026/09/21 09:28:10 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4024worker1 # [ 42.453674] nix-grpc-daemon[853]: time=2026-09-21T09:28:10.085Z level=INFO msg="Upload complete. (487ms)"4025lb # [ 42.466094] niks3-server[1023]: 2026/09/21 09:28:10 INFO Completed upload id=34026lb # [ 42.469735] niks3-server[1023]: 2026/09/21 09:28:10 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4027worker1 # [ 42.460687] nix-grpc-daemon[775]: ts=2026-09-21T09:28:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=0 paths=1 nar_bytes_in=6244028worker1 # [ 42.473444] nix-grpc-daemon[775]: ts=2026-09-21T09:28:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376564029worker1 # [ 42.485691] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4030worker1 # [ 42.495984] nix-daemon[839]: reaped child process 899, status = succeeded4031client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4032client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4033worker1 # [ 42.512946] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4034client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4035client # farm-b> LOG-farm-b-t14036worker1 # [ 43.739135] nix-daemon[839]: reaped child process 913, status = succeeded4037worker1 # [ 43.894484] nix-daemon[839]: accepted connection from pid 930, user nix-grpc-daemon (trusted)4038worker1 # [ 43.912244] nix-daemon[839]: reaped child process 933, status = succeeded4039lb # [ 43.933168] niks3-server[1023]: 2026/09/21 09:28:11 INFO Received uploads request method=POST path=/api/pending_closures4040worker1 # [ 43.937106] nix-grpc-daemon[853]: time=2026-09-21T09:28:11.568Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4041worker1 # [ 43.939455] nix-grpc-daemon[853]: time=2026-09-21T09:28:11.570Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4042lb # [ 43.965303] niks3-server[1023]: 2026/09/21 09:28:11 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4043worker1 # [ 43.980966] nix-grpc-daemon[853]: time=2026-09-21T09:28:11.612Z level=INFO msg="Uploading 1 narinfos"4044lb # [ 43.981544] niks3-server[1023]: 2026/09/21 09:28:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4045lb # [ 43.995511] niks3-server[1023]: 2026/09/21 09:28:11 INFO Signed narinfos id=4 count=14046lb # [ 44.001024] niks3-server[1023]: 2026/09/21 09:28:11 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4047lb # [ 44.006341] niks3-server[1023]: 2026/09/21 09:28:11 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4048lb # [ 44.013803] niks3-server[1023]: 2026/09/21 09:28:11 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4049lb # [ 44.017810] niks3-server[1023]: 2026/09/21 09:28:11 INFO Completed upload id=44050worker1 # [ 44.011642] nix-grpc-daemon[853]: time=2026-09-21T09:28:11.643Z level=INFO msg="Upload complete. (285ms)"4051lb # [ 44.020223] niks3-server[1023]: 2026/09/21 09:28:11 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4052worker1 # [ 44.020980] nix-grpc-daemon[775]: ts=2026-09-21T09:28:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14053worker2 # [ 44.039447] nix-grpc-daemon[777]: ts=2026-09-21T09:28:11Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04054worker1 # [ 44.037589] nix-daemon[839]: reaped child process 911, status = succeeded4055worker2 # [ 44.045034] nix-grpc-daemon[777]: ts=2026-09-21T09:28:11Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14056worker1 # [ 44.075098] nix-grpc-daemon[775]: ts=2026-09-21T09:28:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376564057worker1 # [ 44.090061] nix-grpc-daemon[775]: ts=2026-09-21T09:28:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376564058worker1 # [ 44.105214] nix-grpc-daemon[853]: time=2026-09-21T09:28:11.737Z level=INFO msg="All 2 paths already cached"4059client # copying 1 paths to 'lb:50051'...4060worker1 # [ 44.115282] nix-grpc-daemon[775]: ts=2026-09-21T09:28:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376564061client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4062worker1 # [ 44.138259] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4063worker1 # [ 44.349069] nix-daemon[839]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4064worker1 # [ 44.368984] nix-daemon[839]: reaped child process 940, status = succeeded4065worker1 # [ 44.543286] nix-daemon[839]: accepted connection from pid 942, user nix-grpc-daemon (trusted)4066worker1 # [ 44.558803] nix-daemon[839]: reaped child process 945, status = succeeded4067lb # [ 44.580142] niks3-server[1023]: 2026/09/21 09:28:12 INFO Received uploads request method=POST path=/api/pending_closures4068worker1 # [ 44.584091] nix-grpc-daemon[853]: time=2026-09-21T09:28:12.215Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4069worker1 # [ 44.585601] nix-grpc-daemon[853]: time=2026-09-21T09:28:12.215Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4070lb # [ 44.612142] niks3-server[1023]: 2026/09/21 09:28:12 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4071lb # [ 44.623176] niks3-server[1023]: 2026/09/21 09:28:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4072worker1 # [ 44.623396] nix-grpc-daemon[853]: time=2026-09-21T09:28:12.255Z level=INFO msg="Uploading 1 narinfos"4073lb # [ 44.632813] niks3-server[1023]: 2026/09/21 09:28:12 INFO Signed narinfos id=5 count=14074lb # [ 44.642159] niks3-server[1023]: 2026/09/21 09:28:12 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4075lb # [ 44.651057] niks3-server[1023]: 2026/09/21 09:28:12 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4076worker1 # [ 44.651883] nix-grpc-daemon[853]: time=2026-09-21T09:28:12.283Z level=INFO msg="Upload complete. (495ms)"4077lb # [ 44.660291] niks3-server[1023]: 2026/09/21 09:28:12 INFO Completed upload id=54078lb # [ 44.664513] niks3-server[1023]: 2026/09/21 09:28:12 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4079worker1 # [ 44.660297] nix-grpc-daemon[775]: ts=2026-09-21T09:28:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=0 paths=1 nar_bytes_in=7524080worker1 # [ 44.674500] nix-grpc-daemon[775]: ts=2026-09-21T09:28:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376564081worker1 # [ 44.681193] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4082worker1 # [ 44.686297] nix-daemon[839]: reaped child process 935, status = succeeded4083client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4084worker1 # [ 44.697797] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4085client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4086client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4087client # farm-top> LOG-farm-top-t14088worker1 # [ 46.128986] nix-daemon[839]: reaped child process 949, status = succeeded4089worker1 # [ 46.290521] nix-daemon[839]: accepted connection from pid 966, user nix-grpc-daemon (trusted)4090worker1 # [ 46.308577] nix-daemon[839]: reaped child process 969, status = succeeded4091lb # [ 46.329774] niks3-server[1023]: 2026/09/21 09:28:13 INFO Received uploads request method=POST path=/api/pending_closures4092worker1 # [ 46.335245] nix-grpc-daemon[853]: time=2026-09-21T09:28:13.965Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4093worker1 # [ 46.336999] nix-grpc-daemon[853]: time=2026-09-21T09:28:13.965Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4094lb # [ 46.368838] niks3-server[1023]: 2026/09/21 09:28:13 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4095lb # [ 46.381000] niks3-server[1023]: 2026/09/21 09:28:13 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4096lb # [ 46.391531] niks3-server[1023]: 2026/09/21 09:28:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4097worker1 # [ 46.391309] nix-grpc-daemon[853]: time=2026-09-21T09:28:14.023Z level=INFO msg="Uploading 1 narinfos"4098lb # [ 46.400291] niks3-server[1023]: 2026/09/21 09:28:14 INFO Signed narinfos id=6 count=14099lb # [ 46.402735] niks3-server[1023]: 2026/09/21 09:28:14 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4100lb # [ 46.413583] niks3-server[1023]: 2026/09/21 09:28:14 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4101lb # [ 46.421428] niks3-server[1023]: 2026/09/21 09:28:14 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4102lb # [ 46.426211] niks3-server[1023]: 2026/09/21 09:28:14 INFO Completed upload id=64103worker1 # [ 46.420220] nix-grpc-daemon[853]: time=2026-09-21T09:28:14.052Z level=INFO msg="Upload complete. (310ms)"4104worker1 # [ 46.433133] nix-grpc-daemon[775]: ts=2026-09-21T09:28:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14105worker1 # [ 46.449949] nix-daemon[839]: reaped child process 947, status = succeeded4106client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.57 seconds)4107client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4108client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.23 seconds)4109client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4110client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4111client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4112client # copying 3 paths...4113client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4114client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4115client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4116client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4117(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.24 seconds)4118subtest: repeat is answered Cached by the scheduler without building4119worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4120worker1 # finding garbage collector roots...4121worker1 # deleting specified paths...4122worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4123worker1 # deleting unused links...4124worker1 # note: hard linking is currently saving 0.0 KiB4125worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.28 seconds)4126worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4127worker2 # finding garbage collector roots...4128worker2 # deleting specified paths...4129worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4130worker2 # deleting unused links...4131worker2 # note: hard linking is currently saving 0.0 KiB4132worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.27 seconds)4133client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24134worker1 # [ 47.961447] nix-grpc-daemon[775]: ts=2026-09-21T09:28:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376564135client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4136client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4137worker1 # [ 47.979873] nix-grpc-daemon[775]: ts=2026-09-21T09:28:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376564138worker1 # [ 47.991129] nix-grpc-daemon[853]: time=2026-09-21T09:28:15.622Z level=INFO msg="All 2 paths already cached"4139worker2 # [ 48.021149] nix-grpc-daemon[777]: ts=2026-09-21T09:28:15Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14140client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.35 seconds)4141client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4142client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.21 seconds)4143lb: must succeed: curl -sf localhost:9901/clusters4144lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4145lb: must succeed: curl -sf localhost:9901/clusters4146lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4147worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4148worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4149(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.42 seconds)4150subtest: inputs already in the cache are not sent to the scheduler4151lb: must succeed: curl -sf localhost:9901/clusters4152lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4153lb: must succeed: curl -sf localhost:9901/clusters4154lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4155worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4156worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4157client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24158worker1 # [ 49.038007] nix-grpc-daemon[775]: ts=2026-09-21T09:28:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376564159client # don't know how to build these paths:4160client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4161worker2 # [ 49.061622] nix-grpc-daemon[777]: ts=2026-09-21T09:28:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:402984162worker2 # [ 49.087496] nix-grpc-daemon[777]: ts=2026-09-21T09:28:16Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04163worker2 # [ 49.093616] nix-grpc-daemon[777]: ts=2026-09-21T09:28:16Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14164worker1 # [ 49.123706] nix-grpc-daemon[775]: ts=2026-09-21T09:28:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376564165worker1 # [ 49.138488] nix-grpc-daemon[775]: ts=2026-09-21T09:28:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376564166client # copying 1 paths to 'lb:50051'...4167worker1 # [ 49.155962] nix-grpc-daemon[853]: time=2026-09-21T09:28:16.787Z level=INFO msg="All 2 paths already cached"4168client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4169worker1 # [ 49.162126] nix-grpc-daemon[775]: ts=2026-09-21T09:28:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376564170worker1 # [ 49.177380] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4171worker1 # [ 49.386153] nix-daemon[839]: accepted connection from pid 986, user nix-grpc-daemon (trusted)4172worker1 # [ 49.403756] nix-daemon[839]: reaped child process 989, status = succeeded4173worker1 # [ 49.583184] nix-daemon[839]: accepted connection from pid 991, user nix-grpc-daemon (trusted)4174worker1 # [ 49.599114] nix-daemon[839]: reaped child process 994, status = succeeded4175lb # [ 49.616819] niks3-server[1023]: 2026/09/21 09:28:17 INFO Received uploads request method=POST path=/api/pending_closures4176worker1 # [ 49.620131] nix-grpc-daemon[853]: time=2026-09-21T09:28:17.252Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4177worker1 # [ 49.621986] nix-grpc-daemon[853]: time=2026-09-21T09:28:17.253Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4178lb # [ 49.646699] niks3-server[1023]: 2026/09/21 09:28:17 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4179worker1 # [ 49.652415] nix-grpc-daemon[853]: time=2026-09-21T09:28:17.284Z level=INFO msg="Uploading 1 narinfos"4180lb # [ 49.660338] niks3-server[1023]: 2026/09/21 09:28:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4181lb # [ 49.671273] niks3-server[1023]: 2026/09/21 09:28:17 INFO Signed narinfos id=7 count=14182lb # [ 49.681085] niks3-server[1023]: 2026/09/21 09:28:17 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4183worker1 # [ 49.679124] nix-grpc-daemon[853]: time=2026-09-21T09:28:17.311Z level=INFO msg="Upload complete. (484ms)"4184lb # [ 49.685565] niks3-server[1023]: 2026/09/21 09:28:17 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4185lb # [ 49.690270] niks3-server[1023]: 2026/09/21 09:28:17 INFO Completed upload id=74186lb # [ 49.693228] niks3-server[1023]: 2026/09/21 09:28:17 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4187worker1 # [ 49.687961] nix-grpc-daemon[775]: ts=2026-09-21T09:28:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=0 paths=1 nar_bytes_in=7524188worker1 # [ 49.691454] nix-grpc-daemon[775]: ts=2026-09-21T09:28:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376564189worker1 # [ 49.703485] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4190worker1 # [ 49.706135] nix-daemon[839]: reaped child process 984, status = succeeded4191client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4192worker1 # [ 49.718227] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4193client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4194client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4195lb # [ 50.857041] envoy[913]: conn peer=[2001:db8:1::1]:32900 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9605 tx=4083 ms=72974196lb # [ 50.862009] envoy[913]: conn peer=[2001:db8:1::1]:45518 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=476 ms=664197client # farm-top> LOG-farm-top-t1b4198worker1 # [ 50.941766] nix-daemon[839]: reaped child process 998, status = succeeded4199worker1 # [ 51.098855] nix-daemon[839]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)4200worker1 # [ 51.115818] nix-daemon[839]: reaped child process 1017, status = succeeded4201lb # [ 51.136654] niks3-server[1023]: 2026/09/21 09:28:18 INFO Received uploads request method=POST path=/api/pending_closures4202worker1 # [ 51.140537] nix-grpc-daemon[853]: time=2026-09-21T09:28:18.771Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4203worker1 # [ 51.144837] nix-grpc-daemon[853]: time=2026-09-21T09:28:18.771Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4204lb # [ 51.171540] niks3-server[1023]: 2026/09/21 09:28:18 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4205lb # [ 51.188880] niks3-server[1023]: 2026/09/21 09:28:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4206worker1 # [ 51.191107] nix-grpc-daemon[853]: time=2026-09-21T09:28:18.822Z level=INFO msg="Uploading 1 narinfos"4207lb # [ 51.202026] niks3-server[1023]: 2026/09/21 09:28:18 INFO Signed narinfos id=8 count=14208lb # [ 51.207986] niks3-server[1023]: 2026/09/21 09:28:18 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4209lb # [ 51.214253] niks3-server[1023]: 2026/09/21 09:28:18 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4210lb # [ 51.221898] niks3-server[1023]: 2026/09/21 09:28:18 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4211lb # [ 51.226629] niks3-server[1023]: 2026/09/21 09:28:18 INFO Completed upload id=84212worker1 # [ 51.221010] nix-grpc-daemon[853]: time=2026-09-21T09:28:18.852Z level=INFO msg="Upload complete. (294ms)"4213lb # [ 51.229452] niks3-server[1023]: 2026/09/21 09:28:18 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4214worker1 # [ 51.233132] nix-grpc-daemon[775]: ts=2026-09-21T09:28:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14215worker1 # [ 51.246651] nix-daemon[839]: reaped child process 996, status = succeeded4216client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.51 seconds)4217client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4218client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.22 seconds)4219lb: must succeed: curl -sf localhost:9901/clusters4220lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4221lb: must succeed: curl -sf localhost:9901/clusters4222lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4223worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4224worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4225(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.14 seconds)4226subtest: two clients wanting the same drv build it once4227client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4228client # [ 51.786825] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4229client # Running as unit: dedup1.service; invocation ID: 9de2cbf73c4c494fb551edf10a7c58974230client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4231client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4232worker1 # [ 52.059662] nix-grpc-daemon[775]: ts=2026-09-21T09:28:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376564233client # [ 52.060054] nix[886]: don't know how to build these paths:4234client # [ 52.061174] nix[886]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4235worker2 # [ 52.093623] nix-grpc-daemon[777]: ts=2026-09-21T09:28:19Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04236worker2 # [ 52.099394] nix-grpc-daemon[777]: ts=2026-09-21T09:28:19Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14237worker2 # [ 52.106069] nix-grpc-daemon[777]: ts=2026-09-21T09:28:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:402984238worker2 # [ 52.122791] nix-grpc-daemon[777]: ts=2026-09-21T09:28:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:402984239client # [ 52.120152] nix[886]: copying 1 paths to 'lb:50051'...4240client # [ 52.122002] nix[886]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4241worker2 # [ 52.137228] nix-grpc-daemon[777]: ts=2026-09-21T09:28:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:402984242worker2 # [ 52.152763] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4243worker2 # [ 52.179886] nix-grpc-daemon[777]: ts=2026-09-21T09:28:19Z level=info event=niks3_push_started pid=9154244worker2 # [ 52.444288] nix-grpc-daemon[915]: time=2026-09-21T09:28:20.070Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4245worker2 # [ 52.637642] nix-daemon[861]: accepted connection from pid 928, user nix-grpc-daemon (trusted)4246worker2 # [ 52.655439] nix-daemon[861]: reaped child process 931, status = succeeded4247worker2 # [ 52.828470] nix-daemon[861]: accepted connection from pid 933, user nix-grpc-daemon (trusted)4248worker2 # [ 52.846081] nix-daemon[861]: reaped child process 936, status = succeeded4249lb # [ 52.854736] niks3-server[1023]: 2026/09/21 09:28:20 INFO Received uploads request method=POST path=/api/pending_closures4250worker2 # [ 52.869040] nix-grpc-daemon[915]: time=2026-09-21T09:28:20.496Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4251worker2 # [ 52.872331] nix-grpc-daemon[915]: time=2026-09-21T09:28:20.498Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4252lb # [ 52.890442] niks3-server[1023]: 2026/09/21 09:28:20 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4253worker2 # [ 52.915919] nix-grpc-daemon[915]: time=2026-09-21T09:28:20.543Z level=INFO msg="Uploading 1 narinfos"4254lb # [ 52.905009] niks3-server[1023]: 2026/09/21 09:28:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4255lb # [ 52.916651] niks3-server[1023]: 2026/09/21 09:28:20 INFO Signed narinfos id=9 count=14256client # [ 52.902500] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4257lb # [ 52.924136] niks3-server[1023]: 2026/09/21 09:28:20 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4258lb # [ 52.936661] niks3-server[1023]: 2026/09/21 09:28:20 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4259lb # [ 52.942957] niks3-server[1023]: 2026/09/21 09:28:20 INFO Completed upload id=94260worker2 # [ 52.946248] nix-grpc-daemon[915]: time=2026-09-21T09:28:20.573Z level=INFO msg="Upload complete. (493ms)"4261client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.13 seconds)4262lb # [ 52.945756] niks3-server[1023]: 2026/09/21 09:28:20 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4263worker2 # [ 52.956766] nix-grpc-daemon[777]: ts=2026-09-21T09:28:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40298 duration_s=0 paths=1 nar_bytes_in=4324264worker2 # [ 52.966250] nix-grpc-daemon[777]: ts=2026-09-21T09:28:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:402984265worker2 # [ 52.972972] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4266worker2 # [ 52.978270] nix-daemon[861]: reaped child process 913, status = succeeded4267client # Running as unit: dedup2.service; invocation ID: f446fc22365042b583c425c90535c67f4268client # [ 52.971182] nix[886]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4269worker2 # [ 52.991261] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4270client # [ 53.261671] nix[902]: don't know how to build these paths:4271client # [ 53.262745] nix[902]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4272worker1 # [ 53.267298] nix-grpc-daemon[775]: ts=2026-09-21T09:28:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376564273worker2 # [ 53.289147] nix-grpc-daemon[777]: ts=2026-09-21T09:28:20Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04274client # [ 53.281216] nix[902]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4275worker2 # [ 53.297058] nix-grpc-daemon[777]: ts=2026-09-21T09:28:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:402984276client: must succeed: sleep 24277client: (finished: must succeed: sleep 2, in 2.06 seconds)4278client: waiting for success: ! systemctl is-active dedup1 dedup24279worker2 # [ 56.629054] nix-daemon[861]: reaped child process 940, status = succeeded4280worker2 # [ 56.804627] nix-daemon[861]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4281worker2 # [ 56.821636] nix-daemon[861]: reaped child process 975, status = succeeded4282lb # [ 56.830330] niks3-server[1023]: 2026/09/21 09:28:24 INFO Received uploads request method=POST path=/api/pending_closures4283worker2 # [ 56.844427] nix-grpc-daemon[915]: time=2026-09-21T09:28:24.471Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4284worker2 # [ 56.845912] nix-grpc-daemon[915]: time=2026-09-21T09:28:24.471Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4285lb # [ 56.873437] niks3-server[1023]: 2026/09/21 09:28:24 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4286lb # [ 56.885284] niks3-server[1023]: 2026/09/21 09:28:24 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4287worker2 # [ 56.898295] nix-grpc-daemon[915]: time=2026-09-21T09:28:24.524Z level=INFO msg="Uploading 1 narinfos"4288lb # [ 56.896100] niks3-server[1023]: 2026/09/21 09:28:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4289lb # [ 56.902520] niks3-server[1023]: 2026/09/21 09:28:24 INFO Signed narinfos id=10 count=14290lb # [ 56.907646] niks3-server[1023]: 2026/09/21 09:28:24 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4291lb # [ 56.915479] niks3-server[1023]: 2026/09/21 09:28:24 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4292lb # [ 56.922002] niks3-server[1023]: 2026/09/21 09:28:24 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4293lb # [ 56.927548] niks3-server[1023]: 2026/09/21 09:28:24 INFO Completed upload id=104294worker2 # [ 56.931493] nix-grpc-daemon[915]: time=2026-09-21T09:28:24.559Z level=INFO msg="Upload complete. (328ms)"4295worker2 # [ 56.945115] nix-grpc-daemon[777]: ts=2026-09-21T09:28:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40298 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14296worker2 # [ 56.959010] nix-grpc-daemon[777]: ts=2026-09-21T09:28:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40298 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14297worker2 # [ 56.966050] nix-daemon[861]: reaped child process 938, status = succeeded4298client # [ 56.966737] systemd[1]: dedup2.service: Deactivated successfully.4299client # [ 56.972756] systemd[1]: dedup2.service: Consumed 205ms CPU time over 4.068s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic.4300client # [ 57.021422] systemd[1]: dedup1.service: Deactivated successfully.4301client # [ 57.025069] systemd[1]: dedup1.service: Consumed 174ms CPU time over 5.237s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.5K outgoing IP traffic.4302client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds)4303client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4304client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4305worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4306worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4307worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4308worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4309worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4310worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4311worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4312worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4313(finished: subtest: two clients wanting the same drv build it once, in 6.28 seconds)4314subtest: build hook: nix-daemon with builders = grpc://lb4315client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4316client # [ 58.218565] systemd[1]: Started Nix Daemon.4317client # [ 58.402119] nix-daemon[944]: accepted connection from pid 941, user root (trusted)4318worker1 # [ 58.624877] nix-grpc-daemon[775]: ts=2026-09-21T09:28:26Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376564319worker2 # [ 58.638727] nix-grpc-daemon[777]: ts=2026-09-21T09:28:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:402984320worker2 # [ 58.657653] nix-grpc-daemon[777]: ts=2026-09-21T09:28:26Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04321worker2 # [ 58.663534] nix-grpc-daemon[777]: ts=2026-09-21T09:28:26Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14322worker1 # [ 58.694240] nix-grpc-daemon[775]: ts=2026-09-21T09:28:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376564323worker1 # [ 58.710040] nix-grpc-daemon[775]: ts=2026-09-21T09:28:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376564324worker1 # [ 58.725116] nix-grpc-daemon[775]: ts=2026-09-21T09:28:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376564325worker1 # [ 58.740249] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4326worker1 # [ 58.955259] nix-daemon[839]: accepted connection from pid 1042, user nix-grpc-daemon (trusted)4327worker1 # [ 58.973127] nix-daemon[839]: reaped child process 1045, status = succeeded4328worker1 # [ 59.149126] nix-daemon[839]: accepted connection from pid 1047, user nix-grpc-daemon (trusted)4329worker1 # [ 59.165088] nix-daemon[839]: reaped child process 1050, status = succeeded4330lb # [ 59.182851] niks3-server[1023]: 2026/09/21 09:28:26 INFO Received uploads request method=POST path=/api/pending_closures4331worker1 # [ 59.186597] nix-grpc-daemon[853]: time=2026-09-21T09:28:26.818Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4332worker1 # [ 59.188185] nix-grpc-daemon[853]: time=2026-09-21T09:28:26.818Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4333lb # [ 59.210987] niks3-server[1023]: 2026/09/21 09:28:26 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4334worker1 # [ 59.218808] nix-grpc-daemon[853]: time=2026-09-21T09:28:26.850Z level=INFO msg="Uploading 1 narinfos"4335lb # [ 59.224385] niks3-server[1023]: 2026/09/21 09:28:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4336lb # [ 59.235384] niks3-server[1023]: 2026/09/21 09:28:26 INFO Signed narinfos id=11 count=14337lb # [ 59.242078] niks3-server[1023]: 2026/09/21 09:28:26 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4338lb # [ 59.254717] niks3-server[1023]: 2026/09/21 09:28:26 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4339worker1 # [ 59.253331] nix-grpc-daemon[853]: time=2026-09-21T09:28:26.885Z level=INFO msg="Upload complete. (495ms)"4340lb # [ 59.259733] niks3-server[1023]: 2026/09/21 09:28:26 INFO Completed upload id=114341lb # [ 59.263746] niks3-server[1023]: 2026/09/21 09:28:26 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4342worker1 # [ 59.259354] nix-grpc-daemon[775]: ts=2026-09-21T09:28:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=0 paths=1 nar_bytes_in=5124343worker1 # [ 59.272849] nix-grpc-daemon[775]: ts=2026-09-21T09:28:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376564344worker1 # [ 59.279051] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4345worker1 # [ 59.283423] nix-daemon[839]: reaped child process 1040, status = succeeded4346worker1 # [ 59.296546] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4347worker1 # [ 60.542448] nix-daemon[839]: reaped child process 1054, status = succeeded4348worker1 # [ 60.719402] nix-daemon[839]: accepted connection from pid 1070, user nix-grpc-daemon (trusted)4349worker1 # [ 60.737622] nix-daemon[839]: reaped child process 1073, status = succeeded4350lb # [ 60.756896] niks3-server[1023]: 2026/09/21 09:28:28 INFO Received uploads request method=POST path=/api/pending_closures4351worker1 # [ 60.763724] nix-grpc-daemon[853]: time=2026-09-21T09:28:28.392Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4352worker1 # [ 60.765427] nix-grpc-daemon[853]: time=2026-09-21T09:28:28.392Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4353lb # [ 60.792936] niks3-server[1023]: 2026/09/21 09:28:28 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4354lb # [ 60.805906] niks3-server[1023]: 2026/09/21 09:28:28 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4355lb # [ 60.815324] niks3-server[1023]: 2026/09/21 09:28:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4356worker1 # [ 60.815677] nix-grpc-daemon[853]: time=2026-09-21T09:28:28.447Z level=INFO msg="Uploading 1 narinfos"4357lb # [ 60.821382] niks3-server[1023]: 2026/09/21 09:28:28 INFO Signed narinfos id=12 count=14358lb # [ 60.827632] niks3-server[1023]: 2026/09/21 09:28:28 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4359lb # [ 60.834063] niks3-server[1023]: 2026/09/21 09:28:28 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4360lb # [ 60.839854] niks3-server[1023]: 2026/09/21 09:28:28 INFO Completed upload id=124361worker1 # [ 60.833362] nix-grpc-daemon[853]: time=2026-09-21T09:28:28.465Z level=INFO msg="Upload complete. (326ms)"4362lb # [ 60.842347] niks3-server[1023]: 2026/09/21 09:28:28 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4363worker1 # [ 60.844935] nix-grpc-daemon[775]: ts=2026-09-21T09:28:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14364lb # [ 60.855324] envoy[913]: conn peer=[2001:db8:1::1]:45520 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4215 tx=1821 ms=22074365worker2 # [ 60.858058] nix-grpc-daemon[777]: ts=2026-09-21T09:28:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:402984366worker1 # [ 60.858457] nix-daemon[839]: reaped child process 1052, status = succeeded4367lb # [ 60.859878] envoy[913]: conn peer=[2001:db8:1::1]:56412 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1085 ms=36834368lb # [ 60.867375] envoy[913]: conn peer=[2001:db8:1::1]:56396 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1452 ms=49384369worker2 # [ 60.880539] nix-grpc-daemon[777]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4370worker1 # [ 60.915006] nix-grpc-daemon[775]: ts=2026-09-21T09:28:28Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:376564371worker1 # [ 60.921981] nix-grpc-daemon[775]: ts=2026-09-21T09:28:28Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=0 paths=1 nar_bytes_out=1284372worker1 # [ 61.188927] nix-grpc-daemon[775]: ts=2026-09-21T09:28:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376564373worker2 # [ 61.204430] nix-grpc-daemon[777]: ts=2026-09-21T09:28:28Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:402984374worker2 # [ 61.218767] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4375worker1 # [ 61.276064] nix-grpc-daemon[775]: ts=2026-09-21T09:28:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376564376worker1 # [ 61.286628] nix-grpc-daemon[853]: time=2026-09-21T09:28:28.918Z level=INFO msg="All 1 paths already cached"4377worker2 # [ 61.317044] nix-grpc-daemon[777]: ts=2026-09-21T09:28:28Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04378worker2 # [ 61.323518] nix-grpc-daemon[777]: ts=2026-09-21T09:28:28Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14379worker1 # [ 61.322639] nix-grpc-daemon[775]: ts=2026-09-21T09:28:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376564380worker1 # [ 61.338163] nix-grpc-daemon[775]: ts=2026-09-21T09:28:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376564381worker1 # [ 61.352354] nix-grpc-daemon[853]: time=2026-09-21T09:28:28.984Z level=INFO msg="All 1 paths already cached"4382worker1 # [ 61.361390] nix-grpc-daemon[775]: ts=2026-09-21T09:28:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376564383worker1 # [ 61.384886] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4384worker1 # [ 61.595579] nix-daemon[839]: accepted connection from pid 1077, user nix-grpc-daemon (trusted)4385worker1 # [ 61.613256] nix-daemon[839]: reaped child process 1080, status = succeeded4386worker1 # [ 61.782487] nix-daemon[839]: accepted connection from pid 1082, user nix-grpc-daemon (trusted)4387worker1 # [ 61.798486] nix-daemon[839]: reaped child process 1085, status = succeeded4388lb # [ 61.817935] niks3-server[1023]: 2026/09/21 09:28:29 INFO Received uploads request method=POST path=/api/pending_closures4389worker1 # [ 61.819905] nix-grpc-daemon[853]: time=2026-09-21T09:28:29.451Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4390worker1 # [ 61.821536] nix-grpc-daemon[853]: time=2026-09-21T09:28:29.451Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4391lb # [ 61.843210] niks3-server[1023]: 2026/09/21 09:28:29 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4392worker1 # [ 61.849199] nix-grpc-daemon[853]: time=2026-09-21T09:28:29.481Z level=INFO msg="Uploading 1 narinfos"4393lb # [ 61.858948] niks3-server[1023]: 2026/09/21 09:28:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4394lb # [ 61.868848] niks3-server[1023]: 2026/09/21 09:28:29 INFO Signed narinfos id=13 count=14395lb # [ 61.873660] niks3-server[1023]: 2026/09/21 09:28:29 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4396lb # [ 61.881873] niks3-server[1023]: 2026/09/21 09:28:29 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4397worker1 # [ 61.880402] nix-grpc-daemon[853]: time=2026-09-21T09:28:29.512Z level=INFO msg="Upload complete. (478ms)"4398lb # [ 61.886729] niks3-server[1023]: 2026/09/21 09:28:29 INFO Completed upload id=134399lb # [ 61.890592] niks3-server[1023]: 2026/09/21 09:28:29 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4400worker1 # [ 61.888624] nix-grpc-daemon[775]: ts=2026-09-21T09:28:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=0 paths=1 nar_bytes_in=6244401worker1 # [ 61.898862] nix-grpc-daemon[775]: ts=2026-09-21T09:28:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376564402worker1 # [ 61.906578] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4403worker1 # [ 61.908835] nix-daemon[839]: reaped child process 1075, status = succeeded4404worker1 # [ 61.918972] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4405worker1 # [ 63.381936] nix-daemon[839]: reaped child process 1089, status = succeeded4406worker1 # [ 63.683338] nix-daemon[839]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)4407worker1 # [ 63.710651] nix-daemon[839]: reaped child process 1108, status = succeeded4408lb # [ 63.734946] niks3-server[1023]: 2026/09/21 09:28:31 INFO Received uploads request method=POST path=/api/pending_closures4409worker1 # [ 63.750661] nix-grpc-daemon[853]: time=2026-09-21T09:28:31.379Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4410worker1 # [ 63.762920] nix-grpc-daemon[853]: time=2026-09-21T09:28:31.384Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4411lb # [ 63.804831] niks3-server[1023]: 2026/09/21 09:28:31 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4412lb # [ 63.825366] niks3-server[1023]: 2026/09/21 09:28:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4413worker1 # [ 63.821232] nix-grpc-daemon[853]: time=2026-09-21T09:28:31.451Z level=INFO msg="Uploading 1 narinfos"4414lb # [ 63.836176] niks3-server[1023]: 2026/09/21 09:28:31 INFO Signed narinfos id=14 count=14415lb # [ 63.847791] niks3-server[1023]: 2026/09/21 09:28:31 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4416lb # [ 63.860848] niks3-server[1023]: 2026/09/21 09:28:31 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4417lb # [ 63.870646] niks3-server[1023]: 2026/09/21 09:28:31 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4418lb # [ 63.881077] niks3-server[1023]: 2026/09/21 09:28:31 INFO Completed upload id=144419lb # [ 63.884189] niks3-server[1023]: 2026/09/21 09:28:31 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4420worker1 # [ 63.873333] nix-grpc-daemon[853]: time=2026-09-21T09:28:31.505Z level=INFO msg="Upload complete. (541ms)"4421worker1 # [ 63.891049] nix-grpc-daemon[775]: ts=2026-09-21T09:28:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14422worker1 # [ 63.912603] nix-grpc-daemon[775]: ts=2026-09-21T09:28:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376564423worker1 # [ 63.929231] nix-daemon[839]: reaped child process 1087, status = succeeded4424worker1 # [ 63.956542] nix-grpc-daemon[775]: ts=2026-09-21T09:28:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:376564425worker1 # [ 63.963462] nix-grpc-daemon[775]: ts=2026-09-21T09:28:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=0 paths=1 nar_bytes_out=1764426worker2 # [ 64.035197] nix-grpc-daemon[777]: ts=2026-09-21T09:28:31Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:40298 duration_s=2 bytes_in=312 bytes_out=2004427worker2 # [ 64.044788] nix-daemon[861]: reaped child process 990, status = succeeded4428worker1 # [ 64.342331] nix-grpc-daemon[775]: ts=2026-09-21T09:28:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376564429worker2 # [ 64.359957] nix-grpc-daemon[777]: ts=2026-09-21T09:28:31Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:402984430worker2 # [ 64.373199] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4431worker1 # [ 64.432634] nix-grpc-daemon[775]: ts=2026-09-21T09:28:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376564432worker1 # [ 64.442939] nix-grpc-daemon[853]: time=2026-09-21T09:28:32.074Z level=INFO msg="All 2 paths already cached"4433worker2 # [ 64.474448] nix-grpc-daemon[777]: ts=2026-09-21T09:28:32Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04434worker2 # [ 64.480856] nix-grpc-daemon[777]: ts=2026-09-21T09:28:32Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14435worker1 # [ 64.476097] nix-grpc-daemon[775]: ts=2026-09-21T09:28:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376564436worker1 # [ 64.492156] nix-grpc-daemon[775]: ts=2026-09-21T09:28:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376564437worker1 # [ 64.505932] nix-grpc-daemon[853]: time=2026-09-21T09:28:32.137Z level=INFO msg="All 2 paths already cached"4438worker1 # [ 64.515016] nix-grpc-daemon[775]: ts=2026-09-21T09:28:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376564439worker1 # [ 64.531428] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4440worker1 # [ 64.746332] nix-daemon[839]: accepted connection from pid 1112, user nix-grpc-daemon (trusted)4441worker1 # [ 64.765782] nix-daemon[839]: reaped child process 1115, status = succeeded4442worker1 # [ 64.935313] nix-daemon[839]: accepted connection from pid 1117, user nix-grpc-daemon (trusted)4443worker1 # [ 64.952831] nix-daemon[839]: reaped child process 1120, status = succeeded4444lb # [ 64.971470] niks3-server[1023]: 2026/09/21 09:28:32 INFO Received uploads request method=POST path=/api/pending_closures4445worker1 # [ 64.976211] nix-grpc-daemon[853]: time=2026-09-21T09:28:32.607Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4446worker1 # [ 64.977665] nix-grpc-daemon[853]: time=2026-09-21T09:28:32.607Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4447lb # [ 64.997073] niks3-server[1023]: 2026/09/21 09:28:32 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4448worker1 # [ 65.001203] nix-grpc-daemon[853]: time=2026-09-21T09:28:32.633Z level=INFO msg="Uploading 1 narinfos"4449lb # [ 65.006894] niks3-server[1023]: 2026/09/21 09:28:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4450lb # [ 65.014547] niks3-server[1023]: 2026/09/21 09:28:32 INFO Signed narinfos id=15 count=14451lb # [ 65.018496] niks3-server[1023]: 2026/09/21 09:28:32 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4452worker1 # [ 65.019807] nix-grpc-daemon[853]: time=2026-09-21T09:28:32.651Z level=INFO msg="Upload complete. (469ms)"4453lb # [ 65.026282] niks3-server[1023]: 2026/09/21 09:28:32 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4454lb # [ 65.030851] niks3-server[1023]: 2026/09/21 09:28:32 INFO Completed upload id=154455worker1 # [ 65.025378] nix-grpc-daemon[775]: ts=2026-09-21T09:28:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=0 paths=1 nar_bytes_in=7524456lb # [ 65.033605] niks3-server[1023]: 2026/09/21 09:28:32 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4457worker1 # [ 65.035360] nix-grpc-daemon[775]: ts=2026-09-21T09:28:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376564458worker1 # [ 65.038161] nix-daemon[839]: reaped child process 1110, status = succeeded4459worker1 # [ 65.044106] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4460worker1 # [ 65.055296] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4461worker1 # [ 66.276253] nix-daemon[839]: reaped child process 1124, status = succeeded4462worker1 # [ 66.430169] nix-daemon[839]: accepted connection from pid 1140, user nix-grpc-daemon (trusted)4463worker1 # [ 66.448399] nix-daemon[839]: reaped child process 1143, status = succeeded4464lb # [ 66.467181] niks3-server[1023]: 2026/09/21 09:28:34 INFO Received uploads request method=POST path=/api/pending_closures4465worker1 # [ 66.472676] nix-grpc-daemon[853]: time=2026-09-21T09:28:34.104Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4466worker1 # [ 66.478876] nix-grpc-daemon[853]: time=2026-09-21T09:28:34.106Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4467lb # [ 66.510261] niks3-server[1023]: 2026/09/21 09:28:34 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4468lb # [ 66.523061] niks3-server[1023]: 2026/09/21 09:28:34 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4469worker1 # [ 66.528862] nix-grpc-daemon[853]: time=2026-09-21T09:28:34.160Z level=INFO msg="Uploading 1 narinfos"4470lb # [ 66.534348] niks3-server[1023]: 2026/09/21 09:28:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4471lb # [ 66.542021] niks3-server[1023]: 2026/09/21 09:28:34 INFO Signed narinfos id=16 count=14472lb # [ 66.548280] niks3-server[1023]: 2026/09/21 09:28:34 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4473lb # [ 66.557198] niks3-server[1023]: 2026/09/21 09:28:34 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4474lb # [ 66.566834] niks3-server[1023]: 2026/09/21 09:28:34 INFO Completed upload id=164475worker1 # [ 66.561155] nix-grpc-daemon[853]: time=2026-09-21T09:28:34.193Z level=INFO msg="Upload complete. (297ms)"4476lb # [ 66.570113] niks3-server[1023]: 2026/09/21 09:28:34 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4477worker1 # [ 66.571372] nix-grpc-daemon[775]: ts=2026-09-21T09:28:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14478worker2 # [ 66.586526] nix-grpc-daemon[777]: ts=2026-09-21T09:28:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:402984479worker1 # [ 66.587692] nix-daemon[839]: reaped child process 1122, status = succeeded4480worker2 # [ 66.608720] nix-grpc-daemon[777]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4481worker2 # [ 66.626963] nix-grpc-daemon[777]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4482worker2 # [ 66.668117] nix-grpc-daemon[777]: ts=2026-09-21T09:28:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:402984483worker2 # [ 66.674647] nix-grpc-daemon[777]: ts=2026-09-21T09:28:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40298 duration_s=0 paths=1 nar_bytes_out=2324484worker2 # [ 66.738577] nix-grpc-daemon[777]: ts=2026-09-21T09:28:34Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:40298 duration_s=2 bytes_in=376 bytes_out=2004485worker2 # [ 66.746106] nix-daemon[861]: reaped child process 995, status = succeeded4486client # [ 66.791808] nix-daemon[944]: reaped child process 946, status = succeeded4487client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.83 seconds)4488client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4489client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.05 seconds)4490client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4491client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4492client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4493client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4494(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.96 seconds)4495subtest: an input only one worker has still reaches the builder4496client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4497client # this derivation will be built:4498client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4499client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4500client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.44 seconds)4501client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4502client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)4503worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4504worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.24 seconds)4505worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4506worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4507client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24508client # [ 68.084460] nix-daemon[944]: accepted connection from pid 1029, user root (trusted)4509client # this derivation will be built:4510client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4511worker2 # [ 68.318747] nix-grpc-daemon[777]: ts=2026-09-21T09:28:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:402984512client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4513worker1 # [ 68.315240] nix-grpc-daemon[775]: ts=2026-09-21T09:28:35Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:376564514worker1 # [ 68.329090] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4515worker2 # [ 68.405918] nix-grpc-daemon[777]: ts=2026-09-21T09:28:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:402984516client # copying 1 paths...4517client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4518worker2 # [ 68.423516] nix-grpc-daemon[777]: ts=2026-09-21T09:28:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:402984519worker2 # [ 68.439833] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4520worker2 # [ 68.654560] nix-daemon[861]: accepted connection from pid 1007, user nix-grpc-daemon (trusted)4521worker2 # [ 68.674301] nix-daemon[861]: reaped child process 1010, status = succeeded4522lb # [ 68.683940] niks3-server[1023]: 2026/09/21 09:28:36 INFO Received uploads request method=POST path=/api/pending_closures4523worker2 # [ 68.697100] nix-grpc-daemon[915]: time=2026-09-21T09:28:36.324Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4524worker2 # [ 68.698568] nix-grpc-daemon[915]: time=2026-09-21T09:28:36.324Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4525lb # [ 68.712544] niks3-server[1023]: 2026/09/21 09:28:36 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4526worker2 # [ 68.728421] nix-grpc-daemon[915]: time=2026-09-21T09:28:36.355Z level=INFO msg="Uploading 1 narinfos"4527lb # [ 68.725046] niks3-server[1023]: 2026/09/21 09:28:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4528lb # [ 68.735751] niks3-server[1023]: 2026/09/21 09:28:36 INFO Signed narinfos id=17 count=14529lb # [ 68.743457] niks3-server[1023]: 2026/09/21 09:28:36 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4530lb # [ 68.751704] niks3-server[1023]: 2026/09/21 09:28:36 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4531lb # [ 68.758673] niks3-server[1023]: 2026/09/21 09:28:36 INFO Completed upload id=174532worker2 # [ 68.762474] nix-grpc-daemon[915]: time=2026-09-21T09:28:36.390Z level=INFO msg="Upload complete. (306ms)"4533lb # [ 68.761707] niks3-server[1023]: 2026/09/21 09:28:36 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4534worker2 # [ 68.772247] nix-grpc-daemon[777]: ts=2026-09-21T09:28:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40298 duration_s=0 paths=1 nar_bytes_in=1204535worker2 # [ 68.785181] nix-daemon[861]: reaped child process 1005, status = succeeded4536worker2 # [ 68.790446] nix-grpc-daemon[777]: ts=2026-09-21T09:28:36Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04537worker2 # [ 68.794841] nix-grpc-daemon[777]: ts=2026-09-21T09:28:36Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14538worker2 # [ 68.836277] nix-grpc-daemon[777]: ts=2026-09-21T09:28:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:402984539worker2 # [ 68.850758] nix-grpc-daemon[777]: ts=2026-09-21T09:28:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:402984540client # copying 2 paths to 'lb:50051'...4541client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4542worker2 # [ 68.869194] nix-grpc-daemon[777]: ts=2026-09-21T09:28:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:402984543client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4544worker2 # [ 68.883026] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4545worker2 # [ 69.082063] nix-daemon[861]: accepted connection from pid 1016, user nix-grpc-daemon (trusted)4546worker2 # [ 69.098835] nix-daemon[861]: reaped child process 1019, status = succeeded4547worker2 # [ 69.269116] nix-daemon[861]: accepted connection from pid 1021, user nix-grpc-daemon (trusted)4548worker2 # [ 69.285401] nix-daemon[861]: reaped child process 1024, status = succeeded4549lb # [ 69.292538] niks3-server[1023]: 2026/09/21 09:28:36 INFO Received uploads request method=POST path=/api/pending_closures4550lb # [ 69.302866] niks3-server[1023]: 2026/09/21 09:28:36 INFO Received uploads request method=POST path=/api/pending_closures4551worker2 # [ 69.315708] nix-grpc-daemon[915]: time=2026-09-21T09:28:36.942Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4552worker2 # [ 69.317372] nix-grpc-daemon[915]: time=2026-09-21T09:28:36.942Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4553worker2 # [ 69.319221] nix-grpc-daemon[915]: time=2026-09-21T09:28:36.943Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4554lb # [ 69.336819] niks3-server[1023]: 2026/09/21 09:28:36 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4555lb # [ 69.350651] niks3-server[1023]: 2026/09/21 09:28:36 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4556lb # [ 69.364284] niks3-server[1023]: 2026/09/21 09:28:36 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4557lb # [ 69.370206] niks3-server[1023]: 2026/09/21 09:28:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4558worker2 # [ 69.382776] nix-grpc-daemon[915]: time=2026-09-21T09:28:37.009Z level=INFO msg="Uploading 3 narinfos"4559lb # [ 69.380341] niks3-server[1023]: 2026/09/21 09:28:36 INFO Signed narinfos id=18 count=14560lb # [ 69.382644] niks3-server[1023]: 2026/09/21 09:28:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4561lb # [ 69.393252] niks3-server[1023]: 2026/09/21 09:28:36 INFO Signed narinfos id=19 count=24562lb # [ 69.395404] niks3-server[1023]: 2026/09/21 09:28:36 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4563lb # [ 69.406002] niks3-server[1023]: 2026/09/21 09:28:37 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4564lb # [ 69.417277] niks3-server[1023]: 2026/09/21 09:28:37 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4565lb # [ 69.424757] niks3-server[1023]: 2026/09/21 09:28:37 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4566lb # [ 69.427573] niks3-server[1023]: 2026/09/21 09:28:37 INFO Completed upload id=184567lb # [ 69.431344] niks3-server[1023]: 2026/09/21 09:28:37 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4568worker2 # [ 69.435236] nix-grpc-daemon[915]: time=2026-09-21T09:28:37.062Z level=INFO msg="Upload complete. (534ms)"4569lb # [ 69.434416] niks3-server[1023]: 2026/09/21 09:28:37 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4570lb # [ 69.438026] niks3-server[1023]: 2026/09/21 09:28:37 INFO Completed upload id=194571worker2 # [ 69.449293] nix-grpc-daemon[777]: ts=2026-09-21T09:28:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40298 duration_s=0 paths=2 nar_bytes_in=9844572worker2 # [ 69.460940] nix-grpc-daemon[777]: ts=2026-09-21T09:28:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:402984573worker2 # [ 69.475134] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4574worker2 # [ 69.479941] nix-daemon[861]: reaped child process 1014, status = succeeded4575client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4576worker2 # [ 69.493567] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4577client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4578client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4579lb # [ 70.857494] envoy[913]: conn peer=[2001:db8:1::1]:50574 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=294580lb # [ 70.862767] envoy[913]: conn peer=[2001:db8:1::1]:56428 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3407 tx=1830 ms=23154581lb # [ 70.869182] envoy[913]: conn peer=[2001:db8:1::1]:50592 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=554582lb # [ 70.873906] envoy[913]: conn peer=[2001:db8:1::1]:50584 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4444 tx=2268 ms=28054583lb # [ 70.878997] envoy[913]: conn peer=[2001:db8:1::1]:50614 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=694584lb # [ 70.883380] envoy[913]: conn peer=[2001:db8:1::1]:50606 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4915 tx=2596 ms=23824585worker2 # [ 70.944287] nix-daemon[861]: reaped child process 1028, status = succeeded4586worker2 # [ 71.127456] nix-daemon[861]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)4587worker2 # [ 71.143093] nix-daemon[861]: reaped child process 1047, status = succeeded4588lb # [ 71.155461] niks3-server[1023]: 2026/09/21 09:28:38 INFO Received uploads request method=POST path=/api/pending_closures4589worker2 # [ 71.168437] nix-grpc-daemon[915]: time=2026-09-21T09:28:38.795Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4590lb # [ 71.187025] niks3-server[1023]: 2026/09/21 09:28:38 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4591worker2 # [ 71.204535] nix-grpc-daemon[915]: time=2026-09-21T09:28:38.831Z level=INFO msg="Uploading 1 narinfos"4592lb # [ 71.201443] niks3-server[1023]: 2026/09/21 09:28:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4593lb # [ 71.210059] niks3-server[1023]: 2026/09/21 09:28:38 INFO Signed narinfos id=20 count=14594lb # [ 71.214861] niks3-server[1023]: 2026/09/21 09:28:38 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4595lb # [ 71.225723] niks3-server[1023]: 2026/09/21 09:28:38 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4596lb # [ 71.230425] niks3-server[1023]: 2026/09/21 09:28:38 INFO Completed upload id=204597worker2 # [ 71.234123] nix-grpc-daemon[915]: time=2026-09-21T09:28:38.861Z level=INFO msg="Upload complete. (326ms)"4598lb # [ 71.232762] niks3-server[1023]: 2026/09/21 09:28:38 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4599client # copying 1 paths...4600worker2 # [ 71.246228] nix-grpc-daemon[777]: ts=2026-09-21T09:28:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40298 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14601worker1 # [ 71.241672] nix-grpc-daemon[775]: ts=2026-09-21T09:28:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376564602worker2 # [ 71.263425] nix-daemon[861]: reaped child process 1026, status = succeeded4603worker1 # [ 71.258011] nix-grpc-daemon[775]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4604client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4605worker1 # [ 71.308111] nix-grpc-daemon[775]: ts=2026-09-21T09:28:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:376564606worker1 # [ 71.313991] nix-grpc-daemon[775]: ts=2026-09-21T09:28:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=0 paths=1 nar_bytes_out=1204607worker1 # [ 71.377101] nix-grpc-daemon[775]: ts=2026-09-21T09:28:39Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=3 bytes_in=320 bytes_out=2004608worker1 # [ 71.384700] nix-daemon[839]: reaped child process 1151, status = succeeded4609client # [ 71.430731] nix-daemon[944]: reaped child process 1032, status = succeeded4610client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.58 seconds)4611client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24612client # [ 71.640937] nix-daemon[944]: accepted connection from pid 1053, user root (trusted)4613client # this derivation will be built:4614client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4615worker2 # [ 71.865068] nix-grpc-daemon[777]: ts=2026-09-21T09:28:39Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:402984616client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4617worker2 # [ 71.873412] nix-grpc-daemon[777]: ts=2026-09-21T09:28:39Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:402984618worker2 # [ 71.885871] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4619worker1 # [ 71.944519] nix-grpc-daemon[775]: ts=2026-09-21T09:28:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376564620worker1 # [ 71.955176] nix-grpc-daemon[853]: time=2026-09-21T09:28:39.586Z level=INFO msg="All 1 paths already cached"4621worker2 # [ 71.986267] nix-grpc-daemon[777]: ts=2026-09-21T09:28:39Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04622worker2 # [ 71.993510] nix-grpc-daemon[777]: ts=2026-09-21T09:28:39Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14623worker1 # [ 71.987095] nix-grpc-daemon[775]: ts=2026-09-21T09:28:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376564624worker1 # [ 72.002480] nix-grpc-daemon[775]: ts=2026-09-21T09:28:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376564625client # copying 1 paths to 'lb:50051'...4626client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4627worker1 # [ 72.023878] nix-grpc-daemon[775]: ts=2026-09-21T09:28:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376564628worker1 # [ 72.037562] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4629worker1 # [ 72.060133] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 1/5)4630worker1 # [ 72.136775] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 2/5)4631worker1 # [ 72.186093] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 135 ms (attempt 3/5)4632worker1 # [ 72.329737] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 539 ms (attempt 4/5)4633worker1 # [ 72.876806] nix-grpc-daemon[775]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4634worker1 # [ 72.884387] nix-grpc-daemon[775]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4635worker1 # [ 72.891665] nix-grpc-daemon[775]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4636worker1 # [ 73.118471] nix-daemon[839]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)4637worker1 # [ 73.135855] nix-daemon[839]: reaped child process 1165, status = succeeded4638worker1 # [ 73.306753] nix-daemon[839]: accepted connection from pid 1167, user nix-grpc-daemon (trusted)4639worker1 # [ 73.322739] nix-daemon[839]: reaped child process 1170, status = succeeded4640lb # [ 73.340242] niks3-server[1023]: 2026/09/21 09:28:40 INFO Received uploads request method=POST path=/api/pending_closures4641worker1 # [ 73.345415] nix-grpc-daemon[853]: time=2026-09-21T09:28:40.976Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4642worker1 # [ 73.346830] nix-grpc-daemon[853]: time=2026-09-21T09:28:40.976Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4643lb # [ 73.367174] niks3-server[1023]: 2026/09/21 09:28:40 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4644lb # [ 73.380764] niks3-server[1023]: 2026/09/21 09:28:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4645worker1 # [ 73.380496] nix-grpc-daemon[853]: time=2026-09-21T09:28:41.012Z level=INFO msg="Uploading 1 narinfos"4646lb # [ 73.392285] niks3-server[1023]: 2026/09/21 09:28:41 INFO Signed narinfos id=21 count=14647lb # [ 73.399140] niks3-server[1023]: 2026/09/21 09:28:41 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4648worker1 # [ 73.406134] nix-grpc-daemon[853]: time=2026-09-21T09:28:41.038Z level=INFO msg="Upload complete. (494ms)"4649lb # [ 73.413734] niks3-server[1023]: 2026/09/21 09:28:41 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4650lb # [ 73.420155] niks3-server[1023]: 2026/09/21 09:28:41 INFO Completed upload id=214651worker1 # [ 73.416064] nix-grpc-daemon[775]: ts=2026-09-21T09:28:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=1 paths=1 nar_bytes_in=5684652lb # [ 73.424111] niks3-server[1023]: 2026/09/21 09:28:41 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4653worker1 # [ 73.423251] nix-grpc-daemon[775]: ts=2026-09-21T09:28:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376564654worker1 # [ 73.434831] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4655worker1 # [ 73.439415] nix-daemon[839]: reaped child process 1155, status = succeeded4656client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4657worker1 # [ 73.451857] nix-daemon[839]: accepted connection from pid 775, user nix-grpc-daemon (trusted)4658client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4659client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4660worker1 # [ 74.886108] nix-daemon[839]: reaped child process 1174, status = succeeded4661worker1 # [ 75.048698] nix-daemon[839]: accepted connection from pid 1190, user nix-grpc-daemon (trusted)4662worker1 # [ 75.063775] nix-daemon[839]: reaped child process 1193, status = succeeded4663lb # [ 75.084696] niks3-server[1023]: 2026/09/21 09:28:42 INFO Received uploads request method=POST path=/api/pending_closures4664worker1 # [ 75.090244] nix-grpc-daemon[853]: time=2026-09-21T09:28:42.720Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4665lb # [ 75.114184] niks3-server[1023]: 2026/09/21 09:28:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4666worker1 # [ 75.119098] nix-grpc-daemon[853]: time=2026-09-21T09:28:42.751Z level=INFO msg="Uploading 1 narinfos"4667lb # [ 75.128430] niks3-server[1023]: 2026/09/21 09:28:42 INFO Signed narinfos id=22 count=14668lb # [ 75.131625] niks3-server[1023]: 2026/09/21 09:28:42 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4669lb # [ 75.144075] niks3-server[1023]: 2026/09/21 09:28:42 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4670lb # [ 75.152617] niks3-server[1023]: 2026/09/21 09:28:42 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4671lb # [ 75.156218] niks3-server[1023]: 2026/09/21 09:28:42 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4672lb # [ 75.160216] niks3-server[1023]: 2026/09/21 09:28:42 INFO Completed upload id=224673worker1 # [ 75.154495] nix-grpc-daemon[853]: time=2026-09-21T09:28:42.786Z level=INFO msg="Upload complete. (292ms)"4674client # copying 1 paths...4675worker1 # [ 75.164915] nix-grpc-daemon[775]: ts=2026-09-21T09:28:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14676worker2 # [ 75.178679] nix-grpc-daemon[777]: ts=2026-09-21T09:28:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:402984677worker1 # [ 75.177640] nix-daemon[839]: reaped child process 1172, status = succeeded4678worker2 # [ 75.194184] nix-grpc-daemon[777]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4679client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4680worker1 # [ 75.231614] nix-grpc-daemon[775]: ts=2026-09-21T09:28:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:376564681worker1 # [ 75.237159] nix-grpc-daemon[775]: ts=2026-09-21T09:28:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=0 paths=1 nar_bytes_out=1204682worker2 # [ 75.262735] nix-grpc-daemon[777]: ts=2026-09-21T09:28:42Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:40298 duration_s=3 bytes_in=320 bytes_out=2004683worker2 # [ 75.269811] nix-daemon[861]: reaped child process 1050, status = succeeded4684client # [ 75.311135] nix-daemon[944]: reaped child process 1056, status = succeeded4685client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.88 seconds)4686client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24687client # [ 75.520375] nix-daemon[944]: accepted connection from pid 1077, user root (trusted)4688client # this derivation will be built:4689client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4690worker1 # [ 75.738194] nix-grpc-daemon[775]: ts=2026-09-21T09:28:43Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:376564691client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4692worker2 # [ 75.752145] nix-grpc-daemon[777]: ts=2026-09-21T09:28:43Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:402984693worker2 # [ 75.767808] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4694worker1 # [ 75.824979] nix-grpc-daemon[775]: ts=2026-09-21T09:28:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376564695worker1 # [ 75.835035] nix-grpc-daemon[853]: time=2026-09-21T09:28:43.467Z level=INFO msg="All 1 paths already cached"4696worker2 # [ 75.866259] nix-grpc-daemon[777]: ts=2026-09-21T09:28:43Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04697worker2 # [ 75.874125] nix-grpc-daemon[777]: ts=2026-09-21T09:28:43Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14698worker2 # [ 75.883662] nix-grpc-daemon[777]: ts=2026-09-21T09:28:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:402984699worker2 # [ 75.895035] nix-grpc-daemon[777]: ts=2026-09-21T09:28:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:402984700client # copying 1 paths to 'lb:50051'...4701worker2 # [ 75.910495] nix-grpc-daemon[915]: time=2026-09-21T09:28:43.536Z level=INFO msg="All 1 paths already cached"4702client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4703worker2 # [ 75.916206] nix-grpc-daemon[777]: ts=2026-09-21T09:28:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:402984704worker2 # [ 75.930976] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4705worker2 # [ 76.139212] nix-daemon[861]: accepted connection from pid 1064, user nix-grpc-daemon (trusted)4706worker2 # [ 76.159347] nix-daemon[861]: reaped child process 1067, status = succeeded4707worker2 # [ 76.331358] nix-daemon[861]: accepted connection from pid 1069, user nix-grpc-daemon (trusted)4708worker2 # [ 76.347652] nix-daemon[861]: reaped child process 1072, status = succeeded4709lb # [ 76.361349] niks3-server[1023]: 2026/09/21 09:28:43 INFO Received uploads request method=POST path=/api/pending_closures4710worker2 # [ 76.375817] nix-grpc-daemon[915]: time=2026-09-21T09:28:44.003Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4711worker2 # [ 76.377486] nix-grpc-daemon[915]: time=2026-09-21T09:28:44.003Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4712lb # [ 76.389802] niks3-server[1023]: 2026/09/21 09:28:44 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4713lb # [ 76.403013] niks3-server[1023]: 2026/09/21 09:28:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4714worker2 # [ 76.412489] nix-grpc-daemon[915]: time=2026-09-21T09:28:44.040Z level=INFO msg="Uploading 1 narinfos"4715lb # [ 76.415458] niks3-server[1023]: 2026/09/21 09:28:44 INFO Signed narinfos id=23 count=14716lb # [ 76.421237] niks3-server[1023]: 2026/09/21 09:28:44 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4717lb # [ 76.430547] niks3-server[1023]: 2026/09/21 09:28:44 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4718lb # [ 76.438849] niks3-server[1023]: 2026/09/21 09:28:44 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4719lb # [ 76.443781] niks3-server[1023]: 2026/09/21 09:28:44 INFO Completed upload id=234720worker2 # [ 76.448183] nix-grpc-daemon[915]: time=2026-09-21T09:28:44.075Z level=INFO msg="Upload complete. (500ms)"4721worker2 # [ 76.456775] nix-grpc-daemon[777]: ts=2026-09-21T09:28:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40298 duration_s=0 paths=1 nar_bytes_in=5684722worker2 # [ 76.470160] nix-grpc-daemon[777]: ts=2026-09-21T09:28:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:402984723worker2 # [ 76.478848] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4724worker2 # [ 76.481461] nix-daemon[861]: reaped child process 1062, status = succeeded4725client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4726worker2 # [ 76.495176] nix-daemon[861]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4727client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4728client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4729worker2 # [ 77.314204] nix-daemon[861]: reaped child process 1076, status = succeeded4730worker2 # [ 77.476493] nix-daemon[861]: accepted connection from pid 1092, user nix-grpc-daemon (trusted)4731worker2 # [ 77.494528] nix-daemon[861]: reaped child process 1095, status = succeeded4732lb # [ 77.505373] niks3-server[1023]: 2026/09/21 09:28:45 INFO Received uploads request method=POST path=/api/pending_closures4733worker2 # [ 77.522216] nix-grpc-daemon[915]: time=2026-09-21T09:28:45.148Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4734lb # [ 77.535323] niks3-server[1023]: 2026/09/21 09:28:45 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4735lb # [ 77.549167] niks3-server[1023]: 2026/09/21 09:28:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4736worker2 # [ 77.558783] nix-grpc-daemon[915]: time=2026-09-21T09:28:45.186Z level=INFO msg="Uploading 1 narinfos"4737lb # [ 77.559611] niks3-server[1023]: 2026/09/21 09:28:45 INFO Signed narinfos id=24 count=14738lb # [ 77.565412] niks3-server[1023]: 2026/09/21 09:28:45 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4739lb # [ 77.572513] niks3-server[1023]: 2026/09/21 09:28:45 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4740lb # [ 77.580977] niks3-server[1023]: 2026/09/21 09:28:45 INFO Completed upload id=244741worker2 # [ 77.584355] nix-grpc-daemon[915]: time=2026-09-21T09:28:45.212Z level=INFO msg="Upload complete. (279ms)"4742lb # [ 77.583576] niks3-server[1023]: 2026/09/21 09:28:45 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4743worker2 # [ 77.592425] nix-grpc-daemon[777]: ts=2026-09-21T09:28:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40298 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14744worker2 # [ 77.604298] nix-daemon[861]: reaped child process 1074, status = succeeded4745client # copying 1 paths...4746worker1 # [ 77.632869] nix-grpc-daemon[775]: ts=2026-09-21T09:28:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376564747worker1 # [ 77.647495] nix-grpc-daemon[775]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4748client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4749worker1 # [ 77.692111] nix-grpc-daemon[775]: ts=2026-09-21T09:28:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:376564750worker1 # [ 77.697883] nix-grpc-daemon[775]: ts=2026-09-21T09:28:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37656 duration_s=0 paths=1 nar_bytes_out=1204751worker2 # [ 77.769071] nix-grpc-daemon[777]: ts=2026-09-21T09:28:45Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:40298 duration_s=2 bytes_in=320 bytes_out=2004752worker2 # [ 77.775721] nix-daemon[861]: reaped child process 1057, status = succeeded4753client # [ 77.817836] nix-daemon[944]: reaped child process 1080, status = succeeded4754client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.51 seconds)4755(finished: subtest: an input only one worker has still reaches the builder, in 10.88 seconds)4756subtest: upload whose reference the worker lacks is completed from the cache4757client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4758client # this derivation will be built:4759client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4760client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4761client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.44 seconds)4762client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4763client # this derivation will be built:4764client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4765client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4766client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.42 seconds)4767client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4768client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.19 seconds)4769worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4770worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.24 seconds)4771worker2 # [ 79.227423] systemd[1]: Stopping Nix worker-protocol over gRPC...4772worker2 # [ 79.346977] nix-grpc-daemon[777]: ts=2026-09-21T09:28:46Z level=info event=signal_exit4773worker2 # [ 79.571685] nix-grpc-daemon[777]: ts=2026-09-21T09:28:47Z level=info event=scheduler_yield4774worker2 # [ 79.581786] nix-grpc-daemon[777]: ts=2026-09-21T09:28:47Z level=info event=scheduler_restarting addr=lb:500514775lb # [ 79.578743] niks3-server[1023]: 2026/09/21 09:28:47 INFO lead: released remote=192.168.1.4:568444776worker2 # [ 79.587271] nix-grpc-daemon[777]: ts=2026-09-21T09:28:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:402864777worker1 # [ 79.580446] nix-grpc-daemon[775]: ts=2026-09-21T09:28:47Z level=info event=scheduler_restarting addr=lb:500514778worker2 # [ 79.593819] nix-grpc-daemon[777]: ts=2026-09-21T09:28:47Z level=info event=worker_gone addr=192.168.1.4:500514779worker2 # [ 79.597998] nix-grpc-daemon[777]: ts=2026-09-21T09:28:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:402864780worker2 # [ 79.602517] nix-grpc-daemon[777]: ts=2026-09-21T09:28:47Z level=info event=worker_gone addr=192.168.1.3:500514781worker2 # [ 79.783319] nix-grpc-daemon[777]: ts=2026-09-21T09:28:47Z level=info event=scheduler_connected addr=lb:500514782worker2 # [ 79.788636] nix-grpc-daemon[777]: ts=2026-09-21T09:28:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4783worker1 # [ 79.780253] nix-grpc-daemon[775]: ts=2026-09-21T09:28:47Z level=info event=scheduler_connected addr=lb:500514784worker1 # [ 79.785323] nix-grpc-daemon[775]: ts=2026-09-21T09:28:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4785worker2 # [ 79.989964] nix-grpc-daemon[777]: ts=2026-09-21T09:28:47Z level=info event=scheduler_connected addr=lb:500514786worker1 # [ 79.986263] nix-grpc-daemon[775]: ts=2026-09-21T09:28:47Z level=info event=scheduler_connected addr=lb:500514787worker2 # [ 79.995487] nix-grpc-daemon[777]: ts=2026-09-21T09:28:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4788worker1 # [ 79.991553] nix-grpc-daemon[775]: ts=2026-09-21T09:28:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4789worker2 # [ 80.196811] nix-grpc-daemon[777]: ts=2026-09-21T09:28:47Z level=info event=scheduler_connected addr=lb:500514790worker1 # [ 80.192526] nix-grpc-daemon[775]: ts=2026-09-21T09:28:47Z level=info event=scheduler_connected addr=lb:500514791worker2 # [ 80.202196] nix-grpc-daemon[777]: ts=2026-09-21T09:28:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4792worker1 # [ 80.197690] nix-grpc-daemon[775]: ts=2026-09-21T09:28:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4793worker2 # [ 80.406194] nix-grpc-daemon[777]: ts=2026-09-21T09:28:48Z level=info event=scheduler_connected addr=lb:500514794worker1 # [ 80.398378] nix-grpc-daemon[775]: ts=2026-09-21T09:28:48Z level=info event=scheduler_connected addr=lb:500514795worker2 # [ 80.411014] nix-grpc-daemon[777]: ts=2026-09-21T09:28:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4796worker1 # [ 80.403709] nix-grpc-daemon[775]: ts=2026-09-21T09:28:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4797lb # [ 80.437230] niks3-server[1023]: 2026/09/21 09:28:48 INFO lead: acquired remote=192.168.1.3:351864798worker1 # [ 80.431845] nix-grpc-daemon[775]: ts=2026-09-21T09:28:48Z level=info event=scheduler_take_over4799worker2 # [ 80.577223] nix-grpc-daemon[777]: ts=2026-09-21T09:28:48Z level=info event=scheduler_restarting4800worker1 # [ 80.606275] nix-grpc-daemon[775]: ts=2026-09-21T09:28:48Z level=info event=scheduler_connected addr=lb:500514801worker1 # [ 80.610444] nix-grpc-daemon[775]: ts=2026-09-21T09:28:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4802worker2 # [ 80.634482] nix-daemon[861]: reaped child process 863, status = succeeded4803worker2 # [ 80.636870] nix-grpc-daemon[777]: ts=2026-09-21T09:28:48Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4804worker1 # [ 80.806673] nix-grpc-daemon[775]: ts=2026-09-21T09:28:48Z level=info event=scheduler_connected addr=lb:500514805worker1 # [ 80.811992] nix-grpc-daemon[775]: ts=2026-09-21T09:28:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4806lb # [ 80.856590] envoy[913]: conn peer=[2001:db8:1::1]:38316 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=684807lb # [ 80.861492] envoy[913]: conn peer=[2001:db8:1::1]:50616 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4786 tx=2281 ms=30654808lb # [ 80.866457] envoy[913]: conn peer=[2001:db8:1::1]:38328 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=214809lb # [ 80.871930] envoy[913]: conn peer=[2001:db8:1::1]:38318 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4317 tx=2305 ms=33954810lb # [ 80.876812] envoy[913]: conn peer=[2001:db8:1::1]:38356 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=664811lb # [ 80.881320] envoy[913]: conn peer=[2001:db8:1::1]:38342 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4336 tx=2306 ms=20174812lb # [ 80.886025] envoy[913]: conn peer=[2001:db8:1::4]:58790 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=42118 tx=2265 ms=592284813worker1 # [ 81.012731] nix-grpc-daemon[775]: ts=2026-09-21T09:28:48Z level=info event=scheduler_connected addr=lb:500514814worker1 # [ 81.017723] nix-grpc-daemon[775]: ts=2026-09-21T09:28:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4815worker1 # [ 81.218585] nix-grpc-daemon[775]: ts=2026-09-21T09:28:48Z level=info event=scheduler_connected addr=lb:500514816worker1 # [ 81.223569] nix-grpc-daemon[775]: ts=2026-09-21T09:28:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4817worker1 # [ 81.424338] nix-grpc-daemon[775]: ts=2026-09-21T09:28:49Z level=info event=scheduler_connected addr=lb:500514818worker1 # [ 81.444113] nix-grpc-daemon[775]: ts=2026-09-21T09:28:49Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:384204819worker2 # [ 81.455309] nix-grpc-daemon[777]: E0000 00:00:1789982929.081544 822 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4820worker1 # [ 81.449784] nix-grpc-daemon[775]: ts=2026-09-21T09:28:49Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04821worker2 # [ 81.463287] nix-grpc-daemon[777]: E0000 00:00:1789982929.090915 822 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4822worker2 # [ 82.068610] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4823worker2 # [ 82.076844] systemd[1]: Stopped Nix worker-protocol over gRPC.4824worker2 # [ 82.079549] systemd[1]: nix-grpc-daemon.service: Consumed 2.015s CPU time over 1min 1.066s wall clock time, 56.3M memory peak, 58.2K incoming IP traffic, 157.2K outgoing IP traffic.4825worker2 # [ 82.098441] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4826worker2 # [ 82.104392] systemd[1]: Closed Nix worker-protocol over gRPC.4827worker2 # [ 82.107122] systemd[1]: nix-grpc-daemon.socket: Consumed 582us CPU time over 1min 10.918s wall clock time, 256K memory peak, 81K incoming IP traffic, 68K outgoing IP traffic.4828lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4829lb: must succeed: curl -sf localhost:9901/clusters4830lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4831lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)4832client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24833worker1 # [ 82.441492] nix-grpc-daemon[775]: ts=2026-09-21T09:28:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376564834client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4835worker1 # [ 82.448838] nix-grpc-daemon[775]: ts=2026-09-21T09:28:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:376564836client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.29 seconds)4837worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4838worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4839worker2 # [ 82.589835] systemd[1]: Listening on Nix worker-protocol over gRPC.4840worker1 # [ 82.658781] systemd[1]: Stopping Nix worker-protocol over gRPC...4841worker2 # [ 83.059315] systemd[1]: Starting scheduler-token.service...4842worker2 # [ 83.163389] systemd[1]: scheduler-token.service: Deactivated successfully.4843worker2 # [ 83.168389] systemd[1]: Finished scheduler-token.service.4844worker2 # [ 83.170386] systemd[1]: scheduler-token.service: Consumed 52ms CPU time over 107ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4845worker2 # [ 83.180983] systemd[1]: Starting Nix worker-protocol over gRPC...4846worker1 # [ 83.184463] nix-grpc-daemon[775]: ts=2026-09-21T09:28:50Z level=info event=signal_exit4847worker2 # [ 83.364568] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:50Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4848worker2 # [ 83.367386] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4849worker2 # [ 83.371660] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14850worker2 # [ 83.379512] nix-grpc-daemon[1119]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4851worker2 # [ 83.391930] nix-grpc-daemon[1119]: E0000 00:00:1789982931.006426 1119 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4852worker2 # [ 83.395123] nix-grpc-daemon[1119]: E0000 00:00:1789982931.010346 1119 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4853worker2 # [ 83.399913] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:51Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4854worker2 # [ 83.408202] systemd[1]: Started Nix worker-protocol over gRPC.4855worker2 # [ 83.409045] nix-grpc-daemon[1119]: E0000 00:00:1789982931.032133 1159 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4856worker2 # [ 83.438652] nix-grpc-daemon[1119]: E0000 00:00:1789982931.050919 1159 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4857worker2 # [ 83.441639] nix-grpc-daemon[1119]: E0000 00:00:1789982931.051068 1159 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4858worker2 # [ 83.444395] nix-grpc-daemon[1119]: E0000 00:00:1789982931.065786 1159 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4859worker2 # [ 83.452056] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:51Z level=info event=scheduler_connected addr=lb:500514860worker2 # [ 83.453483] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4861worker1 # [ 83.443907] nix-grpc-daemon[775]: ts=2026-09-21T09:28:51Z level=info event=scheduler_yield4862lb # [ 83.451192] niks3-server[1023]: 2026/09/21 09:28:51 INFO lead: released remote=192.168.1.3:351864863worker1 # [ 83.449023] nix-grpc-daemon[775]: ts=2026-09-21T09:28:51Z level=info event=scheduler_restarting addr=lb:500514864worker1 # [ 83.459045] nix-grpc-daemon[775]: ts=2026-09-21T09:28:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:384204865worker1 # [ 83.463989] nix-grpc-daemon[775]: ts=2026-09-21T09:28:51Z level=info event=worker_gone addr=192.168.1.3:500514866worker2 # [ 83.653615] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:51Z level=info event=scheduler_connected addr=lb:500514867worker1 # [ 83.649667] nix-grpc-daemon[775]: ts=2026-09-21T09:28:51Z level=info event=scheduler_connected addr=lb:500514868worker2 # [ 83.658622] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4869worker1 # [ 83.654660] nix-grpc-daemon[775]: ts=2026-09-21T09:28:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4870worker2 # [ 83.859982] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:51Z level=info event=scheduler_connected addr=lb:500514871worker1 # [ 83.855472] nix-grpc-daemon[775]: ts=2026-09-21T09:28:51Z level=info event=scheduler_connected addr=lb:500514872worker2 # [ 83.865234] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4873worker1 # [ 83.860765] nix-grpc-daemon[775]: ts=2026-09-21T09:28:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4874worker2 # [ 84.066421] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:51Z level=info event=scheduler_connected addr=lb:500514875worker1 # [ 84.061502] nix-grpc-daemon[775]: ts=2026-09-21T09:28:51Z level=info event=scheduler_connected addr=lb:500514876worker2 # [ 84.071725] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4877worker1 # [ 84.066479] nix-grpc-daemon[775]: ts=2026-09-21T09:28:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4878worker2 # [ 84.272947] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:51Z level=info event=scheduler_connected addr=lb:500514879worker1 # [ 84.267294] nix-grpc-daemon[775]: ts=2026-09-21T09:28:51Z level=info event=scheduler_connected addr=lb:500514880worker2 # [ 84.278183] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4881worker1 # [ 84.270334] nix-grpc-daemon[775]: ts=2026-09-21T09:28:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4882worker1 # [ 84.453990] nix-grpc-daemon[775]: ts=2026-09-21T09:28:52Z level=info event=scheduler_restarting4883worker2 # [ 84.481204] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:52Z level=info event=scheduler_connected addr=lb:500514884worker2 # [ 84.485501] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4885worker1 # [ 84.502586] nix-grpc-daemon[775]: ts=2026-09-21T09:28:52Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4886worker1 # [ 84.515984] nix-daemon[839]: reaped child process 841, status = succeeded4887worker2 # [ 84.681792] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:52Z level=info event=scheduler_connected addr=lb:500514888worker2 # [ 84.686988] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4889worker2 # [ 84.888496] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:52Z level=info event=scheduler_connected addr=lb:500514890worker2 # [ 84.893710] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4891worker2 # [ 85.095064] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:52Z level=info event=scheduler_connected addr=lb:500514892worker2 # [ 85.100322] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4893worker1 # [ 85.274086] nix-grpc-daemon[775]: E0000 00:00:1789982932.905814 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true4894worker2 # [ 85.301431] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:52Z level=info event=scheduler_connected addr=lb:500514895worker2 # [ 85.306366] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4896worker1 # [ 85.327137] nix-grpc-daemon[775]: E0000 00:00:1789982932.959029 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4897worker2 # [ 85.508852] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:53Z level=info event=scheduler_connected addr=lb:500514898worker2 # [ 85.513521] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4899worker2 # [ 85.714533] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:53Z level=info event=scheduler_connected addr=lb:500514900worker2 # [ 85.719223] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4901worker2 # [ 85.920373] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:53Z level=info event=scheduler_connected addr=lb:500514902worker2 # [ 85.925451] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4903worker1 # [ 85.983229] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4904worker1 # [ 85.986661] systemd[1]: Stopped Nix worker-protocol over gRPC.4905worker1 # [ 85.994633] systemd[1]: nix-grpc-daemon.service: Consumed 3.617s CPU time over 1min 5.131s wall clock time, 63.5M memory peak, 86.3K incoming IP traffic, 160.2K outgoing IP traffic.4906worker1 # [ 86.007315] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4907worker1 # [ 86.010190] systemd[1]: Closed Nix worker-protocol over gRPC.4908worker1 # [ 86.011939] systemd[1]: nix-grpc-daemon.socket: Consumed 593us CPU time over 1min 14.651s wall clock time, 256K memory peak, 85.7K incoming IP traffic, 72.3K outgoing IP traffic.4909lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4910lb: must succeed: curl -sf localhost:9901/clusters4911lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4912lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4913client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24914worker2 # [ 86.126602] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:53Z level=info event=scheduler_connected addr=lb:500514915worker2 # [ 86.131640] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4916worker2 # [ 86.333022] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:53Z level=info event=scheduler_connected addr=lb:500514917worker2 # [ 86.338128] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4918worker2 # [ 86.386926] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:54Z level=debug event=rpc_start method=Connect cn=ci-1 peer=ipv4:192.168.1.2:572924919worker2 # [ 86.402376] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)4920worker2 # [ 86.476418] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572924921worker2 # [ 86.490410] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)4922worker2 # [ 86.519791] nix-grpc-daemon[1119]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 1/5)4923worker2 # [ 86.527764] nix-grpc-daemon[1119]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 2/5)4924worker2 # [ 86.539045] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:54Z level=info event=scheduler_connected addr=lb:500514925worker2 # [ 86.545106] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4926worker2 # [ 86.550148] nix-grpc-daemon[1119]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 136 ms (attempt 3/5)4927worker2 # [ 86.681084] nix-grpc-daemon[1119]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 550 ms (attempt 4/5)4928worker2 # [ 86.745127] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:54Z level=info event=scheduler_connected addr=lb:500514929worker2 # [ 86.750294] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4930worker2 # [ 86.951418] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:54Z level=info event=scheduler_connected addr=lb:500514931worker2 # [ 86.956647] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4932worker2 # [ 87.157599] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:54Z level=info event=scheduler_connected addr=lb:500514933worker2 # [ 87.162577] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4934worker2 # [ 87.239026] nix-grpc-daemon[1119]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4935worker2 # [ 87.244734] nix-grpc-daemon[1119]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4936client # copying 2 paths...4937client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4938worker2 # [ 87.266528] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:572924939client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4940worker2 # [ 87.285020] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)4941worker2 # [ 87.312785] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:54Z level=info event=niks3_push_started pid=11834942worker2 # [ 87.363472] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:54Z level=info event=scheduler_connected addr=lb:500514943worker2 # [ 87.367696] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4944worker2 # [ 87.541127] nix-grpc-daemon[1183]: time=2026-09-21T09:28:55.168Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4945worker2 # [ 87.568467] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:55Z level=info event=scheduler_connected addr=lb:500514946worker2 # [ 87.570384] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4947worker2 # [ 87.719309] nix-daemon[861]: accepted connection from pid 1196, user nix-grpc-daemon (trusted)4948worker2 # [ 87.734682] nix-daemon[861]: reaped child process 1199, status = succeeded4949lb # [ 87.750117] niks3-server[1023]: 2026/09/21 09:28:55 INFO Received uploads request method=POST path=/api/pending_closures4950lb # [ 87.758473] niks3-server[1023]: 2026/09/21 09:28:55 INFO Received uploads request method=POST path=/api/pending_closures4951worker2 # [ 87.772079] nix-grpc-daemon[1183]: time=2026-09-21T09:28:55.398Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4952worker2 # [ 87.773593] nix-grpc-daemon[1183]: time=2026-09-21T09:28:55.398Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4953worker2 # [ 87.775263] nix-grpc-daemon[1183]: time=2026-09-21T09:28:55.398Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4954worker2 # [ 87.779732] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:55Z level=info event=scheduler_connected addr=lb:500514955worker2 # [ 87.781529] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4956lb # [ 87.824936] niks3-server[1023]: 2026/09/21 09:28:55 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4957lb # [ 87.836413] niks3-server[1023]: 2026/09/21 09:28:55 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4958worker2 # [ 87.851065] nix-grpc-daemon[1183]: time=2026-09-21T09:28:55.477Z level=INFO msg="Uploading 3 narinfos"4959lb # [ 87.847249] niks3-server[1023]: 2026/09/21 09:28:55 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4960lb # [ 87.855653] niks3-server[1023]: 2026/09/21 09:28:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4961lb # [ 87.863503] niks3-server[1023]: 2026/09/21 09:28:55 INFO Signed narinfos id=25 count=14962lb # [ 87.868407] niks3-server[1023]: 2026/09/21 09:28:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4963lb # [ 87.871440] niks3-server[1023]: 2026/09/21 09:28:55 INFO Signed narinfos id=26 count=24964lb # [ 87.879959] niks3-server[1023]: 2026/09/21 09:28:55 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4965lb # [ 87.887132] niks3-server[1023]: 2026/09/21 09:28:55 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4966lb # [ 87.892786] niks3-server[1023]: 2026/09/21 09:28:55 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4967lb # [ 87.897220] niks3-server[1023]: 2026/09/21 09:28:55 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4968lb # [ 87.901294] niks3-server[1023]: 2026/09/21 09:28:55 INFO Completed upload id=254969lb # [ 87.902803] niks3-server[1023]: 2026/09/21 09:28:55 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4970lb # [ 87.906596] niks3-server[1023]: 2026/09/21 09:28:55 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4971lb # [ 87.909995] niks3-server[1023]: 2026/09/21 09:28:55 INFO Completed upload id=264972worker2 # [ 87.913889] nix-grpc-daemon[1183]: time=2026-09-21T09:28:55.541Z level=INFO msg="Upload complete. (369ms)"4973worker2 # [ 87.925609] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57292 duration_s=0 paths=2 nar_bytes_in=3124974worker2 # [ 87.955054] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:55Z level=info event=rpc method=Connect cn=ci-1 peer=ipv4:192.168.1.2:57292 duration_s=1 bytes_in=400 bytes_out=2004975worker2 # [ 87.959809] nix-daemon[861]: reaped child process 1181, status = succeeded4976worker2 # [ 87.962084] nix-daemon[861]: reaped child process 1172, status = succeeded4977client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.85 seconds)4978worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4979worker2 # [ 87.980606] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:55Z level=info event=scheduler_connected addr=lb:500514980worker2 # [ 87.987210] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4981worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds)4982worker1 # [ 88.088882] systemd[1]: Listening on Nix worker-protocol over gRPC.4983worker1 # [ 88.096903] systemd[1]: Starting Nix worker-protocol over gRPC...4984worker2 # [ 88.186697] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:55Z level=info event=scheduler_connected addr=lb:500514985worker2 # [ 88.191941] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4986worker1 # [ 88.295500] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:55Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4987worker1 # [ 88.298092] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:55Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4988worker1 # [ 88.302314] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:55Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14989worker1 # [ 88.310873] nix-grpc-daemon[1217]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4990worker1 # [ 88.317811] nix-grpc-daemon[1217]: E0000 00:00:1789982935.940794 1217 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4991worker1 # [ 88.322667] nix-grpc-daemon[1217]: E0000 00:00:1789982935.942778 1217 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4992lb # [ 88.329823] niks3-server[1023]: 2026/09/21 09:28:55 INFO lead: acquired remote=192.168.1.3:344104993worker1 # [ 88.327010] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:55Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4994worker1 # [ 88.329538] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:55Z level=info event=scheduler_take_over4995worker1 # [ 88.330932] systemd[1]: Started Nix worker-protocol over gRPC.4996lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4997lb: must succeed: curl -sf localhost:9901/clusters4998worker1 # [ 88.352170] nix-grpc-daemon[1217]: E0000 00:00:1789982935.964929 1257 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4999worker1 # [ 88.354526] nix-grpc-daemon[1217]: E0000 00:00:1789982935.980275 1257 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5000worker1 # [ 88.368590] nix-grpc-daemon[1217]: E0000 00:00:1789982935.980435 1257 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5001worker1 # [ 88.371413] nix-grpc-daemon[1217]: E0000 00:00:1789982935.999627 1257 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5002worker2 # [ 88.392997] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:56Z level=info event=scheduler_connected addr=lb:500515003worker1 # [ 88.387607] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:56Z level=info event=scheduler_connected addr=lb:500515004worker1 # [ 88.389455] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5005worker2 # [ 88.399139] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5006lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5007worker1 # [ 88.590633] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:56Z level=info event=scheduler_connected addr=lb:500515008worker2 # [ 88.602363] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:56Z level=info event=scheduler_connected addr=lb:500515009worker1 # [ 88.596145] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5010worker2 # [ 88.606582] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5011worker2 # [ 88.803205] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:56Z level=info event=scheduler_connected addr=lb:500515012worker1 # [ 88.796910] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:56Z level=info event=scheduler_connected addr=lb:500515013worker2 # [ 88.808465] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5014worker1 # [ 88.802171] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5015worker2 # [ 89.009438] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:56Z level=info event=scheduler_connected addr=lb:500515016worker1 # [ 89.002816] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:56Z level=info event=scheduler_connected addr=lb:500515017worker1 # [ 89.010877] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:56Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:384925018worker1 # [ 89.016984] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:56Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05019worker1 # [ 89.028167] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:56Z level=info event=oidc_jwks provider=mock keys=15020worker1 # [ 89.033479] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:56Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:384925021worker1 # [ 89.038813] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:56Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05022lb: must succeed: curl -sf localhost:9901/clusters5023lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5024lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.18 seconds)5025(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.69 seconds)5026subtest: scheduler down: the other node takes the lock and keeps it5027lb: must succeed: curl -sf localhost:9901/clusters5028lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5029lb: must succeed: curl -sf localhost:9901/clusters5030lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5031lb: must succeed: curl -sf localhost:9901/clusters5032lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5033lb: must succeed: curl -sf localhost:9901/clusters5034lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5035worker1 # [ 89.785881] systemd[1]: Stopping Nix worker-protocol over gRPC...5036worker1 # [ 90.316698] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:57Z level=info event=signal_exit5037worker1 # [ 90.521967] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:58Z level=info event=scheduler_yield5038lb # [ 90.537023] niks3-server[1023]: 2026/09/21 09:28:58 INFO lead: released remote=192.168.1.3:344105039worker2 # [ 90.542619] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:58Z level=info event=scheduler_restarting addr=lb:500515040worker1 # [ 90.536799] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:384925041worker1 # [ 90.543323] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:58Z level=info event=worker_gone addr=192.168.1.4:500515042worker1 # [ 90.547520] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:58Z level=info event=scheduler_restarting addr=lb:500515043worker1 # [ 90.551682] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:384925044worker1 # [ 90.556310] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:58Z level=info event=worker_gone addr=192.168.1.3:500515045worker2 # [ 90.742808] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:58Z level=info event=scheduler_connected addr=lb:500515046worker2 # [ 90.748156] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5047worker1 # [ 90.743065] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:58Z level=info event=scheduler_connected addr=lb:500515048worker1 # [ 90.749343] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5049lb # [ 90.852941] envoy[913]: conn peer=[2001:db8:1::1]:55126 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=225050lb # [ 90.857888] envoy[913]: conn peer=[2001:db8:1::3]:39888 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=10561 tx=2995 ms=632505051lb # [ 90.862876] envoy[913]: conn peer=[2001:db8:1::1]:55128 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1611 tx=640 ms=15545052worker2 # [ 90.949225] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:58Z level=info event=scheduler_connected addr=lb:500515053worker2 # [ 90.954302] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5054worker1 # [ 90.950166] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:58Z level=info event=scheduler_connected addr=lb:500515055worker1 # [ 90.956477] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:58Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5056worker2 # [ 91.155341] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:58Z level=info event=scheduler_connected addr=lb:500515057worker2 # [ 91.160508] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5058worker1 # [ 91.157407] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:58Z level=info event=scheduler_connected addr=lb:500515059worker1 # [ 91.162522] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5060worker2 # [ 91.362358] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:58Z level=info event=scheduler_connected addr=lb:500515061worker2 # [ 91.367838] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5062worker1 # [ 91.365087] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:58Z level=info event=scheduler_connected addr=lb:500515063worker1 # [ 91.369273] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5064worker1 # [ 91.527083] nix-grpc-daemon[1217]: ts=2026-09-21T09:28:59Z level=info event=scheduler_restarting5065worker2 # [ 91.571057] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:59Z level=info event=scheduler_connected addr=lb:500515066worker2 # [ 91.575450] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5067worker2 # [ 91.772122] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:59Z level=info event=scheduler_connected addr=lb:500515068worker2 # [ 91.777480] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5069worker2 # [ 91.978774] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:59Z level=info event=scheduler_connected addr=lb:500515070worker2 # [ 91.984130] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5071worker1 # [ 91.986376] nix-grpc-daemon[1217]: E0000 00:00:1789982939.618125 1262 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5072worker1 # [ 91.993110] nix-grpc-daemon[1217]: E0000 00:00:1789982939.618330 1262 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5073worker2 # [ 92.185150] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:59Z level=info event=scheduler_connected addr=lb:500515074worker2 # [ 92.190380] nix-grpc-daemon[1119]: ts=2026-09-21T09:28:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5075worker1 # [ 92.341811] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5076worker1 # [ 92.348460] systemd[1]: Stopped Nix worker-protocol over gRPC.5077worker1 # [ 92.355428] systemd[1]: nix-grpc-daemon.service: Consumed 231ms CPU time over 4.241s wall clock time, 13.3M memory peak, 9.8K incoming IP traffic, 9.1K outgoing IP traffic.5078worker1 # [ 92.371810] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5079worker1 # [ 92.375430] systemd[1]: Closed Nix worker-protocol over gRPC.5080worker2 # [ 92.391053] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:00Z level=info event=scheduler_connected addr=lb:500515081worker1 # [ 92.380465] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 4.294s wall clock time, 256K memory peak, 16.2K incoming IP traffic, 11.5K outgoing IP traffic.5082worker2 # [ 92.394139] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5083worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5084worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.06 seconds)5085worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5086worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5087worker2 # [ 92.597216] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:00Z level=info event=scheduler_connected addr=lb:500515088worker2 # [ 92.601486] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5089worker2 # [ 92.797971] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:00Z level=info event=scheduler_connected addr=lb:500515090worker2 # [ 92.803036] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5091worker2 # [ 93.004393] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:00Z level=info event=scheduler_connected addr=lb:500515092worker2 # [ 93.009425] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5093worker2 # [ 93.210650] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:00Z level=info event=scheduler_connected addr=lb:500515094worker2 # [ 93.215786] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5095worker2 # [ 93.396054] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:01Z level=info event=scheduler_take_over5096lb # [ 93.392974] niks3-server[1023]: 2026/09/21 09:29:01 INFO lead: acquired remote=192.168.1.4:344765097worker2 # [ 93.416850] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:01Z level=info event=scheduler_connected addr=lb:500515098worker2 # [ 93.423445] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5099worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5100worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5101worker2 # [ 93.624427] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:01Z level=info event=scheduler_connected addr=lb:500515102worker2 # [ 93.652233] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:01Z level=info event=oidc_jwks provider=mock keys=15103worker2 # [ 93.656648] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:01Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:601925104worker2 # [ 93.661920] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:01Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05105worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5106worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5107client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25108worker2 # [ 94.915912] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:572925109client # don't know how to build these paths:5110client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5111worker2 # [ 94.925445] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572925112worker2 # [ 94.950679] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:02Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05113worker2 # [ 94.957528] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:02Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15114worker2 # [ 94.995348] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572925115worker2 # [ 95.010942] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572925116client # copying 1 paths to 'lb:50051'...5117client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5118worker2 # [ 95.026262] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:572925119worker2 # [ 95.041452] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5120worker2 # [ 95.252887] nix-daemon[861]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5121worker2 # [ 95.268755] nix-daemon[861]: reaped child process 1226, status = succeeded5122worker2 # [ 95.443964] nix-daemon[861]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5123worker2 # [ 95.460700] nix-daemon[861]: reaped child process 1231, status = succeeded5124lb # [ 95.469541] niks3-server[1023]: 2026/09/21 09:29:03 INFO Received uploads request method=POST path=/api/pending_closures5125worker2 # [ 95.483006] nix-grpc-daemon[1183]: time=2026-09-21T09:29:03.110Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5126worker2 # [ 95.484639] nix-grpc-daemon[1183]: time=2026-09-21T09:29:03.110Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5127lb # [ 95.503801] niks3-server[1023]: 2026/09/21 09:29:03 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5128lb # [ 95.515731] niks3-server[1023]: 2026/09/21 09:29:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5129lb # [ 95.527565] niks3-server[1023]: 2026/09/21 09:29:03 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5130lb # [ 95.539956] niks3-server[1023]: 2026/09/21 09:29:03 INFO Signed narinfos id=27 count=15131worker2 # [ 95.544189] nix-grpc-daemon[1183]: time=2026-09-21T09:29:03.171Z level=INFO msg="Uploading 1 narinfos"5132lb # [ 95.549894] niks3-server[1023]: 2026/09/21 09:29:03 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5133lb # [ 95.560564] niks3-server[1023]: 2026/09/21 09:29:03 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5134lb # [ 95.571196] niks3-server[1023]: 2026/09/21 09:29:03 INFO Completed upload id=275135worker2 # [ 95.575205] nix-grpc-daemon[1183]: time=2026-09-21T09:29:03.202Z level=INFO msg="Upload complete. (517ms)"5136worker2 # [ 95.588611] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57292 duration_s=0 paths=1 nar_bytes_in=5205137worker2 # [ 95.602362] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572925138worker2 # [ 95.612864] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5139worker2 # [ 95.621302] nix-daemon[861]: reaped child process 1221, status = succeeded5140client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5141client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5142worker2 # [ 95.628644] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5143client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5144client # farm-a> LOG-farm-a-on-standby5145worker2 # [ 97.070731] nix-daemon[861]: reaped child process 1235, status = succeeded5146worker2 # [ 97.252487] nix-daemon[861]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5147worker2 # [ 97.270395] nix-daemon[861]: reaped child process 1254, status = succeeded5148lb # [ 97.282038] niks3-server[1023]: 2026/09/21 09:29:04 INFO Received uploads request method=POST path=/api/pending_closures5149worker2 # [ 97.300441] nix-grpc-daemon[1183]: time=2026-09-21T09:29:04.926Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5150worker2 # [ 97.302056] nix-grpc-daemon[1183]: time=2026-09-21T09:29:04.926Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5151lb # [ 97.318884] niks3-server[1023]: 2026/09/21 09:29:04 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5152lb # [ 97.335949] niks3-server[1023]: 2026/09/21 09:29:04 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5153lb # [ 97.345014] niks3-server[1023]: 2026/09/21 09:29:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5154worker2 # [ 97.354029] nix-grpc-daemon[1183]: time=2026-09-21T09:29:04.980Z level=INFO msg="Uploading 1 narinfos"5155lb # [ 97.350112] niks3-server[1023]: 2026/09/21 09:29:04 INFO Signed narinfos id=28 count=15156lb # [ 97.353675] niks3-server[1023]: 2026/09/21 09:29:04 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5157lb # [ 97.363784] niks3-server[1023]: 2026/09/21 09:29:04 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5158lb # [ 97.371658] niks3-server[1023]: 2026/09/21 09:29:04 INFO Completed upload id=285159worker2 # [ 97.373844] nix-grpc-daemon[1183]: time=2026-09-21T09:29:05.001Z level=INFO msg="Upload complete. (335ms)"5160lb # [ 97.374453] niks3-server[1023]: 2026/09/21 09:29:04 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5161worker2 # [ 97.394518] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57292 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15162worker2 # [ 97.410220] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:05Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05163worker2 # [ 97.413762] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:05Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15164worker2 # [ 97.417474] nix-daemon[861]: reaped child process 1233, status = succeeded5165worker2 # [ 97.440869] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572925166worker2 # [ 97.456847] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572925167worker2 # [ 97.471642] nix-grpc-daemon[1183]: time=2026-09-21T09:29:05.099Z level=INFO msg="All 1 paths already cached"5168client # copying 1 paths to 'lb:50051'...5169client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5170worker2 # [ 97.482346] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:572925171worker2 # [ 97.496404] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5172worker2 # [ 97.712346] nix-daemon[861]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5173worker2 # [ 97.732924] nix-daemon[861]: reaped child process 1263, status = succeeded5174worker2 # [ 97.916204] nix-daemon[861]: accepted connection from pid 1265, user nix-grpc-daemon (trusted)5175worker2 # [ 97.933443] nix-daemon[861]: reaped child process 1268, status = succeeded5176lb # [ 97.942673] niks3-server[1023]: 2026/09/21 09:29:05 INFO Received uploads request method=POST path=/api/pending_closures5177worker2 # [ 97.956176] nix-grpc-daemon[1183]: time=2026-09-21T09:29:05.583Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5178worker2 # [ 97.958255] nix-grpc-daemon[1183]: time=2026-09-21T09:29:05.585Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5179lb # [ 97.976349] niks3-server[1023]: 2026/09/21 09:29:05 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5180worker2 # [ 97.993171] nix-grpc-daemon[1183]: time=2026-09-21T09:29:05.620Z level=INFO msg="Uploading 1 narinfos"5181lb # [ 97.993390] niks3-server[1023]: 2026/09/21 09:29:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5182lb # [ 98.004325] niks3-server[1023]: 2026/09/21 09:29:05 INFO Signed narinfos id=29 count=15183lb # [ 98.008631] niks3-server[1023]: 2026/09/21 09:29:05 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5184lb # [ 98.016526] niks3-server[1023]: 2026/09/21 09:29:05 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5185lb # [ 98.022901] niks3-server[1023]: 2026/09/21 09:29:05 INFO Completed upload id=295186worker2 # [ 98.025956] nix-grpc-daemon[1183]: time=2026-09-21T09:29:05.653Z level=INFO msg="Upload complete. (512ms)"5187lb # [ 98.025865] niks3-server[1023]: 2026/09/21 09:29:05 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5188worker2 # [ 98.036238] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57292 duration_s=0 paths=1 nar_bytes_in=6405189worker2 # [ 98.047032] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572925190worker2 # [ 98.054318] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5191worker2 # [ 98.059047] nix-daemon[861]: reaped child process 1258, status = succeeded5192client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5193client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5194worker2 # [ 98.073186] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5195client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5196client # farm-b> LOG-farm-b-on-standby5197worker2 # [ 99.121456] nix-daemon[861]: reaped child process 1272, status = succeeded5198worker2 # [ 99.297685] nix-daemon[861]: accepted connection from pid 1288, user nix-grpc-daemon (trusted)5199worker2 # [ 99.314075] nix-daemon[861]: reaped child process 1291, status = succeeded5200lb # [ 99.322905] niks3-server[1023]: 2026/09/21 09:29:06 INFO Received uploads request method=POST path=/api/pending_closures5201worker2 # [ 99.341257] nix-grpc-daemon[1183]: time=2026-09-21T09:29:06.968Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5202worker2 # [ 99.342897] nix-grpc-daemon[1183]: time=2026-09-21T09:29:06.968Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5203lb # [ 99.360792] niks3-server[1023]: 2026/09/21 09:29:06 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5204lb # [ 99.371195] niks3-server[1023]: 2026/09/21 09:29:06 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5205lb # [ 99.381182] niks3-server[1023]: 2026/09/21 09:29:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5206lb # [ 99.386747] niks3-server[1023]: 2026/09/21 09:29:07 INFO Signed narinfos id=30 count=15207worker2 # [ 99.390982] nix-grpc-daemon[1183]: time=2026-09-21T09:29:07.018Z level=INFO msg="Uploading 1 narinfos"5208lb # [ 99.389700] niks3-server[1023]: 2026/09/21 09:29:07 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5209lb # [ 99.399917] niks3-server[1023]: 2026/09/21 09:29:07 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5210worker2 # [ 99.408439] nix-grpc-daemon[1183]: time=2026-09-21T09:29:07.036Z level=INFO msg="Upload complete. (325ms)"5211lb # [ 99.406253] niks3-server[1023]: 2026/09/21 09:29:07 INFO Completed upload id=305212lb # [ 99.408873] niks3-server[1023]: 2026/09/21 09:29:07 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5213worker2 # [ 99.426437] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57292 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15214worker2 # [ 99.441136] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:07Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05215worker2 # [ 99.445389] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:07Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15216worker2 # [ 99.449229] nix-daemon[861]: reaped child process 1270, status = succeeded5217worker2 # [ 99.471792] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572925218worker2 # [ 99.486639] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572925219worker2 # [ 99.500633] nix-grpc-daemon[1183]: time=2026-09-21T09:29:07.128Z level=INFO msg="All 2 paths already cached"5220client # copying 1 paths to 'lb:50051'...5221client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5222worker2 # [ 99.509232] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:572925223worker2 # [ 99.522626] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5224worker2 # [ 99.720908] nix-daemon[861]: accepted connection from pid 1297, user nix-grpc-daemon (trusted)5225worker2 # [ 99.737952] nix-daemon[861]: reaped child process 1300, status = succeeded5226worker2 # [ 99.913580] nix-daemon[861]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5227worker2 # [ 99.930154] nix-daemon[861]: reaped child process 1305, status = succeeded5228lb # [ 99.937319] niks3-server[1023]: 2026/09/21 09:29:07 INFO Received uploads request method=POST path=/api/pending_closures5229worker2 # [ 99.951012] nix-grpc-daemon[1183]: time=2026-09-21T09:29:07.578Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5230worker2 # [ 99.952593] nix-grpc-daemon[1183]: time=2026-09-21T09:29:07.578Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5231lb # [ 99.962421] niks3-server[1023]: 2026/09/21 09:29:07 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5232lb # [ 99.977195] niks3-server[1023]: 2026/09/21 09:29:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5233worker2 # [ 99.986850] nix-grpc-daemon[1183]: time=2026-09-21T09:29:07.614Z level=INFO msg="Uploading 1 narinfos"5234lb # [ 99.989459] niks3-server[1023]: 2026/09/21 09:29:07 INFO Signed narinfos id=31 count=15235lb # [ 99.996463] niks3-server[1023]: 2026/09/21 09:29:07 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5236lb # [ 100.006356] niks3-server[1023]: 2026/09/21 09:29:07 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5237lb # [ 100.011431] niks3-server[1023]: 2026/09/21 09:29:07 INFO Completed upload id=315238worker2 # [ 100.015344] nix-grpc-daemon[1183]: time=2026-09-21T09:29:07.643Z level=INFO msg="Upload complete. (476ms)"5239lb # [ 100.014311] niks3-server[1023]: 2026/09/21 09:29:07 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5240worker2 # [ 100.023667] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57292 duration_s=0 paths=1 nar_bytes_in=7685241worker2 # [ 100.037490] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572925242worker2 # [ 100.043403] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5243worker2 # [ 100.048319] nix-daemon[861]: reaped child process 1295, status = succeeded5244client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5245client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5246worker2 # [ 100.062096] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5247client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5248client # farm-top> LOG-farm-top-on-standby5249worker2 # [ 100.704380] nix-daemon[861]: reaped child process 1309, status = succeeded5250lb # [ 100.856935] envoy[913]: conn peer=[2001:db8:1::3]:35310 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1433 tx=485 ms=31945251worker2 # [ 100.877534] nix-daemon[861]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5252worker2 # [ 100.895316] nix-daemon[861]: reaped child process 1328, status = succeeded5253lb # [ 100.903684] niks3-server[1023]: 2026/09/21 09:29:08 INFO Received uploads request method=POST path=/api/pending_closures5254worker2 # [ 100.919758] nix-grpc-daemon[1183]: time=2026-09-21T09:29:08.545Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5255worker2 # [ 100.921627] nix-grpc-daemon[1183]: time=2026-09-21T09:29:08.545Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5256lb # [ 100.938171] niks3-server[1023]: 2026/09/21 09:29:08 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5257lb # [ 100.951816] niks3-server[1023]: 2026/09/21 09:29:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5258worker2 # [ 100.962021] nix-grpc-daemon[1183]: time=2026-09-21T09:29:08.589Z level=INFO msg="Uploading 1 narinfos"5259lb # [ 100.961920] niks3-server[1023]: 2026/09/21 09:29:08 INFO Signed narinfos id=32 count=15260lb # [ 100.967591] niks3-server[1023]: 2026/09/21 09:29:08 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5261lb # [ 100.972440] niks3-server[1023]: 2026/09/21 09:29:08 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5262worker2 # [ 100.983298] nix-grpc-daemon[1183]: time=2026-09-21T09:29:08.610Z level=INFO msg="Upload complete. (311ms)"5263lb # [ 100.979790] niks3-server[1023]: 2026/09/21 09:29:08 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5264lb # [ 100.983673] niks3-server[1023]: 2026/09/21 09:29:08 INFO Completed upload id=325265lb # [ 100.986511] niks3-server[1023]: 2026/09/21 09:29:08 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5266worker2 # [ 100.992971] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57292 duration_s=0 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15267worker2 # [ 101.009174] nix-daemon[861]: reaped child process 1307, status = succeeded5268client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 6.40 seconds)5269client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5270client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.24 seconds)5271worker1 # [ 101.385043] systemd[1]: Listening on Nix worker-protocol over gRPC.5272worker1 # [ 101.394411] systemd[1]: Starting Nix worker-protocol over gRPC...5273worker1 # [ 101.589645] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5274worker1 # [ 101.592240] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5275worker1 # [ 101.596374] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15276worker1 # [ 101.604068] nix-grpc-daemon[1284]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5277worker1 # [ 101.605572] nix-grpc-daemon[1284]: E0000 00:00:1789982949.235634 1284 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5278worker1 # [ 101.615716] nix-grpc-daemon[1284]: E0000 00:00:1789982949.241849 1284 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5279worker1 # [ 101.620362] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5280worker1 # [ 101.622589] systemd[1]: Started Nix worker-protocol over gRPC.5281worker1 # [ 101.628171] nix-grpc-daemon[1284]: E0000 00:00:1789982949.257876 1324 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5282lb: must succeed: curl -sf localhost:9901/clusters5283worker1 # [ 101.656125] nix-grpc-daemon[1284]: E0000 00:00:1789982949.276208 1324 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5284worker1 # [ 101.658423] nix-grpc-daemon[1284]: E0000 00:00:1789982949.276368 1324 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5285worker1 # [ 101.664896] nix-grpc-daemon[1284]: E0000 00:00:1789982949.295855 1324 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5286worker1 # [ 101.679406] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:09Z level=info event=scheduler_connected addr=lb:500515287worker2 # [ 101.690756] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:601925288worker2 # [ 101.691219] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05289lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5290worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5291worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5292lb: must succeed: curl -sf localhost:9901/clusters5293lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5294lb: must succeed: curl -sf localhost:9901/clusters5295lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5296worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5297worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5298(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.49 seconds)5299subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5300client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5301client # [ 102.074391] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5302client # Running as unit: rs.service; invocation ID: d88dc99db4764170bbf552daed435a795303client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.09 seconds)5304client # [ 102.322981] nix[1185]: don't know how to build these paths:5305client # [ 102.324122] nix[1185]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5306worker2 # [ 102.338968] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:572925307worker2 # [ 102.352288] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:09Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05308worker2 # [ 102.358169] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:09Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=15309worker2 # [ 102.403464] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572925310worker2 # [ 102.419056] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:572925311client # [ 102.417452] nix[1185]: copying 1 paths to 'lb:50051'...5312client # [ 102.421248] nix[1185]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5313worker2 # [ 102.438549] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:572925314worker2 # [ 102.455426] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5315worker2 # [ 102.654978] nix-daemon[861]: accepted connection from pid 1341, user nix-grpc-daemon (trusted)5316worker2 # [ 102.674254] nix-daemon[861]: reaped child process 1344, status = succeeded5317worker2 # [ 102.843998] nix-daemon[861]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5318worker2 # [ 102.861730] nix-daemon[861]: reaped child process 1349, status = succeeded5319lb # [ 102.871650] niks3-server[1023]: 2026/09/21 09:29:10 INFO Received uploads request method=POST path=/api/pending_closures5320worker2 # [ 102.886372] nix-grpc-daemon[1183]: time=2026-09-21T09:29:10.513Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5321worker2 # [ 102.887814] nix-grpc-daemon[1183]: time=2026-09-21T09:29:10.513Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5322lb # [ 102.898408] niks3-server[1023]: 2026/09/21 09:29:10 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5323lb # [ 102.913563] niks3-server[1023]: 2026/09/21 09:29:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5324worker2 # [ 102.923234] nix-grpc-daemon[1183]: time=2026-09-21T09:29:10.550Z level=INFO msg="Uploading 1 narinfos"5325lb # [ 102.925839] niks3-server[1023]: 2026/09/21 09:29:10 INFO Signed narinfos id=33 count=15326lb # [ 102.932676] niks3-server[1023]: 2026/09/21 09:29:10 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5327lb # [ 102.942197] niks3-server[1023]: 2026/09/21 09:29:10 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5328lb # [ 102.947755] niks3-server[1023]: 2026/09/21 09:29:10 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5329lb # [ 102.952054] niks3-server[1023]: 2026/09/21 09:29:10 INFO Completed upload id=335330worker2 # [ 102.955637] nix-grpc-daemon[1183]: time=2026-09-21T09:29:10.583Z level=INFO msg="Upload complete. (482ms)"5331worker2 # [ 102.963759] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57292 duration_s=0 paths=1 nar_bytes_in=4165332worker2 # [ 102.978231] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:572925333worker2 # [ 102.984942] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5334worker2 # [ 102.989906] nix-daemon[861]: reaped child process 1339, status = succeeded5335client # [ 102.983108] nix[1185]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5336worker2 # [ 103.003507] nix-daemon[861]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)5337lb: must succeed: curl -sf localhost:9901/clusters5338lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5339lb: must succeed: curl -sf localhost:9901/clusters5340lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5341lb: must succeed: curl -sf localhost:9901/clusters5342lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5343lb: must succeed: curl -sf localhost:9901/clusters5344lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5345worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5346worker1 # [ 105.020880] systemd-journald[275]: Received client request to rotate journal, rotating.5347worker1 # [ 105.040415] systemd-journald[275]: Vacuuming done, freed 0B of archived journals from /var/log/journal/fc8e72aa0fd0405d859837f3842fb8c2.5348worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5349worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5350worker2 # [ 105.105039] systemd[1]: Stopping Nix worker-protocol over gRPC...5351worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5352client: must succeed: sleep 5.05353worker2 # [ 105.398222] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:13Z level=info event=draining5354client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5355worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5356worker2 # [ 110.654680] nix-daemon[861]: reaped child process 1353, status = succeeded5357worker2 # [ 110.831395] nix-daemon[861]: accepted connection from pid 1394, user nix-grpc-daemon (trusted)5358worker2 # [ 110.846750] nix-daemon[861]: reaped child process 1397, status = succeeded5359lb # [ 110.856782] envoy[913]: conn peer=[2001:db8:1::1]:37180 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9606 tx=4058 ms=61005360lb # [ 110.863571] niks3-server[1023]: 2026/09/21 09:29:18 INFO Received uploads request method=POST path=/api/pending_closures5361worker2 # [ 110.880471] nix-grpc-daemon[1183]: time=2026-09-21T09:29:18.505Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5362worker2 # [ 110.882375] nix-grpc-daemon[1183]: time=2026-09-21T09:29:18.505Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5363lb # [ 110.906694] niks3-server[1023]: 2026/09/21 09:29:18 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5364lb # [ 110.924738] niks3-server[1023]: 2026/09/21 09:29:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5365lb # [ 110.934881] niks3-server[1023]: 2026/09/21 09:29:18 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5366lb # [ 110.941274] niks3-server[1023]: 2026/09/21 09:29:18 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5367worker2 # [ 110.945594] nix-grpc-daemon[1183]: time=2026-09-21T09:29:18.572Z level=INFO msg="Uploading 1 narinfos"5368lb # [ 110.946963] niks3-server[1023]: 2026/09/21 09:29:18 INFO Signed narinfos id=34 count=15369lb # [ 110.953773] niks3-server[1023]: 2026/09/21 09:29:18 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5370lb # [ 110.964516] niks3-server[1023]: 2026/09/21 09:29:18 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5371lb # [ 110.974849] niks3-server[1023]: 2026/09/21 09:29:18 INFO Completed upload id=345372worker2 # [ 110.978842] nix-grpc-daemon[1183]: time=2026-09-21T09:29:18.606Z level=INFO msg="Upload complete. (359ms)"5373worker2 # [ 110.993234] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57292 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15374worker2 # [ 111.011471] nix-daemon[861]: reaped child process 1351, status = succeeded5375client # [ 111.016283] systemd[1]: rs.service: Deactivated successfully.5376client # [ 111.020571] systemd[1]: rs.service: Consumed 175ms CPU time over 8.945s wall clock time, 41.2M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.5377worker2 # [ 111.399661] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:19Z level=info event=signal_exit5378worker2 # [ 111.623456] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:19Z level=info event=scheduler_yield5379worker1 # [ 111.623923] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:19Z level=info event=scheduler_restarting addr=lb:500515380worker2 # [ 111.632936] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:19Z level=info event=scheduler_restarting addr=lb:500515381worker1 # [ 111.627464] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:19Z level=info event=scheduler_take_over5382lb # [ 111.631307] niks3-server[1023]: 2026/09/21 09:29:19 INFO lead: released remote=192.168.1.4:344765383lb # [ 111.632386] niks3-server[1023]: 2026/09/21 09:29:19 INFO lead: acquired remote=192.168.1.3:382365384worker2 # [ 111.639175] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:601925385worker2 # [ 111.644321] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:19Z level=info event=worker_gone addr=192.168.1.3:500515386worker2 # [ 111.649157] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:601925387worker2 # [ 111.653513] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:19Z level=info event=worker_gone addr=192.168.1.4:500515388worker2 # [ 111.834172] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:19Z level=info event=scheduler_connected addr=lb:500515389worker1 # [ 111.824431] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:19Z level=info event=scheduler_connected addr=lb:500515390worker1 # [ 111.829416] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5391worker2 # [ 111.839075] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5392worker1 # [ 112.029909] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:19Z level=info event=scheduler_connected addr=lb:500515393worker2 # [ 112.041899] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:19Z level=info event=scheduler_connected addr=lb:500515394worker1 # [ 112.033420] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5395worker2 # [ 112.046083] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5396worker2 # [ 112.242755] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:19Z level=info event=scheduler_connected addr=lb:500515397worker1 # [ 112.234378] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:19Z level=info event=scheduler_connected addr=lb:500515398worker2 # [ 112.248103] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5399worker1 # [ 112.240101] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5400worker1 # [ 112.440782] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:20Z level=info event=scheduler_connected addr=lb:500515401worker2 # [ 112.450447] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:20Z level=info event=scheduler_connected addr=lb:500515402worker2 # [ 112.455420] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5403worker1 # [ 112.446345] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5404worker2 # [ 112.629397] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:20Z level=info event=scheduler_restarting5405worker1 # [ 112.649461] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:20Z level=info event=scheduler_connected addr=lb:500515406worker1 # [ 112.655966] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5407worker2 # [ 112.674035] nix-grpc-daemon[1119]: ts=2026-09-21T09:29:20Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5408worker2 # [ 112.690026] nix-daemon[861]: reaped child process 1175, status = succeeded5409worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.24 seconds)5410client: waiting for success: ! systemctl is-active rs5411client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5412client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5413client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5414client: must fail: journalctl -u rs | grep -q 'reconnecting'5415worker1 # [ 112.849509] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:20Z level=info event=scheduler_connected addr=lb:500515416client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5417lb: must succeed: curl -sf localhost:9901/clusters5418worker1 # [ 112.854681] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5419lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5420worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5421worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5422worker1 # [ 113.055517] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:20Z level=info event=scheduler_connected addr=lb:500515423worker1 # [ 113.060891] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5424worker1 # [ 113.261685] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:20Z level=info event=scheduler_connected addr=lb:500515425worker1 # [ 113.266905] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5426worker2 # [ 113.451991] nix-grpc-daemon[1119]: E0000 00:00:1789982961.078359 1164 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5427worker2 # [ 113.452376] nix-grpc-daemon[1119]: E0000 00:00:1789982961.078643 1164 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5428worker1 # [ 113.467567] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:21Z level=info event=scheduler_connected addr=lb:500515429worker1 # [ 113.476530] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:21Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:428845430worker1 # [ 113.482473] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:21Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05431worker2 # [ 113.870782] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5432worker2 # [ 113.877214] systemd[1]: Stopped Nix worker-protocol over gRPC.5433worker2 # [ 113.879718] systemd[1]: nix-grpc-daemon.service: Consumed 2.054s CPU time over 30.686s wall clock time, 36.8M memory peak, 62.9K incoming IP traffic, 162.7K outgoing IP traffic.5434worker2 # [ 113.896899] systemd[1]: Starting scheduler-token.service...5435worker2 # [ 113.993535] systemd[1]: scheduler-token.service: Deactivated successfully.5436worker2 # [ 113.995800] systemd[1]: Finished scheduler-token.service.5437worker2 # [ 114.000950] systemd[1]: scheduler-token.service: Consumed 42ms CPU time over 99ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5438worker2 # [ 114.005540] systemd[1]: Starting Nix worker-protocol over gRPC...5439lb: must succeed: curl -sf localhost:9901/clusters5440lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5441worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5442worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5443worker2 # [ 114.188230] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:21Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5444worker2 # [ 114.190900] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:21Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5445worker2 # [ 114.195023] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:21Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15446worker2 # [ 114.202438] nix-grpc-daemon[1407]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5447worker2 # [ 114.203916] nix-grpc-daemon[1407]: E0000 00:00:1789982961.829404 1407 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5448worker2 # [ 114.211117] nix-grpc-daemon[1407]: E0000 00:00:1789982961.835412 1407 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5449worker2 # [ 114.214861] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:21Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5450worker2 # [ 114.217557] systemd[1]: Started Nix worker-protocol over gRPC.5451worker2 # [ 114.224063] nix-grpc-daemon[1407]: E0000 00:00:1789982961.848973 1447 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5452worker2 # [ 114.252144] nix-grpc-daemon[1407]: E0000 00:00:1789982961.866561 1447 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5453worker2 # [ 114.256679] nix-grpc-daemon[1407]: E0000 00:00:1789982961.866712 1447 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5454worker2 # [ 114.259523] nix-grpc-daemon[1407]: E0000 00:00:1789982961.883607 1447 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5455worker2 # [ 114.269418] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:21Z level=info event=scheduler_connected addr=lb:500515456worker1 # [ 114.269249] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:21Z level=info event=oidc_jwks provider=mock keys=15457worker1 # [ 114.274737] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:21Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:428845458worker1 # [ 114.280194] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:21Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05459lb: must succeed: curl -sf localhost:9901/clusters5460lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5461worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5462worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5463(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.32 seconds)5464subtest: scheduler restart mid-build: build finishes, no double build5465client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5466client # [ 115.395815] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5467client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)5468worker2 # [ 115.663856] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:538245469worker2 # [ 115.678341] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5470client # Running as unit: mid.service; invocation ID: 0c773a1a9c4b4b9b9299a8845ff365b25471client # [ 115.678935] nix[1221]: don't know how to build these paths:5472client # [ 115.682710] nix[1221]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5473worker1 # [ 115.701489] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:23Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05474worker1 # [ 115.701774] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:23Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15475worker1 # [ 115.759523] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428865476worker1 # [ 115.773650] nix-daemon[839]: accepted connection from pid 1284, user nix-grpc-daemon (trusted)5477worker1 # [ 115.798320] nix-grpc-daemon[1284]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 1/5)5478worker1 # [ 115.807812] nix-grpc-daemon[1284]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 2/5)5479worker1 # [ 115.853568] nix-grpc-daemon[1284]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 207 ms (attempt 3/5)5480worker1 # [ 116.067778] nix-grpc-daemon[1284]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 720 ms (attempt 4/5)5481worker1 # [ 116.789821] nix-grpc-daemon[1284]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5482worker1 # [ 116.790089] nix-grpc-daemon[1284]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5483worker1 # [ 116.815435] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428865484client # [ 116.825319] nix[1221]: copying 1 paths to 'lb:50051'...5485worker1 # [ 116.830609] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428865486client # [ 116.825639] nix[1221]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5487worker1 # [ 116.846137] nix-daemon[839]: accepted connection from pid 1284, user nix-grpc-daemon (trusted)5488worker1 # [ 116.874229] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:24Z level=info event=niks3_push_started pid=14135489worker1 # [ 117.085563] nix-grpc-daemon[1413]: time=2026-09-21T09:29:24.716Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5490worker1 # [ 117.257923] nix-daemon[839]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5491worker1 # [ 117.274861] nix-daemon[839]: reaped child process 1429, status = succeeded5492worker1 # [ 117.448742] nix-daemon[839]: accepted connection from pid 1431, user nix-grpc-daemon (trusted)5493worker1 # [ 117.466753] nix-daemon[839]: reaped child process 1434, status = succeeded5494lb # [ 117.484386] niks3-server[1023]: 2026/09/21 09:29:25 INFO Received uploads request method=POST path=/api/pending_closures5495worker1 # [ 117.488104] nix-grpc-daemon[1413]: time=2026-09-21T09:29:25.119Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5496worker1 # [ 117.489588] nix-grpc-daemon[1413]: time=2026-09-21T09:29:25.119Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5497lb # [ 117.520642] niks3-server[1023]: 2026/09/21 09:29:25 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5498worker1 # [ 117.528278] nix-grpc-daemon[1413]: time=2026-09-21T09:29:25.159Z level=INFO msg="Uploading 1 narinfos"5499lb # [ 117.533514] niks3-server[1023]: 2026/09/21 09:29:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5500lb # [ 117.545940] niks3-server[1023]: 2026/09/21 09:29:25 INFO Signed narinfos id=35 count=15501lb # [ 117.554615] niks3-server[1023]: 2026/09/21 09:29:25 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5502lb # [ 117.565020] niks3-server[1023]: 2026/09/21 09:29:25 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5503worker1 # [ 117.565742] nix-grpc-daemon[1413]: time=2026-09-21T09:29:25.197Z level=INFO msg="Upload complete. (476ms)"5504lb # [ 117.572278] niks3-server[1023]: 2026/09/21 09:29:25 INFO Completed upload id=355505lb # [ 117.576248] niks3-server[1023]: 2026/09/21 09:29:25 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5506worker1 # [ 117.572277] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42886 duration_s=0 paths=1 nar_bytes_in=4245507worker1 # [ 117.587511] nix-grpc-daemon[1284]: ts=2026-09-21T09:29:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428865508worker1 # [ 117.593744] nix-daemon[839]: accepted connection from pid 1284, user nix-grpc-daemon (trusted)5509worker1 # [ 117.598766] nix-daemon[839]: reaped child process 1411, status = succeeded5510worker1 # [ 117.609575] nix-daemon[839]: accepted connection from pid 1284, user nix-grpc-daemon (trusted)5511client # [ 117.601293] nix[1221]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5512lb: must succeed: curl -sf localhost:9901/clusters5513lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5514lb: must succeed: curl -sf localhost:9901/clusters5515lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5516worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5517worker1 # [ 119.415349] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1284 (nix-grpc-daemon) on client request.5518worker1 # [ 119.417018] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5519lb # [ 119.438718] niks3-server[1023]: 2026/09/21 09:29:27 INFO lead: released remote=192.168.1.3:382365520client # [ 119.439675] nix[1221]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5521worker2 # [ 119.455933] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:27Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5522worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.08 seconds)5523client: must succeed: sleep 5.05524client # [ 119.446010] nix[1221]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5525worker1 # [ 119.460530] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5526worker1 # [ 119.462089] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5527worker1 # [ 119.463321] systemd[1]: nix-grpc-daemon.service: Consumed 595ms CPU time over 18.065s wall clock time, 34.3M memory peak, 17.1K incoming IP traffic, 27.5K outgoing IP traffic.5528worker1 # [ 119.479021] nix-daemon[839]: reaped child process 1436, status = succeeded5529worker1 # [ 119.480659] nix-daemon[839]: reaped child process 1401, status = succeeded5530worker1 # [ 119.488608] nix-daemon[1438]: unexpected Nix daemon error: error: interrupted by the user5531worker1 # [ 119.496458] nix-daemon[839]: reaped child process 1438, status = failed with exit code 15532worker2 # [ 119.656467] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:27Z level=info event=scheduler_connected addr=lb:500515533worker1 # [ 119.649915] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5534worker1 # [ 119.658656] systemd[1]: Starting Nix worker-protocol over gRPC...5535worker1 # [ 119.859112] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:27Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5536worker1 # [ 119.861751] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5537worker1 # [ 119.865887] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15538worker1 # [ 119.873266] nix-grpc-daemon[1475]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5539worker1 # [ 119.874693] nix-grpc-daemon[1475]: E0000 00:00:1789982967.504436 1475 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5540worker1 # [ 119.883162] nix-grpc-daemon[1475]: E0000 00:00:1789982967.510283 1475 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5541worker1 # [ 119.889375] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:27Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5542worker1 # [ 119.892142] systemd[1]: Started Nix worker-protocol over gRPC.5543lb # [ 119.896163] niks3-server[1023]: 2026/09/21 09:29:27 INFO lead: acquired remote=192.168.1.3:497945544worker1 # [ 119.895056] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:27Z level=info event=scheduler_take_over5545worker2 # [ 119.905171] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5546worker1 # [ 119.900069] nix-grpc-daemon[1475]: E0000 00:00:1789982967.530633 1515 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5547worker1 # [ 119.928053] nix-grpc-daemon[1475]: E0000 00:00:1789982967.548480 1515 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5548worker1 # [ 119.930325] nix-grpc-daemon[1475]: E0000 00:00:1789982967.548632 1515 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5549worker1 # [ 119.936722] nix-grpc-daemon[1475]: E0000 00:00:1789982967.567961 1515 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5550worker1 # [ 119.948544] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:27Z level=info event=scheduler_connected addr=lb:500515551worker1 # [ 119.951048] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:27Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:428945552worker1 # [ 119.953022] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05553worker1 # [ 119.958133] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:27Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05554worker1 # [ 119.960494] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:27Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15555worker1 # [ 120.015851] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429085556worker1 # [ 120.024315] nix-daemon[839]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5557worker1 # [ 120.037159] nix-daemon[839]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5558client # [ 120.038743] nix[1221]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5559worker1 # [ 120.048113] nix-daemon[839]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5560worker2 # [ 120.106517] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:27Z level=info event=scheduler_connected addr=lb:500515561worker1 # [ 120.104800] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:27Z level=info event=oidc_jwks provider=mock keys=15562worker1 # [ 120.109690] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:27Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:428945563worker1 # [ 120.114129] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:27Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05564lb # [ 120.856936] envoy[913]: conn peer=[2001:db8:1::1]:50748 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1430 ms=86655565lb # [ 120.861831] envoy[913]: conn peer=[2001:db8:1::4]:34572 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=36957 tx=2136 ms=292145566lb # [ 120.866907] envoy[913]: conn peer=[2001:db8:1::3]:53010 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1746 tx=595 ms=177765567client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5568client: waiting for success: ! systemctl is-active mid5569worker1 # [ 124.869886] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:32Z level=info event=niks3_push_started pid=15585570worker1 # [ 124.914070] nix-daemon[839]: reaped child process 1533, status = succeeded5571worker1 # [ 125.097440] nix-grpc-daemon[1558]: time=2026-09-21T09:29:32.729Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5572worker1 # [ 125.270335] nix-daemon[839]: accepted connection from pid 1571, user nix-grpc-daemon (trusted)5573worker1 # [ 125.286810] nix-daemon[839]: reaped child process 1574, status = succeeded5574lb # [ 125.305774] niks3-server[1023]: 2026/09/21 09:29:32 INFO Received uploads request method=POST path=/api/pending_closures5575worker1 # [ 125.311010] nix-grpc-daemon[1558]: time=2026-09-21T09:29:32.941Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5576worker1 # [ 125.313764] nix-grpc-daemon[1558]: time=2026-09-21T09:29:32.941Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5577lb # [ 125.348615] niks3-server[1023]: 2026/09/21 09:29:32 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5578lb # [ 125.366001] niks3-server[1023]: 2026/09/21 09:29:32 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5579worker1 # [ 125.368118] nix-grpc-daemon[1558]: time=2026-09-21T09:29:32.998Z level=INFO msg="Uploading 1 narinfos"5580lb # [ 125.377829] niks3-server[1023]: 2026/09/21 09:29:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5581lb # [ 125.393103] niks3-server[1023]: 2026/09/21 09:29:32 INFO Signed narinfos id=36 count=15582lb # [ 125.402385] niks3-server[1023]: 2026/09/21 09:29:33 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5583lb # [ 125.413700] niks3-server[1023]: 2026/09/21 09:29:33 INFO Completed upload id=365584worker1 # [ 125.406899] nix-grpc-daemon[1558]: time=2026-09-21T09:29:33.038Z level=INFO msg="Upload complete. (305ms)"5585worker1 # [ 125.421709] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42908 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15586worker1 # [ 125.440090] nix-grpc-daemon[1558]: time=2026-09-21T09:29:33.048Z level=WARN msg="Failed to register uploaded object" key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls error="server returned 404: object is not pending upload\n"5587worker1 # [ 125.445550] nix-grpc-daemon[1558]: time=2026-09-21T09:29:33.051Z level=WARN msg="Failed to register uploaded object" key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo error="server returned 404: object is not pending upload\n"5588worker1 # [ 125.450556] nix-daemon[839]: reaped child process 1531, status = succeeded5589client # [ 125.471257] systemd[1]: mid.service: Deactivated successfully.5590client # [ 125.476488] systemd[1]: mid.service: Consumed 184ms CPU time over 10.079s wall clock time, 39.2M memory peak, 8.5K incoming IP traffic, 10.4K outgoing IP traffic.5591client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5592client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5593client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5594(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.68 seconds)5595subtest: interrupting the client stops the build on the worker5596client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5597client # [ 126.045896] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5598client # Running as unit: intr.service; invocation ID: e876ef6278bb4cd48efa9c7b634583755599client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5600worker1 # [ 126.285979] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429085601client # [ 126.286720] nix[1257]: don't know how to build these paths:5602client # [ 126.288414] nix[1257]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5603worker1 # [ 126.306931] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:33Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05604worker1 # [ 126.312640] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:33Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15605worker2 # [ 126.365765] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538245606worker2 # [ 126.382844] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 1/5)5607worker2 # [ 126.472542] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 2/5)5608worker2 # [ 126.484723] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 3/5)5609worker2 # [ 126.572947] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 235 ms (attempt 4/5)5610worker2 # [ 126.816064] nix-grpc-daemon[1407]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5611worker2 # [ 126.819773] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5612worker2 # [ 126.840863] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538245613client # [ 126.839803] nix[1257]: copying 1 paths to 'lb:50051'...5614worker2 # [ 126.855926] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538245615client # [ 126.842394] nix[1257]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5616worker2 # [ 126.871588] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5617worker2 # [ 126.901184] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:34Z level=info event=niks3_push_started pid=14995618worker2 # [ 127.117616] nix-grpc-daemon[1499]: time=2026-09-21T09:29:34.744Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5619worker2 # [ 127.291719] nix-daemon[861]: accepted connection from pid 1512, user nix-grpc-daemon (trusted)5620worker2 # [ 127.307275] nix-daemon[861]: reaped child process 1515, status = succeeded5621worker2 # [ 127.497980] nix-daemon[861]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)5622worker2 # [ 127.514157] nix-daemon[861]: reaped child process 1524, status = succeeded5623lb # [ 127.522899] niks3-server[1023]: 2026/09/21 09:29:35 INFO Received uploads request method=POST path=/api/pending_closures5624worker2 # [ 127.541210] nix-grpc-daemon[1499]: time=2026-09-21T09:29:35.167Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5625worker2 # [ 127.542706] nix-grpc-daemon[1499]: time=2026-09-21T09:29:35.167Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5626lb # [ 127.572079] niks3-server[1023]: 2026/09/21 09:29:35 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5627lb # [ 127.580272] niks3-server[1023]: 2026/09/21 09:29:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5628worker2 # [ 127.592619] nix-grpc-daemon[1499]: time=2026-09-21T09:29:35.219Z level=INFO msg="Uploading 1 narinfos"5629lb # [ 127.592941] niks3-server[1023]: 2026/09/21 09:29:35 INFO Signed narinfos id=37 count=15630lb # [ 127.614608] niks3-server[1023]: 2026/09/21 09:29:35 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5631lb # [ 127.619477] niks3-server[1023]: 2026/09/21 09:29:35 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5632lb # [ 127.624872] niks3-server[1023]: 2026/09/21 09:29:35 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5633lb # [ 127.628204] niks3-server[1023]: 2026/09/21 09:29:35 INFO Completed upload id=375634worker2 # [ 127.632182] nix-grpc-daemon[1499]: time=2026-09-21T09:29:35.259Z level=INFO msg="Upload complete. (510ms)"5635worker2 # [ 127.645022] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53824 duration_s=0 paths=1 nar_bytes_in=4245636worker2 # [ 127.660219] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538245637worker2 # [ 127.669117] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5638worker2 # [ 127.672686] nix-daemon[861]: reaped child process 1497, status = succeeded5639client # [ 127.665811] nix[1257]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5640worker2 # [ 127.684528] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5641client: must succeed: systemctl kill -s INT intr5642client # [ 128.575511] systemd[1]: intr.service: Sent signal SIGINT to main process 1257 (nix) on client request.5643worker2 # [ 128.594105] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:36Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5644client # [ 128.581436] nix[1257]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5645client # [ 128.587570] nix[1257]: error: interrupted by the user5646client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)5647client # [ 128.633773] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5648client # [ 128.638721] systemd[1]: intr.service: Failed with result 'exit-code'.5649client # [ 128.641032] systemd[1]: intr.service: Consumed 157ms CPU time over 2.590s wall clock time, 41.4M memory peak, 6.3K incoming IP traffic, 8.1K outgoing IP traffic.5650worker2 # [ 128.722983] nix-daemon[861]: reaped child process 1526, status = succeeded5651worker2 # [ 128.725263] nix-daemon[1528]: unexpected Nix daemon error: error: interrupted by the user5652worker2 # [ 128.735686] nix-daemon[861]: reaped child process 1528, status = failed with exit code 15653(finished: subtest: interrupting the client stops the build on the worker, in 2.74 seconds)5654subtest: a deploy mid-build drains: the build finishes, the new generation takes over5655client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5656client # [ 128.806856] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5657client # Running as unit: sw.service; invocation ID: 49547860f51c4250904439e25c3fb2f85658client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5659worker1 # [ 129.066241] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429085660client # [ 129.066626] nix[1277]: don't know how to build these paths:5661client # [ 129.068283] nix[1277]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5662worker1 # [ 129.084283] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:36Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05663worker1 # [ 129.090148] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:36Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15664worker1 # [ 129.134311] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429085665worker1 # [ 129.150781] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 1/5)5666worker1 # [ 129.217621] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 2/5)5667worker1 # [ 129.269112] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 215 ms (attempt 3/5)5668worker1 # [ 129.491812] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 329 ms (attempt 4/5)5669worker1 # [ 129.829059] nix-grpc-daemon[1475]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5670worker1 # [ 129.834471] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5671worker1 # [ 129.855125] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429085672client # [ 129.864181] nix[1277]: copying 1 paths to 'lb:50051'...5673worker1 # [ 129.870383] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429085674client # [ 129.864541] nix[1277]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5675worker1 # [ 129.888403] nix-daemon[839]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5676worker1 # [ 130.104064] nix-daemon[839]: accepted connection from pid 1606, user nix-grpc-daemon (trusted)5677worker1 # [ 130.120584] nix-daemon[839]: reaped child process 1613, status = succeeded5678worker1 # [ 130.297095] nix-daemon[839]: accepted connection from pid 1615, user nix-grpc-daemon (trusted)5679worker1 # [ 130.313637] nix-daemon[839]: reaped child process 1618, status = succeeded5680lb # [ 130.333043] niks3-server[1023]: 2026/09/21 09:29:37 INFO Received uploads request method=POST path=/api/pending_closures5681worker1 # [ 130.339206] nix-grpc-daemon[1558]: time=2026-09-21T09:29:37.970Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5682worker1 # [ 130.340888] nix-grpc-daemon[1558]: time=2026-09-21T09:29:37.970Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5683lb # [ 130.363943] niks3-server[1023]: 2026/09/21 09:29:37 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5684worker1 # [ 130.371970] nix-grpc-daemon[1558]: time=2026-09-21T09:29:38.003Z level=INFO msg="Uploading 1 narinfos"5685lb # [ 130.378591] niks3-server[1023]: 2026/09/21 09:29:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5686lb # [ 130.389740] niks3-server[1023]: 2026/09/21 09:29:37 INFO Signed narinfos id=38 count=15687worker1 # [ 130.394413] nix-grpc-daemon[1558]: time=2026-09-21T09:29:38.026Z level=INFO msg="Upload complete. (487ms)"5688lb # [ 130.400671] niks3-server[1023]: 2026/09/21 09:29:38 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5689lb # [ 130.405462] niks3-server[1023]: 2026/09/21 09:29:38 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5690lb # [ 130.410034] niks3-server[1023]: 2026/09/21 09:29:38 INFO Completed upload id=385691worker1 # [ 130.405469] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42908 duration_s=0 paths=1 nar_bytes_in=4165692lb # [ 130.412374] niks3-server[1023]: 2026/09/21 09:29:38 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5693worker1 # [ 130.412857] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429085694worker1 # [ 130.420871] nix-daemon[839]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5695worker1 # [ 130.425162] nix-daemon[839]: reaped child process 1604, status = succeeded5696client # [ 130.428871] nix[1277]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5697worker1 # [ 130.437840] nix-daemon[839]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5698lb # [ 130.860327] envoy[913]: conn peer=[2001:db8:1::1]:44028 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3940 tx=1851 ms=98005699lb # [ 130.865245] envoy[913]: conn peer=[2001:db8:1::1]:57130 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2944 tx=835 ms=23295700worker1: must succeed: systemctl reload nix-grpc-daemon.service5701worker1 # [ 132.650751] systemd[1]: Reloading Nix worker-protocol over gRPC...5702worker1 # [ 132.705301] systemd[1]: Reloaded Nix worker-protocol over gRPC.5703worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.11 seconds)5704worker1: must succeed: pgrep -f 'read -t [9]0 x'5705worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5706client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25707worker1 # [ 132.890181] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:40Z level=info event=draining5708worker2 # [ 133.042666] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:538245709client # don't know how to build these paths:5710client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5711worker2 # [ 133.053233] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538245712worker1 # [ 133.067469] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:40Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05713worker1 # [ 133.069553] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:40Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=15714worker2 # [ 133.123815] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538245715worker2 # [ 133.139071] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538245716client # copying 1 paths to 'lb:50051'...5717client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5718worker2 # [ 133.155789] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538245719worker2 # [ 133.170968] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5720worker2 # [ 133.378320] nix-daemon[861]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)5721worker2 # [ 133.395613] nix-daemon[861]: reaped child process 1578, status = succeeded5722worker2 # [ 133.567244] nix-daemon[861]: accepted connection from pid 1580, user nix-grpc-daemon (trusted)5723worker2 # [ 133.583656] nix-daemon[861]: reaped child process 1583, status = succeeded5724lb # [ 133.594274] niks3-server[1023]: 2026/09/21 09:29:41 INFO Received uploads request method=POST path=/api/pending_closures5725worker2 # [ 133.609189] nix-grpc-daemon[1499]: time=2026-09-21T09:29:41.236Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5726worker2 # [ 133.610790] nix-grpc-daemon[1499]: time=2026-09-21T09:29:41.236Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5727lb # [ 133.628576] niks3-server[1023]: 2026/09/21 09:29:41 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5728worker2 # [ 133.644817] nix-grpc-daemon[1499]: time=2026-09-21T09:29:41.272Z level=INFO msg="Uploading 1 narinfos"5729lb # [ 133.645228] niks3-server[1023]: 2026/09/21 09:29:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5730lb # [ 133.655570] niks3-server[1023]: 2026/09/21 09:29:41 INFO Signed narinfos id=39 count=15731lb # [ 133.661993] niks3-server[1023]: 2026/09/21 09:29:41 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5732lb # [ 133.670106] niks3-server[1023]: 2026/09/21 09:29:41 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5733lb # [ 133.677380] niks3-server[1023]: 2026/09/21 09:29:41 INFO Completed upload id=395734worker2 # [ 133.680960] nix-grpc-daemon[1499]: time=2026-09-21T09:29:41.308Z level=INFO msg="Upload complete. (492ms)"5735lb # [ 133.680349] niks3-server[1023]: 2026/09/21 09:29:41 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5736worker2 # [ 133.688217] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53824 duration_s=0 paths=1 nar_bytes_in=5285737worker2 # [ 133.702818] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538245738worker2 # [ 133.709979] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5739worker2 # [ 133.714865] nix-daemon[861]: reaped child process 1573, status = succeeded5740client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5741client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5742worker2 # [ 133.726745] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5743client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5744client # farm-a> LOG-farm-a-during-drain5745worker2 # [ 134.772918] nix-daemon[861]: reaped child process 1587, status = succeeded5746worker2 # [ 134.946976] nix-daemon[861]: accepted connection from pid 1603, user nix-grpc-daemon (trusted)5747worker2 # [ 134.963501] nix-daemon[861]: reaped child process 1606, status = succeeded5748lb # [ 134.971896] niks3-server[1023]: 2026/09/21 09:29:42 INFO Received uploads request method=POST path=/api/pending_closures5749worker2 # [ 134.988140] nix-grpc-daemon[1499]: time=2026-09-21T09:29:42.612Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5750worker2 # [ 134.989637] nix-grpc-daemon[1499]: time=2026-09-21T09:29:42.612Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5751lb # [ 135.003761] niks3-server[1023]: 2026/09/21 09:29:42 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5752lb # [ 135.016309] niks3-server[1023]: 2026/09/21 09:29:42 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5753lb # [ 135.032827] niks3-server[1023]: 2026/09/21 09:29:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5754worker2 # [ 135.043518] nix-grpc-daemon[1499]: time=2026-09-21T09:29:42.670Z level=INFO msg="Uploading 1 narinfos"5755lb # [ 135.044951] niks3-server[1023]: 2026/09/21 09:29:42 INFO Signed narinfos id=40 count=15756lb # [ 135.051210] niks3-server[1023]: 2026/09/21 09:29:42 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5757lb # [ 135.061286] niks3-server[1023]: 2026/09/21 09:29:42 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5758worker2 # [ 135.071911] nix-grpc-daemon[1499]: time=2026-09-21T09:29:42.699Z level=INFO msg="Upload complete. (332ms)"5759lb # [ 135.071819] niks3-server[1023]: 2026/09/21 09:29:42 INFO Completed upload id=405760worker1 # [ 135.075186] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:42Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05761lb # [ 135.079999] niks3-server[1023]: 2026/09/21 09:29:42 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5762worker1 # [ 135.077136] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:42Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=15763worker2 # [ 135.085077] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53824 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=15764worker2 # [ 135.099931] nix-daemon[861]: reaped child process 1585, status = succeeded5765worker2 # [ 135.136613] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538245766worker2 # [ 135.151718] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538245767worker2 # [ 135.165945] nix-grpc-daemon[1499]: time=2026-09-21T09:29:42.793Z level=INFO msg="All 1 paths already cached"5768client # copying 1 paths to 'lb:50051'...5769client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5770worker2 # [ 135.176071] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538245771worker2 # [ 135.190076] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5772worker2 # [ 135.403407] nix-daemon[861]: accepted connection from pid 1610, user nix-grpc-daemon (trusted)5773worker2 # [ 135.419994] nix-daemon[861]: reaped child process 1613, status = succeeded5774worker2 # [ 135.594237] nix-daemon[861]: accepted connection from pid 1615, user nix-grpc-daemon (trusted)5775worker2 # [ 135.610033] nix-daemon[861]: reaped child process 1618, status = succeeded5776lb # [ 135.617705] niks3-server[1023]: 2026/09/21 09:29:43 INFO Received uploads request method=POST path=/api/pending_closures5777worker2 # [ 135.630837] nix-grpc-daemon[1499]: time=2026-09-21T09:29:43.258Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5778worker2 # [ 135.632451] nix-grpc-daemon[1499]: time=2026-09-21T09:29:43.258Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5779lb # [ 135.646299] niks3-server[1023]: 2026/09/21 09:29:43 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5780lb # [ 135.659857] niks3-server[1023]: 2026/09/21 09:29:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5781worker2 # [ 135.670371] nix-grpc-daemon[1499]: time=2026-09-21T09:29:43.297Z level=INFO msg="Uploading 1 narinfos"5782lb # [ 135.670679] niks3-server[1023]: 2026/09/21 09:29:43 INFO Signed narinfos id=41 count=15783lb # [ 135.677237] niks3-server[1023]: 2026/09/21 09:29:43 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5784lb # [ 135.684089] niks3-server[1023]: 2026/09/21 09:29:43 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5785lb # [ 135.690553] niks3-server[1023]: 2026/09/21 09:29:43 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5786lb # [ 135.694740] niks3-server[1023]: 2026/09/21 09:29:43 INFO Completed upload id=415787worker2 # [ 135.698681] nix-grpc-daemon[1499]: time=2026-09-21T09:29:43.326Z level=INFO msg="Upload complete. (492ms)"5788worker2 # [ 135.708395] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53824 duration_s=0 paths=1 nar_bytes_in=6405789worker2 # [ 135.719729] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538245790worker2 # [ 135.726495] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5791worker2 # [ 135.731166] nix-daemon[861]: reaped child process 1608, status = succeeded5792client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5793client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5794worker2 # [ 135.745098] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5795client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5796client # farm-b> LOG-farm-b-during-drain5797worker2 # [ 136.975641] nix-daemon[861]: reaped child process 1622, status = succeeded5798worker2 # [ 137.135221] nix-daemon[861]: accepted connection from pid 1638, user nix-grpc-daemon (trusted)5799worker2 # [ 137.152895] nix-daemon[861]: reaped child process 1641, status = succeeded5800lb # [ 137.162150] niks3-server[1023]: 2026/09/21 09:29:44 INFO Received uploads request method=POST path=/api/pending_closures5801worker2 # [ 137.175759] nix-grpc-daemon[1499]: time=2026-09-21T09:29:44.802Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5802worker2 # [ 137.177491] nix-grpc-daemon[1499]: time=2026-09-21T09:29:44.802Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5803lb # [ 137.197240] niks3-server[1023]: 2026/09/21 09:29:44 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5804lb # [ 137.211608] niks3-server[1023]: 2026/09/21 09:29:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5805worker2 # [ 137.220380] nix-grpc-daemon[1499]: time=2026-09-21T09:29:44.847Z level=INFO msg="Uploading 1 narinfos"5806lb # [ 137.222607] niks3-server[1023]: 2026/09/21 09:29:44 INFO Signed narinfos id=42 count=15807lb # [ 137.233357] niks3-server[1023]: 2026/09/21 09:29:44 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5808lb # [ 137.242108] niks3-server[1023]: 2026/09/21 09:29:44 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5809worker2 # [ 137.249981] nix-grpc-daemon[1499]: time=2026-09-21T09:29:44.877Z level=INFO msg="Upload complete. (295ms)"5810lb # [ 137.249804] niks3-server[1023]: 2026/09/21 09:29:44 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5811lb # [ 137.256088] niks3-server[1023]: 2026/09/21 09:29:44 INFO Completed upload id=425812lb # [ 137.258155] niks3-server[1023]: 2026/09/21 09:29:44 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5813worker2 # [ 137.261042] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53824 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=15814worker1 # [ 137.257292] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:44Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05815worker1 # [ 137.260159] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:44Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=15816worker2 # [ 137.276416] nix-daemon[861]: reaped child process 1620, status = succeeded5817worker2 # [ 137.311542] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538245818worker2 # [ 137.327453] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538245819worker2 # [ 137.340898] nix-grpc-daemon[1499]: time=2026-09-21T09:29:44.968Z level=INFO msg="All 2 paths already cached"5820client # copying 1 paths to 'lb:50051'...5821worker2 # [ 137.350828] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538245822client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5823worker2 # [ 137.376190] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5824worker2 # [ 137.580149] nix-daemon[861]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)5825worker2 # [ 137.596272] nix-daemon[861]: reaped child process 1648, status = succeeded5826worker2 # [ 137.771908] nix-daemon[861]: accepted connection from pid 1650, user nix-grpc-daemon (trusted)5827worker2 # [ 137.787268] nix-daemon[861]: reaped child process 1653, status = succeeded5828lb # [ 137.798206] niks3-server[1023]: 2026/09/21 09:29:45 INFO Received uploads request method=POST path=/api/pending_closures5829worker2 # [ 137.810829] nix-grpc-daemon[1499]: time=2026-09-21T09:29:45.438Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5830worker2 # [ 137.812348] nix-grpc-daemon[1499]: time=2026-09-21T09:29:45.438Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5831lb # [ 137.825878] niks3-server[1023]: 2026/09/21 09:29:45 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5832lb # [ 137.838186] niks3-server[1023]: 2026/09/21 09:29:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5833worker2 # [ 137.848438] nix-grpc-daemon[1499]: time=2026-09-21T09:29:45.475Z level=INFO msg="Uploading 1 narinfos"5834lb # [ 137.849530] niks3-server[1023]: 2026/09/21 09:29:45 INFO Signed narinfos id=43 count=15835lb # [ 137.855039] niks3-server[1023]: 2026/09/21 09:29:45 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5836lb # [ 137.863365] niks3-server[1023]: 2026/09/21 09:29:45 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5837worker2 # [ 137.872291] nix-grpc-daemon[1499]: time=2026-09-21T09:29:45.500Z level=INFO msg="Upload complete. (475ms)"5838lb # [ 137.867987] niks3-server[1023]: 2026/09/21 09:29:45 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5839lb # [ 137.871906] niks3-server[1023]: 2026/09/21 09:29:45 INFO Completed upload id=435840worker2 # [ 137.880125] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53824 duration_s=0 paths=1 nar_bytes_in=7685841worker2 # [ 137.890145] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538245842worker2 # [ 137.896933] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5843worker2 # [ 137.901569] nix-daemon[861]: reaped child process 1643, status = succeeded5844client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5845worker2 # [ 137.914134] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5846client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5847client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5848client # farm-top> LOG-farm-top-during-drain5849worker2 # [ 139.334846] nix-daemon[861]: reaped child process 1657, status = succeeded5850worker2 # [ 139.490816] nix-daemon[861]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)5851worker2 # [ 139.507035] nix-daemon[861]: reaped child process 1676, status = succeeded5852lb # [ 139.517931] niks3-server[1023]: 2026/09/21 09:29:47 INFO Received uploads request method=POST path=/api/pending_closures5853worker2 # [ 139.534485] nix-grpc-daemon[1499]: time=2026-09-21T09:29:47.160Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5854worker2 # [ 139.535936] nix-grpc-daemon[1499]: time=2026-09-21T09:29:47.160Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5855lb # [ 139.555922] niks3-server[1023]: 2026/09/21 09:29:47 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5856lb # [ 139.568504] niks3-server[1023]: 2026/09/21 09:29:47 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5857worker2 # [ 139.585922] nix-grpc-daemon[1499]: time=2026-09-21T09:29:47.213Z level=INFO msg="Uploading 1 narinfos"5858lb # [ 139.576984] niks3-server[1023]: 2026/09/21 09:29:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5859lb # [ 139.585622] niks3-server[1023]: 2026/09/21 09:29:47 INFO Signed narinfos id=44 count=15860lb # [ 139.587865] niks3-server[1023]: 2026/09/21 09:29:47 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5861lb # [ 139.597622] niks3-server[1023]: 2026/09/21 09:29:47 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5862lb # [ 139.604464] niks3-server[1023]: 2026/09/21 09:29:47 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5863lb # [ 139.608707] niks3-server[1023]: 2026/09/21 09:29:47 INFO Completed upload id=445864worker2 # [ 139.612444] nix-grpc-daemon[1499]: time=2026-09-21T09:29:47.240Z level=INFO msg="Upload complete. (288ms)"5865worker2 # [ 139.624940] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53824 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=15866worker2 # [ 139.637711] nix-daemon[861]: reaped child process 1655, status = succeeded5867client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.89 seconds)5868client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5869client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)5870worker1: must succeed: pkill -f 'read -t [9]0 x'5871worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5872client: waiting for success: ! systemctl is-active sw5873worker1 # [ 140.073962] nix-daemon[839]: reaped child process 1622, status = succeeded5874worker1 # [ 140.243824] nix-daemon[839]: accepted connection from pid 1669, user nix-grpc-daemon (trusted)5875worker1 # [ 140.259298] nix-daemon[839]: reaped child process 1672, status = succeeded5876lb # [ 140.279675] niks3-server[1023]: 2026/09/21 09:29:47 INFO Received uploads request method=POST path=/api/pending_closures5877worker1 # [ 140.283929] nix-grpc-daemon[1558]: time=2026-09-21T09:29:47.915Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5878worker1 # [ 140.287182] nix-grpc-daemon[1558]: time=2026-09-21T09:29:47.918Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5879lb # [ 140.319549] niks3-server[1023]: 2026/09/21 09:29:47 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5880lb # [ 140.331226] niks3-server[1023]: 2026/09/21 09:29:47 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5881worker1 # [ 140.333346] nix-grpc-daemon[1558]: time=2026-09-21T09:29:47.964Z level=INFO msg="Uploading 1 narinfos"5882lb # [ 140.341066] niks3-server[1023]: 2026/09/21 09:29:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5883lb # [ 140.349705] niks3-server[1023]: 2026/09/21 09:29:47 INFO Signed narinfos id=45 count=15884lb # [ 140.360676] niks3-server[1023]: 2026/09/21 09:29:47 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5885worker1 # [ 140.354551] nix-grpc-daemon[1558]: time=2026-09-21T09:29:47.986Z level=INFO msg="Upload complete. (300ms)"5886lb # [ 140.364792] niks3-server[1023]: 2026/09/21 09:29:47 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5887lb # [ 140.370075] niks3-server[1023]: 2026/09/21 09:29:47 INFO Completed upload id=455888lb # [ 140.373174] niks3-server[1023]: 2026/09/21 09:29:47 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5889worker1 # [ 140.368793] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42908 duration_s=9 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=15890worker1 # [ 140.384778] nix-daemon[839]: reaped child process 1620, status = succeeded5891client # [ 140.406672] systemd[1]: sw.service: Deactivated successfully.5892client # [ 140.411969] systemd[1]: sw.service: Consumed 177ms CPU time over 11.602s wall clock time, 39.2M memory peak, 7.5K incoming IP traffic, 8.5K outgoing IP traffic.5893lb # [ 140.858271] envoy[913]: conn peer=[2001:db8:1::1]:50176 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9651 tx=4135 ms=65925894lb # [ 140.863513] envoy[913]: conn peer=[2001:db8:1::1]:57136 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2982 tx=1447 ms=113195895worker1 # [ 140.892275] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:48Z level=info event=signal_exit5896client: (finished: waiting for success: ! systemctl is-active sw, in 1.13 seconds)5897client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5898worker1 # [ 141.113104] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:48Z level=info event=scheduler_yield5899lb # [ 141.121374] niks3-server[1023]: 2026/09/21 09:29:48 INFO lead: released remote=192.168.1.3:497945900worker2 # [ 141.131405] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:48Z level=info event=scheduler_restarting addr=lb:500515901worker1 # [ 141.126995] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:428945902worker1 # [ 141.133514] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:48Z level=info event=worker_gone addr=192.168.1.4:500515903worker1 # [ 141.137684] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:48Z level=info event=scheduler_restarting addr=lb:500515904worker1 # [ 141.141678] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:48Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:428945905client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)5906worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5907worker1 # [ 141.146072] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:48Z level=info event=worker_gone addr=192.168.1.3:500515908worker2 # [ 141.331556] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:48Z level=info event=scheduler_connected addr=lb:500515909worker2 # [ 141.337340] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5910worker1 # [ 141.333977] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:48Z level=info event=scheduler_connected addr=lb:500515911worker1 # [ 141.340876] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5912worker2 # [ 141.537861] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:49Z level=info event=scheduler_connected addr=lb:500515913worker2 # [ 141.542952] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5914worker1 # [ 141.543561] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:49Z level=info event=scheduler_connected addr=lb:500515915worker1 # [ 141.547736] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5916worker2 # [ 141.744056] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:49Z level=info event=scheduler_connected addr=lb:500515917worker2 # [ 141.749057] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5918worker1 # [ 141.744070] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:49Z level=info event=scheduler_connected addr=lb:500515919worker1 # [ 141.749090] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5920worker2 # [ 141.950181] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:49Z level=info event=scheduler_connected addr=lb:500515921worker2 # [ 141.955224] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5922worker1 # [ 141.949587] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:49Z level=info event=scheduler_connected addr=lb:500515923worker1 # [ 141.954656] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5924worker1 # [ 142.114181] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:49Z level=info event=scheduler_restarting5925worker1 # [ 142.144409] nix-daemon[839]: reaped child process 1529, status = succeeded5926worker2 # [ 142.158056] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:49Z level=info event=scheduler_connected addr=lb:500515927worker2 # [ 142.161956] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5928worker1 # [ 142.175280] nix-grpc-daemon[1475]: ts=2026-09-21T09:29:49Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5929worker2 # [ 142.358927] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:49Z level=info event=scheduler_connected addr=lb:500515930worker2 # [ 142.363781] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5931worker1 # [ 142.429215] nix-grpc-daemon[1475]: E0000 00:00:1789982990.060978 1520 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5932worker1 # [ 142.450287] nix-grpc-daemon[1475]: E0000 00:00:1789982990.082197 1520 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5933worker2 # [ 142.564765] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:50Z level=info event=scheduler_connected addr=lb:500515934worker2 # [ 142.569508] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5935worker2 # [ 142.770448] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:50Z level=info event=scheduler_connected addr=lb:500515936worker2 # [ 142.775491] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5937worker2 # [ 142.976325] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:50Z level=info event=scheduler_connected addr=lb:500515938worker2 # [ 142.981189] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5939worker2 # [ 143.184139] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:50Z level=info event=scheduler_connected addr=lb:500515940worker2 # [ 143.187978] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5941worker2 # [ 143.384970] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:51Z level=info event=scheduler_connected addr=lb:500515942worker2 # [ 143.389791] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5943worker2 # [ 143.590828] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:51Z level=info event=scheduler_connected addr=lb:500515944worker2 # [ 143.595949] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5945worker2 # [ 143.797078] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:51Z level=info event=scheduler_connected addr=lb:500515946worker2 # [ 143.802146] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5947worker1 # [ 143.934556] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5948worker1 # [ 143.940545] systemd[1]: nix-grpc-daemon.service: Consumed 893ms CPU time over 24.472s wall clock time, 35.1M memory peak, 26.2K incoming IP traffic, 33.1K outgoing IP traffic.5949worker1 # [ 143.959388] systemd[1]: Starting Nix worker-protocol over gRPC...5950worker2 # [ 144.003280] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:51Z level=info event=scheduler_connected addr=lb:500515951worker2 # [ 144.008761] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5952worker1 # [ 144.165482] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5953worker1 # [ 144.168241] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5954worker1 # [ 144.172572] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15955worker1 # [ 144.180063] nix-grpc-daemon[1686]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5956worker1 # [ 144.181535] nix-grpc-daemon[1686]: E0000 00:00:1789982991.811661 1686 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5957worker1 # [ 144.192444] nix-grpc-daemon[1686]: E0000 00:00:1789982991.818190 1686 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5958worker1 # [ 144.196214] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:51Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5959lb # [ 144.202299] niks3-server[1023]: 2026/09/21 09:29:51 INFO lead: acquired remote=192.168.1.3:575245960worker1 # [ 144.199311] systemd[1]: Started Nix worker-protocol over gRPC.5961worker1 # [ 144.200722] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:51Z level=info event=scheduler_take_over5962worker2 # [ 144.211669] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:51Z level=info event=scheduler_connected addr=lb:500515963worker1 # [ 144.208126] nix-grpc-daemon[1686]: E0000 00:00:1789982991.836408 1726 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5964worker2 # [ 144.215993] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5965worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.08 seconds)5966lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5967lb: must succeed: curl -sf localhost:9901/clusters5968worker1 # [ 144.236056] nix-grpc-daemon[1686]: E0000 00:00:1789982991.854389 1726 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5969worker1 # [ 144.241312] nix-grpc-daemon[1686]: E0000 00:00:1789982991.855297 1726 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5970worker1 # [ 144.244278] nix-grpc-daemon[1686]: E0000 00:00:1789982991.872316 1726 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5971worker1 # [ 144.260202] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:51Z level=info event=scheduler_connected addr=lb:500515972worker1 # [ 144.262972] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5973lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5974worker2 # [ 144.413262] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:52Z level=info event=scheduler_connected addr=lb:500515975worker2 # [ 144.419326] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5976worker1 # [ 144.464214] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:52Z level=info event=scheduler_connected addr=lb:500515977worker1 # [ 144.469713] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5978worker2 # [ 144.620514] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:52Z level=info event=scheduler_connected addr=lb:500515979worker2 # [ 144.625937] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5980worker1 # [ 144.670608] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:52Z level=info event=scheduler_connected addr=lb:500515981worker1 # [ 144.676179] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5982worker2 # [ 144.827108] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:52Z level=info event=scheduler_connected addr=lb:500515983worker2 # [ 144.832573] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5984worker1 # [ 144.877024] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:52Z level=info event=scheduler_connected addr=lb:500515985worker1 # [ 144.882386] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5986worker2 # [ 145.033682] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:52Z level=info event=scheduler_connected addr=lb:500515987worker1 # [ 145.036151] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:52Z level=info event=oidc_jwks provider=mock keys=15988worker1 # [ 145.041496] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:52Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:581225989worker1 # [ 145.046835] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:52Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05990worker1 # [ 145.083044] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:52Z level=info event=scheduler_connected addr=lb:500515991worker1 # [ 145.088497] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:52Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:581225992worker1 # [ 145.093303] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05993lb: must succeed: curl -sf localhost:9901/clusters5994lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5995lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.16 seconds)5996lb: must succeed: curl -sf localhost:9901/clusters5997lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5998worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5999worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6000(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 16.78 seconds)6001subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6002client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16003worker2 # [ 145.810669] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:538246004worker2 # [ 145.818737] nix-grpc-daemon[1407]: ts=2026-09-21T09:29:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538246005worker1 # [ 145.850336] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:53Z level=debug event=want client=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06006worker1 # [ 145.856745] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:53Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16007worker1 # [ 145.864324] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:53Z level=debug event=want client=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06008worker1 # [ 145.869550] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:53Z level=debug event=want client=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06009worker1 # [ 145.906802] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326010worker1 # [ 145.921835] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6011worker1 # [ 145.946030] nix-grpc-daemon[1686]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 91 ms (attempt 1/5)6012worker1 # [ 146.043108] nix-grpc-daemon[1686]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 2/5)6013worker1 # [ 146.055583] nix-grpc-daemon[1686]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 3/5)6014worker1 # [ 146.180329] nix-grpc-daemon[1686]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 4/5)6015worker1 # [ 146.286851] nix-grpc-daemon[1686]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6016worker1 # [ 146.292569] nix-grpc-daemon[1686]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6017worker1 # [ 146.310632] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581326018worker1 # [ 146.325986] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581326019worker1 # [ 146.340226] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6020worker1 # [ 146.367484] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:53Z level=info event=niks3_push_started pid=17516021worker1 # [ 146.587876] nix-grpc-daemon[1751]: time=2026-09-21T09:29:54.219Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""6022worker1 # [ 146.760669] nix-daemon[839]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)6023worker1 # [ 146.776791] nix-daemon[839]: reaped child process 1767, status = succeeded6024worker1 # [ 146.950886] nix-daemon[839]: accepted connection from pid 1769, user nix-grpc-daemon (trusted)6025worker1 # [ 146.966446] nix-daemon[839]: reaped child process 1772, status = succeeded6026lb # [ 146.984689] niks3-server[1023]: 2026/09/21 09:29:54 INFO Received uploads request method=POST path=/api/pending_closures6027worker1 # [ 146.989652] nix-grpc-daemon[1751]: time=2026-09-21T09:29:54.620Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6028worker1 # [ 146.991136] nix-grpc-daemon[1751]: time=2026-09-21T09:29:54.620Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6029lb # [ 147.018595] niks3-server[1023]: 2026/09/21 09:29:54 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6030worker1 # [ 147.025041] nix-grpc-daemon[1751]: time=2026-09-21T09:29:54.656Z level=INFO msg="Uploading 1 narinfos"6031lb # [ 147.030693] niks3-server[1023]: 2026/09/21 09:29:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6032lb # [ 147.039181] niks3-server[1023]: 2026/09/21 09:29:54 INFO Signed narinfos id=46 count=16033lb # [ 147.048629] niks3-server[1023]: 2026/09/21 09:29:54 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6034lb # [ 147.058587] niks3-server[1023]: 2026/09/21 09:29:54 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6035lb # [ 147.068097] niks3-server[1023]: 2026/09/21 09:29:54 INFO Completed upload id=466036worker1 # [ 147.062459] nix-grpc-daemon[1751]: time=2026-09-21T09:29:54.694Z level=INFO msg="Upload complete. (470ms)"6037lb # [ 147.071029] niks3-server[1023]: 2026/09/21 09:29:54 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6038worker1 # [ 147.073274] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=0 paths=1 nar_bytes_in=5126039worker1 # [ 147.082118] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326040worker1 # [ 147.091084] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6041worker1 # [ 147.093286] nix-daemon[839]: reaped child process 1749, status = succeeded6042worker1 # [ 147.105387] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6043worker1 # [ 148.526137] nix-daemon[839]: reaped child process 1776, status = succeeded6044worker1 # [ 148.685703] nix-daemon[839]: accepted connection from pid 1792, user nix-grpc-daemon (trusted)6045worker1 # [ 148.703443] nix-daemon[839]: reaped child process 1795, status = succeeded6046lb # [ 148.721914] niks3-server[1023]: 2026/09/21 09:29:56 INFO Received uploads request method=POST path=/api/pending_closures6047worker1 # [ 148.731620] nix-grpc-daemon[1751]: time=2026-09-21T09:29:56.360Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6048worker1 # [ 148.733742] nix-grpc-daemon[1751]: time=2026-09-21T09:29:56.360Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6049lb # [ 148.756090] niks3-server[1023]: 2026/09/21 09:29:56 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6050lb # [ 148.771546] niks3-server[1023]: 2026/09/21 09:29:56 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6051lb # [ 148.787397] niks3-server[1023]: 2026/09/21 09:29:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6052worker1 # [ 148.787808] nix-grpc-daemon[1751]: time=2026-09-21T09:29:56.418Z level=INFO msg="Uploading 1 narinfos"6053lb # [ 148.801293] niks3-server[1023]: 2026/09/21 09:29:56 INFO Signed narinfos id=47 count=16054lb # [ 148.807130] niks3-server[1023]: 2026/09/21 09:29:56 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6055lb # [ 148.815884] niks3-server[1023]: 2026/09/21 09:29:56 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6056lb # [ 148.821189] niks3-server[1023]: 2026/09/21 09:29:56 INFO Completed upload id=476057worker1 # [ 148.814802] nix-grpc-daemon[1751]: time=2026-09-21T09:29:56.446Z level=INFO msg="Upload complete. (300ms)"6058lb # [ 148.824181] niks3-server[1023]: 2026/09/21 09:29:56 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6059worker1 # [ 148.830975] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=1 outputs=16060worker1 # [ 148.846981] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:56Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16061worker1 # [ 148.851732] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:56Z level=debug event=want client=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06062worker1 # [ 148.855493] nix-daemon[839]: reaped child process 1774, status = succeeded6063worker1 # [ 148.874484] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326064worker1 # [ 148.889030] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581326065worker1 # [ 148.902378] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581326066worker1 # [ 148.915932] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6067worker1 # [ 149.112592] nix-daemon[839]: accepted connection from pid 1801, user nix-grpc-daemon (trusted)6068worker1 # [ 149.128841] nix-daemon[839]: reaped child process 1804, status = succeeded6069worker1 # [ 149.303785] nix-daemon[839]: accepted connection from pid 1806, user nix-grpc-daemon (trusted)6070worker1 # [ 149.318575] nix-daemon[839]: reaped child process 1809, status = succeeded6071lb # [ 149.339445] niks3-server[1023]: 2026/09/21 09:29:56 INFO Received uploads request method=POST path=/api/pending_closures6072worker1 # [ 149.343384] nix-grpc-daemon[1751]: time=2026-09-21T09:29:56.975Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6073worker1 # [ 149.344999] nix-grpc-daemon[1751]: time=2026-09-21T09:29:56.975Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6074lb # [ 149.367757] niks3-server[1023]: 2026/09/21 09:29:56 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6075lb # [ 149.381049] niks3-server[1023]: 2026/09/21 09:29:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6076worker1 # [ 149.382452] nix-grpc-daemon[1751]: time=2026-09-21T09:29:57.014Z level=INFO msg="Uploading 1 narinfos"6077lb # [ 149.390364] niks3-server[1023]: 2026/09/21 09:29:57 INFO Signed narinfos id=48 count=16078lb # [ 149.396614] niks3-server[1023]: 2026/09/21 09:29:57 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6079worker1 # [ 149.401090] nix-grpc-daemon[1751]: time=2026-09-21T09:29:57.033Z level=INFO msg="Upload complete. (471ms)"6080lb # [ 149.406990] niks3-server[1023]: 2026/09/21 09:29:57 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6081lb # [ 149.411786] niks3-server[1023]: 2026/09/21 09:29:57 INFO Completed upload id=486082worker1 # [ 149.408147] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=0 paths=1 nar_bytes_in=5126083lb # [ 149.415225] niks3-server[1023]: 2026/09/21 09:29:57 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6084worker1 # [ 149.422429] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326085worker1 # [ 149.425788] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6086worker1 # [ 149.430713] nix-daemon[839]: reaped child process 1799, status = succeeded6087worker1 # [ 149.441149] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6088worker1 # [ 150.267588] nix-daemon[839]: reaped child process 1813, status = succeeded6089worker1 # [ 150.427846] nix-daemon[839]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)6090worker1 # [ 150.445873] nix-daemon[839]: reaped child process 1832, status = succeeded6091lb # [ 150.465476] niks3-server[1023]: 2026/09/21 09:29:58 INFO Received uploads request method=POST path=/api/pending_closures6092worker1 # [ 150.468178] nix-grpc-daemon[1751]: time=2026-09-21T09:29:58.099Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6093worker1 # [ 150.469616] nix-grpc-daemon[1751]: time=2026-09-21T09:29:58.099Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6094lb # [ 150.496188] niks3-server[1023]: 2026/09/21 09:29:58 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6095lb # [ 150.510203] niks3-server[1023]: 2026/09/21 09:29:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6096worker1 # [ 150.511348] nix-grpc-daemon[1751]: time=2026-09-21T09:29:58.143Z level=INFO msg="Uploading 1 narinfos"6097lb # [ 150.522218] niks3-server[1023]: 2026/09/21 09:29:58 INFO Signed narinfos id=49 count=16098lb # [ 150.530674] niks3-server[1023]: 2026/09/21 09:29:58 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6099lb # [ 150.539266] niks3-server[1023]: 2026/09/21 09:29:58 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6100lb # [ 150.550917] niks3-server[1023]: 2026/09/21 09:29:58 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6101lb # [ 150.554895] niks3-server[1023]: 2026/09/21 09:29:58 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6102lb # [ 150.559481] niks3-server[1023]: 2026/09/21 09:29:58 INFO Completed upload id=496103worker1 # [ 150.553490] nix-grpc-daemon[1751]: time=2026-09-21T09:29:58.185Z level=INFO msg="Upload complete. (304ms)"6104worker1 # [ 150.569363] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=2 outputs=16105worker1 # [ 150.584844] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:58Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16106worker1 # [ 150.591856] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326107worker1 # [ 150.595910] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:58Z level=debug event=want client=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06108worker1 # [ 150.599652] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581326109worker1 # [ 150.603520] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581326110worker1 # [ 150.615250] nix-daemon[839]: reaped child process 1811, status = succeeded6111worker1 # [ 150.617742] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6112worker1 # [ 150.788463] nix-daemon[839]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)6113worker1 # [ 150.806285] nix-daemon[839]: reaped child process 1841, status = succeeded6114lb # [ 150.860445] envoy[913]: conn peer=[2001:db8:1::3]:48548 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1426 tx=568 ms=222166115worker1 # [ 150.978918] nix-daemon[839]: accepted connection from pid 1843, user nix-grpc-daemon (trusted)6116worker1 # [ 150.994753] nix-daemon[839]: reaped child process 1846, status = succeeded6117lb # [ 151.011890] niks3-server[1023]: 2026/09/21 09:29:58 INFO Received uploads request method=POST path=/api/pending_closures6118worker1 # [ 151.015183] nix-grpc-daemon[1751]: time=2026-09-21T09:29:58.646Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6119worker1 # [ 151.016760] nix-grpc-daemon[1751]: time=2026-09-21T09:29:58.647Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6120lb # [ 151.038827] niks3-server[1023]: 2026/09/21 09:29:58 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6121worker1 # [ 151.045656] nix-grpc-daemon[1751]: time=2026-09-21T09:29:58.677Z level=INFO msg="Uploading 1 narinfos"6122lb # [ 151.053031] niks3-server[1023]: 2026/09/21 09:29:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6123lb # [ 151.061586] niks3-server[1023]: 2026/09/21 09:29:58 INFO Signed narinfos id=50 count=16124lb # [ 151.067380] niks3-server[1023]: 2026/09/21 09:29:58 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6125lb # [ 151.072971] niks3-server[1023]: 2026/09/21 09:29:58 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6126worker1 # [ 151.072264] nix-grpc-daemon[1751]: time=2026-09-21T09:29:58.704Z level=INFO msg="Upload complete. (451ms)"6127lb # [ 151.079199] niks3-server[1023]: 2026/09/21 09:29:58 INFO Completed upload id=506128lb # [ 151.082530] niks3-server[1023]: 2026/09/21 09:29:58 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6129worker1 # [ 151.080233] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=0 paths=1 nar_bytes_in=5126130worker1 # [ 151.090441] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326131worker1 # [ 151.096696] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6132worker1 # [ 151.101455] nix-daemon[839]: reaped child process 1836, status = succeeded6133worker1 # [ 151.114770] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6134worker1 # [ 151.962538] nix-daemon[839]: reaped child process 1850, status = succeeded6135worker1 # [ 152.137952] nix-daemon[839]: accepted connection from pid 1866, user nix-grpc-daemon (trusted)6136worker1 # [ 152.158056] nix-daemon[839]: reaped child process 1869, status = succeeded6137lb # [ 152.178134] niks3-server[1023]: 2026/09/21 09:29:59 INFO Received uploads request method=POST path=/api/pending_closures6138worker1 # [ 152.181993] nix-grpc-daemon[1751]: time=2026-09-21T09:29:59.813Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6139worker1 # [ 152.185758] nix-grpc-daemon[1751]: time=2026-09-21T09:29:59.816Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6140lb # [ 152.214083] niks3-server[1023]: 2026/09/21 09:29:59 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6141lb # [ 152.230430] niks3-server[1023]: 2026/09/21 09:29:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6142worker1 # [ 152.230380] nix-grpc-daemon[1751]: time=2026-09-21T09:29:59.861Z level=INFO msg="Uploading 1 narinfos"6143lb # [ 152.243390] niks3-server[1023]: 2026/09/21 09:29:59 INFO Signed narinfos id=51 count=16144lb # [ 152.248328] niks3-server[1023]: 2026/09/21 09:29:59 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6145lb # [ 152.251697] niks3-server[1023]: 2026/09/21 09:29:59 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6146lb # [ 152.258056] niks3-server[1023]: 2026/09/21 09:29:59 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6147lb # [ 152.263585] niks3-server[1023]: 2026/09/21 09:29:59 INFO Completed upload id=516148worker1 # [ 152.257511] nix-grpc-daemon[1751]: time=2026-09-21T09:29:59.889Z level=INFO msg="Upload complete. (333ms)"6149lb # [ 152.266002] niks3-server[1023]: 2026/09/21 09:29:59 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6150worker1 # [ 152.277275] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=3 outputs=16151worker1 # [ 152.292530] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:59Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16152worker1 # [ 152.298457] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:59Z level=debug event=want client=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06153worker1 # [ 152.303119] nix-daemon[839]: reaped child process 1848, status = succeeded6154worker1 # [ 152.310638] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326155worker1 # [ 152.325470] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581326156worker1 # [ 152.339862] nix-grpc-daemon[1751]: time=2026-09-21T09:29:59.971Z level=INFO msg="All 1 paths already cached"6157worker1 # [ 152.348716] nix-grpc-daemon[1686]: ts=2026-09-21T09:29:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581326158worker1 # [ 152.361904] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6159worker1 # [ 152.551802] nix-daemon[839]: accepted connection from pid 1875, user nix-grpc-daemon (trusted)6160worker1 # [ 152.572500] nix-daemon[839]: reaped child process 1878, status = succeeded6161worker1 # [ 152.750837] nix-daemon[839]: accepted connection from pid 1880, user nix-grpc-daemon (trusted)6162worker1 # [ 152.769705] nix-daemon[839]: reaped child process 1883, status = succeeded6163lb # [ 152.786307] niks3-server[1023]: 2026/09/21 09:30:00 INFO Received uploads request method=POST path=/api/pending_closures6164worker1 # [ 152.790052] nix-grpc-daemon[1751]: time=2026-09-21T09:30:00.421Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6165worker1 # [ 152.791569] nix-grpc-daemon[1751]: time=2026-09-21T09:30:00.421Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6166lb # [ 152.813088] niks3-server[1023]: 2026/09/21 09:30:00 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6167worker1 # [ 152.822226] nix-grpc-daemon[1751]: time=2026-09-21T09:30:00.454Z level=INFO msg="Uploading 1 narinfos"6168lb # [ 152.827773] niks3-server[1023]: 2026/09/21 09:30:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6169lb # [ 152.839995] niks3-server[1023]: 2026/09/21 09:30:00 INFO Signed narinfos id=52 count=16170lb # [ 152.847562] niks3-server[1023]: 2026/09/21 09:30:00 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6171lb # [ 152.855589] niks3-server[1023]: 2026/09/21 09:30:00 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6172worker1 # [ 152.856734] nix-grpc-daemon[1751]: time=2026-09-21T09:30:00.488Z level=INFO msg="Upload complete. (481ms)"6173lb # [ 152.863278] niks3-server[1023]: 2026/09/21 09:30:00 INFO Completed upload id=526174lb # [ 152.867027] niks3-server[1023]: 2026/09/21 09:30:00 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6175worker1 # [ 152.864170] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=0 paths=1 nar_bytes_in=6246176worker1 # [ 152.876256] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326177worker1 # [ 152.882333] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6178worker1 # [ 152.887200] nix-daemon[839]: reaped child process 1873, status = succeeded6179worker1 # [ 152.900973] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6180worker1 # [ 154.147361] nix-daemon[839]: reaped child process 1887, status = succeeded6181worker1 # [ 154.339482] nix-daemon[839]: accepted connection from pid 1903, user nix-grpc-daemon (trusted)6182worker1 # [ 154.355644] nix-daemon[839]: reaped child process 1906, status = succeeded6183lb # [ 154.376935] niks3-server[1023]: 2026/09/21 09:30:01 INFO Received uploads request method=POST path=/api/pending_closures6184worker1 # [ 154.383619] nix-grpc-daemon[1751]: time=2026-09-21T09:30:02.012Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6185worker1 # [ 154.385461] nix-grpc-daemon[1751]: time=2026-09-21T09:30:02.012Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6186lb # [ 154.414614] niks3-server[1023]: 2026/09/21 09:30:02 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6187lb # [ 154.427889] niks3-server[1023]: 2026/09/21 09:30:02 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6188lb # [ 154.436563] niks3-server[1023]: 2026/09/21 09:30:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6189worker1 # [ 154.436537] nix-grpc-daemon[1751]: time=2026-09-21T09:30:02.068Z level=INFO msg="Uploading 1 narinfos"6190lb # [ 154.446172] niks3-server[1023]: 2026/09/21 09:30:02 INFO Signed narinfos id=53 count=16191lb # [ 154.452281] niks3-server[1023]: 2026/09/21 09:30:02 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6192lb # [ 154.460143] niks3-server[1023]: 2026/09/21 09:30:02 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6193lb # [ 154.466417] niks3-server[1023]: 2026/09/21 09:30:02 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6194lb # [ 154.472182] niks3-server[1023]: 2026/09/21 09:30:02 INFO Completed upload id=536195worker1 # [ 154.466578] nix-grpc-daemon[1751]: time=2026-09-21T09:30:02.098Z level=INFO msg="Upload complete. (355ms)"6196worker1 # [ 154.485079] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=4 outputs=16197worker1 # [ 154.505356] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:02Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16198worker1 # [ 154.510019] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326199worker1 # [ 154.514186] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:02Z level=debug event=want client=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06200worker1 # [ 154.518943] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581326201worker1 # [ 154.523302] nix-daemon[839]: reaped child process 1885, status = succeeded6202worker1 # [ 154.525282] nix-grpc-daemon[1751]: time=2026-09-21T09:30:02.154Z level=INFO msg="All 1 paths already cached"6203worker1 # [ 154.529731] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581326204worker1 # [ 154.543159] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6205worker1 # [ 154.725082] nix-daemon[839]: accepted connection from pid 1912, user nix-grpc-daemon (trusted)6206worker1 # [ 154.741741] nix-daemon[839]: reaped child process 1915, status = succeeded6207worker1 # [ 154.914520] nix-daemon[839]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)6208worker1 # [ 154.930633] nix-daemon[839]: reaped child process 1920, status = succeeded6209lb # [ 154.947222] niks3-server[1023]: 2026/09/21 09:30:02 INFO Received uploads request method=POST path=/api/pending_closures6210worker1 # [ 154.950639] nix-grpc-daemon[1751]: time=2026-09-21T09:30:02.582Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6211worker1 # [ 154.952245] nix-grpc-daemon[1751]: time=2026-09-21T09:30:02.582Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6212lb # [ 154.974741] niks3-server[1023]: 2026/09/21 09:30:02 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6213lb # [ 154.986526] niks3-server[1023]: 2026/09/21 09:30:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6214worker1 # [ 154.986501] nix-grpc-daemon[1751]: time=2026-09-21T09:30:02.618Z level=INFO msg="Uploading 1 narinfos"6215lb # [ 154.996714] niks3-server[1023]: 2026/09/21 09:30:02 INFO Signed narinfos id=54 count=16216lb # [ 155.004524] niks3-server[1023]: 2026/09/21 09:30:02 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6217lb # [ 155.017715] niks3-server[1023]: 2026/09/21 09:30:02 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6218worker1 # [ 155.020331] nix-grpc-daemon[1751]: time=2026-09-21T09:30:02.652Z level=INFO msg="Upload complete. (471ms)"6219lb # [ 155.026033] niks3-server[1023]: 2026/09/21 09:30:02 INFO Completed upload id=546220lb # [ 155.029571] niks3-server[1023]: 2026/09/21 09:30:02 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6221worker1 # [ 155.028205] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=0 paths=1 nar_bytes_in=6246222worker1 # [ 155.042766] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326223worker1 # [ 155.046321] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6224worker1 # [ 155.051899] nix-daemon[839]: reaped child process 1910, status = succeeded6225worker1 # [ 155.063141] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6226worker1 # [ 156.306830] nix-daemon[839]: reaped child process 1924, status = succeeded6227worker1 # [ 156.478552] nix-daemon[839]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)6228worker1 # [ 156.496622] nix-daemon[839]: reaped child process 1943, status = succeeded6229lb # [ 156.517322] niks3-server[1023]: 2026/09/21 09:30:04 INFO Received uploads request method=POST path=/api/pending_closures6230worker1 # [ 156.528410] nix-grpc-daemon[1751]: time=2026-09-21T09:30:04.157Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6231worker1 # [ 156.529930] nix-grpc-daemon[1751]: time=2026-09-21T09:30:04.157Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6232lb # [ 156.557669] niks3-server[1023]: 2026/09/21 09:30:04 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6233lb # [ 156.570670] niks3-server[1023]: 2026/09/21 09:30:04 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6234lb # [ 156.582629] niks3-server[1023]: 2026/09/21 09:30:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6235worker1 # [ 156.582108] nix-grpc-daemon[1751]: time=2026-09-21T09:30:04.213Z level=INFO msg="Uploading 1 narinfos"6236lb # [ 156.592115] niks3-server[1023]: 2026/09/21 09:30:04 INFO Signed narinfos id=55 count=16237lb # [ 156.594582] niks3-server[1023]: 2026/09/21 09:30:04 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6238lb # [ 156.602756] niks3-server[1023]: 2026/09/21 09:30:04 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6239lb # [ 156.611891] niks3-server[1023]: 2026/09/21 09:30:04 INFO Completed upload id=556240worker1 # [ 156.606275] nix-grpc-daemon[1751]: time=2026-09-21T09:30:04.238Z level=INFO msg="Upload complete. (333ms)"6241lb # [ 156.614963] niks3-server[1023]: 2026/09/21 09:30:04 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6242worker1 # [ 156.628453] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=5 outputs=16243worker1 # [ 156.638267] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:04Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16244worker1 # [ 156.647707] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326245worker1 # [ 156.652926] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:04Z level=debug event=want client=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06246worker1 # [ 156.657510] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581326247worker1 # [ 156.661702] nix-daemon[839]: reaped child process 1922, status = succeeded6248worker1 # [ 156.663501] nix-grpc-daemon[1751]: time=2026-09-21T09:30:04.293Z level=INFO msg="All 1 paths already cached"6249worker1 # [ 156.666983] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581326250worker1 # [ 156.679258] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6251worker1 # [ 156.864502] nix-daemon[839]: accepted connection from pid 1949, user nix-grpc-daemon (trusted)6252worker1 # [ 156.883178] nix-daemon[839]: reaped child process 1952, status = succeeded6253worker1 # [ 157.053961] nix-daemon[839]: accepted connection from pid 1954, user nix-grpc-daemon (trusted)6254worker1 # [ 157.070298] nix-daemon[839]: reaped child process 1957, status = succeeded6255lb # [ 157.087052] niks3-server[1023]: 2026/09/21 09:30:04 INFO Received uploads request method=POST path=/api/pending_closures6256worker1 # [ 157.089956] nix-grpc-daemon[1751]: time=2026-09-21T09:30:04.721Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6257worker1 # [ 157.091497] nix-grpc-daemon[1751]: time=2026-09-21T09:30:04.721Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6258lb # [ 157.111856] niks3-server[1023]: 2026/09/21 09:30:04 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6259lb # [ 157.125239] niks3-server[1023]: 2026/09/21 09:30:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6260worker1 # [ 157.127149] nix-grpc-daemon[1751]: time=2026-09-21T09:30:04.758Z level=INFO msg="Uploading 1 narinfos"6261lb # [ 157.136661] niks3-server[1023]: 2026/09/21 09:30:04 INFO Signed narinfos id=56 count=16262lb # [ 157.139607] niks3-server[1023]: 2026/09/21 09:30:04 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6263lb # [ 157.150599] niks3-server[1023]: 2026/09/21 09:30:04 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6264lb # [ 157.157564] niks3-server[1023]: 2026/09/21 09:30:04 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6265lb # [ 157.161444] niks3-server[1023]: 2026/09/21 09:30:04 INFO Completed upload id=566266worker1 # [ 157.155857] nix-grpc-daemon[1751]: time=2026-09-21T09:30:04.787Z level=INFO msg="Upload complete. (464ms)"6267worker1 # [ 157.164715] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=0 paths=1 nar_bytes_in=6246268worker1 # [ 157.177540] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326269worker1 # [ 157.184427] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6270worker1 # [ 157.192262] nix-daemon[839]: reaped child process 1947, status = succeeded6271worker1 # [ 157.199180] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6272worker1 # [ 158.426487] nix-daemon[839]: reaped child process 1961, status = succeeded6273worker1 # [ 158.600710] nix-daemon[839]: accepted connection from pid 1977, user nix-grpc-daemon (trusted)6274worker1 # [ 158.617083] nix-daemon[839]: reaped child process 1980, status = succeeded6275lb # [ 158.635268] niks3-server[1023]: 2026/09/21 09:30:06 INFO Received uploads request method=POST path=/api/pending_closures6276worker1 # [ 158.640551] nix-grpc-daemon[1751]: time=2026-09-21T09:30:06.272Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6277worker1 # [ 158.643106] nix-grpc-daemon[1751]: time=2026-09-21T09:30:06.274Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6278lb # [ 158.676184] niks3-server[1023]: 2026/09/21 09:30:06 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6279lb # [ 158.689640] niks3-server[1023]: 2026/09/21 09:30:06 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6280lb # [ 158.696114] niks3-server[1023]: 2026/09/21 09:30:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6281worker1 # [ 158.695102] nix-grpc-daemon[1751]: time=2026-09-21T09:30:06.326Z level=INFO msg="Uploading 1 narinfos"6282lb # [ 158.704803] niks3-server[1023]: 2026/09/21 09:30:06 INFO Signed narinfos id=57 count=16283lb # [ 158.709866] niks3-server[1023]: 2026/09/21 09:30:06 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6284lb # [ 158.716453] niks3-server[1023]: 2026/09/21 09:30:06 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6285lb # [ 158.724739] niks3-server[1023]: 2026/09/21 09:30:06 INFO Completed upload id=576286worker1 # [ 158.719171] nix-grpc-daemon[1751]: time=2026-09-21T09:30:06.351Z level=INFO msg="Upload complete. (311ms)"6287lb # [ 158.727384] niks3-server[1023]: 2026/09/21 09:30:06 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6288worker1 # [ 158.737306] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=6 outputs=16289worker1 # [ 158.758823] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:06Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16290worker1 # [ 158.763994] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326291worker1 # [ 158.769262] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:06Z level=debug event=want client=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06292worker1 # [ 158.774616] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581326293worker1 # [ 158.779555] nix-daemon[839]: reaped child process 1959, status = succeeded6294worker1 # [ 158.781793] nix-grpc-daemon[1751]: time=2026-09-21T09:30:06.410Z level=INFO msg="All 2 paths already cached"6295worker1 # [ 158.788124] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581326296worker1 # [ 158.798022] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6297worker1 # [ 158.980398] nix-daemon[839]: accepted connection from pid 1986, user nix-grpc-daemon (trusted)6298worker1 # [ 158.998920] nix-daemon[839]: reaped child process 1989, status = succeeded6299worker1 # [ 159.167877] nix-daemon[839]: accepted connection from pid 1991, user nix-grpc-daemon (trusted)6300worker1 # [ 159.185781] nix-daemon[839]: reaped child process 1994, status = succeeded6301lb # [ 159.203333] niks3-server[1023]: 2026/09/21 09:30:06 INFO Received uploads request method=POST path=/api/pending_closures6302worker1 # [ 159.206352] nix-grpc-daemon[1751]: time=2026-09-21T09:30:06.838Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6303worker1 # [ 159.207845] nix-grpc-daemon[1751]: time=2026-09-21T09:30:06.838Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6304lb # [ 159.228556] niks3-server[1023]: 2026/09/21 09:30:06 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6305lb # [ 159.243014] niks3-server[1023]: 2026/09/21 09:30:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6306worker1 # [ 159.243790] nix-grpc-daemon[1751]: time=2026-09-21T09:30:06.875Z level=INFO msg="Uploading 1 narinfos"6307lb # [ 159.255347] niks3-server[1023]: 2026/09/21 09:30:06 INFO Signed narinfos id=58 count=16308lb # [ 159.260764] niks3-server[1023]: 2026/09/21 09:30:06 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6309lb # [ 159.268271] niks3-server[1023]: 2026/09/21 09:30:06 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6310lb # [ 159.273058] niks3-server[1023]: 2026/09/21 09:30:06 INFO Completed upload id=586311worker1 # [ 159.267270] nix-grpc-daemon[1751]: time=2026-09-21T09:30:06.899Z level=INFO msg="Upload complete. (460ms)"6312lb # [ 159.275764] niks3-server[1023]: 2026/09/21 09:30:06 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6313worker1 # [ 159.275041] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=0 paths=1 nar_bytes_in=7526314worker1 # [ 159.287978] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326315worker1 # [ 159.294091] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6316worker1 # [ 159.298889] nix-daemon[839]: reaped child process 1984, status = succeeded6317worker1 # [ 159.312205] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6318worker1 # [ 160.357614] nix-daemon[839]: reaped child process 1998, status = succeeded6319worker1 # [ 160.536139] nix-daemon[839]: accepted connection from pid 2014, user nix-grpc-daemon (trusted)6320worker1 # [ 160.554142] nix-daemon[839]: reaped child process 2017, status = succeeded6321lb # [ 160.573941] niks3-server[1023]: 2026/09/21 09:30:08 INFO Received uploads request method=POST path=/api/pending_closures6322worker1 # [ 160.576449] nix-grpc-daemon[1751]: time=2026-09-21T09:30:08.207Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6323worker1 # [ 160.578264] nix-grpc-daemon[1751]: time=2026-09-21T09:30:08.207Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6324worker1 # [ 160.615831] nix-grpc-daemon[1751]: time=2026-09-21T09:30:08.247Z level=INFO msg="Uploading 1 narinfos"6325lb # [ 160.615188] niks3-server[1023]: 2026/09/21 09:30:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6326lb # [ 160.632333] niks3-server[1023]: 2026/09/21 09:30:08 INFO Signed narinfos id=59 count=16327lb # [ 160.635415] niks3-server[1023]: 2026/09/21 09:30:08 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6328lb # [ 160.652472] niks3-server[1023]: 2026/09/21 09:30:08 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6329worker1 # [ 160.652307] nix-grpc-daemon[1751]: time=2026-09-21T09:30:08.284Z level=INFO msg="Upload complete. (330ms)"6330lb # [ 160.658135] niks3-server[1023]: 2026/09/21 09:30:08 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6331lb # [ 160.661840] niks3-server[1023]: 2026/09/21 09:30:08 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6332lb # [ 160.666720] niks3-server[1023]: 2026/09/21 09:30:08 INFO Completed upload id=596333lb # [ 160.669689] niks3-server[1023]: 2026/09/21 09:30:08 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6334worker1 # [ 160.664922] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=7 outputs=16335worker1 # [ 160.680422] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:08Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16336worker1 # [ 160.685678] nix-daemon[839]: reaped child process 1996, status = succeeded6337worker1 # [ 160.710418] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326338worker1 # [ 160.725672] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581326339worker1 # [ 160.740317] nix-grpc-daemon[1751]: time=2026-09-21T09:30:08.372Z level=INFO msg="All 2 paths already cached"6340worker1 # [ 160.749236] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581326341worker1 # [ 160.762809] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6342worker1 # [ 160.970115] nix-daemon[839]: accepted connection from pid 2021, user nix-grpc-daemon (trusted)6343worker1 # [ 160.988829] nix-daemon[839]: reaped child process 2024, status = succeeded6344worker1 # [ 161.157617] nix-daemon[839]: accepted connection from pid 2026, user nix-grpc-daemon (trusted)6345worker1 # [ 161.174963] nix-daemon[839]: reaped child process 2029, status = succeeded6346lb # [ 161.192144] niks3-server[1023]: 2026/09/21 09:30:08 INFO Received uploads request method=POST path=/api/pending_closures6347worker1 # [ 161.196058] nix-grpc-daemon[1751]: time=2026-09-21T09:30:08.827Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6348worker1 # [ 161.198048] nix-grpc-daemon[1751]: time=2026-09-21T09:30:08.827Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6349lb # [ 161.223054] niks3-server[1023]: 2026/09/21 09:30:08 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6350lb # [ 161.234730] niks3-server[1023]: 2026/09/21 09:30:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6351worker1 # [ 161.235379] nix-grpc-daemon[1751]: time=2026-09-21T09:30:08.867Z level=INFO msg="Uploading 1 narinfos"6352lb # [ 161.246096] niks3-server[1023]: 2026/09/21 09:30:08 INFO Signed narinfos id=60 count=16353lb # [ 161.252843] niks3-server[1023]: 2026/09/21 09:30:08 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6354lb # [ 161.264258] niks3-server[1023]: 2026/09/21 09:30:08 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6355worker1 # [ 161.262877] nix-grpc-daemon[1751]: time=2026-09-21T09:30:08.894Z level=INFO msg="Upload complete. (482ms)"6356lb # [ 161.268715] niks3-server[1023]: 2026/09/21 09:30:08 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6357lb # [ 161.272413] niks3-server[1023]: 2026/09/21 09:30:08 INFO Completed upload id=606358worker1 # [ 161.271397] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=0 paths=1 nar_bytes_in=7526359worker1 # [ 161.283238] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326360worker1 # [ 161.289185] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6361worker1 # [ 161.293774] nix-daemon[839]: reaped child process 2019, status = succeeded6362worker1 # [ 161.306672] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6363worker1 # [ 162.940885] nix-daemon[839]: reaped child process 2033, status = succeeded6364worker1 # [ 163.116589] nix-daemon[839]: accepted connection from pid 2049, user nix-grpc-daemon (trusted)6365worker1 # [ 163.132233] nix-daemon[839]: reaped child process 2052, status = succeeded6366lb # [ 163.155138] niks3-server[1023]: 2026/09/21 09:30:10 INFO Received uploads request method=POST path=/api/pending_closures6367worker1 # [ 163.157653] nix-grpc-daemon[1751]: time=2026-09-21T09:30:10.788Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6368worker1 # [ 163.159135] nix-grpc-daemon[1751]: time=2026-09-21T09:30:10.788Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6369lb # [ 163.184400] niks3-server[1023]: 2026/09/21 09:30:10 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6370lb # [ 163.200701] niks3-server[1023]: 2026/09/21 09:30:10 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6371lb # [ 163.210183] niks3-server[1023]: 2026/09/21 09:30:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6372worker1 # [ 163.209954] nix-grpc-daemon[1751]: time=2026-09-21T09:30:10.841Z level=INFO msg="Uploading 1 narinfos"6373lb # [ 163.219240] niks3-server[1023]: 2026/09/21 09:30:10 INFO Signed narinfos id=61 count=16374lb # [ 163.225636] niks3-server[1023]: 2026/09/21 09:30:10 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6375lb # [ 163.233676] niks3-server[1023]: 2026/09/21 09:30:10 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6376lb # [ 163.239965] niks3-server[1023]: 2026/09/21 09:30:10 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6377lb # [ 163.246004] niks3-server[1023]: 2026/09/21 09:30:10 INFO Completed upload id=616378worker1 # [ 163.240412] nix-grpc-daemon[1751]: time=2026-09-21T09:30:10.872Z level=INFO msg="Upload complete. (323ms)"6379worker1 # [ 163.253312] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=8 outputs=16380worker1 # [ 163.269049] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:10Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16381worker1 # [ 163.273798] nix-daemon[839]: reaped child process 2031, status = succeeded6382worker1 # [ 163.294651] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326383worker1 # [ 163.309352] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581326384worker1 # [ 163.323630] nix-grpc-daemon[1751]: time=2026-09-21T09:30:10.955Z level=INFO msg="All 2 paths already cached"6385worker1 # [ 163.332586] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581326386worker1 # [ 163.346045] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6387worker1 # [ 163.542083] nix-daemon[839]: accepted connection from pid 2056, user nix-grpc-daemon (trusted)6388worker1 # [ 163.559991] nix-daemon[839]: reaped child process 2059, status = succeeded6389worker1 # [ 163.734995] nix-daemon[839]: accepted connection from pid 2061, user nix-grpc-daemon (trusted)6390worker1 # [ 163.751027] nix-daemon[839]: reaped child process 2064, status = succeeded6391lb # [ 163.769306] niks3-server[1023]: 2026/09/21 09:30:11 INFO Received uploads request method=POST path=/api/pending_closures6392worker1 # [ 163.774141] nix-grpc-daemon[1751]: time=2026-09-21T09:30:11.405Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6393worker1 # [ 163.775618] nix-grpc-daemon[1751]: time=2026-09-21T09:30:11.405Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6394lb # [ 163.798592] niks3-server[1023]: 2026/09/21 09:30:11 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6395lb # [ 163.812816] niks3-server[1023]: 2026/09/21 09:30:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6396worker1 # [ 163.812951] nix-grpc-daemon[1751]: time=2026-09-21T09:30:11.444Z level=INFO msg="Uploading 1 narinfos"6397lb # [ 163.825291] niks3-server[1023]: 2026/09/21 09:30:11 INFO Signed narinfos id=62 count=16398lb # [ 163.831799] niks3-server[1023]: 2026/09/21 09:30:11 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6399lb # [ 163.841659] niks3-server[1023]: 2026/09/21 09:30:11 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6400worker1 # [ 163.841360] nix-grpc-daemon[1751]: time=2026-09-21T09:30:11.473Z level=INFO msg="Upload complete. (478ms)"6401lb # [ 163.847882] niks3-server[1023]: 2026/09/21 09:30:11 INFO Completed upload id=626402lb # [ 163.850919] niks3-server[1023]: 2026/09/21 09:30:11 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6403worker1 # [ 163.850737] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=0 paths=1 nar_bytes_in=7526404worker1 # [ 163.863684] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581326405worker1 # [ 163.874163] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6406worker1 # [ 163.877078] nix-daemon[839]: reaped child process 2054, status = succeeded6407worker1 # [ 163.890718] nix-daemon[839]: accepted connection from pid 1686, user nix-grpc-daemon (trusted)6408worker1 # [ 165.130669] nix-daemon[839]: reaped child process 2068, status = succeeded6409worker1 # [ 165.296854] nix-daemon[839]: accepted connection from pid 2084, user nix-grpc-daemon (trusted)6410worker1 # [ 165.315338] nix-daemon[839]: reaped child process 2087, status = succeeded6411lb # [ 165.334261] niks3-server[1023]: 2026/09/21 09:30:12 INFO Received uploads request method=POST path=/api/pending_closures6412worker1 # [ 165.340544] nix-grpc-daemon[1751]: time=2026-09-21T09:30:12.971Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6413worker1 # [ 165.342908] nix-grpc-daemon[1751]: time=2026-09-21T09:30:12.971Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6414lb # [ 165.372320] niks3-server[1023]: 2026/09/21 09:30:12 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6415lb # [ 165.384382] niks3-server[1023]: 2026/09/21 09:30:13 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6416lb # [ 165.396874] niks3-server[1023]: 2026/09/21 09:30:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6417worker1 # [ 165.396146] nix-grpc-daemon[1751]: time=2026-09-21T09:30:13.027Z level=INFO msg="Uploading 1 narinfos"6418lb # [ 165.404377] niks3-server[1023]: 2026/09/21 09:30:13 INFO Signed narinfos id=63 count=16419lb # [ 165.412040] niks3-server[1023]: 2026/09/21 09:30:13 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6420lb # [ 165.421716] niks3-server[1023]: 2026/09/21 09:30:13 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6421lb # [ 165.429814] niks3-server[1023]: 2026/09/21 09:30:13 INFO Completed upload id=636422lb # [ 165.433923] niks3-server[1023]: 2026/09/21 09:30:13 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6423worker1 # [ 165.424261] nix-grpc-daemon[1751]: time=2026-09-21T09:30:13.056Z level=INFO msg="Upload complete. (323ms)"6424worker1 # [ 165.440585] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58132 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=9 outputs=16425worker1 # [ 165.455539] nix-daemon[839]: reaped child process 2066, status = succeeded6426client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 19.99 seconds)6427worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6428worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6429client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'6430client # [ 165.653867] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6431client # Running as unit: gpu.service; invocation ID: 7d0370952dbd40c482d1d47cd1deee9e6432client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6433lb: must succeed: curl -sf localhost:9901/clusters6434lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6435lb: must succeed: curl -sf localhost:9901/clusters6436lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6437worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6438worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6439client # [ 165.904783] nix[1357]: don't know how to build these paths:6440client # [ 165.905890] nix[1357]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6441worker1 # [ 165.909900] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:581326442worker2 # [ 165.927580] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538246443client # [ 165.936364] nix[1357]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6444worker1 # [ 165.942351] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:13Z level=debug event=want client=2 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06445lb: must succeed: curl -sf localhost:9901/clusters6446lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6447lb: must succeed: curl -sf localhost:9901/clusters6448lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6449worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6450worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6451client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6452lb # [ 170.858522] envoy[913]: conn peer=[2001:db8:1::1]:51180 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27131 tx=11350 ms=196536453client # [ 195.938440] nix[1357]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6454client # [ 195.952132] nix[1357]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6455client # [ 195.998119] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6456client # [ 196.005936] systemd[1]: gpu.service: Failed with result 'exit-code'.6457client # [ 196.010566] systemd[1]: gpu.service: Consumed 175ms CPU time over 30.349s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6458client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.28 seconds)6459client: must succeed: journalctl -u gpu -o cat6460client: (finished: must succeed: journalctl -u gpu -o cat, in 0.04 seconds)6461lb: must succeed: curl -sf localhost:9901/clusters6462lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6463lb: must succeed: curl -sf localhost:9901/clusters6464lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6465worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6466worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6467(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 51.12 seconds)6468subtest: low disk drains a worker and builds go to the other6469worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6470worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.10 seconds)6471worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6472worker1 # [ 198.215983] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:45Z level=info event=unhealthy reason=min_free6473worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.12 seconds)6474client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26475client # don't know how to build these paths:6476client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6477worker1 # [ 199.139064] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:581326478worker1 # [ 199.145871] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581326479worker1 # [ 199.173883] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:46Z level=debug event=want client=3 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06480worker1 # [ 199.179867] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:46Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16481worker2 # [ 199.230176] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538246482worker2 # [ 199.246399] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538246483client # copying 1 paths to 'lb:50051'...6484client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6485worker2 # [ 199.262717] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538246486worker2 # [ 199.278311] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)6487worker2 # [ 199.482163] nix-daemon[861]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)6488worker2 # [ 199.500951] nix-daemon[861]: reaped child process 1683, status = succeeded6489worker2 # [ 199.675308] nix-daemon[861]: accepted connection from pid 1685, user nix-grpc-daemon (trusted)6490worker2 # [ 199.692455] nix-daemon[861]: reaped child process 1688, status = succeeded6491lb # [ 199.703877] niks3-server[1023]: 2026/09/21 09:30:47 INFO Received uploads request method=POST path=/api/pending_closures6492worker2 # [ 199.719487] nix-grpc-daemon[1499]: time=2026-09-21T09:30:47.346Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6493worker2 # [ 199.721139] nix-grpc-daemon[1499]: time=2026-09-21T09:30:47.346Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6494lb # [ 199.737008] niks3-server[1023]: 2026/09/21 09:30:47 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6495lb # [ 199.752109] niks3-server[1023]: 2026/09/21 09:30:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6496worker2 # [ 199.760879] nix-grpc-daemon[1499]: time=2026-09-21T09:30:47.388Z level=INFO msg="Uploading 1 narinfos"6497lb # [ 199.763186] niks3-server[1023]: 2026/09/21 09:30:47 INFO Signed narinfos id=64 count=16498lb # [ 199.768319] niks3-server[1023]: 2026/09/21 09:30:47 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6499lb # [ 199.776321] niks3-server[1023]: 2026/09/21 09:30:47 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6500lb # [ 199.781136] niks3-server[1023]: 2026/09/21 09:30:47 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6501lb # [ 199.784845] niks3-server[1023]: 2026/09/21 09:30:47 INFO Completed upload id=646502worker2 # [ 199.788438] nix-grpc-daemon[1499]: time=2026-09-21T09:30:47.416Z level=INFO msg="Upload complete. (493ms)"6503worker2 # [ 199.796149] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53824 duration_s=0 paths=1 nar_bytes_in=5126504worker2 # [ 199.811042] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538246505worker2 # [ 199.817737] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)6506worker2 # [ 199.822694] nix-daemon[861]: reaped child process 1678, status = succeeded6507client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6508worker2 # [ 199.833683] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)6509client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6510client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6511client # farm-a> LOG-farm-a-drain6512worker2 # [ 200.455736] nix-daemon[861]: reaped child process 1692, status = succeeded6513worker2 # [ 200.613052] nix-daemon[861]: accepted connection from pid 1708, user nix-grpc-daemon (trusted)6514worker2 # [ 200.628804] nix-daemon[861]: reaped child process 1711, status = succeeded6515lb # [ 200.638186] niks3-server[1023]: 2026/09/21 09:30:48 INFO Received uploads request method=POST path=/api/pending_closures6516worker2 # [ 200.652064] nix-grpc-daemon[1499]: time=2026-09-21T09:30:48.279Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6517worker2 # [ 200.653542] nix-grpc-daemon[1499]: time=2026-09-21T09:30:48.279Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6518lb # [ 200.679524] niks3-server[1023]: 2026/09/21 09:30:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6519worker2 # [ 200.690780] nix-grpc-daemon[1499]: time=2026-09-21T09:30:48.318Z level=INFO msg="Uploading 1 narinfos"6520lb # [ 200.695989] niks3-server[1023]: 2026/09/21 09:30:48 INFO Signed narinfos id=65 count=16521lb # [ 200.699286] niks3-server[1023]: 2026/09/21 09:30:48 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6522lb # [ 200.721828] niks3-server[1023]: 2026/09/21 09:30:48 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6523lb # [ 200.728280] niks3-server[1023]: 2026/09/21 09:30:48 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6524lb # [ 200.734421] niks3-server[1023]: 2026/09/21 09:30:48 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6525lb # [ 200.738504] niks3-server[1023]: 2026/09/21 09:30:48 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6526lb # [ 200.753055] niks3-server[1023]: 2026/09/21 09:30:48 INFO Completed upload id=656527worker2 # [ 200.758078] nix-grpc-daemon[1499]: time=2026-09-21T09:30:48.384Z level=INFO msg="Upload complete. (312ms)"6528worker2 # [ 200.769627] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53824 duration_s=0 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=10 outputs=16529worker2 # [ 200.785464] nix-daemon[861]: reaped child process 1690, status = succeeded6530worker1 # [ 200.807801] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:48Z level=debug event=want client=3 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06531worker1 # [ 200.813784] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:48Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16532lb # [ 200.859367] envoy[913]: conn peer=[2001:db8:1::1]:53486 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300576533worker2 # [ 200.862839] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538246534worker2 # [ 200.871524] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538246535worker2 # [ 200.879527] nix-grpc-daemon[1499]: time=2026-09-21T09:30:48.507Z level=INFO msg="All 1 paths already cached"6536client # copying 1 paths to 'lb:50051'...6537client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6538worker2 # [ 200.886400] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538246539worker2 # [ 200.894421] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)6540worker2 # [ 201.075458] nix-daemon[861]: accepted connection from pid 1715, user nix-grpc-daemon (trusted)6541worker2 # [ 201.094954] nix-daemon[861]: reaped child process 1718, status = succeeded6542worker2 # [ 201.269991] nix-daemon[861]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6543worker2 # [ 201.286044] nix-daemon[861]: reaped child process 1723, status = succeeded6544lb # [ 201.294701] niks3-server[1023]: 2026/09/21 09:30:48 INFO Received uploads request method=POST path=/api/pending_closures6545worker2 # [ 201.309672] nix-grpc-daemon[1499]: time=2026-09-21T09:30:48.937Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6546worker2 # [ 201.311158] nix-grpc-daemon[1499]: time=2026-09-21T09:30:48.937Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6547lb # [ 201.327018] niks3-server[1023]: 2026/09/21 09:30:48 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6548worker2 # [ 201.340861] nix-grpc-daemon[1499]: time=2026-09-21T09:30:48.968Z level=INFO msg="Uploading 1 narinfos"6549lb # [ 201.343237] niks3-server[1023]: 2026/09/21 09:30:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6550worker2 # [ 201.358700] nix-grpc-daemon[1499]: time=2026-09-21T09:30:48.986Z level=INFO msg="Upload complete. (452ms)"6551lb # [ 201.356974] niks3-server[1023]: 2026/09/21 09:30:48 INFO Signed narinfos id=66 count=16552lb # [ 201.360501] niks3-server[1023]: 2026/09/21 09:30:48 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6553lb # [ 201.364626] niks3-server[1023]: 2026/09/21 09:30:48 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6554lb # [ 201.367905] niks3-server[1023]: 2026/09/21 09:30:48 INFO Completed upload id=666555worker2 # [ 201.371252] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53824 duration_s=0 paths=1 nar_bytes_in=6326556lb # [ 201.370044] niks3-server[1023]: 2026/09/21 09:30:48 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6557worker2 # [ 201.375456] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538246558worker2 # [ 201.378497] nix-daemon[861]: reaped child process 1713, status = succeeded6559worker2 # [ 201.384849] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)6560client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6561worker2 # [ 201.396449] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)6562client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6563client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6564client # farm-b> LOG-farm-b-drain6565worker2 # [ 202.637886] nix-daemon[861]: reaped child process 1727, status = succeeded6566worker2 # [ 202.831927] nix-daemon[861]: accepted connection from pid 1743, user nix-grpc-daemon (trusted)6567worker2 # [ 202.850312] nix-daemon[861]: reaped child process 1746, status = succeeded6568lb # [ 202.862021] niks3-server[1023]: 2026/09/21 09:30:50 INFO Received uploads request method=POST path=/api/pending_closures6569worker2 # [ 202.876576] nix-grpc-daemon[1499]: time=2026-09-21T09:30:50.501Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6570worker2 # [ 202.878116] nix-grpc-daemon[1499]: time=2026-09-21T09:30:50.501Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6571lb # [ 202.903175] niks3-server[1023]: 2026/09/21 09:30:50 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6572lb # [ 202.915928] niks3-server[1023]: 2026/09/21 09:30:50 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6573lb # [ 202.923368] niks3-server[1023]: 2026/09/21 09:30:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6574worker2 # [ 202.932078] nix-grpc-daemon[1499]: time=2026-09-21T09:30:50.559Z level=INFO msg="Uploading 1 narinfos"6575lb # [ 202.933735] niks3-server[1023]: 2026/09/21 09:30:50 INFO Signed narinfos id=67 count=16576lb # [ 202.938736] niks3-server[1023]: 2026/09/21 09:30:50 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6577lb # [ 202.944114] niks3-server[1023]: 2026/09/21 09:30:50 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6578lb # [ 202.950266] niks3-server[1023]: 2026/09/21 09:30:50 INFO Completed upload id=676579worker2 # [ 202.953670] nix-grpc-daemon[1499]: time=2026-09-21T09:30:50.581Z level=INFO msg="Upload complete. (344ms)"6580lb # [ 202.953653] niks3-server[1023]: 2026/09/21 09:30:50 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6581worker2 # [ 202.965324] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53824 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=11 outputs=16582worker1 # [ 202.971046] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:50Z level=debug event=want client=3 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06583worker2 # [ 202.985699] nix-daemon[861]: reaped child process 1725, status = succeeded6584worker1 # [ 202.971648] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:50Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16585worker2 # [ 203.024147] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538246586worker2 # [ 203.040882] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538246587worker2 # [ 203.056512] nix-grpc-daemon[1499]: time=2026-09-21T09:30:50.684Z level=INFO msg="All 2 paths already cached"6588client # copying 1 paths to 'lb:50051'...6589client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6590worker2 # [ 203.066112] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538246591worker2 # [ 203.080550] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)6592worker2 # [ 203.289952] nix-daemon[861]: accepted connection from pid 1750, user nix-grpc-daemon (trusted)6593worker2 # [ 203.310027] nix-daemon[861]: reaped child process 1753, status = succeeded6594worker2 # [ 203.482408] nix-daemon[861]: accepted connection from pid 1755, user nix-grpc-daemon (trusted)6595worker2 # [ 203.499432] nix-daemon[861]: reaped child process 1758, status = succeeded6596lb # [ 203.510596] niks3-server[1023]: 2026/09/21 09:30:51 INFO Received uploads request method=POST path=/api/pending_closures6597worker2 # [ 203.524114] nix-grpc-daemon[1499]: time=2026-09-21T09:30:51.151Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6598worker2 # [ 203.525610] nix-grpc-daemon[1499]: time=2026-09-21T09:30:51.151Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6599lb # [ 203.542359] niks3-server[1023]: 2026/09/21 09:30:51 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6600lb # [ 203.555915] niks3-server[1023]: 2026/09/21 09:30:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6601worker2 # [ 203.567228] nix-grpc-daemon[1499]: time=2026-09-21T09:30:51.194Z level=INFO msg="Uploading 1 narinfos"6602lb # [ 203.569496] niks3-server[1023]: 2026/09/21 09:30:51 INFO Signed narinfos id=68 count=16603lb # [ 203.575257] niks3-server[1023]: 2026/09/21 09:30:51 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6604lb # [ 203.582739] niks3-server[1023]: 2026/09/21 09:30:51 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6605lb # [ 203.589222] niks3-server[1023]: 2026/09/21 09:30:51 INFO Completed upload id=686606worker2 # [ 203.592883] nix-grpc-daemon[1499]: time=2026-09-21T09:30:51.220Z level=INFO msg="Upload complete. (493ms)"6607lb # [ 203.591661] niks3-server[1023]: 2026/09/21 09:30:51 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6608worker2 # [ 203.600222] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53824 duration_s=0 paths=1 nar_bytes_in=7606609worker2 # [ 203.614891] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538246610worker2 # [ 203.621687] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)6611worker2 # [ 203.626588] nix-daemon[861]: reaped child process 1748, status = succeeded6612client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6613client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6614worker2 # [ 203.640687] nix-daemon[861]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)6615client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6616client # farm-top> LOG-farm-top-drain6617worker2 # [ 204.688402] nix-daemon[861]: reaped child process 1762, status = succeeded6618worker2 # [ 204.867682] nix-daemon[861]: accepted connection from pid 1778, user nix-grpc-daemon (trusted)6619worker2 # [ 204.884703] nix-daemon[861]: reaped child process 1781, status = succeeded6620lb # [ 204.897780] niks3-server[1023]: 2026/09/21 09:30:52 INFO Received uploads request method=POST path=/api/pending_closures6621worker2 # [ 204.920276] nix-grpc-daemon[1499]: time=2026-09-21T09:30:52.546Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6622worker2 # [ 204.921746] nix-grpc-daemon[1499]: time=2026-09-21T09:30:52.546Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6623lb # [ 204.932820] niks3-server[1023]: 2026/09/21 09:30:52 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6624lb # [ 204.947304] niks3-server[1023]: 2026/09/21 09:30:52 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6625lb # [ 204.960078] niks3-server[1023]: 2026/09/21 09:30:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6626worker2 # [ 204.969108] nix-grpc-daemon[1499]: time=2026-09-21T09:30:52.596Z level=INFO msg="Uploading 1 narinfos"6627lb # [ 204.969263] niks3-server[1023]: 2026/09/21 09:30:52 INFO Signed narinfos id=69 count=16628lb # [ 204.971596] niks3-server[1023]: 2026/09/21 09:30:52 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6629lb # [ 204.979565] niks3-server[1023]: 2026/09/21 09:30:52 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6630lb # [ 204.986946] niks3-server[1023]: 2026/09/21 09:30:52 INFO Completed upload id=696631worker2 # [ 204.990246] nix-grpc-daemon[1499]: time=2026-09-21T09:30:52.617Z level=INFO msg="Upload complete. (339ms)"6632lb # [ 204.989777] niks3-server[1023]: 2026/09/21 09:30:52 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6633worker2 # [ 205.001085] nix-grpc-daemon[1407]: ts=2026-09-21T09:30:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53824 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=12 outputs=16634worker2 # [ 205.016424] nix-daemon[861]: reaped child process 1760, status = succeeded6635client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 6.22 seconds)6636client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6637client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.23 seconds)6638worker1: must succeed: rm /nix/.rw-store/fill6639worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)6640worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6641worker1 # [ 206.220399] nix-grpc-daemon[1686]: ts=2026-09-21T09:30:53Z level=info event=healthy reason=min_free6642worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)6643(finished: subtest: low disk drains a worker and builds go to the other, in 10.05 seconds)6644(finished: run the VM test script, in 207.60 seconds)6645test script finished in 207.89s6646cleanup6647kill QemuMachine (pid 50)6648client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6649kill QemuMachine (pid 49)6650lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6651kill QemuMachine (pid 51)6652worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6653kill QemuMachine (pid 52)6654worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6655(finished: cleanup, in 1.05 seconds)6656additionally exposed symbols:6657 client, lb, worker1, worker2,6658 vlan1,6659 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh