vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #405
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2 # Formatting '/build/vm-state-worker2/tmp.xbKNgWvx7j', fmt=raw size=107374182414worker2 # mke2fs 1.47.4 (6-Mar-2025)15worker2 # Discarding device blocks: 0/262144 done16worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes17worker2: QEMU running (pid 52)18worker2 # Filesystem UUID: b7b4459a-11fe-4d9c-a4fa-50942b2d7ba119worker2 # Superblock backups stored on blocks:20worker2 # 32768, 98304, 163840, 22937621worker2 # 22worker2 # Allocating group tables: 0/8 done23worker2 # Writing inode tables: 0/8 done24worker2 # Creating journal (8192 blocks): done25worker2 # Writing superblocks and filesystem accounting information: 0/8 done26worker2 # 27worker2 # Virtualisation disk image created.28worker1 # Disk image does not exist, creating the virtualisation disk image...29worker1 # Formatting '/build/vm-state-worker1/tmp.2cnN9ThnkV', fmt=raw size=107374182430worker1 # mke2fs 1.47.4 (6-Mar-2025)31worker1 # Discarding device blocks: 0/262144 done32worker1: QEMU running (pid 51)33worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes34worker1 # Filesystem UUID: 58cc546a-53d5-4182-bdc5-d2b6ffbff31b35client # Disk image does not exist, creating the virtualisation disk image...36worker1 # Superblock backups stored on blocks:37client # Formatting '/build/vm-state-client/tmp.Hu8vxJk9Zx', fmt=raw size=107374182438client # mke2fs 1.47.4 (6-Mar-2025)39client # Discarding device blocks: 0/262144 done40client # Creating filesystem with 262144 4k blocks and 65536 inodes41client # Filesystem UUID: 807334f1-aa14-4768-ae30-1d40f9408b0c42client # Superblock backups stored on blocks:43client # 32768, 98304, 163840, 22937644client # 45client # Allocating group tables: 0/8 done46client # Writing inode tables: 0/8 done47client # Creating journal (8192 blocks): done48client # Writing superblocks and filesystem accounting information: 0/8 done49client # 50client # Virtualisation disk image created.51client: QEMU running (pid 49)52worker1 # 32768, 98304, 163840, 22937653worker1 # 54worker1 # Allocating group tables: 0/8 done55worker1 # Writing inode tables: 0/8 done56worker1 # Creating journal (8192 blocks): done57worker1 # Writing superblocks and filesystem accounting information: 0/8 done58worker1 # 59worker1 # Virtualisation disk image created.60lb # Disk image does not exist, creating the virtualisation disk image...61lb: QEMU running (pid 50)62lb # Formatting '/build/vm-state-lb/tmp.EO9P07008r', fmt=raw size=107374182463lb # mke2fs 1.47.4 (6-Mar-2025)64lb # Discarding device blocks: 0/262144 done65lb # Creating filesystem with 262144 4k blocks and 65536 inodes66lb # Filesystem UUID: 8bad2d3e-c252-47c5-9313-7471e750c08c67lb # Superblock backups stored on blocks:68lb # 32768, 98304, 163840, 22937669lb # 70lb # Allocating group tables: 0/8 done71lb # Writing inode tables: 0/8 done72lb # Creating journal (8192 blocks): done73lb # Writing superblocks and filesystem accounting information: 0/8 done74lb # 75lb # Virtualisation disk image created.76(finished: start all VMs, in 0.58 seconds)77client: waiting for unit network-addresses-eth1.service78client: waiting for the VM to finish booting79worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]81client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]82worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202683worker1 # [ 0.000000] KASLR enabled84worker1 # [ 0.000000] random: crng init done85worker1 # [ 0.000000] Machine model: linux,dummy-virt86client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202687worker1 # [ 0.000000] efi: UEFI not found.88client # [ 0.000000] KASLR enabled89client # [ 0.000000] random: crng init done90worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT91client # [ 0.000000] Machine model: linux,dummy-virt92worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202693worker2 # [ 0.000000] KASLR enabled94worker2 # [ 0.000000] random: crng init done95worker2 # [ 0.000000] Machine model: linux,dummy-virt96worker2 # [ 0.000000] efi: UEFI not found.97worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT98worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]99worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]100client # [ 0.000000] efi: UEFI not found.101worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]102client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT103worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]104worker2 # [ 0.000000] Zone ranges:105worker1 # [ 0.000000] Zone ranges:106worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]107client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]108worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]109worker2 # [ 0.000000] DMA32 empty110worker1 # [ 0.000000] DMA32 empty111client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]112worker2 # [ 0.000000] Normal empty113worker1 # [ 0.000000] Normal empty114worker1 # [ 0.000000] Device empty115client # [ 0.000000] Zone ranges:116client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]117worker1 # [ 0.000000] Movable zone start for each node118client # [ 0.000000] DMA32 empty119worker2 # [ 0.000000] Device empty120worker2 # [ 0.000000] Movable zone start for each node121worker2 # [ 0.000000] Early memory node ranges122worker1 # [ 0.000000] Early memory node ranges123worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]124worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]125client # [ 0.000000] Normal empty126client # [ 0.000000] Device empty127client # [ 0.000000] Movable zone start for each node128client # [ 0.000000] Early memory node ranges129worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]130worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]131client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]132worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000133worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000134worker2 # [ 0.000000] psci: probing for conduit method from DT.135worker1 # [ 0.000000] psci: probing for conduit method from DT.136client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]137worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.138worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.139client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000140worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs141worker2 # [ 0.000000] psci: Trusted OS migration not required142worker2 # [ 0.000000] psci: SMC Calling Convention v1.1143worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs144worker1 # [ 0.000000] psci: Trusted OS migration not required145worker1 # [ 0.000000] psci: SMC Calling Convention v1.1146client # [ 0.000000] psci: probing for conduit method from DT.147client # [ 0.000000] psci: PSCIv1.3 detected in firmware.148client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs149lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]150client # [ 0.000000] psci: Trusted OS migration not required151worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)152worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296153lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026154worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)155worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296156worker2 # [ 0.000000] Detected PIPT I-cache on CPU0157worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)158worker1 # [ 0.000000] Detected PIPT I-cache on CPU0159worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface160worker2 # [ 0.000000] CPU features: detected: Spectre-v4161lb # [ 0.000000] KASLR enabled162lb # [ 0.000000] random: crng init done163lb # [ 0.000000] Machine model: linux,dummy-virt164client # [ 0.000000] psci: SMC Calling Convention v1.1165lb # [ 0.000000] efi: UEFI not found.166worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)167client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)168worker2 # [ 0.000000] CPU features: detected: Spectre-BHB169client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296170worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface171client # [ 0.000000] Detected PIPT I-cache on CPU0172worker1 # [ 0.000000] CPU features: detected: Spectre-v4173worker1 # [ 0.000000] CPU features: detected: Spectre-BHB174client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)175lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT176client # [ 0.000000] CPU features: detected: GICv3 CPU interface177worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38178worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38179client # [ 0.000000] CPU features: detected: Spectre-v4180lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]181worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23182client # [ 0.000000] CPU features: detected: Spectre-BHB183worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23184lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]185worker2 # [ 0.000000] alternatives: applying boot alternatives186worker1 # [ 0.000000] alternatives: applying boot alternatives187client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38188lb # [ 0.000000] Zone ranges:189client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23190lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]191lb # [ 0.000000] DMA32 empty192client # [ 0.000000] alternatives: applying boot alternatives193lb # [ 0.000000] Normal empty194lb # [ 0.000000] Device empty195lb # [ 0.000000] Movable zone start for each node196lb # [ 0.000000] Early memory node ranges197worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qxlzavd9kkrahyk8swqs03fcck49n6hv-nixos-system-worker2-test/init regInfo=/nix/store/7rckirrzx1fwryj29dsnagwq3640l4xa-closure-info/registration console=ttyAMA0,115200n8 console=tty0198worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/cph26q37ffk3xyndbrr93rc9zdmd200a-nixos-system-worker1-test/init regInfo=/nix/store/0nkb7lf7rz5cf5ifxzwf7akkpap456gx-closure-info/registration console=ttyAMA0,115200n8 console=tty0199lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]200lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]201client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/f1p01b8r1g0vwpnb6kbil89vcdp0b0k0-nixos-system-client-test/init regInfo=/nix/store/mhc4xibdb98493vap7vldlyl5w0nrkkw-closure-info/registration console=ttyAMA0,115200n8 console=tty0202lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000203worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/7rckirrzx1fwryj29dsnagwq3640l4xa-closure-info/registration", will be passed to user space.204worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/0nkb7lf7rz5cf5ifxzwf7akkpap456gx-closure-info/registration", will be passed to user space.205lb # [ 0.000000] psci: probing for conduit method from DT.206worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes207worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes208lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.209client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/mhc4xibdb98493vap7vldlyl5w0nrkkw-closure-info/registration", will be passed to user space.210lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs211worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)212worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)213lb # [ 0.000000] psci: Trusted OS migration not required214client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes215worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)216lb # [ 0.000000] psci: SMC Calling Convention v1.1217worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)218worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB219lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)220client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)221worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB222worker2 # [ 0.000000] software IO TLB: area num 1.223worker1 # [ 0.000000] software IO TLB: area num 1.224client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)225lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296226worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB227worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB228client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB229lb # [ 0.000000] Detected PIPT I-cache on CPU0230client # [ 0.000000] software IO TLB: area num 1.231worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)232worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)233lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)234worker2 # [ 0.000000] Fallback order for Node 0: 0235client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB236worker1 # [ 0.000000] Fallback order for Node 0: 0237lb # [ 0.000000] CPU features: detected: GICv3 CPU interface238worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216239worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216240client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)241lb # [ 0.000000] CPU features: detected: Spectre-v4242worker2 # [ 0.000000] Policy zone: DMA243worker1 # [ 0.000000] Policy zone: DMA244client # [ 0.000000] Fallback order for Node 0: 0245lb # [ 0.000000] CPU features: detected: Spectre-BHB246worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off247worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off248client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216249lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38250client # [ 0.000000] Policy zone: DMA251worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1252worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1253lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23254worker2 # [ 0.000000] allocated 3145728 bytes of page_ext255worker1 # [ 0.000000] allocated 3145728 bytes of page_ext256client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off257lb # [ 0.000000] alternatives: applying boot alternatives258worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages259worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages260client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1261worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups262worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups263client # [ 0.000000] allocated 3145728 bytes of page_ext264worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.265worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.266client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages267worker2 # [ 0.000000] rcu: RCU event tracing is enabled.268worker1 # [ 0.000000] rcu: RCU event tracing is enabled.269client # [ 0.000000] ftrace: allocated 294 pages with 4 groups270worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.271worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.272client # [ 0.000000] rcu: Hierarchical RCU implementation.273worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.274client # [ 0.000000] rcu: RCU event tracing is enabled.275worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.276lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lai4n7chb0cnvnygfis27sipihspygia-nixos-system-lb-test/init regInfo=/nix/store/qdg5w64gnxykmz82pw25fgmx46q7lp99-closure-info/registration console=ttyAMA0,115200n8 console=tty0277worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.278worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.279client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.280worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.281worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.282client # [ 0.000000] Trampoline variant of Tasks RCU enabled.283lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/qdg5w64gnxykmz82pw25fgmx46q7lp99-closure-info/registration", will be passed to user space.284worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.285client # [ 0.000000] Rude variant of Tasks RCU enabled.286worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.287client # [ 0.000000] Tracing variant of Tasks RCU enabled.288worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1289lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes290worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1291client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.292lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)293worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.294worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.295client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1296lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)297worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.298worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.299client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.300lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB301lb # [ 0.000000] software IO TLB: area num 1.302worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.303worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.304client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.305worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0306lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB307worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0308worker2 # [ 0.000000] GICv3: 256 SPIs implemented309worker1 # [ 0.000000] GICv3: 256 SPIs implemented310client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.311lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)312worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented313worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented314lb # [ 0.000000] Fallback order for Node 0: 0315client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0316worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq317worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq318client # [ 0.000000] GICv3: 256 SPIs implemented319worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI320worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI321lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216322client # [ 0.000000] GICv3: 0 Extended SPIs implemented323worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0324worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0325lb # [ 0.000000] Policy zone: DMA326client # [ 0.000000] Root IRQ handler: gic_handle_irq327lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off328worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000329client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI330worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000331worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]332client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0333lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1334worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]335lb # [ 0.000000] allocated 3145728 bytes of page_ext336client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000337worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)338worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)339lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages340client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]341lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups342worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)343worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)344lb # [ 0.000000] rcu: Hierarchical RCU implementation.345client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)346worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000347lb # [ 0.000000] rcu: RCU event tracing is enabled.348worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000349worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000350client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)351lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.352worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000353lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.354client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000355worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.356worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.357lb # [ 0.000000] Rude variant of Tasks RCU enabled.358client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000359lb # [ 0.000000] Tracing variant of Tasks RCU enabled.360worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns361worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns362client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.363lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.364worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).365worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).366lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1367client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns368worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns369worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns370client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).371lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.372worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns373worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns374worker2 # [ 0.000036] arm-pv: using stolen time PV375client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns376worker1 # [ 0.000040] arm-pv: using stolen time PV377lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.378client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns379worker2 # [ 0.000480] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)380lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.381worker1 # [ 0.000517] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)382client # [ 0.000035] arm-pv: using stolen time PV383worker2 # [ 0.000700] Console: colour dummy device 80x25384worker1 # [ 0.000735] Console: colour dummy device 80x25385lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0386worker2 # [ 0.000710] printk: legacy console [tty0] enabled387lb # [ 0.000000] GICv3: 256 SPIs implemented388worker1 # [ 0.000744] printk: legacy console [tty0] enabled389client # [ 0.000560] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)390lb # [ 0.000000] GICv3: 0 Extended SPIs implemented391client # [ 0.000743] Console: colour dummy device 80x25392lb # [ 0.000000] Root IRQ handler: gic_handle_irq393worker2 # [ 0.000922] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)394worker1 # [ 0.000955] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)395client # [ 0.000751] printk: legacy console [tty0] enabled396lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI397worker2 # [ 0.000930] pid_max: default: 32768 minimum: 301398worker1 # [ 0.000963] pid_max: default: 32768 minimum: 301399lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0400worker2 # [ 0.001043] LSM: initializing lsm=capability,landlock,yama,bpf,ima401worker1 # [ 0.001040] LSM: initializing lsm=capability,landlock,yama,bpf,ima402client # [ 0.000955] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)403worker2 # [ 0.001267] landlock: Up and running.404lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000405worker1 # [ 0.001183] landlock: Up and running.406worker2 # [ 0.001270] Yama: becoming mindful.407client # [ 0.000962] pid_max: default: 32768 minimum: 301408worker1 # [ 0.001185] Yama: becoming mindful.409lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]410worker2 # [ 0.001792] LSM support for eBPF active411worker1 # [ 0.001801] LSM support for eBPF active412client # [ 0.001038] LSM: initializing lsm=capability,landlock,yama,bpf,ima413client # [ 0.001185] landlock: Up and running.414lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)415worker2 # [ 0.001942] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)416worker1 # [ 0.001935] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)417client # [ 0.001187] Yama: becoming mindful.418client # [ 0.001716] LSM support for eBPF active419worker2 # [ 0.001976] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)420worker1 # [ 0.001973] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)421lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)422client # [ 0.001914] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)423worker2 # [ 0.003193] cacheinfo: Unable to detect cache hierarchy for CPU 0424worker1 # [ 0.003204] cacheinfo: Unable to detect cache hierarchy for CPU 0425lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000426worker2 # [ 0.004062] rcu: Hierarchical SRCU implementation.427worker1 # [ 0.003991] rcu: Hierarchical SRCU implementation.428client # [ 0.001950] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)429lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000430worker2 # [ 0.004068] rcu: Max phase no-delay instances is 1000.431worker1 # [ 0.004022] rcu: Max phase no-delay instances is 1000.432client # [ 0.003265] cacheinfo: Unable to detect cache hierarchy for CPU 0433worker2 # [ 0.005382] fsl-mc MSI: its@8080000 domain created434lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.435worker1 # [ 0.005368] fsl-mc MSI: its@8080000 domain created436client # [ 0.004149] rcu: Hierarchical SRCU implementation.437worker2 # [ 0.005482] EFI services will not be available.438worker1 # [ 0.005468] EFI services will not be available.439client # [ 0.004155] rcu: Max phase no-delay instances is 1000.440worker2 # [ 0.005562] smp: Bringing up secondary CPUs ...441worker1 # [ 0.005552] smp: Bringing up secondary CPUs ...442lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns443client # [ 0.005449] fsl-mc MSI: its@8080000 domain created444worker2 # [ 0.005572] smp: Brought up 1 node, 1 CPU445worker1 # [ 0.005563] smp: Brought up 1 node, 1 CPU446lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).447client # [ 0.005550] EFI services will not be available.448worker2 # [ 0.005576] SMP: Total of 1 processors activated.449worker1 # [ 0.005567] SMP: Total of 1 processors activated.450client # [ 0.005630] smp: Bringing up secondary CPUs ...451worker2 # [ 0.005582] CPU: All CPU(s) started at EL1452worker1 # [ 0.005574] CPU: All CPU(s) started at EL1453client # [ 0.005640] smp: Brought up 1 node, 1 CPU454lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns455worker2 # [ 0.005596] CPU features: detected: Branch Target Identification456worker1 # [ 0.005586] CPU features: detected: Branch Target Identification457client # [ 0.005643] SMP: Total of 1 processors activated.458worker2 # [ 0.005600] CPU features: detected: ARMv8.4 Translation Table Level459lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns460client # [ 0.005650] CPU: All CPU(s) started at EL1461worker1 # [ 0.005592] CPU features: detected: ARMv8.4 Translation Table Level462lb # [ 0.000035] arm-pv: using stolen time PV463client # [ 0.005663] CPU features: detected: Branch Target Identification464worker2 # [ 0.005603] CPU features: detected: Instruction cache invalidation not required for I/D coherence465worker1 # [ 0.005595] CPU features: detected: Instruction cache invalidation not required for I/D coherence466client # [ 0.005669] CPU features: detected: ARMv8.4 Translation Table Level467lb # [ 0.000428] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)468worker2 # [ 0.005607] CPU features: detected: Data cache clean to the PoU not required for I/D coherence469worker1 # [ 0.005599] CPU features: detected: Data cache clean to the PoU not required for I/D coherence470lb # [ 0.000648] Console: colour dummy device 80x25471client # [ 0.005673] CPU features: detected: Instruction cache invalidation not required for I/D coherence472lb # [ 0.000657] printk: legacy console [tty0] enabled473worker2 # [ 0.005611] CPU features: detected: Common not Private translations474worker1 # [ 0.005602] CPU features: detected: Common not Private translations475worker2 # [ 0.005614] CPU features: detected: CRC32 instructions476worker1 # [ 0.005606] CPU features: detected: CRC32 instructions477client # [ 0.005676] CPU features: detected: Data cache clean to the PoU not required for I/D coherence478lb # [ 0.000866] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)479worker2 # [ 0.005617] CPU features: detected: Data cache clean to Point of Deep Persistence480client # [ 0.005680] CPU features: detected: Common not Private translations481worker1 # [ 0.005608] CPU features: detected: Data cache clean to Point of Deep Persistence482lb # [ 0.000874] pid_max: default: 32768 minimum: 301483client # [ 0.005684] CPU features: detected: CRC32 instructions484worker2 # [ 0.005621] CPU features: detected: Data cache clean to Point of Persistence485worker1 # [ 0.005613] CPU features: detected: Data cache clean to Point of Persistence486lb # [ 0.000967] LSM: initializing lsm=capability,landlock,yama,bpf,ima487lb # [ 0.001159] landlock: Up and running.488client # [ 0.005687] CPU features: detected: Data cache clean to Point of Deep Persistence489worker2 # [ 0.005624] CPU features: detected: Data independent timing control (DIT)490worker1 # [ 0.005616] CPU features: detected: Data independent timing control (DIT)491lb # [ 0.001162] Yama: becoming mindful.492worker2 # [ 0.005627] CPU features: detected: E0PD493worker1 # [ 0.005619] CPU features: detected: E0PD494lb # [ 0.001724] LSM support for eBPF active495client # [ 0.005690] CPU features: detected: Data cache clean to Point of Persistence496worker2 # [ 0.005630] CPU features: detected: Enhanced Counter Virtualization497worker1 # [ 0.005622] CPU features: detected: Enhanced Counter Virtualization498client # [ 0.005694] CPU features: detected: Data independent timing control (DIT)499lb # [ 0.001885] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)500client # [ 0.005697] CPU features: detected: E0PD501worker2 # [ 0.005633] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)502worker1 # [ 0.005625] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)503lb # [ 0.001918] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)504client # [ 0.005699] CPU features: detected: Enhanced Counter Virtualization505worker2 # [ 0.005636] CPU features: detected: Enhanced Virtualization Traps506worker1 # [ 0.005629] CPU features: detected: Enhanced Virtualization Traps507worker1 # [ 0.005632] CPU features: detected: Fine Grained Traps508lb # [ 0.003140] cacheinfo: Unable to detect cache hierarchy for CPU 0509worker2 # [ 0.005639] CPU features: detected: Fine Grained Traps510client # [ 0.005702] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)511lb # [ 0.003892] rcu: Hierarchical SRCU implementation.512client # [ 0.005706] CPU features: detected: Enhanced Virtualization Traps513worker1 # [ 0.005636] CPU features: detected: Generic authentication (architected QARMA5 algorithm)514worker2 # [ 0.005644] CPU features: detected: Generic authentication (architected QARMA5 algorithm)515lb # [ 0.003897] rcu: Max phase no-delay instances is 1000.516client # [ 0.005708] CPU features: detected: Fine Grained Traps517lb # [ 0.005186] fsl-mc MSI: its@8080000 domain created518worker2 # [ 0.005649] CPU features: detected: RCpc load-acquire (LDAPR)519worker1 # [ 0.005641] CPU features: detected: RCpc load-acquire (LDAPR)520lb # [ 0.005291] EFI services will not be available.521worker1 # [ 0.005645] CPU features: detected: LSE atomic instructions522worker2 # [ 0.005652] CPU features: detected: LSE atomic instructions523client # [ 0.005713] CPU features: detected: Generic authentication (architected QARMA5 algorithm)524lb # [ 0.005366] smp: Bringing up secondary CPUs ...525worker1 # [ 0.005648] CPU features: detected: Privileged Access Never526worker2 # [ 0.005655] CPU features: detected: Privileged Access Never527lb # [ 0.005375] smp: Brought up 1 node, 1 CPU528client # [ 0.005718] CPU features: detected: RCpc load-acquire (LDAPR)529worker2 # [ 0.005658] CPU features: detected: PMUv3530worker1 # [ 0.005651] CPU features: detected: PMUv3531lb # [ 0.005412] SMP: Total of 1 processors activated.532client # [ 0.005721] CPU features: detected: LSE atomic instructions533worker1 # [ 0.005653] CPU features: detected: RAS Extension Support534worker2 # [ 0.005661] CPU features: detected: RAS Extension Support535lb # [ 0.005419] CPU: All CPU(s) started at EL1536client # [ 0.005725] CPU features: detected: Privileged Access Never537worker1 # [ 0.005656] CPU features: detected: RASv1p1 Extension Support538worker2 # [ 0.005664] CPU features: detected: RASv1p1 Extension Support539lb # [ 0.005434] CPU features: detected: Branch Target Identification540client # [ 0.005727] CPU features: detected: PMUv3541worker1 # [ 0.005659] CPU features: detected: Random Number Generator542worker2 # [ 0.005667] CPU features: detected: Random Number Generator543client # [ 0.005730] CPU features: detected: RAS Extension Support544lb # [ 0.005439] CPU features: detected: ARMv8.4 Translation Table Level545worker2 # [ 0.005669] CPU features: detected: Speculation barrier (SB)546worker1 # [ 0.005661] CPU features: detected: Speculation barrier (SB)547client # [ 0.005733] CPU features: detected: RASv1p1 Extension Support548lb # [ 0.005443] CPU features: detected: Instruction cache invalidation not required for I/D coherence549worker2 # [ 0.005673] CPU features: detected: Stage-2 Force Write-Back550worker1 # [ 0.005665] CPU features: detected: Stage-2 Force Write-Back551client # [ 0.005736] CPU features: detected: Random Number Generator552worker2 # [ 0.005676] CPU features: detected: TLB range maintenance instructions553client # [ 0.005739] CPU features: detected: Speculation barrier (SB)554worker1 # [ 0.005668] CPU features: detected: TLB range maintenance instructions555lb # [ 0.005447] CPU features: detected: Data cache clean to the PoU not required for I/D coherence556client # [ 0.005742] CPU features: detected: Stage-2 Force Write-Back557worker2 # [ 0.005680] CPU features: detected: Speculative Store Bypassing Safe (SSBS)558lb # [ 0.005451] CPU features: detected: Common not Private translations559worker1 # [ 0.005673] CPU features: detected: Speculative Store Bypassing Safe (SSBS)560client # [ 0.005745] CPU features: detected: TLB range maintenance instructions561lb # [ 0.005454] CPU features: detected: CRC32 instructions562worker1 # [ 0.005715] alternatives: applying system-wide alternatives563worker2 # [ 0.005724] alternatives: applying system-wide alternatives564client # [ 0.005750] CPU features: detected: Speculative Store Bypassing Safe (SSBS)565worker2 # [ 0.009186] CPU features: detected: BBM Level 2 without TLB conflict abort566worker1 # [ 0.009196] CPU features: detected: BBM Level 2 without TLB conflict abort567lb # [ 0.005457] CPU features: detected: Data cache clean to Point of Deep Persistence568client # [ 0.005793] alternatives: applying system-wide alternatives569lb # [ 0.005461] CPU features: detected: Data cache clean to Point of Persistence570client # [ 0.009281] CPU features: detected: BBM Level 2 without TLB conflict abort571lb # [ 0.005465] CPU features: detected: Data independent timing control (DIT)572worker1 # [ 0.009383] Memory: 1406536K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)573worker2 # [ 0.009369] Memory: 1406504K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)574lb # [ 0.005468] CPU features: detected: E0PD575lb # [ 0.005470] CPU features: detected: Enhanced Counter Virtualization576worker1 # [ 0.009810] devtmpfs: initialized577worker2 # [ 0.009835] devtmpfs: initialized578client # [ 0.009464] Memory: 1406536K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)579lb # [ 0.005473] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)580worker1 # [ 0.011668] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)581worker2 # [ 0.011849] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)582lb # [ 0.005477] CPU features: detected: Enhanced Virtualization Traps583client # [ 0.009923] devtmpfs: initialized584worker1 # [ 0.011693] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).585worker2 # [ 0.011878] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).586lb # [ 0.005480] CPU features: detected: Fine Grained Traps587client # [ 0.011972] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)588worker1 # [ 0.011924] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL589worker2 # [ 0.012110] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL590lb # [ 0.005484] CPU features: detected: Generic authentication (architected QARMA5 algorithm)591worker1 # [ 0.011930] 0 pages in range for non-PLT usage592worker2 # [ 0.012115] 0 pages in range for non-PLT usage593client # [ 0.011997] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).594worker1 # [ 0.011931] 508288 pages in range for PLT usage595worker2 # [ 0.012116] 508288 pages in range for PLT usage596client # [ 0.012229] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL597worker1 # [ 0.012060] pinctrl core: initialized pinctrl subsystem598worker2 # [ 0.012229] pinctrl core: initialized pinctrl subsystem599client # [ 0.012234] 0 pages in range for non-PLT usage600lb # [ 0.005489] CPU features: detected: RCpc load-acquire (LDAPR)601worker1 # [ 0.012895] DMI not present or invalid.602worker2 # [ 0.013111] DMI not present or invalid.603client # [ 0.012235] 508288 pages in range for PLT usage604lb # [ 0.005492] CPU features: detected: LSE atomic instructions605worker1 # [ 0.016298] NET: Registered PF_NETLINK/PF_ROUTE protocol family606worker2 # [ 0.016930] NET: Registered PF_NETLINK/PF_ROUTE protocol family607client # [ 0.012364] pinctrl core: initialized pinctrl subsystem608lb # [ 0.005495] CPU features: detected: Privileged Access Never609client # [ 0.013196] DMI not present or invalid.610worker2 # [ 0.019312] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations611lb # [ 0.005498] CPU features: detected: PMUv3612worker1 # [ 0.018855] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations613client # [ 0.016624] NET: Registered PF_NETLINK/PF_ROUTE protocol family614lb # [ 0.005501] CPU features: detected: RAS Extension Support615worker1 # [ 0.019051] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations616worker2 # [ 0.019510] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations617client # [ 0.019153] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations618lb # [ 0.005504] CPU features: detected: RASv1p1 Extension Support619worker1 # [ 0.019270] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations620worker2 # [ 0.019728] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations621lb # [ 0.005507] CPU features: detected: Random Number Generator622worker1 # [ 0.019299] audit: initializing netlink subsys (disabled)623worker2 # [ 0.019756] audit: initializing netlink subsys (disabled)624lb # [ 0.005509] CPU features: detected: Speculation barrier (SB)625client # [ 0.019375] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations626worker1 # [ 0.019999] thermal_sys: Registered thermal governor 'fair_share'627worker2 # [ 0.020422] thermal_sys: Registered thermal governor 'fair_share'628lb # [ 0.005513] CPU features: detected: Stage-2 Force Write-Back629client # [ 0.019596] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations630worker1 # [ 0.020001] thermal_sys: Registered thermal governor 'bang_bang'631worker2 # [ 0.020424] thermal_sys: Registered thermal governor 'bang_bang'632lb # [ 0.005516] CPU features: detected: TLB range maintenance instructions633client # [ 0.019634] audit: initializing netlink subsys (disabled)634worker2 # [ 0.020429] thermal_sys: Registered thermal governor 'step_wise'635worker1 # [ 0.020006] thermal_sys: Registered thermal governor 'step_wise'636lb # [ 0.005520] CPU features: detected: Speculative Store Bypassing Safe (SSBS)637client # [ 0.020258] thermal_sys: Registered thermal governor 'fair_share'638worker1 # [ 0.020012] thermal_sys: Registered thermal governor 'user_space'639worker2 # [ 0.020434] thermal_sys: Registered thermal governor 'user_space'640lb # [ 0.005565] alternatives: applying system-wide alternatives641client # [ 0.020260] thermal_sys: Registered thermal governor 'bang_bang'642worker1 # [ 0.020014] thermal_sys: Registered thermal governor 'power_allocator'643worker2 # [ 0.020437] thermal_sys: Registered thermal governor 'power_allocator'644lb # [ 0.009083] CPU features: detected: BBM Level 2 without TLB conflict abort645client # [ 0.020264] thermal_sys: Registered thermal governor 'step_wise'646worker1 # [ 0.020054] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1647worker2 # [ 0.020465] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1648client # [ 0.020270] thermal_sys: Registered thermal governor 'user_space'649worker2 # [ 0.020476] cpuidle: using governor ladder650worker1 # [ 0.020063] cpuidle: using governor ladder651client # [ 0.020273] thermal_sys: Registered thermal governor 'power_allocator'652lb # [ 0.009287] Memory: 1406524K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)653worker2 # [ 0.020482] cpuidle: using governor menu654worker1 # [ 0.020069] cpuidle: using governor menu655lb # [ 0.009754] devtmpfs: initialized656client # [ 0.020302] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1657worker1 # [ 0.020292] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.658worker2 # [ 0.020725] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.659lb # [ 0.011595] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)660client # [ 0.020312] cpuidle: using governor ladder661worker1 # [ 0.020311] ASID allocator initialised with 65536 entries662worker2 # [ 0.020744] ASID allocator initialised with 65536 entries663client # [ 0.020318] cpuidle: using governor menu664worker1 # [ 0.021558] Serial: AMBA PL011 UART driver665worker2 # [ 0.022043] Serial: AMBA PL011 UART driver666lb # [ 0.011618] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).667client # [ 0.020520] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.668lb # [ 0.011852] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL669worker2 # [ 0.027605] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1670worker1 # [ 0.026967] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1671client # [ 0.020538] ASID allocator initialised with 65536 entries672lb # [ 0.011857] 0 pages in range for non-PLT usage673worker2 # [ 0.027802] printk: console [ttyAMA0] enabled674worker1 # [ 0.027123] printk: console [ttyAMA0] enabled675client # [ 0.021749] Serial: AMBA PL011 UART driver676lb # [ 0.011858] 508288 pages in range for PLT usage677lb # [ 0.011991] pinctrl core: initialized pinctrl subsystem678client # [ 0.027188] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1679lb # [ 0.012860] DMI not present or invalid.680client # [ 0.027345] printk: console [ttyAMA0] enabled681lb # [ 0.016106] NET: Registered PF_NETLINK/PF_ROUTE protocol family682lb # [ 0.018554] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations683lb # [ 0.018776] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations684lb # [ 0.018995] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations685lb # [ 0.019033] audit: initializing netlink subsys (disabled)686lb # [ 0.019687] thermal_sys: Registered thermal governor 'fair_share'687lb # [ 0.019688] thermal_sys: Registered thermal governor 'bang_bang'688worker2 # [ 0.151894] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages689worker1 # [ 0.151150] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages690lb # [ 0.019692] thermal_sys: Registered thermal governor 'step_wise'691worker2 # [ 0.151917] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page692worker1 # [ 0.151175] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page693lb # [ 0.019698] thermal_sys: Registered thermal governor 'user_space'694worker1 # [ 0.151181] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages695worker2 # [ 0.151923] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages696client # [ 0.151763] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages697lb # [ 0.019700] thermal_sys: Registered thermal governor 'power_allocator'698worker1 # [ 0.151185] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page699worker2 # [ 0.151928] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page700client # [ 0.151788] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page701client # [ 0.151794] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages702worker2 # [ 0.151933] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages703worker2 # [ 0.151937] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page704client # [ 0.151798] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page705lb # [ 0.019737] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1706lb # [ 0.019745] cpuidle: using governor ladder707lb # [ 0.019751] cpuidle: using governor menu708worker1 # [ 0.151190] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages709worker1 # [ 0.151194] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page710lb # [ 0.019961] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.711worker1 # [ 0.151199] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages712worker2 # [ 0.151942] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages713client # [ 0.151802] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages714lb # [ 0.019980] ASID allocator initialised with 65536 entries715worker1 # [ 0.151203] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page716worker2 # [ 0.151946] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page717client # [ 0.151806] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page718lb # [ 0.021249] Serial: AMBA PL011 UART driver719client # [ 0.151811] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages720lb # [ 0.026633] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1721client # [ 0.151815] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page722lb # [ 0.026817] printk: console [ttyAMA0] enabled723worker2 # [ 0.159617] fbcon: Taking over console724worker1 # [ 0.158865] fbcon: Taking over console725worker1 # [ 0.158887] ACPI: Interpreter disabled.726client # [ 0.159951] fbcon: Taking over console727worker2 # [ 0.159638] ACPI: Interpreter disabled.728client # [ 0.159974] ACPI: Interpreter disabled.729worker1 # [ 0.167975] iommu: Default domain type: Translated730lb # [ 0.153291] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages731worker1 # [ 0.167992] iommu: DMA domain TLB invalidation policy: strict mode732worker2 # [ 0.168750] iommu: Default domain type: Translated733lb # [ 0.153319] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page734worker2 # [ 0.168763] iommu: DMA domain TLB invalidation policy: strict mode735worker2 # [ 0.170443] SCSI subsystem initialized736worker1 # [ 0.168400] SCSI subsystem initialized737client # [ 0.168112] iommu: Default domain type: Translated738lb # [ 0.153325] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages739client # [ 0.168124] iommu: DMA domain TLB invalidation policy: strict mode740lb # [ 0.153329] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page741client # [ 0.168549] SCSI subsystem initialized742worker2 # [ 0.171170] usbcore: registered new interface driver usbfs743worker1 # [ 0.170410] usbcore: registered new interface driver usbfs744worker1 # [ 0.170443] usbcore: registered new interface driver hub745worker2 # [ 0.171204] usbcore: registered new interface driver hub746lb # [ 0.153334] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages747worker1 # [ 0.170470] usbcore: registered new device driver usb748worker2 # [ 0.171231] usbcore: registered new device driver usb749client # [ 0.170588] usbcore: registered new interface driver usbfs750lb # [ 0.153338] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page751worker1 # [ 0.170771] pps_core: LinuxPPS API ver. 1 registered752worker2 # [ 0.171509] pps_core: LinuxPPS API ver. 1 registered753client # [ 0.170622] usbcore: registered new interface driver hub754lb # [ 0.153344] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages755client # [ 0.170647] usbcore: registered new device driver usb756worker1 # [ 0.170777] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>757worker2 # [ 0.171515] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>758lb # [ 0.153348] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page759client # [ 0.170983] pps_core: LinuxPPS API ver. 1 registered760worker1 # [ 0.170787] PTP clock support registered761worker2 # [ 0.171526] PTP clock support registered762worker1 # [ 0.170838] EDAC MC: Ver: 3.0.0763worker2 # [ 0.171586] EDAC MC: Ver: 3.0.0764client # [ 0.170989] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>765worker1 # [ 0.175457] scmi_core: SCMI protocol bus registered766client # [ 0.171000] PTP clock support registered767client # [ 0.171086] EDAC MC: Ver: 3.0.0768worker2 # [ 0.176204] scmi_core: SCMI protocol bus registered769lb # [ 0.160877] fbcon: Taking over console770lb # [ 0.160898] ACPI: Interpreter disabled.771worker2 # [ 0.177211] FPGA manager framework772worker1 # [ 0.176487] FPGA manager framework773client # [ 0.175739] scmi_core: SCMI protocol bus registered774worker2 # [ 0.178211] vgaarb: loaded775worker1 # [ 0.177541] vgaarb: loaded776client # [ 0.176789] FPGA manager framework777worker2 # [ 0.178857] clocksource: Switched to clocksource arch_sys_counter778worker1 # [ 0.178225] clocksource: Switched to clocksource arch_sys_counter779client # [ 0.177808] vgaarb: loaded780client # [ 0.178474] clocksource: Switched to clocksource arch_sys_counter781worker2 # [ 0.182718] VFS: Disk quotas dquot_6.6.0782worker2 # [ 0.182757] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)783client # [ 0.183338] VFS: Disk quotas dquot_6.6.0784worker1 # [ 0.184591] VFS: Disk quotas dquot_6.6.0785lb # [ 0.169626] iommu: Default domain type: Translated786client # [ 0.183380] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)787worker1 # [ 0.184666] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)788lb # [ 0.169642] iommu: DMA domain TLB invalidation policy: strict mode789worker2 # [ 0.186716] netfs: FS-Cache loaded790lb # [ 0.170052] SCSI subsystem initialized791worker1 # [ 0.186432] netfs: FS-Cache loaded792worker2 # [ 0.187321] pnp: PnP ACPI: disabled793worker1 # [ 0.186587] pnp: PnP ACPI: disabled794lb # [ 0.172072] usbcore: registered new interface driver usbfs795lb # [ 0.172105] usbcore: registered new interface driver hub796client # [ 0.187408] netfs: FS-Cache loaded797lb # [ 0.172123] usbcore: registered new device driver usb798client # [ 0.187570] pnp: PnP ACPI: disabled799lb # [ 0.172424] pps_core: LinuxPPS API ver. 1 registered800lb # [ 0.172430] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>801lb # [ 0.172441] PTP clock support registered802lb # [ 0.172496] EDAC MC: Ver: 3.0.0803lb # [ 0.177113] scmi_core: SCMI protocol bus registered804lb # [ 0.178147] FPGA manager framework805lb # [ 0.179153] vgaarb: loaded806worker2 # [ 0.191021] NET: Registered PF_INET protocol family807worker1 # [ 0.190677] NET: Registered PF_INET protocol family808lb # [ 0.179818] clocksource: Switched to clocksource arch_sys_counter809worker2 # [ 0.191310] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)810worker1 # [ 0.190961] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)811client # [ 0.191591] NET: Registered PF_INET protocol family812client # [ 0.192254] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)813lb # [ 0.184496] VFS: Disk quotas dquot_6.6.0814lb # [ 0.184567] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)815lb # [ 0.186308] netfs: FS-Cache loaded816lb # [ 0.186460] pnp: PnP ACPI: disabled817lb # [ 0.190493] NET: Registered PF_INET protocol family818lb # [ 0.190787] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)819worker2 # [ 0.226249] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)820worker2 # [ 0.226318] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)821worker2 # [ 0.226357] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)822worker2 # [ 0.226461] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)823worker2 # [ 0.226607] TCP: Hash tables configured (established 16384 bind 16384)824worker1 # [ 0.224132] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)825worker2 # [ 0.226718] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)826worker1 # [ 0.224202] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)827worker2 # [ 0.226761] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)828worker2 # [ 0.226800] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)829worker1 # [ 0.224272] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)830worker1 # [ 0.224374] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)831worker1 # [ 0.224517] TCP: Hash tables configured (established 16384 bind 16384)832worker1 # [ 0.224623] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)833client # [ 0.225722] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)834worker1 # [ 0.224666] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)835worker1 # [ 0.224706] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)836client # [ 0.225791] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)837worker1 # [ 0.224841] NET: Registered PF_UNIX/PF_LOCAL protocol family838client # [ 0.225829] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)839worker1 # [ 0.224930] NET: Registered PF_XDP protocol family840worker1 # [ 0.224951] PCI: CLS 0 bytes, default 64841client # [ 0.225933] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)842client # [ 0.226077] TCP: Hash tables configured (established 16384 bind 16384)843worker1 # [ 0.225237] Trying to unpack rootfs image as initramfs...844client # [ 0.226188] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)845client # [ 0.226235] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)846worker1 # [ 0.240466] kvm [1]: HYP mode not available847client # [ 0.226274] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)848worker2 # [ 0.234282] NET: Registered PF_UNIX/PF_LOCAL protocol family849client # [ 0.226378] NET: Registered PF_UNIX/PF_LOCAL protocol family850worker2 # [ 0.234323] NET: Registered PF_XDP protocol family851worker2 # [ 0.234344] PCI: CLS 0 bytes, default 64852client # [ 0.226406] NET: Registered PF_XDP protocol family853worker2 # [ 0.234607] Trying to unpack rootfs image as initramfs...854client # [ 0.226426] PCI: CLS 0 bytes, default 64855client # [ 0.226601] Trying to unpack rootfs image as initramfs...856worker2 # [ 0.245300] kvm [1]: HYP mode not available857lb # [ 0.224423] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)858lb # [ 0.224492] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)859lb # [ 0.224529] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)860lb # [ 0.224631] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)861lb # [ 0.224774] TCP: Hash tables configured (established 16384 bind 16384)862lb # [ 0.224876] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)863client # [ 0.244452] kvm [1]: HYP mode not available864lb # [ 0.224921] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)865lb # [ 0.224962] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)866lb # [ 0.225060] NET: Registered PF_UNIX/PF_LOCAL protocol family867lb # [ 0.225086] NET: Registered PF_XDP protocol family868lb # [ 0.225107] PCI: CLS 0 bytes, default 64869lb # [ 0.225441] Trying to unpack rootfs image as initramfs...870lb # [ 0.241469] kvm [1]: HYP mode not available871worker1 # [ 0.463982] Initialise system trusted keyrings872worker1 # [ 0.464823] workingset: timestamp_bits=42 max_order=19 bucket_order=0873client # [ 0.468776] Initialise system trusted keyrings874client # [ 0.469549] workingset: timestamp_bits=42 max_order=19 bucket_order=0875worker2 # [ 0.471606] Initialise system trusted keyrings876worker2 # [ 0.472383] workingset: timestamp_bits=42 max_order=19 bucket_order=0877client # [ 0.470856] squashfs: version 4.0 (2009/01/31) Phillip Lougher878worker2 # [ 0.473730] squashfs: version 4.0 (2009/01/31) Phillip Lougher879worker2 # [ 0.474455] 9p: Installing v9fs 9p2000 file system support880client # [ 0.471652] 9p: Installing v9fs 9p2000 file system support881lb # [ 0.465570] Initialise system trusted keyrings882lb # [ 0.466337] workingset: timestamp_bits=42 max_order=19 bucket_order=0883lb # [ 0.467607] squashfs: version 4.0 (2009/01/31) Phillip Lougher884worker1 # [ 0.474659] squashfs: version 4.0 (2009/01/31) Phillip Lougher885worker1 # [ 0.474779] 9p: Installing v9fs 9p2000 file system support886lb # [ 0.468383] 9p: Installing v9fs 9p2000 file system support887worker1 # [ 0.488142] Key type asymmetric registered888worker1 # [ 0.488155] Asymmetric key parser 'x509' registered889worker1 # [ 0.488234] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)890client # [ 0.495743] Key type asymmetric registered891client # [ 0.495784] Asymmetric key parser 'x509' registered892worker1 # [ 0.498315] io scheduler mq-deadline registered893client # [ 0.495884] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)894worker1 # [ 0.498351] io scheduler kyber registered895client # [ 0.498008] io scheduler mq-deadline registered896client # [ 0.498020] io scheduler kyber registered897worker2 # [ 0.503260] Key type asymmetric registered898worker2 # [ 0.503298] Asymmetric key parser 'x509' registered899worker2 # [ 0.503387] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)900worker2 # [ 0.505447] io scheduler mq-deadline registered901worker2 # [ 0.505461] io scheduler kyber registered902lb # [ 0.496452] Key type asymmetric registered903client # [ 0.511832] pl061_gpio 9030000.pl061: PL061 GPIO chip registered904lb # [ 0.496493] Asymmetric key parser 'x509' registered905lb # [ 0.496584] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)906lb # [ 0.498709] io scheduler mq-deadline registered907lb # [ 0.498724] io scheduler kyber registered908worker2 # [ 0.515122] pl061_gpio 9030000.pl061: PL061 GPIO chip registered909worker2 # [ 0.516706] ledtrig-cpu: registered to indicate activity on CPUs910worker1 # [ 0.510464] pl061_gpio 9030000.pl061: PL061 GPIO chip registered911worker1 # [ 0.511196] ledtrig-cpu: registered to indicate activity on CPUs912worker2 # [ 0.517206] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:913client # [ 0.513198] ledtrig-cpu: registered to indicate activity on CPUs914worker1 # [ 0.511578] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:915worker2 # [ 0.517227] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000916client # [ 0.513583] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:917worker1 # [ 0.511598] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000918worker2 # [ 0.517241] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000919client # [ 0.513602] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000920worker1 # [ 0.511611] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000921worker2 # [ 0.517249] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000922client # [ 0.513614] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000923worker1 # [ 0.511620] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000924worker2 # [ 0.517281] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits925client # [ 0.513623] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000926worker1 # [ 0.511644] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits927worker2 # [ 0.517310] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]928client # [ 0.513646] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits929worker1 # [ 0.511677] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]930worker2 # [ 0.517399] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00931client # [ 0.513670] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]932worker1 # [ 0.511770] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00933worker2 # [ 0.517408] pci_bus 0000:00: root bus resource [bus 00-ff]934client # [ 0.513758] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00935worker2 # [ 0.517415] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]936worker1 # [ 0.511780] pci_bus 0000:00: root bus resource [bus 00-ff]937client # [ 0.513768] pci_bus 0000:00: root bus resource [bus 00-ff]938worker1 # [ 0.511787] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]939worker2 # [ 0.517420] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]940lb # [ 0.512333] pl061_gpio 9030000.pl061: PL061 GPIO chip registered941client # [ 0.513775] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]942worker1 # [ 0.511793] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]943worker2 # [ 0.517425] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]944client # [ 0.513780] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]945worker1 # [ 0.511798] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]946worker2 # [ 0.517498] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint947client # [ 0.513785] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]948worker1 # [ 0.511895] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint949lb # [ 0.513852] ledtrig-cpu: registered to indicate activity on CPUs950worker2 # [ 0.517965] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint951client # [ 0.513906] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint952worker1 # [ 0.512397] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint953worker2 # [ 0.518153] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]954lb # [ 0.514298] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:955client # [ 0.514354] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint956worker1 # [ 0.512583] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]957worker2 # [ 0.518170] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]958client # [ 0.514574] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]959worker1 # [ 0.512601] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]960lb # [ 0.514318] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000961worker2 # [ 0.518200] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]962client # [ 0.514591] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]963worker1 # [ 0.512630] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]964lb # [ 0.514345] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000965worker2 # [ 0.518217] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]966client # [ 0.514621] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]967worker1 # [ 0.512646] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]968client # [ 0.514637] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]969lb # [ 0.514353] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000970worker2 # [ 0.518683] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint971worker1 # [ 0.513102] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint972client # [ 0.515137] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint973worker1 # [ 0.513286] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]974lb # [ 0.514382] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits975client # [ 0.515318] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]976lb # [ 0.514406] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]977worker1 # [ 0.513302] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]978client # [ 0.515334] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]979worker1 # [ 0.513331] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]980lb # [ 0.514494] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00981worker1 # [ 0.513786] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint982client # [ 0.515363] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]983client # [ 0.515834] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint984client # [ 0.516016] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]985lb # [ 0.514503] pci_bus 0000:00: root bus resource [bus 00-ff]986lb # [ 0.514509] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]987worker1 # [ 0.513967] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]988worker1 # [ 0.513983] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]989lb # [ 0.514515] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]990client # [ 0.516031] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]991worker1 # [ 0.514012] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]992lb # [ 0.514520] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]993client # [ 0.516061] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]994worker1 # [ 0.514497] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint995lb # [ 0.514616] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint996client # [ 0.516509] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint997worker1 # [ 0.514678] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]998client # [ 0.516694] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]999worker1 # [ 0.514694] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1000lb # [ 0.515068] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1001client # [ 0.516710] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1002lb # [ 0.515264] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1003worker1 # [ 0.514726] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1004lb # [ 0.515281] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1005client # [ 0.516745] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1006worker1 # [ 0.515176] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1007lb # [ 0.515310] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1008client # [ 0.517211] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1009worker1 # [ 0.515366] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1010lb # [ 0.515326] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1011client # [ 0.517393] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1012worker1 # [ 0.515382] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1013client # [ 0.517409] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1014lb # [ 0.515793] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1015worker1 # [ 0.515411] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1016client # [ 0.517438] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1017worker1 # [ 0.515898] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1018client # [ 0.517889] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1019worker1 # [ 0.516087] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1020client # [ 0.518070] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1021worker1 # [ 0.516103] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1022client # [ 0.518086] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1023worker2 # [ 0.542994] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1024worker1 # [ 0.516132] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025lb # [ 0.533034] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1026client # [ 0.518115] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1027worker2 # [ 0.543041] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1028lb # [ 0.533055] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1029worker1 # [ 0.516623] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1030worker2 # [ 0.543072] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1031worker1 # [ 0.516804] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1032lb # [ 0.533085] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1033worker1 # [ 0.516824] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1034worker2 # [ 0.543646] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1035lb # [ 0.533608] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1036worker1 # [ 0.516854] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1037worker2 # [ 0.543839] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1038lb # [ 0.533792] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1039worker1 # [ 0.516870] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1040worker2 # [ 0.543856] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1041lb # [ 0.533808] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1042worker1 # [ 0.517321] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1043lb # [ 0.533837] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1044worker2 # [ 0.543885] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1045worker1 # [ 0.517507] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1046lb # [ 0.534294] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1047worker2 # [ 0.544368] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1048worker1 # [ 0.517537] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1049lb # [ 0.534475] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1050worker2 # [ 0.544552] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1051worker2 # [ 0.544568] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1052worker1 # [ 0.517999] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1053lb # [ 0.534491] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1054lb # [ 0.534523] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1055worker1 # [ 0.518185] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1056worker2 # [ 0.544601] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1057worker1 # [ 0.518215] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1058lb # [ 0.534984] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1059client # [ 0.554731] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1060worker2 # [ 0.545097] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1061lb # [ 0.535165] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1062client # [ 0.554950] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1063worker2 # [ 0.545280] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1064lb # [ 0.535180] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1065client # [ 0.554968] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1066worker2 # [ 0.545296] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1067lb # [ 0.535209] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1068client # [ 0.554997] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1069worker2 # [ 0.545325] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1070client # [ 0.555014] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1071lb # [ 0.535665] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1072worker2 # [ 0.545794] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1073client # [ 0.555641] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1074worker2 # [ 0.545978] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1075client # [ 0.555830] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1076worker2 # [ 0.545993] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1077client # [ 0.555860] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1078worker2 # [ 0.546022] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1079client # [ 0.556315] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1080client # [ 0.556501] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1081worker2 # [ 0.546472] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1082worker2 # [ 0.546655] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1083client # [ 0.556530] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1084worker2 # [ 0.546671] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1085client # [ 0.556917] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1086worker2 # [ 0.546701] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1087client # [ 0.557097] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1088worker1 # [ 0.566834] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1089worker2 # [ 0.546717] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1090client # [ 0.557342] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1091worker1 # [ 0.567054] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1092client # [ 0.557586] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1093worker2 # [ 0.547223] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1094worker1 # [ 0.567315] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1095client # [ 0.557603] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1096worker2 # [ 0.547411] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1097worker1 # [ 0.567652] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1098client # [ 0.557633] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1099worker2 # [ 0.547441] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100worker1 # [ 0.567671] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1101worker2 # [ 0.547906] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1102client # [ 0.558100] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1103worker1 # [ 0.567701] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1104client # [ 0.558284] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1105worker2 # [ 0.548112] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1106worker1 # [ 0.568166] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1107client # [ 0.558299] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1108worker2 # [ 0.548143] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1109worker1 # [ 0.568360] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1110client # [ 0.558328] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1111worker1 # [ 0.568376] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1112worker2 # [ 0.548587] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1113worker2 # [ 0.548771] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1114worker1 # [ 0.568405] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1115worker1 # [ 0.569022] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1116worker2 # [ 0.549045] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1117lb # [ 0.555997] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1118worker2 # [ 0.549371] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1119worker1 # [ 0.569035] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1120lb # [ 0.556039] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1121worker2 # [ 0.549390] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1122worker1 # [ 0.569041] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1123worker2 # [ 0.549420] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1124lb # [ 0.556069] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1125worker2 # [ 0.549889] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1126worker1 # [ 0.569087] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1127lb # [ 0.556652] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1128lb # [ 0.556845] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1129worker1 # [ 0.569135] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1130worker2 # [ 0.550091] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1131lb # [ 0.556861] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1132worker2 # [ 0.550107] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1133worker1 # [ 0.569183] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1134lb # [ 0.556891] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1135worker2 # [ 0.550137] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1136worker1 # [ 0.569230] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1137lb # [ 0.556907] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1138worker2 # [ 0.550725] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1139worker1 # [ 0.569277] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1140lb # [ 0.557482] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1141worker2 # [ 0.550737] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1142lb # [ 0.557671] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1143worker1 # [ 0.569331] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1144worker2 # [ 0.550742] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1145lb # [ 0.557701] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1146worker1 # [ 0.569380] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1147worker2 # [ 0.550789] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1148lb # [ 0.558168] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1149client # [ 0.583223] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1150worker1 # [ 0.569430] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1151lb # [ 0.558364] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1152worker2 # [ 0.550849] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1153client # [ 0.583264] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1154worker1 # [ 0.569477] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1155lb # [ 0.558393] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1156client # [ 0.583271] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1157worker1 # [ 0.569570] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1158lb # [ 0.558792] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1159client # [ 0.583331] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1160worker1 # [ 0.569617] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1161lb # [ 0.558975] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1162worker1 # [ 0.569638] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1163client # [ 0.583381] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1164lb # [ 0.559228] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1165worker1 # [ 0.569660] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1166lb # [ 0.559545] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1167client # [ 0.583429] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1168worker1 # [ 0.569682] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1169lb # [ 0.559563] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1170client # [ 0.583477] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1171lb # [ 0.559592] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1172worker1 # [ 0.569707] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1173client # [ 0.583526] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1174worker1 # [ 0.569730] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1175lb # [ 0.560098] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1176lb # [ 0.560284] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1177client # [ 0.583574] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1178worker1 # [ 0.569752] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1179lb # [ 0.560300] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1180worker1 # [ 0.569775] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1181client # [ 0.583623] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1182lb # [ 0.560329] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1183worker1 # [ 0.569797] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1184client # [ 0.583671] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1185lb # [ 0.560959] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1186worker1 # [ 0.569819] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1187client # [ 0.583720] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1188lb # [ 0.560970] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1189worker1 # [ 0.569842] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1190worker2 # [ 0.595028] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1191worker2 # [ 0.595113] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1192worker1 # [ 0.569865] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1193client # [ 0.583898] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1194lb # [ 0.560976] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1195worker2 # [ 0.595163] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1196client # [ 0.583948] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1197worker1 # [ 0.569886] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1198lb # [ 0.561023] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1199worker2 # [ 0.595214] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1200client # [ 0.583969] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1201worker1 # [ 0.569908] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1202lb # [ 0.561071] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1203client # [ 0.583991] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1204worker1 # [ 0.569929] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1205worker2 # [ 0.595265] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1206lb # [ 0.561123] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1207worker1 # [ 0.569950] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1208client # [ 0.584012] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1209worker2 # [ 0.595315] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1210worker1 # [ 0.569972] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1211client # [ 0.584043] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1212lb # [ 0.561170] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1213worker2 # [ 0.595365] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1214worker1 # [ 0.569993] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1215client # [ 0.584065] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1216lb # [ 0.561218] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1217worker1 # [ 0.570015] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1218worker2 # [ 0.595469] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1219client # [ 0.584089] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1220lb # [ 0.561266] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1221worker1 # [ 0.570037] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1222worker2 # [ 0.595579] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1223client # [ 0.584111] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1224worker1 # [ 0.570059] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1225lb # [ 0.561314] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1226client # [ 0.584133] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1227worker1 # [ 0.570088] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1228worker2 # [ 0.595604] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1229worker2 # [ 0.595629] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1230worker2 # [ 0.595651] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1231lb # [ 0.561361] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1232worker1 # [ 0.570098] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1233lb # [ 0.561421] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1234worker2 # [ 0.595678] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1235client # [ 0.584154] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1236client # [ 0.584178] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1237client # [ 0.584201] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1238worker1 # [ 0.570102] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1239lb # [ 0.561520] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1240lb # [ 0.561566] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1241client # [ 0.584222] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1242worker2 # [ 0.595700] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1243client # [ 0.584243] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1244lb # [ 0.561588] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1245client # [ 0.584267] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1246worker2 # [ 0.595723] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1247lb # [ 0.561609] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1248client # [ 0.584291] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1249worker2 # [ 0.595745] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1250worker1 # [ 0.610473] pci 0000:00:0a.0: enabling device (0000 -> 0002)1251lb # [ 0.561631] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1252client # [ 0.584313] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1253worker2 # [ 0.595768] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1254client # [ 0.584334] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1255lb # [ 0.561657] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1256worker2 # [ 0.595791] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1257client # [ 0.584356] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1258worker2 # [ 0.595815] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1259lb # [ 0.561678] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1260client # [ 0.584380] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1261lb # [ 0.561701] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1262worker2 # [ 0.595838] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1263client # [ 0.584402] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1264lb # [ 0.561723] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1265worker2 # [ 0.595878] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1266client # [ 0.584431] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1267lb # [ 0.561745] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1268worker2 # [ 0.595900] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1269client # [ 0.584440] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1270lb # [ 0.561767] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1271worker2 # [ 0.595921] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1272client # [ 0.584445] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1273lb # [ 0.561790] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1274worker2 # [ 0.595943] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1275client # [ 0.585628] pci 0000:00:0a.0: enabling device (0000 -> 0002)1276lb # [ 0.561812] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1277worker2 # [ 0.595965] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1278lb # [ 0.561836] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1279worker2 # [ 0.595986] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1280lb # [ 0.561857] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1281worker2 # [ 0.596009] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1282lb # [ 0.561878] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1283worker2 # [ 0.596031] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1284lb # [ 0.561899] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1285worker2 # [ 0.596054] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1286lb # [ 0.561921] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1287worker2 # [ 0.596087] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1288lb # [ 0.561942] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1289worker2 # [ 0.596097] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1290lb # [ 0.561964] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1291worker2 # [ 0.596102] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1292lb # [ 0.561986] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1293worker2 # [ 0.597341] pci 0000:00:0a.0: enabling device (0000 -> 0002)1294lb # [ 0.562008] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1295lb # [ 0.562038] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1296lb # [ 0.562047] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1297lb # [ 0.562052] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1298lb # [ 0.563226] pci 0000:00:0a.0: enabling device (0000 -> 0002)1299worker1 # [ 0.637725] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1300client # [ 0.642428] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1301worker1 # [ 0.647965] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1302worker2 # [ 0.650684] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1303worker1 # [ 0.652130] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1304client # [ 0.652093] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1305lb # [ 0.640473] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1306lb # [ 0.642941] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1307worker2 # [ 0.660470] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1308client # [ 0.659288] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1309client # [ 0.661641] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1310worker2 # [ 0.664769] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1311lb # [ 0.653570] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1312worker1 # [ 0.662857] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1313lb # [ 0.655813] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1314worker1 # [ 0.665154] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1315worker2 # [ 0.671134] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1316worker1 # [ 0.667381] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1317worker2 # [ 0.673542] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1318worker1 # [ 0.669817] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1319client # [ 0.672365] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1320client # [ 0.675503] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1321client # [ 0.678002] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1322lb # [ 0.666633] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1323worker1 # [ 0.683947] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1324worker2 # [ 0.684262] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1325worker1 # [ 0.687269] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1326client # [ 0.688166] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1327worker1 # [ 0.689284] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1328lb # [ 0.676908] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1329lb # [ 0.679285] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1330client # [ 0.694701] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1331client # [ 0.696832] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1332worker1 # [ 0.700462] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1333worker2 # [ 0.695209] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1334worker2 # [ 0.698594] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1335worker2 # [ 0.701195] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1336lb # [ 0.686061] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1337client # [ 0.708356] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1338worker2 # [ 0.711247] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1339lb # [ 0.696140] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1340lb # [ 0.698420] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1341worker1 # [ 0.715620] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1342worker2 # [ 0.715835] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1343worker1 # [ 0.718611] msm_serial: driver initialized1344worker1 # [ 0.718772] SuperH (H)SCI(F) driver initialized1345worker1 # [ 0.718825] STM32 USART driver initialized1346client # [ 0.721589] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1347lb # [ 0.710107] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1348client # [ 0.724457] msm_serial: driver initialized1349client # [ 0.724602] SuperH (H)SCI(F) driver initialized1350client # [ 0.724655] STM32 USART driver initialized1351worker2 # [ 0.729960] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1352worker2 # [ 0.733003] msm_serial: driver initialized1353worker2 # [ 0.733154] SuperH (H)SCI(F) driver initialized1354worker2 # [ 0.733211] STM32 USART driver initialized1355lb # [ 0.723591] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1356lb # [ 0.727271] msm_serial: driver initialized1357lb # [ 0.727441] SuperH (H)SCI(F) driver initialized1358lb # [ 0.727498] STM32 USART driver initialized1359worker1 # [ 0.758720] loop: module loaded1360worker1 # [ 0.758963] virtio_blk virtio5: 1/0/0 default/read/poll queues1361worker1 # [ 0.759840] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1362client # [ 0.767621] loop: module loaded1363worker1 # [ 0.767101] megasas: 07.734.00.00-rc11364worker1 # [ 0.767902] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1365client # [ 0.767870] virtio_blk virtio5: 1/0/0 default/read/poll queues1366client # [ 0.768747] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1367worker1 # [ 0.770053] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001368worker1 # [ 0.770093] Intel/Sharp Extended Query Table at 0x00311369worker2 # [ 0.773601] loop: module loaded1370worker2 # [ 0.773876] virtio_blk virtio5: 1/0/0 default/read/poll queues1371worker2 # [ 0.774725] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1372worker1 # [ 0.775063] Using buffer write method1373client # [ 0.775499] megasas: 07.734.00.00-rc11374worker1 # [ 0.775174] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1375client # [ 0.776276] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1376worker1 # [ 0.776966] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001377worker1 # [ 0.777009] Intel/Sharp Extended Query Table at 0x00311378lb # [ 0.765969] loop: module loaded1379lb # [ 0.766235] virtio_blk virtio5: 1/0/0 default/read/poll queues1380lb # [ 0.767227] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1381client # [ 0.780114] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001382client # [ 0.780148] Intel/Sharp Extended Query Table at 0x00311383client # [ 0.782245] Using buffer write method1384client # [ 0.782363] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1385worker2 # [ 0.787835] megasas: 07.734.00.00-rc11386worker1 # [ 0.794979] Using buffer write method1387worker2 # [ 0.788701] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1388worker1 # [ 0.795036] Concatenating MTD devices:1389worker1 # [ 0.795040] (0): "0.flash"1390worker1 # [ 0.795045] (1): "0.flash"1391worker2 # [ 0.791133] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001392worker1 # [ 0.795048] into device "0.flash"1393worker2 # [ 0.791173] Intel/Sharp Extended Query Table at 0x00311394worker2 # [ 0.793146] Using buffer write method1395worker2 # [ 0.793295] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1396client # [ 0.796434] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001397client # [ 0.796498] Intel/Sharp Extended Query Table at 0x00311398client # [ 0.798383] Using buffer write method1399client # [ 0.798427] Concatenating MTD devices:1400client # [ 0.798431] (0): "0.flash"1401client # [ 0.798436] (1): "0.flash"1402lb # [ 0.780755] megasas: 07.734.00.00-rc11403client # [ 0.798439] into device "0.flash"1404lb # [ 0.781669] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1405lb # [ 0.784031] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001406lb # [ 0.784072] Intel/Sharp Extended Query Table at 0x00311407worker2 # [ 0.803745] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001408lb # [ 0.785949] Using buffer write method1409worker2 # [ 0.803812] Intel/Sharp Extended Query Table at 0x00311410worker2 # [ 0.805708] Using buffer write method1411worker2 # [ 0.805759] Concatenating MTD devices:1412lb # [ 0.786093] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1413worker2 # [ 0.805764] (0): "0.flash"1414worker2 # [ 0.805769] (1): "0.flash"1415worker2 # [ 0.805772] into device "0.flash"1416lb # [ 0.800503] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001417lb # [ 0.800573] Intel/Sharp Extended Query Table at 0x00311418lb # [ 0.802368] Using buffer write method1419lb # [ 0.802417] Concatenating MTD devices:1420lb # [ 0.802422] (0): "0.flash"1421lb # [ 0.802427] (1): "0.flash"1422lb # [ 0.802430] into device "0.flash"1423worker1 # [ 1.080276] Freeing initrd memory: 26148K1424worker1 # [ 1.086907] tun: Universal TUN/TAP device driver, 1.61425client # [ 1.092676] Freeing initrd memory: 26160K1426worker1 # [ 1.091085] thunder_xcv, ver 1.01427worker1 # [ 1.091139] thunder_bgx, ver 1.01428worker1 # [ 1.091163] nicpf, ver 1.01429worker1 # [ 1.091720] e1000: Intel(R) PRO/1000 Network Driver1430worker1 # [ 1.091728] e1000: Copyright (c) 1999-2006 Intel Corporation.1431worker1 # [ 1.091752] e1000e: Intel(R) PRO/1000 Network Driver1432worker1 # [ 1.091761] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1433worker1 # [ 1.091795] igb: Intel(R) Gigabit Ethernet Network Driver1434worker1 # [ 1.091800] igb: Copyright (c) 2007-2014 Intel Corporation.1435worker1 # [ 1.091824] igbvf: Intel(R) Gigabit Virtual Function Network Driver1436client # [ 1.099411] tun: Universal TUN/TAP device driver, 1.61437worker1 # [ 1.091830] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1438worker1 # [ 1.091963] sky2: driver version 1.301439worker1 # [ 1.093601] usbcore: registered new interface driver usb-storage1440worker1 # [ 1.093745] usbcore: registered new interface driver usbserial_generic1441worker1 # [ 1.093760] usbserial: USB Serial support registered for generic1442worker1 # [ 1.095083] ehci-pci 0000:00:0a.0: EHCI Host Controller1443worker1 # [ 1.095119] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11444worker1 # [ 1.095397] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001445client # [ 1.103799] thunder_xcv, ver 1.01446client # [ 1.103855] thunder_bgx, ver 1.01447worker1 # [ 1.107099] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001448client # [ 1.103878] nicpf, ver 1.01449worker1 # [ 1.107509] hub 1-0:1.0: USB hub found1450client # [ 1.104469] e1000: Intel(R) PRO/1000 Network Driver1451worker1 # [ 1.107531] hub 1-0:1.0: 6 ports detected1452client # [ 1.104478] e1000: Copyright (c) 1999-2006 Intel Corporation.1453worker1 # [ 1.110013] hv_vmbus: registering driver hyperv_keyboard1454client # [ 1.104504] e1000e: Intel(R) PRO/1000 Network Driver1455lb # [ 1.095661] Freeing initrd memory: 26140K1456client # [ 1.104514] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1457client # [ 1.104545] igb: Intel(R) Gigabit Ethernet Network Driver1458client # [ 1.104551] igb: Copyright (c) 2007-2014 Intel Corporation.1459worker1 # [ 1.111696] rtc-pl031 9010000.pl031: registered as rtc01460client # [ 1.104576] igbvf: Intel(R) Gigabit Virtual Function Network Driver1461client # [ 1.104582] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1462worker1 # [ 1.111738] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T13:32:34 UTC (1789911154)1463client # [ 1.104725] sky2: driver version 1.301464worker2 # [ 1.113396] Freeing initrd memory: 26148K1465worker1 # [ 1.112061] i2c_dev: i2c /dev entries driver1466client # [ 1.106387] usbcore: registered new interface driver usb-storage1467lb # [ 1.102324] tun: Universal TUN/TAP device driver, 1.61468worker1 # [ 1.117300] sdhci: Secure Digital Host Controller Interface driver1469client # [ 1.114653] usbcore: registered new interface driver usbserial_generic1470worker1 # [ 1.117313] sdhci: Copyright(c) Pierre Ossman1471worker1 # [ 1.117586] Synopsys Designware Multimedia Card Interface Driver1472client # [ 1.114681] usbserial: USB Serial support registered for generic1473worker2 # [ 1.120714] tun: Universal TUN/TAP device driver, 1.61474worker1 # [ 1.117960] sdhci-pltfm: SDHCI platform and OF driver helper1475client # [ 1.115392] hv_vmbus: registering driver hyperv_keyboard1476client # [ 1.116861] ehci-pci 0000:00:0a.0: EHCI Host Controller1477client # [ 1.116899] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11478client # [ 1.117194] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001479lb # [ 1.106691] thunder_xcv, ver 1.01480lb # [ 1.106752] thunder_bgx, ver 1.01481lb # [ 1.106775] nicpf, ver 1.01482client # [ 1.122039] rtc-pl031 9010000.pl031: registered as rtc01483worker1 # [ 1.122437] hid: raw HID events driver (C) Jiri Kosina1484lb # [ 1.107356] e1000: Intel(R) PRO/1000 Network Driver1485worker1 # [ 1.122702] usbcore: registered new interface driver usbhid1486lb # [ 1.107364] e1000: Copyright (c) 1999-2006 Intel Corporation.1487client # [ 1.122077] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T13:32:34 UTC (1789911154)1488worker1 # [ 1.122708] usbhid: USB HID core driver1489lb # [ 1.107389] e1000e: Intel(R) PRO/1000 Network Driver1490lb # [ 1.107399] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1491worker2 # [ 1.125321] thunder_xcv, ver 1.01492client # [ 1.124271] i2c_dev: i2c /dev entries driver1493worker2 # [ 1.125386] thunder_bgx, ver 1.01494lb # [ 1.107428] igb: Intel(R) Gigabit Ethernet Network Driver1495worker1 # [ 1.125600] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1496worker2 # [ 1.125417] nicpf, ver 1.01497lb # [ 1.107433] igb: Copyright (c) 2007-2014 Intel Corporation.1498worker2 # [ 1.126014] e1000: Intel(R) PRO/1000 Network Driver1499worker2 # [ 1.126021] e1000: Copyright (c) 1999-2006 Intel Corporation.1500lb # [ 1.107457] igbvf: Intel(R) Gigabit Virtual Function Network Driver1501worker2 # [ 1.126046] e1000e: Intel(R) PRO/1000 Network Driver1502worker1 # [ 1.128319] drop_monitor: Initializing network drop monitor service1503lb # [ 1.107463] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1504lb # [ 1.107598] sky2: driver version 1.301505worker1 # [ 1.128498] NET: Registered PF_INET6 protocol family1506worker2 # [ 1.126056] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1507worker2 # [ 1.126090] igb: Intel(R) Gigabit Ethernet Network Driver1508worker1 # [ 1.130739] Segment Routing with IPv61509worker2 # [ 1.126096] igb: Copyright (c) 2007-2014 Intel Corporation.1510worker1 # [ 1.130759] In-situ OAM (IOAM) with IPv61511worker2 # [ 1.126123] igbvf: Intel(R) Gigabit Virtual Function Network Driver1512client # [ 1.127740] sdhci: Secure Digital Host Controller Interface driver1513client # [ 1.127752] sdhci: Copyright(c) Pierre Ossman1514worker2 # [ 1.126129] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1515worker1 # [ 1.130791] NET: Registered PF_PACKET protocol family1516worker2 # [ 1.126271] sky2: driver version 1.301517client # [ 1.128024] Synopsys Designware Multimedia Card Interface Driver1518client # [ 1.128401] sdhci-pltfm: SDHCI platform and OF driver helper1519worker1 # [ 1.132604] 9pnet: Installing 9P2000 support1520lb # [ 1.116657] usbcore: registered new interface driver usb-storage1521client # [ 1.130224] hid: raw HID events driver (C) Jiri Kosina1522client # [ 1.133762] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001523lb # [ 1.116772] usbcore: registered new interface driver usbserial_generic1524lb # [ 1.116786] usbserial: USB Serial support registered for generic1525client # [ 1.134912] hub 1-0:1.0: USB hub found1526client # [ 1.134933] hub 1-0:1.0: 6 ports detected1527lb # [ 1.117451] hv_vmbus: registering driver hyperv_keyboard1528lb # [ 1.119007] ehci-pci 0000:00:0a.0: EHCI Host Controller1529worker2 # [ 1.135355] usbcore: registered new interface driver usb-storage1530lb # [ 1.119048] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11531worker1 # [ 1.135521] Key type dns_resolver registered1532worker2 # [ 1.135424] usbcore: registered new interface driver usbserial_generic1533lb # [ 1.119245] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001534client # [ 1.136329] usbcore: registered new interface driver usbhid1535worker2 # [ 1.135442] usbserial: USB Serial support registered for generic1536client # [ 1.136341] usbhid: USB HID core driver1537worker2 # [ 1.136051] hv_vmbus: registering driver hyperv_keyboard1538worker2 # [ 1.137539] ehci-pci 0000:00:0a.0: EHCI Host Controller1539lb # [ 1.124701] rtc-pl031 9010000.pl031: registered as rtc01540worker2 # [ 1.137577] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11541worker2 # [ 1.137923] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001542worker1 # [ 1.141948] registered taskstats version 11543lb # [ 1.124737] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T13:32:34 UTC (1789911154)1544client # [ 1.138776] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1545lb # [ 1.125194] i2c_dev: i2c /dev entries driver1546worker1 # [ 1.142119] Loading compiled-in X.509 certificates1547client # [ 1.140409] drop_monitor: Initializing network drop monitor service1548client # [ 1.140572] NET: Registered PF_INET6 protocol family1549lb # [ 1.128479] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001550worker2 # [ 1.143716] rtc-pl031 9010000.pl031: registered as rtc01551lb # [ 1.128822] hub 1-0:1.0: USB hub found1552client # [ 1.143756] Segment Routing with IPv61553lb # [ 1.128845] hub 1-0:1.0: 6 ports detected1554worker2 # [ 1.143753] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T13:32:34 UTC (1789911154)1555client # [ 1.143778] In-situ OAM (IOAM) with IPv61556worker2 # [ 1.144226] i2c_dev: i2c /dev entries driver1557client # [ 1.143824] NET: Registered PF_PACKET protocol family1558worker2 # [ 1.147437] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001559worker2 # [ 1.147801] hub 1-0:1.0: USB hub found1560client # [ 1.145671] 9pnet: Installing 9P2000 support1561worker2 # [ 1.149129] hub 1-0:1.0: 6 ports detected1562lb # [ 1.133225] sdhci: Secure Digital Host Controller Interface driver1563worker1 # [ 1.150997] Demotion targets for Node 0: null1564lb # [ 1.133238] sdhci: Copyright(c) Pierre Ossman1565client # [ 1.148738] Key type dns_resolver registered1566worker1 # [ 1.151142] Key type .fscrypt registered1567lb # [ 1.133534] Synopsys Designware Multimedia Card Interface Driver1568worker1 # [ 1.151149] Key type fscrypt-provisioning registered1569lb # [ 1.133918] sdhci-pltfm: SDHCI platform and OF driver helper1570worker1 # [ 1.151261] ima: No TPM chip found, activating TPM-bypass!1571lb # [ 1.135755] hid: raw HID events driver (C) Jiri Kosina1572worker1 # [ 1.151281] ima: Allocated hash algorithm: sha11573worker2 # [ 1.152178] sdhci: Secure Digital Host Controller Interface driver1574worker1 # [ 1.151307] ima: No architecture policies found1575lb # [ 1.139358] usbcore: registered new interface driver usbhid1576worker2 # [ 1.152191] sdhci: Copyright(c) Pierre Ossman1577lb # [ 1.139370] usbhid: USB HID core driver1578worker2 # [ 1.152540] Synopsys Designware Multimedia Card Interface Driver1579worker1 # [ 1.155833] input: gpio-keys as /devices/platform/gpio-keys/input/input01580worker2 # [ 1.152933] sdhci-pltfm: SDHCI platform and OF driver helper1581worker2 # [ 1.154785] hid: raw HID events driver (C) Jiri Kosina1582worker2 # [ 1.158446] usbcore: registered new interface driver usbhid1583worker2 # [ 1.158461] usbhid: USB HID core driver1584lb # [ 1.141785] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1585lb # [ 1.143592] drop_monitor: Initializing network drop monitor service1586client # [ 1.155292] registered taskstats version 11587lb # [ 1.143795] NET: Registered PF_INET6 protocol family1588client # [ 1.155462] Loading compiled-in X.509 certificates1589lb # [ 1.146962] Segment Routing with IPv61590lb # [ 1.146985] In-situ OAM (IOAM) with IPv61591worker2 # [ 1.160902] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1592lb # [ 1.147024] NET: Registered PF_PACKET protocol family1593worker2 # [ 1.162516] drop_monitor: Initializing network drop monitor service1594worker2 # [ 1.162699] NET: Registered PF_INET6 protocol family1595worker2 # [ 1.165934] Segment Routing with IPv61596worker2 # [ 1.165974] In-situ OAM (IOAM) with IPv61597worker2 # [ 1.166008] NET: Registered PF_PACKET protocol family1598client # [ 1.164811] Demotion targets for Node 0: null1599lb # [ 1.148732] 9pnet: Installing 9P2000 support1600client # [ 1.164965] Key type .fscrypt registered1601lb # [ 1.151183] Key type dns_resolver registered1602client # [ 1.164971] Key type fscrypt-provisioning registered1603client # [ 1.165086] ima: No TPM chip found, activating TPM-bypass!1604client # [ 1.165107] ima: Allocated hash algorithm: sha11605client # [ 1.165133] ima: No architecture policies found1606worker2 # [ 1.167730] 9pnet: Installing 9P2000 support1607worker2 # [ 1.170412] Key type dns_resolver registered1608client # [ 1.169534] input: gpio-keys as /devices/platform/gpio-keys/input/input01609lb # [ 1.158404] registered taskstats version 11610lb # [ 1.158565] Loading compiled-in X.509 certificates1611worker2 # [ 1.178014] registered taskstats version 11612worker2 # [ 1.178207] Loading compiled-in X.509 certificates1613worker1 # [ 1.182004] clk: Disabling unused clocks1614lb # [ 1.167638] Demotion targets for Node 0: null1615worker1 # [ 1.182051] PM: genpd: Disabling unused power domains1616lb # [ 1.167792] Key type .fscrypt registered1617lb # [ 1.167798] Key type fscrypt-provisioning registered1618lb # [ 1.169694] ima: No TPM chip found, activating TPM-bypass!1619lb # [ 1.169724] ima: Allocated hash algorithm: sha11620lb # [ 1.169751] ima: No architecture policies found1621worker1 # [ 1.186627] Freeing unused kernel memory: 4736K1622worker2 # [ 1.187451] Demotion targets for Node 0: null1623lb # [ 1.172312] input: gpio-keys as /devices/platform/gpio-keys/input/input01624worker1 # [ 1.186890] Run /init as init process1625worker2 # [ 1.187608] Key type .fscrypt registered1626worker2 # [ 1.187617] Key type fscrypt-provisioning registered1627worker2 # [ 1.187734] ima: No TPM chip found, activating TPM-bypass!1628worker2 # [ 1.187757] ima: Allocated hash algorithm: sha11629worker2 # [ 1.187785] ima: No architecture policies found1630worker2 # [ 1.192147] input: gpio-keys as /devices/platform/gpio-keys/input/input01631client # [ 1.195666] clk: Disabling unused clocks1632client # [ 1.195707] PM: genpd: Disabling unused power domains1633client # [ 1.200190] Freeing unused kernel memory: 4736K1634client # [ 1.200466] Run /init as init process1635worker1 # [ 1.204945] systemd[1]: Successfully made /usr/ read-only.1636lb # [ 1.199025] clk: Disabling unused clocks1637lb # [ 1.199074] PM: genpd: Disabling unused power domains1638client # [ 1.217204] systemd[1]: Successfully made /usr/ read-only.1639lb # [ 1.203671] Freeing unused kernel memory: 4736K1640lb # [ 1.204563] Run /init as init process1641worker2 # [ 1.219043] clk: Disabling unused clocks1642worker2 # [ 1.219078] PM: genpd: Disabling unused power domains1643worker2 # [ 1.223581] Freeing unused kernel memory: 4736K1644worker2 # [ 1.223828] Run /init as init process1645lb # [ 1.222561] systemd[1]: Successfully made /usr/ read-only.1646worker2 # [ 1.241731] systemd[1]: Successfully made /usr/ read-only.1647worker1 # [ 1.354274] usb 1-1: new high-speed USB device number 2 using ehci-pci1648client # [ 1.382565] usb 1-1: new high-speed USB device number 2 using ehci-pci1649lb # [ 1.375923] usb 1-1: new high-speed USB device number 2 using ehci-pci1650worker2 # [ 1.394954] usb 1-1: new high-speed USB device number 2 using ehci-pci1651worker1 # [ 1.508956] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11652client # [ 1.541258] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11653worker1 # [ 1.540016] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1654lb # [ 1.531902] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11655worker1 # [ 1.552313] systemd[1]: Detected virtualization qemu.1656worker2 # [ 1.549607] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11657worker1 # [ 1.554358] systemd[1]: Detected architecture arm64.1658worker1 # [ 1.556295] systemd[1]: Running in initrd.1659worker1 # [ 1.558994] systemd[1]: Initializing machine ID from random generator.1660worker1 # [ 1.562016] systemd[1]: Hostname set to <worker1>.1661client # [ 1.552216] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1662client # [ 1.564702] systemd[1]: Detected virtualization qemu.1663client # [ 1.566847] systemd[1]: Detected architecture arm64.1664client # [ 1.568773] systemd[1]: Running in initrd.1665client # [ 1.571555] systemd[1]: Initializing machine ID from random generator.1666client # [ 1.574586] systemd[1]: Hostname set to <client>.1667lb # [ 1.557481] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1668lb # [ 1.570728] systemd[1]: Detected virtualization qemu.1669worker2 # [ 1.576840] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1670lb # [ 1.572960] systemd[1]: Detected architecture arm64.1671worker2 # [ 1.589223] systemd[1]: Detected virtualization qemu.1672lb # [ 1.575030] systemd[1]: Running in initrd.1673worker2 # [ 1.591354] systemd[1]: Detected architecture arm64.1674worker2 # [ 1.593335] systemd[1]: Running in initrd.1675lb # [ 1.578108] systemd[1]: Initializing machine ID from random generator.1676lb # [ 1.581411] systemd[1]: Hostname set to <lb>.1677worker2 # [ 1.596219] systemd[1]: Initializing machine ID from random generator.1678worker2 # [ 1.599291] systemd[1]: Hostname set to <worker2>.1679worker1 # [ 1.602754] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01680client # [ 1.634816] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01681worker2 # [ 1.643325] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682lb # [ 1.632213] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683worker1 # [ 1.726287] usb 1-2: new high-speed USB device number 3 using ehci-pci1684client # [ 1.754597] usb 1-2: new high-speed USB device number 3 using ehci-pci1685worker2 # [ 1.766947] usb 1-2: new high-speed USB device number 3 using ehci-pci1686lb # [ 1.755915] usb 1-2: new high-speed USB device number 3 using ehci-pci1687worker1 # [ 1.889010] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21688worker1 # [ 1.893090] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01689client # [ 1.899289] systemd[1]: bpf-restrict-fs: LSM BPF program attached1690worker1 # [ 1.912983] systemd[1]: bpf-restrict-fs: LSM BPF program attached1691client # [ 1.915006] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21692client # [ 1.920208] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01693lb # [ 1.909099] systemd[1]: bpf-restrict-fs: LSM BPF program attached1694worker2 # [ 1.923707] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21695worker2 # [ 1.931177] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01696lb # [ 1.924663] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21697lb # [ 1.929650] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01698worker2 # [ 1.948095] systemd[1]: bpf-restrict-fs: LSM BPF program attached1699worker1 # [ 1.985865] systemd[1]: Queued start job for default target Initrd Default Target.1700worker1 # [ 1.996907] systemd[1]: Created slice Slice /system/modprobe.1701worker1 # [ 1.998005] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1702worker1 # [ 1.999174] systemd[1]: Expecting device /dev/disk/by-label/nixos...1703worker1 # [ 2.000028] systemd[1]: Reached target Path Units.1704worker1 # [ 2.000740] systemd[1]: Reached target Slice Units.1705worker1 # [ 2.001407] systemd[1]: Reached target Swaps.1706worker1 # [ 2.002016] systemd[1]: Reached target Timer Units.1707worker1 # [ 2.002954] systemd[1]: Listening on D-Bus System Message Bus Socket.1708worker1 # [ 2.004002] systemd[1]: Listening on Journal Socket (/dev/log).1709worker1 # [ 2.004975] systemd[1]: Listening on Journal Sockets.1710worker1 # [ 2.005800] systemd[1]: Listening on udev Control Socket.1711worker1 # [ 2.006729] systemd[1]: Listening on udev Kernel Socket.1712worker1 # [ 2.007476] systemd[1]: Reached target Socket Units.1713client # [ 2.011086] systemd[1]: Queued start job for default target Initrd Default Target.1714worker1 # [ 2.009974] systemd[1]: Starting Create List of Static Device Nodes...1715worker1 # [ 2.018594] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1716worker1 # [ 2.018718] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1717client # [ 2.021005] systemd[1]: Created slice Slice /system/modprobe.1718worker2 # [ 2.022786] systemd[1]: Queued start job for default target Initrd Default Target.1719client # [ 2.022266] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1720client # [ 2.023614] systemd[1]: Expecting device /dev/disk/by-label/nixos...1721client # [ 2.024581] systemd[1]: Reached target Path Units.1722client # [ 2.025340] systemd[1]: Reached target Slice Units.1723worker1 # [ 2.026552] systemd[1]: Mounting Kernel Configuration File System...1724client # [ 2.026074] systemd[1]: Reached target Swaps.1725client # [ 2.026790] systemd[1]: Reached target Timer Units.1726client # [ 2.027775] systemd[1]: Listening on D-Bus System Message Bus Socket.1727lb # [ 2.014662] systemd[1]: Queued start job for default target Initrd Default Target.1728client # [ 2.028954] systemd[1]: Listening on Journal Socket (/dev/log).1729client # [ 2.029946] systemd[1]: Listening on Journal Sockets.1730client # [ 2.030976] systemd[1]: Listening on udev Control Socket.1731worker2 # [ 2.033476] systemd[1]: Created slice Slice /system/modprobe.1732client # [ 2.031830] systemd[1]: Listening on udev Kernel Socket.1733client # [ 2.032585] systemd[1]: Reached target Socket Units.1734worker2 # [ 2.034665] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1735client # [ 2.035115] systemd[1]: Starting Create List of Static Device Nodes...1736worker2 # [ 2.035999] systemd[1]: Expecting device /dev/disk/by-label/nixos...1737worker2 # [ 2.036041] systemd[1]: Reached target Path Units.1738worker2 # [ 2.036062] systemd[1]: Reached target Slice Units.1739worker2 # [ 2.036086] systemd[1]: Reached target Swaps.1740worker2 # [ 2.036107] systemd[1]: Reached target Timer Units.1741lb # [ 2.026322] systemd[1]: Created slice Slice /system/modprobe.1742worker2 # [ 2.036350] systemd[1]: Listening on D-Bus System Message Bus Socket.1743worker2 # [ 2.036508] systemd[1]: Listening on Journal Socket (/dev/log).1744lb # [ 2.027548] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1745worker2 # [ 2.036641] systemd[1]: Listening on Journal Sockets.1746worker2 # [ 2.036773] systemd[1]: Listening on udev Control Socket.1747lb # [ 2.028883] systemd[1]: Expecting device /dev/disk/by-label/nixos...1748client # [ 2.042810] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1749worker2 # [ 2.036885] systemd[1]: Listening on udev Kernel Socket.1750lb # [ 2.029947] systemd[1]: Reached target Path Units.1751worker2 # [ 2.036908] systemd[1]: Reached target Socket Units.1752lb # [ 2.030710] systemd[1]: Reached target Slice Units.1753client # [ 2.043883] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1754lb # [ 2.031583] systemd[1]: Reached target Swaps.1755worker2 # [ 2.047276] systemd[1]: Starting Create List of Static Device Nodes...1756lb # [ 2.032423] systemd[1]: Reached target Timer Units.1757lb # [ 2.033431] systemd[1]: Listening on D-Bus System Message Bus Socket.1758lb # [ 2.034661] systemd[1]: Listening on Journal Socket (/dev/log).1759lb # [ 2.035714] systemd[1]: Listening on Journal Sockets.1760lb # [ 2.036718] systemd[1]: Listening on udev Control Socket.1761lb # [ 2.036845] systemd[1]: Listening on udev Kernel Socket.1762lb # [ 2.036869] systemd[1]: Reached target Socket Units.1763client # [ 2.054726] systemd[1]: Mounting Kernel Configuration File System...1764lb # [ 2.040971] systemd[1]: Starting Create List of Static Device Nodes...1765worker2 # [ 2.058275] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1766worker1 # [ 2.057963] systemd[1]: Starting Journal Service...1767worker2 # [ 2.059573] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1768lb # [ 2.052108] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1769lb # [ 2.053305] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1770worker1 # [ 2.070687] systemd[1]: Starting Load Kernel Modules...1771worker1 # [ 2.071572] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1772lb # [ 2.056921] systemd[1]: Mounting Kernel Configuration File System...1773worker2 # [ 2.075436] systemd[1]: Mounting Kernel Configuration File System...1774worker1 # [ 2.078582] systemd[1]: Starting Coldplug All udev Devices...1775client # [ 2.081999] systemd[1]: Starting Journal Service...1776lb # [ 2.076549] systemd[1]: Starting Journal Service...1777client # [ 2.093832] systemd[1]: Starting Load Kernel Modules...1778client # [ 2.094019] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1779worker1 # [ 2.094544] systemd[1]: Finished Create List of Static Device Nodes.1780worker1 # [ 2.095543] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1781worker2 # [ 2.099208] systemd[1]: Starting Journal Service...1782worker1 # [ 2.095872] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1783worker1 # [ 2.096127] systemd[1]: Mounted Kernel Configuration File System.1784worker2 # [ 2.107187] systemd[1]: Starting Load Kernel Modules...1785worker2 # [ 2.108209] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1786lb # [ 2.096290] systemd[1]: Starting Load Kernel Modules...1787lb # [ 2.097309] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1788client # [ 2.111753] systemd[1]: Starting Coldplug All udev Devices...1789worker1 # [ 2.114926] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1790client # [ 2.118739] systemd[1]: Finished Create List of Static Device Nodes.1791client # [ 2.119940] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1792worker2 # [ 2.123161] systemd[1]: Starting Coldplug All udev Devices...1793lb # [ 2.113493] systemd[1]: Starting Coldplug All udev Devices...1794worker2 # [ 2.131297] systemd[1]: Finished Create List of Static Device Nodes.1795worker2 # [ 2.133364] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1796client # [ 2.139690] systemd-journald[73]: Collecting audit messages is disabled.1797lb # [ 2.128917] systemd[1]: Finished Create List of Static Device Nodes.1798lb # [ 2.130084] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1799lb # [ 2.130430] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1800client # [ 2.146920] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1801client # [ 2.147543] systemd[1]: Mounted Kernel Configuration File System.1802worker1 # [ 2.154549] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1803worker2 # [ 2.159486] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1804client # [ 2.159368] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1805worker1 # [ 2.160843] systemd-journald[73]: Collecting audit messages is disabled.1806lb # [ 2.145939] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1807worker1 # [ 2.167281] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1808worker2 # [ 2.170242] systemd-journald[73]: Collecting audit messages is disabled.1809worker2 # [ 2.171856] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1810worker2 # [ 2.173280] systemd[1]: Mounted Kernel Configuration File System.1811lb # [ 2.160066] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1812client # [ 2.174910] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1813lb # [ 2.162597] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1814lb # [ 2.164561] systemd[1]: Mounted Kernel Configuration File System.1815worker1 # [ 2.182718] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01816worker1 # [ 2.183009] [drm] features: -virgl +edid -resource_blob -host_visible1817worker1 # [ 2.183020] [drm] features: -context_init1818worker1 # [ 2.183811] [drm] number of scanouts: 11819worker1 # [ 2.183836] [drm] number of cap sets: 01820worker1 # [ 2.188100] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1821client # [ 2.186563] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1822lb # [ 2.175092] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01823lb # [ 2.175401] [drm] features: -virgl +edid -resource_blob -host_visible1824lb # [ 2.175409] [drm] features: -context_init1825worker2 # [ 2.197806] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1826client # [ 2.199006] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01827client # [ 2.199290] [drm] features: -virgl +edid -resource_blob -host_visible1828worker1 # [ 2.198888] systemd[1]: Starting Create Static Device Nodes in /dev...1829client # [ 2.199302] [drm] features: -context_init1830client # [ 2.200113] [drm] number of scanouts: 11831client # [ 2.200134] [drm] number of cap sets: 01832lb # [ 2.193891] [drm] number of scanouts: 11833lb # [ 2.193935] [drm] number of cap sets: 01834worker1 # [ 2.206736] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1835worker1 # [ 2.206764] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01836lb # [ 2.196602] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1837worker2 # [ 2.219380] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1838lb # [ 2.196630] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01839client # [ 2.219075] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1840client # [ 2.219113] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01841lb # [ 2.210046] systemd-journald[73]: Collecting audit messages is disabled.1842worker2 # [ 2.228262] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01843worker2 # [ 2.228546] [drm] features: -virgl +edid -resource_blob -host_visible1844worker2 # [ 2.228560] [drm] features: -context_init1845worker2 # [ 2.229432] [drm] number of scanouts: 11846worker2 # [ 2.229456] [drm] number of cap sets: 01847lb # [ 2.224697] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1848worker1 # [ 2.239060] Console: switching to colour frame buffer device 160x501849client # [ 2.243631] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1850lb # [ 2.232368] systemd[1]: Starting Create Static Device Nodes in /dev...1851client # [ 2.255333] systemd[1]: Starting Create Static Device Nodes in /dev...1852lb # [ 2.240389] Console: switching to colour frame buffer device 160x501853worker2 # [ 2.255344] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1854worker2 # [ 2.255385] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01855client # [ 2.256559] Console: switching to colour frame buffer device 160x501856worker2 # [ 2.261505] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1857worker1 # [ 2.263047] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1858worker2 # [ 2.264688] systemd[1]: Starting Create Static Device Nodes in /dev...1859worker1 # [ 2.270180] systemd[1]: Finished Load Kernel Modules.1860lb # [ 2.260564] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1861client # [ 2.275230] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1862worker1 # [ 2.281325] systemd[1]: Starting Apply Kernel Variables...1863client # [ 2.281753] systemd[1]: Finished Load Kernel Modules.1864worker2 # [ 2.291325] Console: switching to colour frame buffer device 160x501865lb # [ 2.276294] systemd[1]: Finished Load Kernel Modules.1866client # [ 2.295089] systemd[1]: Starting Apply Kernel Variables...1867worker2 # [ 2.292052] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1868lb # [ 2.284442] systemd[1]: Starting Apply Kernel Variables...1869worker2 # [ 2.312889] systemd[1]: Finished Load Kernel Modules.1870worker1 # [ 2.314831] systemd[1]: Finished Create Static Device Nodes in /dev.1871worker1 # [ 2.315232] systemd[1]: Reached target Preparation for Local File Systems.1872worker1 # [ 2.315279] systemd[1]: Reached target Local File Systems.1873worker2 # [ 2.320023] systemd[1]: Starting Apply Kernel Variables...1874client # [ 2.331103] systemd[1]: Finished Create Static Device Nodes in /dev.1875client # [ 2.331743] systemd[1]: Reached target Preparation for Local File Systems.1876worker1 # [ 2.334934] systemd[1]: Starting Rule-based Manager for Device Events and Files...1877client # [ 2.331790] systemd[1]: Reached target Local File Systems.1878client # [ 2.339166] systemd[1]: Starting Rule-based Manager for Device Events and Files...1879lb # [ 2.324010] systemd[1]: Finished Create Static Device Nodes in /dev.1880lb # [ 2.324359] systemd[1]: Reached target Preparation for Local File Systems.1881lb # [ 2.324403] systemd[1]: Reached target Local File Systems.1882lb # [ 2.330453] systemd[1]: Starting Rule-based Manager for Device Events and Files...1883client # [ 2.338028] systemd-modules-load[74]: Inserted module 'dm_mod'1884worker2 # [ 2.359457] systemd[1]: Finished Create Static Device Nodes in /dev.1885worker2 # [ 2.359821] systemd[1]: Reached target Preparation for Local File Systems.1886worker2 # [ 2.359869] systemd[1]: Reached target Local File Systems.1887worker1 # [ 2.362599] systemd[1]: Finished Apply Kernel Variables.1888worker2 # [ 2.367415] systemd[1]: Starting Rule-based Manager for Device Events and Files...1889client # [ 2.366825] systemd[1]: Started Journal Service.1890client # [ 2.352356] systemd-modules-load[74]: Module 'virtio_balloon' is built in1891client # [ 2.353774] systemd-modules-load[74]: Module 'virtio_console' is built in1892client # [ 2.354988] systemd-modules-load[74]: Inserted module 'virtio_gpu'1893client # [ 2.358851] systemd-modules-load[74]: Module 'virtio_rng' is built in1894worker1 # [ 2.378509] systemd[1]: Started Journal Service.1895worker1 # [ 2.364647] systemd-modules-load[74]: Inserted module 'dm_mod'1896worker1 # [ 2.366617] systemd-modules-load[74]: Module 'virtio_balloon' is built in1897worker2 # [ 2.370237] systemd-modules-load[74]: Inserted module 'dm_mod'1898client # [ 2.368946] systemd[1]: Starting Create System Files and Directories...1899client # [ 2.370412] systemd[1]: Finished Apply Kernel Variables.1900worker1 # [ 2.372275] systemd-modules-load[74]: Module 'virtio_console' is built in1901lb # [ 2.376030] systemd[1]: Finished Apply Kernel Variables.1902worker2 # [ 2.395219] systemd[1]: Started Journal Service.1903worker1 # [ 2.380549] systemd-modules-load[74]: Inserted module 'virtio_gpu'1904worker1 # [ 2.381760] systemd-modules-load[74]: Module 'virtio_rng' is built in1905worker2 # [ 2.382455] systemd-modules-load[74]: Module 'virtio_balloon' is built in1906worker1 # [ 2.382862] systemd[1]: Starting Create System Files and Directories...1907worker2 # [ 2.383771] systemd-modules-load[74]: Module 'virtio_console' is built in1908worker2 # [ 2.388972] systemd-modules-load[74]: Inserted module 'virtio_gpu'1909worker2 # [ 2.393203] systemd-modules-load[74]: Module 'virtio_rng' is built in1910lb # [ 2.389367] systemd-modules-load[74]: Inserted module 'dm_mod'1911worker2 # [ 2.404461] systemd[1]: Starting Create System Files and Directories...1912lb # [ 2.406678] systemd[1]: Started Journal Service.1913worker2 # [ 2.407103] systemd[1]: Finished Apply Kernel Variables.1914lb # [ 2.401787] systemd-modules-load[74]: Module 'virtio_balloon' is built in1915lb # [ 2.403034] systemd-modules-load[74]: Module 'virtio_console' is built in1916lb # [ 2.412668] systemd-modules-load[74]: Inserted module 'virtio_gpu'1917lb # [ 2.416489] systemd-modules-load[74]: Module 'virtio_rng' is built in1918client # [ 2.430654] systemd[1]: Finished Create System Files and Directories.1919lb # [ 2.420401] systemd[1]: Starting Create System Files and Directories...1920worker1 # [ 2.437869] systemd-udevd[80]: Using default interface naming scheme 'v261'.1921worker1 # [ 2.449815] systemd[1]: Finished Create System Files and Directories.1922client # [ 2.454205] systemd-udevd[81]: Using default interface naming scheme 'v261'.1923lb # [ 2.444518] systemd-udevd[81]: Using default interface naming scheme 'v261'.1924worker2 # [ 2.466403] systemd[1]: Finished Create System Files and Directories.1925worker1 # [ 2.474550] systemd[1]: Started Rule-based Manager for Device Events and Files.1926worker2 # [ 2.477143] systemd-udevd[81]: Using default interface naming scheme 'v261'.1927client # [ 2.487489] systemd[1]: Started Rule-based Manager for Device Events and Files.1928lb # [ 2.483794] systemd[1]: Finished Create System Files and Directories.1929lb # [ 2.490158] systemd[1]: Started Rule-based Manager for Device Events and Files.1930worker2 # [ 2.505502] systemd[1]: Started Rule-based Manager for Device Events and Files.1931worker1 # [ 2.552896] systemd[1]: Starting Virtual Console Setup...1932client # [ 2.565470] systemd[1]: Starting Virtual Console Setup...1933lb # [ 2.564395] systemd[1]: Starting Virtual Console Setup...1934worker2 # [ 2.584130] systemd[1]: Starting Virtual Console Setup...1935worker1 # [ 2.617894] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1936worker1 # [ 2.624351] systemd[1]: Finished Virtual Console Setup.1937worker2 # [ 2.632617] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1938lb # [ 2.624637] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1939client # [ 2.636639] systemd-vconsole-setup[104]: Configuration of first virtual console was skipped, ignoring remaining ones.1940worker2 # [ 2.640707] systemd[1]: Finished Virtual Console Setup.1941lb # [ 2.629097] systemd[1]: Finished Virtual Console Setup.1942client # [ 2.641053] systemd[1]: Finished Virtual Console Setup.1943client # [ 3.213274] systemd[1]: Finished Coldplug All udev Devices.1944client # [ 3.215088] systemd[1]: Reached target System Initialization.1945client # [ 3.220190] systemd[1]: Reached target Basic System.1946worker1 # [ 3.229898] systemd[1]: Finished Coldplug All udev Devices.1947worker1 # [ 3.231077] systemd[1]: Reached target System Initialization.1948worker1 # [ 3.236179] systemd[1]: Reached target Basic System.1949worker2 # [ 3.240412] systemd[1]: Finished Coldplug All udev Devices.1950worker2 # [ 3.241796] systemd[1]: Reached target System Initialization.1951worker2 # [ 3.244149] systemd[1]: Reached target Basic System.1952lb # [ 3.271114] systemd[1]: Finished Coldplug All udev Devices.1953lb # [ 3.272949] systemd[1]: Reached target System Initialization.1954lb # [ 3.276172] systemd[1]: Reached target Basic System.1955worker1 # [ 3.380471] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1956client # [ 3.385905] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1957worker2 # [ 3.395366] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1958client # [ 3.415512] (udev-worker)[103]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1959client # [ 3.420416] (udev-worker)[103]: Network interface NamePolicy= disabled on kernel command line.1960worker1 # [ 3.432739] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1961worker1 # [ 3.436313] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1962worker2 # [ 3.448181] (udev-worker)[102]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1963worker2 # [ 3.453359] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1964lb # [ 3.441423] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1965lb # [ 3.447163] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1966lb # [ 3.466451] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1967client # [ 3.517527] systemd[1]: Found device /dev/disk/by-label/nixos.1968worker1 # [ 3.520672] systemd[1]: Found device /dev/disk/by-label/nixos.1969worker2 # [ 3.523579] systemd[1]: Found device /dev/disk/by-label/nixos.1970client # [ 3.524141] systemd[1]: Reached target Initrd Root Device.1971worker2 # [ 3.525683] systemd[1]: Reached target Initrd Root Device.1972client # [ 3.528815] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1973worker1 # [ 3.531817] systemd[1]: Reached target Initrd Root Device.1974worker2 # [ 3.532625] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1975worker1 # [ 3.534332] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1976lb # [ 3.537374] systemd[1]: Found device /dev/disk/by-label/nixos.1977lb # [ 3.539495] systemd[1]: Reached target Initrd Root Device.1978lb # [ 3.545199] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1979worker2 # [ 3.589437] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1980client # [ 3.592624] systemd-fsck[113]: nixos: clean, 12/65536 files, 13019/262144 blocks1981worker1 # [ 3.594227] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1982worker2 # [ 3.599133] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1983worker1 # [ 3.601531] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1984client # [ 3.603472] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1985worker1 # [ 3.608694] systemd[1]: Mounting /sysroot...1986worker2 # [ 3.612403] systemd[1]: Mounting /sysroot...1987client # [ 3.613387] systemd[1]: Mounting /sysroot...1988lb # [ 3.607630] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1989lb # [ 3.617004] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1990lb # [ 3.625380] systemd[1]: Mounting /sysroot...1991worker1 # [ 3.669243] EXT4-fs (vda): mounted filesystem 58cc546a-53d5-4182-bdc5-d2b6ffbff31b r/w with ordered data mode. Quota mode: none.1992worker1 # [ 3.657432] systemd[1]: Mounted /sysroot.1993worker2 # [ 3.674953] EXT4-fs (vda): mounted filesystem b7b4459a-11fe-4d9c-a4fa-50942b2d7ba1 r/w with ordered data mode. Quota mode: none.1994client # [ 3.675177] EXT4-fs (vda): mounted filesystem 807334f1-aa14-4768-ae30-1d40f9408b0c r/w with ordered data mode. Quota mode: none.1995worker1 # [ 3.661854] systemd[1]: Reached target Initrd Root File System.1996worker2 # [ 3.662707] systemd[1]: Mounted /sysroot.1997worker2 # [ 3.665437] systemd[1]: Reached target Initrd Root File System.1998client # [ 3.665873] systemd[1]: Mounted /sysroot.1999worker1 # [ 3.667065] systemd[1]: Starting Mountpoints Configured in the Real Root...2000client # [ 3.667582] systemd[1]: Reached target Initrd Root File System.
loading 1,809 hidden lines…
3810worker1 # [ 34.120510] nix-grpc-daemon[834]: ts=2026-09-20T13:33:07Z level=info event=scheduler_connected addr=lb:500513811worker2 # [ 34.127927] nix-grpc-daemon[838]: ts=2026-09-20T13:33:07Z level=info event=scheduler_connected addr=lb:500513812worker1 # [ 34.126773] nix-grpc-daemon[834]: ts=2026-09-20T13:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3813worker2 # [ 34.132348] nix-grpc-daemon[838]: ts=2026-09-20T13:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3814worker1 # [ 34.329173] nix-grpc-daemon[834]: ts=2026-09-20T13:33:07Z level=info event=scheduler_connected addr=lb:500513815worker2 # [ 34.328966] nix-grpc-daemon[838]: ts=2026-09-20T13:33:07Z level=info event=scheduler_connected addr=lb:500513816worker1 # [ 34.332640] nix-grpc-daemon[834]: ts=2026-09-20T13:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3817worker2 # [ 34.334898] nix-grpc-daemon[838]: ts=2026-09-20T13:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3818worker1 # [ 34.534217] nix-grpc-daemon[834]: ts=2026-09-20T13:33:07Z level=info event=scheduler_connected addr=lb:500513819worker2 # [ 34.536583] nix-grpc-daemon[838]: ts=2026-09-20T13:33:07Z level=info event=scheduler_connected addr=lb:500513820worker1 # [ 34.540645] nix-grpc-daemon[834]: ts=2026-09-20T13:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3821worker2 # [ 34.542474] nix-grpc-daemon[838]: ts=2026-09-20T13:33:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3822worker1 # [ 34.741213] nix-grpc-daemon[834]: ts=2026-09-20T13:33:08Z level=info event=scheduler_connected addr=lb:500513823worker2 # [ 34.744189] nix-grpc-daemon[838]: ts=2026-09-20T13:33:08Z level=info event=scheduler_connected addr=lb:500513824worker1 # [ 34.747210] nix-grpc-daemon[834]: ts=2026-09-20T13:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3825worker2 # [ 34.749685] nix-grpc-daemon[838]: ts=2026-09-20T13:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3826worker2 # [ 34.947019] nix-grpc-daemon[838]: ts=2026-09-20T13:33:08Z level=info event=scheduler_connected addr=lb:500513827worker1 # [ 34.949278] nix-grpc-daemon[834]: ts=2026-09-20T13:33:08Z level=info event=scheduler_connected addr=lb:500513828worker2 # [ 34.952776] nix-grpc-daemon[838]: ts=2026-09-20T13:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3829worker1 # [ 34.954002] nix-grpc-daemon[834]: ts=2026-09-20T13:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3830worker1 # [ 35.150421] nix-grpc-daemon[834]: ts=2026-09-20T13:33:08Z level=info event=scheduler_connected addr=lb:500513831worker2 # [ 35.154803] nix-grpc-daemon[838]: ts=2026-09-20T13:33:08Z level=info event=scheduler_connected addr=lb:500513832worker1 # [ 35.154326] nix-grpc-daemon[834]: ts=2026-09-20T13:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3833worker2 # [ 35.156696] nix-grpc-daemon[838]: ts=2026-09-20T13:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3834worker1 # [ 35.355311] nix-grpc-daemon[834]: ts=2026-09-20T13:33:08Z level=info event=scheduler_connected addr=lb:500513835worker1 # [ 35.358102] nix-grpc-daemon[834]: ts=2026-09-20T13:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3836worker2 # [ 35.356751] nix-grpc-daemon[838]: ts=2026-09-20T13:33:08Z level=info event=scheduler_connected addr=lb:500513837worker2 # [ 35.362453] nix-grpc-daemon[838]: ts=2026-09-20T13:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3838worker1 # [ 35.559177] nix-grpc-daemon[834]: ts=2026-09-20T13:33:08Z level=info event=scheduler_connected addr=lb:500513839worker2 # [ 35.563952] nix-grpc-daemon[838]: ts=2026-09-20T13:33:08Z level=info event=scheduler_connected addr=lb:500513840worker1 # [ 35.565271] nix-grpc-daemon[834]: ts=2026-09-20T13:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3841worker2 # [ 35.569469] nix-grpc-daemon[838]: ts=2026-09-20T13:33:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3842worker1 # [ 35.765838] nix-grpc-daemon[834]: ts=2026-09-20T13:33:09Z level=info event=scheduler_connected addr=lb:500513843worker1 # [ 35.768155] nix-grpc-daemon[834]: ts=2026-09-20T13:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3844worker2 # [ 35.766896] nix-grpc-daemon[838]: ts=2026-09-20T13:33:09Z level=info event=scheduler_connected addr=lb:500513845worker2 # [ 35.773049] nix-grpc-daemon[838]: ts=2026-09-20T13:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3846worker1 # [ 35.969772] nix-grpc-daemon[834]: ts=2026-09-20T13:33:09Z level=info event=scheduler_connected addr=lb:500513847worker2 # [ 35.974828] nix-grpc-daemon[838]: ts=2026-09-20T13:33:09Z level=info event=scheduler_connected addr=lb:500513848worker1 # [ 35.975870] nix-grpc-daemon[834]: ts=2026-09-20T13:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3849worker2 # [ 35.981016] nix-grpc-daemon[838]: ts=2026-09-20T13:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3850worker1 # [ 36.176720] nix-grpc-daemon[834]: ts=2026-09-20T13:33:09Z level=info event=scheduler_connected addr=lb:500513851worker1 # [ 36.180175] nix-grpc-daemon[834]: ts=2026-09-20T13:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3852worker2 # [ 36.184849] nix-grpc-daemon[838]: ts=2026-09-20T13:33:09Z level=info event=scheduler_connected addr=lb:500513853worker2 # [ 36.189312] nix-grpc-daemon[838]: ts=2026-09-20T13:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3854worker1 # [ 36.383819] nix-grpc-daemon[834]: ts=2026-09-20T13:33:09Z level=info event=scheduler_connected addr=lb:500513855worker2 # [ 36.387172] nix-grpc-daemon[838]: ts=2026-09-20T13:33:09Z level=info event=scheduler_connected addr=lb:500513856worker1 # [ 36.389500] nix-grpc-daemon[834]: ts=2026-09-20T13:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3857worker2 # [ 36.392310] nix-grpc-daemon[838]: ts=2026-09-20T13:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3858worker1 # [ 36.584934] nix-grpc-daemon[834]: ts=2026-09-20T13:33:09Z level=info event=scheduler_connected addr=lb:500513859worker2 # [ 36.588927] nix-grpc-daemon[838]: ts=2026-09-20T13:33:09Z level=info event=scheduler_connected addr=lb:500513860worker1 # [ 36.590969] nix-grpc-daemon[834]: ts=2026-09-20T13:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3861worker2 # [ 36.594947] nix-grpc-daemon[838]: ts=2026-09-20T13:33:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3862worker1 # [ 36.791994] nix-grpc-daemon[834]: ts=2026-09-20T13:33:10Z level=info event=scheduler_connected addr=lb:500513863worker2 # [ 36.796330] nix-grpc-daemon[838]: ts=2026-09-20T13:33:10Z level=info event=scheduler_connected addr=lb:500513864worker1 # [ 36.799323] nix-grpc-daemon[834]: ts=2026-09-20T13:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3865worker2 # [ 36.802132] nix-grpc-daemon[838]: ts=2026-09-20T13:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3866worker1 # [ 37.000418] nix-grpc-daemon[834]: ts=2026-09-20T13:33:10Z level=info event=scheduler_connected addr=lb:500513867worker2 # [ 37.003595] nix-grpc-daemon[838]: ts=2026-09-20T13:33:10Z level=info event=scheduler_connected addr=lb:500513868worker1 # [ 37.006614] nix-grpc-daemon[834]: ts=2026-09-20T13:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3869worker2 # [ 37.010102] nix-grpc-daemon[838]: ts=2026-09-20T13:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3870worker1 # [ 37.207380] nix-grpc-daemon[834]: ts=2026-09-20T13:33:10Z level=info event=scheduler_connected addr=lb:500513871worker2 # [ 37.213131] nix-grpc-daemon[838]: ts=2026-09-20T13:33:10Z level=info event=scheduler_connected addr=lb:500513872worker1 # [ 37.213236] nix-grpc-daemon[834]: ts=2026-09-20T13:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3873worker2 # [ 37.217418] nix-grpc-daemon[838]: ts=2026-09-20T13:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3874worker2 # [ 37.413646] nix-grpc-daemon[838]: ts=2026-09-20T13:33:10Z level=info event=scheduler_connected addr=lb:500513875worker1 # [ 37.416565] nix-grpc-daemon[834]: ts=2026-09-20T13:33:10Z level=info event=scheduler_connected addr=lb:500513876worker1 # [ 37.420943] nix-grpc-daemon[834]: ts=2026-09-20T13:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3877worker2 # [ 37.421738] nix-grpc-daemon[838]: ts=2026-09-20T13:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3878worker1 # [ 37.617786] nix-grpc-daemon[834]: ts=2026-09-20T13:33:11Z level=info event=scheduler_connected addr=lb:500513879worker2 # [ 37.623297] nix-grpc-daemon[838]: ts=2026-09-20T13:33:10Z level=info event=scheduler_connected addr=lb:500513880worker1 # [ 37.623368] nix-grpc-daemon[834]: ts=2026-09-20T13:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3881worker2 # [ 37.628869] nix-grpc-daemon[838]: ts=2026-09-20T13:33:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3882worker1 # [ 37.824354] nix-grpc-daemon[834]: ts=2026-09-20T13:33:11Z level=info event=scheduler_connected addr=lb:500513883worker2 # [ 37.825797] nix-grpc-daemon[838]: ts=2026-09-20T13:33:11Z level=info event=scheduler_connected addr=lb:500513884worker1 # [ 37.830209] nix-grpc-daemon[834]: ts=2026-09-20T13:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3885worker2 # [ 37.831549] nix-grpc-daemon[838]: ts=2026-09-20T13:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3886worker1 # [ 38.031022] nix-grpc-daemon[834]: ts=2026-09-20T13:33:11Z level=info event=scheduler_connected addr=lb:500513887worker2 # [ 38.033346] nix-grpc-daemon[838]: ts=2026-09-20T13:33:11Z level=info event=scheduler_connected addr=lb:500513888worker1 # [ 38.036939] nix-grpc-daemon[834]: ts=2026-09-20T13:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3889worker2 # [ 38.039101] nix-grpc-daemon[838]: ts=2026-09-20T13:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3890worker1 # [ 38.237992] nix-grpc-daemon[834]: ts=2026-09-20T13:33:11Z level=info event=scheduler_connected addr=lb:500513891worker2 # [ 38.243035] nix-grpc-daemon[838]: ts=2026-09-20T13:33:11Z level=info event=scheduler_connected addr=lb:500513892worker1 # [ 38.243471] nix-grpc-daemon[834]: ts=2026-09-20T13:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3893worker2 # [ 38.248533] nix-grpc-daemon[838]: ts=2026-09-20T13:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3894worker2 # [ 38.444661] nix-grpc-daemon[838]: ts=2026-09-20T13:33:11Z level=info event=scheduler_connected addr=lb:500513895worker1 # [ 38.447909] nix-grpc-daemon[834]: ts=2026-09-20T13:33:11Z level=info event=scheduler_connected addr=lb:500513896worker1 # [ 38.456417] nix-grpc-daemon[834]: ts=2026-09-20T13:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3897worker2 # [ 38.457085] nix-grpc-daemon[838]: ts=2026-09-20T13:33:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3898worker1 # [ 38.648301] nix-grpc-daemon[834]: ts=2026-09-20T13:33:12Z level=info event=scheduler_connected addr=lb:500513899worker2 # [ 38.652279] nix-grpc-daemon[838]: ts=2026-09-20T13:33:12Z level=info event=scheduler_connected addr=lb:500513900worker1 # [ 38.654416] nix-grpc-daemon[834]: ts=2026-09-20T13:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3901worker2 # [ 38.657956] nix-grpc-daemon[838]: ts=2026-09-20T13:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3902worker2 # [ 38.855147] nix-grpc-daemon[838]: ts=2026-09-20T13:33:12Z level=info event=scheduler_connected addr=lb:500513903worker1 # [ 38.855294] nix-grpc-daemon[834]: ts=2026-09-20T13:33:12Z level=info event=scheduler_connected addr=lb:500513904worker2 # [ 38.860819] nix-grpc-daemon[838]: ts=2026-09-20T13:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3905worker1 # [ 38.861151] nix-grpc-daemon[834]: ts=2026-09-20T13:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3906worker2 # [ 39.062180] nix-grpc-daemon[838]: ts=2026-09-20T13:33:12Z level=info event=scheduler_connected addr=lb:500513907worker1 # [ 39.062358] nix-grpc-daemon[834]: ts=2026-09-20T13:33:12Z level=info event=scheduler_connected addr=lb:500513908worker2 # [ 39.067774] nix-grpc-daemon[838]: ts=2026-09-20T13:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3909worker1 # [ 39.068191] nix-grpc-daemon[834]: ts=2026-09-20T13:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3910worker1 # [ 39.268882] nix-grpc-daemon[834]: ts=2026-09-20T13:33:12Z level=info event=scheduler_connected addr=lb:500513911worker2 # [ 39.271453] nix-grpc-daemon[838]: ts=2026-09-20T13:33:12Z level=info event=scheduler_connected addr=lb:500513912worker1 # [ 39.274491] nix-grpc-daemon[834]: ts=2026-09-20T13:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3913worker2 # [ 39.275989] nix-grpc-daemon[838]: ts=2026-09-20T13:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3914worker2 # [ 39.473197] nix-grpc-daemon[838]: ts=2026-09-20T13:33:12Z level=info event=scheduler_connected addr=lb:500513915worker1 # [ 39.478002] nix-grpc-daemon[834]: ts=2026-09-20T13:33:12Z level=info event=scheduler_connected addr=lb:500513916worker2 # [ 39.478845] nix-grpc-daemon[838]: ts=2026-09-20T13:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3917worker1 # [ 39.482357] nix-grpc-daemon[834]: ts=2026-09-20T13:33:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3918worker1 # [ 39.677937] nix-grpc-daemon[834]: ts=2026-09-20T13:33:13Z level=info event=scheduler_connected addr=lb:500513919worker2 # [ 39.680308] nix-grpc-daemon[838]: ts=2026-09-20T13:33:13Z level=info event=scheduler_connected addr=lb:500513920worker1 # [ 39.683711] nix-grpc-daemon[834]: ts=2026-09-20T13:33:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3921worker2 # [ 39.685696] nix-grpc-daemon[838]: ts=2026-09-20T13:33:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3922worker1 # [ 39.884727] nix-grpc-daemon[834]: ts=2026-09-20T13:33:13Z level=info event=scheduler_connected addr=lb:500513923worker2 # [ 39.887155] nix-grpc-daemon[838]: ts=2026-09-20T13:33:13Z level=info event=scheduler_connected addr=lb:500513924worker1 # [ 39.890824] nix-grpc-daemon[834]: ts=2026-09-20T13:33:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3925worker2 # [ 39.893060] nix-grpc-daemon[838]: ts=2026-09-20T13:33:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3926worker1 # [ 40.091786] nix-grpc-daemon[834]: ts=2026-09-20T13:33:13Z level=info event=scheduler_connected addr=lb:500513927worker2 # [ 40.094423] nix-grpc-daemon[838]: ts=2026-09-20T13:33:13Z level=info event=scheduler_connected addr=lb:500513928worker1 # [ 40.097796] nix-grpc-daemon[834]: ts=2026-09-20T13:33:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3929worker2 # [ 40.100457] nix-grpc-daemon[838]: ts=2026-09-20T13:33:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3930worker1 # [ 40.298332] nix-grpc-daemon[834]: ts=2026-09-20T13:33:13Z level=info event=scheduler_connected addr=lb:500513931worker2 # [ 40.303832] nix-grpc-daemon[838]: ts=2026-09-20T13:33:13Z level=info event=scheduler_connected addr=lb:500513932worker1 # [ 40.304176] nix-grpc-daemon[834]: ts=2026-09-20T13:33:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3933worker2 # [ 40.308511] nix-grpc-daemon[838]: ts=2026-09-20T13:33:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3934worker2 # [ 40.505322] nix-grpc-daemon[838]: ts=2026-09-20T13:33:13Z level=info event=scheduler_connected addr=lb:500513935worker1 # [ 40.507942] nix-grpc-daemon[834]: ts=2026-09-20T13:33:13Z level=info event=scheduler_connected addr=lb:500513936worker2 # [ 40.510875] nix-grpc-daemon[838]: ts=2026-09-20T13:33:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3937worker1 # [ 40.512308] nix-grpc-daemon[834]: ts=2026-09-20T13:33:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3938worker1 # [ 40.708604] nix-grpc-daemon[834]: ts=2026-09-20T13:33:14Z level=info event=scheduler_connected addr=lb:500513939worker2 # [ 40.712079] nix-grpc-daemon[838]: ts=2026-09-20T13:33:14Z level=info event=scheduler_connected addr=lb:500513940worker1 # [ 40.715090] nix-grpc-daemon[834]: ts=2026-09-20T13:33:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3941worker2 # [ 40.717488] nix-grpc-daemon[838]: ts=2026-09-20T13:33:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3942worker1 # [ 40.854692] nix-grpc-daemon[834]: ts=2026-09-20T13:33:14Z level=info event=scheduler_lock_lost error="error: http://lb:5751/api/farm/lead: Timeout was reached"3943worker1 # [ 40.916140] nix-grpc-daemon[834]: ts=2026-09-20T13:33:14Z level=info event=scheduler_connected addr=lb:500513944worker2 # [ 40.918712] nix-grpc-daemon[838]: ts=2026-09-20T13:33:14Z level=info event=scheduler_connected addr=lb:500513945worker1 # [ 40.921920] nix-grpc-daemon[834]: ts=2026-09-20T13:33:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3946worker2 # [ 40.924317] nix-grpc-daemon[838]: ts=2026-09-20T13:33:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3947worker2 # [ 40.988844] nix-grpc-daemon[838]: ts=2026-09-20T13:33:14Z level=info event=scheduler_lock_lost error="error: http://lb:5751/api/farm/lead: Timeout was reached"3948worker1 # [ 41.123094] nix-grpc-daemon[834]: ts=2026-09-20T13:33:14Z level=info event=scheduler_connected addr=lb:500513949worker2 # [ 41.125668] nix-grpc-daemon[838]: ts=2026-09-20T13:33:14Z level=info event=scheduler_connected addr=lb:500513950worker1 # [ 41.129147] nix-grpc-daemon[834]: ts=2026-09-20T13:33:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3951worker2 # [ 41.131141] nix-grpc-daemon[838]: ts=2026-09-20T13:33:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3952worker1 # [ 41.330066] nix-grpc-daemon[834]: ts=2026-09-20T13:33:14Z level=info event=scheduler_connected addr=lb:500513953worker2 # [ 41.334968] nix-grpc-daemon[838]: ts=2026-09-20T13:33:14Z level=info event=scheduler_connected addr=lb:500513954worker1 # [ 41.335981] nix-grpc-daemon[834]: ts=2026-09-20T13:33:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3955worker2 # [ 41.339397] nix-grpc-daemon[838]: ts=2026-09-20T13:33:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3956worker2 # [ 41.535434] nix-grpc-daemon[838]: ts=2026-09-20T13:33:14Z level=info event=scheduler_connected addr=lb:500513957worker1 # [ 41.539317] nix-grpc-daemon[834]: ts=2026-09-20T13:33:14Z level=info event=scheduler_connected addr=lb:500513958worker2 # [ 41.541211] nix-grpc-daemon[838]: ts=2026-09-20T13:33:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3959worker1 # [ 41.543902] nix-grpc-daemon[834]: ts=2026-09-20T13:33:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3960worker1 # [ 41.741230] nix-grpc-daemon[834]: ts=2026-09-20T13:33:15Z level=info event=scheduler_connected addr=lb:500513961worker2 # [ 41.745486] nix-grpc-daemon[838]: ts=2026-09-20T13:33:15Z level=info event=scheduler_connected addr=lb:500513962worker1 # [ 41.748863] nix-grpc-daemon[834]: ts=2026-09-20T13:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3963worker2 # [ 41.753855] nix-grpc-daemon[838]: ts=2026-09-20T13:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3964worker1 # [ 41.942465] nix-grpc-daemon[834]: ts=2026-09-20T13:33:15Z level=info event=scheduler_connected addr=lb:500513965worker1 # [ 41.948095] nix-grpc-daemon[834]: ts=2026-09-20T13:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3966worker2 # [ 41.952761] nix-grpc-daemon[838]: ts=2026-09-20T13:33:15Z level=info event=scheduler_connected addr=lb:500513967worker2 # [ 41.961944] nix-grpc-daemon[838]: ts=2026-09-20T13:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3968worker1 # [ 42.148943] nix-grpc-daemon[834]: ts=2026-09-20T13:33:15Z level=info event=scheduler_connected addr=lb:500513969worker1 # [ 42.154691] nix-grpc-daemon[834]: ts=2026-09-20T13:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3970worker2 # [ 42.162893] nix-grpc-daemon[838]: ts=2026-09-20T13:33:15Z level=info event=scheduler_connected addr=lb:500513971worker2 # [ 42.168488] nix-grpc-daemon[838]: ts=2026-09-20T13:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3972worker1 # [ 42.355818] nix-grpc-daemon[834]: ts=2026-09-20T13:33:15Z level=info event=scheduler_connected addr=lb:500513973worker1 # [ 42.361369] nix-grpc-daemon[834]: ts=2026-09-20T13:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3974worker2 # [ 42.371701] nix-grpc-daemon[838]: ts=2026-09-20T13:33:15Z level=info event=scheduler_connected addr=lb:500513975worker2 # [ 42.376347] nix-grpc-daemon[838]: ts=2026-09-20T13:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3976worker1 # [ 42.563072] nix-grpc-daemon[834]: ts=2026-09-20T13:33:15Z level=info event=scheduler_connected addr=lb:500513977worker1 # [ 42.566630] nix-grpc-daemon[834]: ts=2026-09-20T13:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3978worker2 # [ 42.573533] nix-grpc-daemon[838]: ts=2026-09-20T13:33:15Z level=info event=scheduler_connected addr=lb:500513979worker2 # [ 42.579078] nix-grpc-daemon[838]: ts=2026-09-20T13:33:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3980worker1 # [ 42.767599] nix-grpc-daemon[834]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500513981worker1 # [ 42.773501] nix-grpc-daemon[834]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3982worker2 # [ 42.776410] nix-grpc-daemon[838]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500513983worker2 # [ 42.782527] nix-grpc-daemon[838]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3984worker1 # [ 42.974305] nix-grpc-daemon[834]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500513985worker1 # [ 42.980509] nix-grpc-daemon[834]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3986worker2 # [ 42.983856] nix-grpc-daemon[838]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500513987worker2 # [ 42.989580] nix-grpc-daemon[838]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3988worker1 # [ 43.181535] nix-grpc-daemon[834]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500513989worker1 # [ 43.187324] nix-grpc-daemon[834]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3990worker2 # [ 43.190802] nix-grpc-daemon[838]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500513991worker2 # [ 43.196774] nix-grpc-daemon[838]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3992worker1 # [ 43.388421] nix-grpc-daemon[834]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500513993worker1 # [ 43.394114] nix-grpc-daemon[834]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3994worker2 # [ 43.400557] nix-grpc-daemon[838]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500513995worker2 # [ 43.404922] nix-grpc-daemon[838]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3996worker1 # [ 43.597562] nix-grpc-daemon[834]: ts=2026-09-20T13:33:17Z level=info event=scheduler_connected addr=lb:500513997worker2 # [ 43.601079] nix-grpc-daemon[838]: ts=2026-09-20T13:33:16Z level=info event=scheduler_connected addr=lb:500513998worker1 # [ 43.602190] nix-grpc-daemon[834]: ts=2026-09-20T13:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3999worker2 # [ 43.606788] nix-grpc-daemon[838]: ts=2026-09-20T13:33:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4000worker1 # [ 43.798216] nix-grpc-daemon[834]: ts=2026-09-20T13:33:17Z level=info event=scheduler_connected addr=lb:500514001worker1 # [ 43.803971] nix-grpc-daemon[834]: ts=2026-09-20T13:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4002worker2 # [ 43.808243] nix-grpc-daemon[838]: ts=2026-09-20T13:33:17Z level=info event=scheduler_connected addr=lb:500514003worker2 # [ 43.813810] nix-grpc-daemon[838]: ts=2026-09-20T13:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4004worker1 # [ 44.005143] nix-grpc-daemon[834]: ts=2026-09-20T13:33:17Z level=info event=scheduler_connected addr=lb:500514005worker1 # [ 44.010857] nix-grpc-daemon[834]: ts=2026-09-20T13:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4006worker2 # [ 44.015081] nix-grpc-daemon[838]: ts=2026-09-20T13:33:17Z level=info event=scheduler_connected addr=lb:500514007worker2 # [ 44.021034] nix-grpc-daemon[838]: ts=2026-09-20T13:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4008worker1 # [ 44.211687] nix-grpc-daemon[834]: ts=2026-09-20T13:33:17Z level=info event=scheduler_connected addr=lb:500514009worker1 # [ 44.217335] nix-grpc-daemon[834]: ts=2026-09-20T13:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4010worker2 # [ 44.222201] nix-grpc-daemon[838]: ts=2026-09-20T13:33:17Z level=info event=scheduler_connected addr=lb:500514011worker2 # [ 44.227792] nix-grpc-daemon[838]: ts=2026-09-20T13:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4012worker1 # [ 44.418100] nix-grpc-daemon[834]: ts=2026-09-20T13:33:17Z level=info event=scheduler_connected addr=lb:500514013worker1 # [ 44.423702] nix-grpc-daemon[834]: ts=2026-09-20T13:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4014worker2 # [ 44.431366] nix-grpc-daemon[838]: ts=2026-09-20T13:33:17Z level=info event=scheduler_connected addr=lb:500514015worker2 # [ 44.435751] nix-grpc-daemon[838]: ts=2026-09-20T13:33:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4016worker1 # [ 44.627085] nix-grpc-daemon[834]: ts=2026-09-20T13:33:18Z level=info event=scheduler_connected addr=lb:500514017worker1 # [ 44.631515] nix-grpc-daemon[834]: ts=2026-09-20T13:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4018worker2 # [ 44.632467] nix-grpc-daemon[838]: ts=2026-09-20T13:33:18Z level=info event=scheduler_connected addr=lb:500514019worker2 # [ 44.638102] nix-grpc-daemon[838]: ts=2026-09-20T13:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4020worker1 # [ 44.828124] nix-grpc-daemon[834]: ts=2026-09-20T13:33:18Z level=info event=scheduler_connected addr=lb:500514021worker1 # [ 44.833908] nix-grpc-daemon[834]: ts=2026-09-20T13:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4022worker2 # [ 44.840664] nix-grpc-daemon[838]: ts=2026-09-20T13:33:18Z level=info event=scheduler_connected addr=lb:500514023worker2 # [ 44.845074] nix-grpc-daemon[838]: ts=2026-09-20T13:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4024worker1 # [ 45.036489] nix-grpc-daemon[834]: ts=2026-09-20T13:33:18Z level=info event=scheduler_connected addr=lb:500514025worker2 # [ 45.046530] nix-grpc-daemon[838]: ts=2026-09-20T13:33:18Z level=info event=scheduler_connected addr=lb:500514026worker1 # [ 45.041448] nix-grpc-daemon[834]: ts=2026-09-20T13:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4027worker2 # [ 45.051361] nix-grpc-daemon[838]: ts=2026-09-20T13:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4028worker1 # [ 45.237881] nix-grpc-daemon[834]: ts=2026-09-20T13:33:18Z level=info event=scheduler_connected addr=lb:500514029worker1 # [ 45.242177] nix-grpc-daemon[834]: ts=2026-09-20T13:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4030worker2 # [ 45.247701] nix-grpc-daemon[838]: ts=2026-09-20T13:33:18Z level=info event=scheduler_connected addr=lb:500514031worker2 # [ 45.253328] nix-grpc-daemon[838]: ts=2026-09-20T13:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4032worker1 # [ 45.443286] nix-grpc-daemon[834]: ts=2026-09-20T13:33:18Z level=info event=scheduler_connected addr=lb:500514033worker1 # [ 45.449161] nix-grpc-daemon[834]: ts=2026-09-20T13:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4034worker2 # [ 45.454424] nix-grpc-daemon[838]: ts=2026-09-20T13:33:18Z level=info event=scheduler_connected addr=lb:500514035worker2 # [ 45.456526] nix-grpc-daemon[838]: ts=2026-09-20T13:33:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4036worker1 # [ 45.652960] nix-grpc-daemon[834]: ts=2026-09-20T13:33:19Z level=info event=scheduler_connected addr=lb:500514037worker2 # [ 45.657555] nix-grpc-daemon[838]: ts=2026-09-20T13:33:19Z level=info event=scheduler_connected addr=lb:500514038worker2 # [ 45.660325] nix-grpc-daemon[838]: ts=2026-09-20T13:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4039worker1 # [ 45.657511] nix-grpc-daemon[834]: ts=2026-09-20T13:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4040worker1 # [ 45.853884] nix-grpc-daemon[834]: ts=2026-09-20T13:33:19Z level=info event=scheduler_connected addr=lb:500514041worker2 # [ 45.861287] nix-grpc-daemon[838]: ts=2026-09-20T13:33:19Z level=info event=scheduler_connected addr=lb:500514042worker1 # [ 45.859877] nix-grpc-daemon[834]: ts=2026-09-20T13:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4043worker2 # [ 45.864551] nix-grpc-daemon[838]: ts=2026-09-20T13:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4044worker1 # [ 46.061298] nix-grpc-daemon[834]: ts=2026-09-20T13:33:19Z level=info event=scheduler_connected addr=lb:500514045worker2 # [ 46.066454] nix-grpc-daemon[838]: ts=2026-09-20T13:33:19Z level=info event=scheduler_connected addr=lb:500514046worker1 # [ 46.067165] nix-grpc-daemon[834]: ts=2026-09-20T13:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4047worker2 # [ 46.072277] nix-grpc-daemon[838]: ts=2026-09-20T13:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4048worker1 # [ 46.267593] nix-grpc-daemon[834]: ts=2026-09-20T13:33:19Z level=info event=scheduler_connected addr=lb:500514049worker1 # [ 46.271402] nix-grpc-daemon[834]: ts=2026-09-20T13:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4050worker2 # [ 46.274092] nix-grpc-daemon[838]: ts=2026-09-20T13:33:19Z level=info event=scheduler_connected addr=lb:500514051worker2 # [ 46.279659] nix-grpc-daemon[838]: ts=2026-09-20T13:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4052worker1 # [ 46.472688] nix-grpc-daemon[834]: ts=2026-09-20T13:33:19Z level=info event=scheduler_connected addr=lb:500514053worker1 # [ 46.478651] nix-grpc-daemon[834]: ts=2026-09-20T13:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4054worker2 # [ 46.483638] nix-grpc-daemon[838]: ts=2026-09-20T13:33:19Z level=info event=scheduler_connected addr=lb:500514055worker2 # [ 46.488120] nix-grpc-daemon[838]: ts=2026-09-20T13:33:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4056worker1 # [ 46.680922] nix-grpc-daemon[834]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500514057worker2 # [ 46.684863] nix-grpc-daemon[838]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500514058worker1 # [ 46.687881] nix-grpc-daemon[834]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4059worker2 # [ 46.690554] nix-grpc-daemon[838]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4060worker1 # [ 46.887689] nix-grpc-daemon[834]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500514061worker2 # [ 46.891524] nix-grpc-daemon[838]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500514062worker1 # [ 46.893867] nix-grpc-daemon[834]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4063worker2 # [ 46.897404] nix-grpc-daemon[838]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4064worker1 # [ 47.094952] nix-grpc-daemon[834]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500514065worker2 # [ 47.098700] nix-grpc-daemon[838]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500514066worker1 # [ 47.100940] nix-grpc-daemon[834]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4067worker2 # [ 47.104557] nix-grpc-daemon[838]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4068worker1 # [ 47.301978] nix-grpc-daemon[834]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500514069worker2 # [ 47.305792] nix-grpc-daemon[838]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500514070worker1 # [ 47.307879] nix-grpc-daemon[834]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4071worker2 # [ 47.311701] nix-grpc-daemon[838]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4072worker1 # [ 47.509039] nix-grpc-daemon[834]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500514073worker2 # [ 47.515595] nix-grpc-daemon[838]: ts=2026-09-20T13:33:20Z level=info event=scheduler_connected addr=lb:500514074worker1 # [ 47.514684] nix-grpc-daemon[834]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4075worker2 # [ 47.520200] nix-grpc-daemon[838]: ts=2026-09-20T13:33:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4076worker2 # [ 47.715734] nix-grpc-daemon[838]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514077worker1 # [ 47.718380] nix-grpc-daemon[834]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514078worker2 # [ 47.719727] nix-grpc-daemon[838]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4079worker1 # [ 47.723055] nix-grpc-daemon[834]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4080worker1 # [ 47.918799] nix-grpc-daemon[834]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514081worker2 # [ 47.921063] nix-grpc-daemon[838]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514082worker1 # [ 47.925244] nix-grpc-daemon[834]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4083worker2 # [ 47.926871] nix-grpc-daemon[838]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4084worker1 # [ 48.126220] nix-grpc-daemon[834]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514085worker2 # [ 48.128208] nix-grpc-daemon[838]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514086worker1 # [ 48.131984] nix-grpc-daemon[834]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4087worker2 # [ 48.133777] nix-grpc-daemon[838]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4088worker1 # [ 48.333297] nix-grpc-daemon[834]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514089worker2 # [ 48.335538] nix-grpc-daemon[838]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514090worker1 # [ 48.339359] nix-grpc-daemon[834]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4091worker2 # [ 48.340955] nix-grpc-daemon[838]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4092worker1 # [ 48.540175] nix-grpc-daemon[834]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514093worker2 # [ 48.543946] nix-grpc-daemon[838]: ts=2026-09-20T13:33:21Z level=info event=scheduler_connected addr=lb:500514094worker1 # [ 48.544475] nix-grpc-daemon[834]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4095worker2 # [ 48.547622] nix-grpc-daemon[838]: ts=2026-09-20T13:33:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4096worker2 # [ 48.745177] nix-grpc-daemon[838]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514097worker1 # [ 48.747836] nix-grpc-daemon[834]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514098worker2 # [ 48.750829] nix-grpc-daemon[838]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4099worker1 # [ 48.752733] nix-grpc-daemon[834]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4100worker1 # [ 48.948753] nix-grpc-daemon[834]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514101worker2 # [ 48.951888] nix-grpc-daemon[838]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514102worker1 # [ 48.954781] nix-grpc-daemon[834]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4103worker2 # [ 48.957550] nix-grpc-daemon[838]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4104worker1 # [ 49.156068] nix-grpc-daemon[834]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514105worker2 # [ 49.159049] nix-grpc-daemon[838]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514106worker1 # [ 49.162286] nix-grpc-daemon[834]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4107worker2 # [ 49.165088] nix-grpc-daemon[838]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4108worker1 # [ 49.363419] nix-grpc-daemon[834]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514109worker2 # [ 49.366541] nix-grpc-daemon[838]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514110worker1 # [ 49.371404] nix-grpc-daemon[834]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4111worker2 # [ 49.372594] nix-grpc-daemon[838]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4112worker1 # [ 49.572914] nix-grpc-daemon[834]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514113worker2 # [ 49.577031] nix-grpc-daemon[838]: ts=2026-09-20T13:33:22Z level=info event=scheduler_connected addr=lb:500514114worker1 # [ 49.579227] nix-grpc-daemon[834]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4115worker2 # [ 49.581567] nix-grpc-daemon[838]: ts=2026-09-20T13:33:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4116worker2 # [ 49.777639] nix-grpc-daemon[838]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514117worker1 # [ 49.781091] nix-grpc-daemon[834]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514118worker1 # [ 49.783597] nix-grpc-daemon[834]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4119worker2 # [ 49.783520] nix-grpc-daemon[838]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4120worker1 # [ 49.981253] nix-grpc-daemon[834]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514121worker1 # [ 49.984152] nix-grpc-daemon[834]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4122worker2 # [ 49.984880] nix-grpc-daemon[838]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514123worker2 # [ 49.990463] nix-grpc-daemon[838]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4124worker2 # [ 50.187854] nix-grpc-daemon[838]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514125worker1 # [ 50.185880] nix-grpc-daemon[834]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514126worker1 # [ 50.192781] nix-grpc-daemon[834]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4127worker2 # [ 50.193524] nix-grpc-daemon[838]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4128worker1 # [ 50.393199] nix-grpc-daemon[834]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514129worker1 # [ 50.396692] nix-grpc-daemon[834]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4130worker2 # [ 50.395058] nix-grpc-daemon[838]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514131worker2 # [ 50.401302] nix-grpc-daemon[838]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4132worker1 # [ 50.597686] nix-grpc-daemon[834]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514133worker1 # [ 50.600174] nix-grpc-daemon[834]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4134worker2 # [ 50.605446] nix-grpc-daemon[838]: ts=2026-09-20T13:33:23Z level=info event=scheduler_connected addr=lb:500514135worker2 # [ 50.609893] nix-grpc-daemon[838]: ts=2026-09-20T13:33:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4136lb # [ 50.752276] rustfs-bucket-start[1170]: mb s3://farm4137lb # [ 50.761260] systemd[1]: Finished rustfs-bucket.service.4138lb # [ 50.773389] systemd[1]: Starting niks3 server...4139worker1 # [ 50.804073] nix-grpc-daemon[834]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514140worker2 # [ 50.806759] nix-grpc-daemon[838]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514141worker1 # [ 50.808497] nix-grpc-daemon[834]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4142worker2 # [ 50.812815] nix-grpc-daemon[838]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4143worker1 # [ 51.005523] nix-grpc-daemon[834]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514144worker1 # [ 51.011602] nix-grpc-daemon[834]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4145worker2 # [ 51.014410] nix-grpc-daemon[838]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514146worker2 # [ 51.020171] nix-grpc-daemon[838]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4147lb # [ 51.068624] postgres[1186]: [1186] ERROR: relation "goose_db_version" does not exist at character 364148lb # [ 51.070362] postgres[1186]: [1186] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4149lb # [ 51.112929] niks3-server[1180]: 2026/09/20 13:33:24 OK 20241026095416_initial_model.sql (25.18ms)4150lb # [ 51.117419] niks3-server[1180]: 2026/09/20 13:33:24 OK 20251210153512_drop_unused_gin_index.sql (4.99ms)4151lb # [ 51.124400] niks3-server[1180]: 2026/09/20 13:33:24 OK 20251218171726_add_pins.sql (6.93ms)4152lb # [ 51.130191] niks3-server[1180]: 2026/09/20 13:33:24 OK 20260628120000_add_object_size_and_stats.sql (5.74ms)4153lb # [ 51.135803] niks3-server[1180]: 2026/09/20 13:33:24 OK 20260905000000_add_claims.sql (5.56ms)4154lb # [ 51.141360] niks3-server[1180]: 2026/09/20 13:33:24 OK 20260920000000_drop_claims.sql (5.4ms)4155lb # [ 51.142676] niks3-server[1180]: 2026/09/20 13:33:24 goose: successfully migrated database to version: 202609200000004156lb # [ 51.148881] niks3-server[1180]: 2026/09/20 13:33:24 OK 1_commit_pending_closure.sql (7.46ms)4157lb # [ 51.152103] niks3-server[1180]: 2026/09/20 13:33:24 OK 2_object_stats_trigger.sql (3.06ms)4158lb # [ 51.153617] niks3-server[1180]: 2026/09/20 13:33:24 goose: up to current file version: 24159lb # [ 51.157912] niks3-server[1180]: 2026/09/20 13:33:24 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key4160worker1 # [ 51.212736] nix-grpc-daemon[834]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514161worker1 # [ 51.218471] nix-grpc-daemon[834]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4162worker2 # [ 51.221563] nix-grpc-daemon[838]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514163worker2 # [ 51.227295] nix-grpc-daemon[838]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4164lb # [ 51.220684] niks3-server[1180]: 2026/09/20 13:33:24 INFO Read proxy enabled — serving cache objects from S34165lb # [ 51.224189] niks3-server[1180]: 2026/09/20 13:33:24 INFO Using socket-activated listener address=0.0.0.0:57514166lb # [ 51.229299] niks3-server[1180]: 2026/09/20 13:33:24 INFO systemd watchdog enabled interval=15s4167lb # [ 51.230561] niks3-server[1180]: 2026/09/20 13:33:24 INFO Starting HTTP server address=0.0.0.0:57514168lb # [ 51.231789] systemd[1]: Started niks3 server.4169lb # [ 51.237906] systemd[1]: Reached target Multi-User System.4170lb # [ 51.238822] systemd[1]: Startup finished in 1.193s (kernel) + 5.728s (initrd) + 44.311s (userspace) = 51.233s.4171lb # [ 51.249235] niks3-server[1180]: 2026/09/20 13:33:24 INFO lead: acquired remote=192.168.1.4:585324172worker2 # [ 51.270281] nix-grpc-daemon[838]: ts=2026-09-20T13:33:24Z level=info event=scheduler_take_over4173worker1 # [ 51.419555] nix-grpc-daemon[834]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514174worker1 # [ 51.425279] nix-grpc-daemon[834]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4175worker2 # [ 51.428762] nix-grpc-daemon[838]: ts=2026-09-20T13:33:24Z level=info event=scheduler_connected addr=lb:500514176worker2 # [ 51.434240] nix-grpc-daemon[838]: ts=2026-09-20T13:33:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4177worker1 # [ 51.626371] nix-grpc-daemon[834]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514178worker1 # [ 51.632416] nix-grpc-daemon[834]: ts=2026-09-20T13:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4179worker2 # [ 51.638248] nix-grpc-daemon[838]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514180worker2 # [ 51.642565] nix-grpc-daemon[838]: ts=2026-09-20T13:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4181worker1 # [ 51.833620] nix-grpc-daemon[834]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514182worker2 # [ 51.838756] nix-grpc-daemon[838]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514183worker1 # [ 51.836439] nix-grpc-daemon[834]: ts=2026-09-20T13:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4184worker2 # [ 51.844325] nix-grpc-daemon[838]: ts=2026-09-20T13:33:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4185lb: (finished: waiting for unit niks3.service, in 35.73 seconds)4186lb: waiting for TCP port 5751 on localhost4187lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4188lb: (finished: waiting for TCP port 5751 on localhost, in 0.12 seconds)4189lb: waiting for unit envoy.service4190worker1 # [ 52.037962] nix-grpc-daemon[834]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514191worker2 # [ 52.051105] nix-grpc-daemon[838]: ts=2026-09-20T13:33:25Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:453244192worker2 # [ 52.059116] nix-grpc-daemon[838]: ts=2026-09-20T13:33:25Z level=info event=scheduler_connected addr=lb:500514193worker2 # [ 52.067764] nix-grpc-daemon[838]: ts=2026-09-20T13:33:25Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04194worker2 # [ 52.077325] nix-grpc-daemon[838]: ts=2026-09-20T13:33:25Z level=info event=oidc_jwks provider=mock keys=14195lb: (finished: waiting for unit envoy.service, in 0.09 seconds)4196lb: waiting for TCP port 50051 on localhost4197worker2 # [ 52.082189] nix-grpc-daemon[838]: ts=2026-09-20T13:33:25Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:453244198worker2 # [ 52.087975] nix-grpc-daemon[838]: ts=2026-09-20T13:33:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04199lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4200lb # [ 52.119140] envoy[911]: conn peer=[::1]:42748 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=44201lb: (finished: waiting for TCP port 50051 on localhost, in 0.05 seconds)4202worker2 # [ 52.305345] systemd[1]: Starting scheduler-token.service...4203worker2 # [ 52.415565] systemd[1]: scheduler-token.service: Deactivated successfully.4204worker2 # [ 52.418932] systemd[1]: Finished scheduler-token.service.4205worker2 # [ 52.419960] systemd[1]: scheduler-token.service: Consumed 46ms CPU time over 114ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4206lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4207??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4208 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284209lb: must succeed: curl -sf localhost:9901/clusters4210lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds)4211lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.11 seconds)4212subtest: one node schedules and both builders hold a WorkerSession on it4213lb: must succeed: curl -sf localhost:9901/clusters4214lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4215worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4216worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)4217lb: must succeed: curl -sf localhost:9901/clusters4218lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4219lb: must succeed: curl -sf localhost:9901/clusters4220lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4221worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4222worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.08 seconds)4223(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.38 seconds)4224subtest: balancer auth: client cert, bearer token, nothing, unknown CN4225worker2 # [ 53.343767] nix-grpc-daemon[838]: ts=2026-09-20T13:33:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:453404226worker2 # [ 53.368363] systemd[1]: Started Nix Daemon.4227worker2 # [ 53.713332] nix-daemon[913]: accepted connection from pid 838, user nix-grpc-daemon (trusted)4228worker2 # [ 53.764389] nix-grpc-daemon[838]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)4229worker2 # [ 53.775873] nix-grpc-daemon[838]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 101 ms (attempt 2/5)4230worker2 # [ 53.877399] nix-grpc-daemon[838]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 167 ms (attempt 3/5)4231worker2 # [ 54.052912] nix-grpc-daemon[838]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 732 ms (attempt 4/5)4232worker2 # [ 54.797812] nix-grpc-daemon[838]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4233worker1 # [ 54.855319] nix-grpc-daemon[834]: ts=2026-09-20T13:33:28Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:494444234worker1 # [ 54.880808] systemd[1]: Started Nix Daemon.4235worker1 # [ 55.209484] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4236worker1 # [ 55.238696] nix-grpc-daemon[834]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 51 ms (attempt 1/5)4237worker1 # [ 55.293784] nix-grpc-daemon[834]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 161 ms (attempt 2/5)4238worker1 # [ 55.461767] nix-grpc-daemon[834]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 389 ms (attempt 3/5)4239worker1 # [ 55.860627] nix-grpc-daemon[834]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 173 ms (attempt 4/5)4240worker1 # [ 56.036589] nix-grpc-daemon[834]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4241worker1 # [ 56.055917] nix-grpc-daemon[834]: ts=2026-09-20T13:33:29Z level=debug event=path_info_miss path=f1p01b8r1g0vwpnb6kbil89vcdp0b0k0-nixos-system-client-test error="error: path '/nix/store/f1p01b8r1g0vwpnb6kbil89vcdp0b0k0-nixos-system-client-test' is required, but there is no substituter that can build it"4242worker2 # [ 56.065523] nix-grpc-daemon[838]: ts=2026-09-20T13:33:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:453404243worker2 # [ 56.084276] nix-grpc-daemon[838]: ts=2026-09-20T13:33:29Z level=debug event=path_info_miss path=f1p01b8r1g0vwpnb6kbil89vcdp0b0k0-nixos-system-client-test error="error: path '/nix/store/f1p01b8r1g0vwpnb6kbil89vcdp0b0k0-nixos-system-client-test' is required, but there is no substituter that can build it"4244client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4245client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.12 seconds)4246client: must succeed: install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4247client: (finished: must succeed: install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.06 seconds)4248worker1 # [ 56.596325] nix-grpc-daemon[834]: ts=2026-09-20T13:33:30Z level=info event=oidc_jwks provider=mock keys=14249worker1 # [ 56.609001] nix-grpc-daemon[834]: ts=2026-09-20T13:33:30Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:494444250worker1 # [ 56.627990] nix-grpc-daemon[834]: ts=2026-09-20T13:33:30Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:494444251worker1 # [ 56.647119] nix-grpc-daemon[834]: ts=2026-09-20T13:33:30Z level=debug event=path_info_miss path=f1p01b8r1g0vwpnb6kbil89vcdp0b0k0-nixos-system-client-test error="error: path '/nix/store/f1p01b8r1g0vwpnb6kbil89vcdp0b0k0-nixos-system-client-test' is required, but there is no substituter that can build it"4252worker2 # [ 56.659503] nix-grpc-daemon[838]: ts=2026-09-20T13:33:30Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:453404253worker2 # [ 56.678867] nix-grpc-daemon[838]: ts=2026-09-20T13:33:30Z level=debug event=path_info_miss path=f1p01b8r1g0vwpnb6kbil89vcdp0b0k0-nixos-system-client-test error="error: path '/nix/store/f1p01b8r1g0vwpnb6kbil89vcdp0b0k0-nixos-system-client-test' is required, but there is no substituter that can build it"4254client: must succeed: rm -r /var/lib/nix-grpc-store4255client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.05 seconds)4256worker1 # [ 57.044567] nix-grpc-daemon[834]: ts=2026-09-20T13:33:30Z level=info event=denied method=QueryMissing cn=- role=none4257worker2 # [ 57.328686] nix-grpc-daemon[838]: ts=2026-09-20T13:33:30Z level=info event=denied method=QueryMissing cn=stranger role=none4258(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.44 seconds)4259subtest: DAG build is scheduled across workers and lands in the cache4260client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24261worker1 # [ 57.610567] nix-grpc-daemon[834]: ts=2026-09-20T13:33:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:494444262client # don't know how to build these paths:4263client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4264worker1 # [ 57.628806] nix-grpc-daemon[834]: ts=2026-09-20T13:33:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444265worker2 # [ 57.687791] nix-grpc-daemon[838]: ts=2026-09-20T13:33:31Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=04266worker2 # [ 57.694632] nix-grpc-daemon[838]: ts=2026-09-20T13:33:31Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=14267worker1 # [ 57.735348] nix-grpc-daemon[834]: ts=2026-09-20T13:33:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444268worker1 # [ 57.761288] nix-grpc-daemon[834]: ts=2026-09-20T13:33:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444269client # copying 1 paths to 'lb:50051'...4270worker1 # [ 57.782314] nix-grpc-daemon[834]: ts=2026-09-20T13:33:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444271client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...4272worker1 # [ 57.813197] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4273worker1 # [ 57.857071] nix-grpc-daemon[834]: ts=2026-09-20T13:33:31Z level=info event=niks3_push_started pid=9074274worker1 # [ 58.208053] nix-grpc-daemon[907]: time=2026-09-20T13:33:31.612Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""4275worker1 # [ 58.419697] nix-daemon[898]: accepted connection from pid 920, user nix-grpc-daemon (trusted)4276worker1 # [ 58.442889] nix-daemon[898]: reaped child process 923, status = succeeded4277worker1 # [ 58.627193] nix-daemon[898]: accepted connection from pid 925, user nix-grpc-daemon (trusted)4278worker1 # [ 58.650813] nix-daemon[898]: reaped child process 928, status = succeeded4279lb # [ 58.646900] niks3-server[1180]: 2026/09/20 13:33:32 INFO Received uploads request method=POST path=/api/pending_closures4280worker1 # [ 58.697104] nix-grpc-daemon[907]: time=2026-09-20T13:33:32.103Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4281worker1 # [ 58.700901] nix-grpc-daemon[907]: time=2026-09-20T13:33:32.105Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"4282lb # [ 58.747232] niks3-server[1180]: 2026/09/20 13:33:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4283lb # [ 58.760170] niks3-server[1180]: 2026/09/20 13:33:32 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst4284lb # [ 58.767725] niks3-server[1180]: 2026/09/20 13:33:32 INFO Signed narinfos id=1 count=14285worker1 # [ 58.780282] nix-grpc-daemon[907]: time=2026-09-20T13:33:32.185Z level=INFO msg="Uploading 1 narinfos"4286lb # [ 58.786487] niks3-server[1180]: 2026/09/20 13:33:32 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls4287lb # [ 58.794314] niks3-server[1180]: 2026/09/20 13:33:32 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4288lb # [ 58.802063] niks3-server[1180]: 2026/09/20 13:33:32 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo4289lb # [ 58.813017] niks3-server[1180]: 2026/09/20 13:33:32 INFO Completed upload id=14290worker1 # [ 58.828552] nix-grpc-daemon[907]: time=2026-09-20T13:33:32.234Z level=INFO msg="Upload complete. (612ms)"4291worker1 # [ 58.842087] nix-grpc-daemon[834]: ts=2026-09-20T13:33:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 paths=1 nar_bytes_in=5044292worker1 # [ 58.860879] nix-grpc-daemon[834]: ts=2026-09-20T13:33:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444293worker1 # [ 58.875211] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4294worker1 # [ 58.879801] nix-daemon[898]: reaped child process 905, status = succeeded4295client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...4296client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4297worker1 # [ 58.898013] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4298client # farm-a> LOG-farm-a-t14299worker1 # [ 59.925159] nix-daemon[898]: reaped child process 932, status = succeeded4300worker1 # [ 60.079979] nix-daemon[898]: accepted connection from pid 947, user nix-grpc-daemon (trusted)4301worker1 # [ 60.100695] nix-daemon[898]: reaped child process 950, status = succeeded4302lb # [ 60.097083] niks3-server[1180]: 2026/09/20 13:33:33 INFO Received uploads request method=POST path=/api/pending_closures4303worker1 # [ 60.121308] nix-grpc-daemon[907]: time=2026-09-20T13:33:33.526Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4304worker1 # [ 60.123323] nix-grpc-daemon[907]: time=2026-09-20T13:33:33.526Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"4305lb # [ 60.154305] niks3-server[1180]: 2026/09/20 13:33:33 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4306lb # [ 60.166495] niks3-server[1180]: 2026/09/20 13:33:33 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4307worker1 # [ 60.187890] nix-grpc-daemon[907]: time=2026-09-20T13:33:33.592Z level=INFO msg="Uploading 1 narinfos"4308lb # [ 60.178493] niks3-server[1180]: 2026/09/20 13:33:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4309lb # [ 60.186996] niks3-server[1180]: 2026/09/20 13:33:33 INFO Signed narinfos id=2 count=14310lb # [ 60.192347] niks3-server[1180]: 2026/09/20 13:33:33 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls4311lb # [ 60.200882] niks3-server[1180]: 2026/09/20 13:33:33 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4312worker1 # [ 60.220596] nix-grpc-daemon[907]: time=2026-09-20T13:33:33.626Z level=INFO msg="Upload complete. (309ms)"4313lb # [ 60.207793] niks3-server[1180]: 2026/09/20 13:33:33 INFO Completed upload id=24314lb # [ 60.211734] niks3-server[1180]: 2026/09/20 13:33:33 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4315worker1 # [ 60.228831] nix-grpc-daemon[834]: ts=2026-09-20T13:33:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14316worker1 # [ 60.236393] nix-daemon[898]: reaped child process 930, status = succeeded4317worker2 # [ 60.238462] nix-grpc-daemon[838]: ts=2026-09-20T13:33:33Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04318worker2 # [ 60.242155] nix-grpc-daemon[838]: ts=2026-09-20T13:33:33Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=14319worker1 # [ 60.291684] nix-grpc-daemon[834]: ts=2026-09-20T13:33:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444320worker1 # [ 60.310166] nix-grpc-daemon[834]: ts=2026-09-20T13:33:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444321worker1 # [ 60.328918] nix-grpc-daemon[907]: time=2026-09-20T13:33:33.735Z level=INFO msg="All 1 paths already cached"4322client # copying 1 paths to 'lb:50051'...4323worker1 # [ 60.338344] nix-grpc-daemon[834]: ts=2026-09-20T13:33:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444324client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4325worker1 # [ 60.359507] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4326worker1 # [ 60.557376] nix-daemon[898]: accepted connection from pid 954, user nix-grpc-daemon (trusted)4327worker1 # [ 60.581166] nix-daemon[898]: reaped child process 957, status = succeeded4328worker1 # [ 60.756232] nix-daemon[898]: accepted connection from pid 959, user nix-grpc-daemon (trusted)4329worker1 # [ 60.777105] nix-daemon[898]: reaped child process 962, status = succeeded4330lb # [ 60.773175] niks3-server[1180]: 2026/09/20 13:33:34 INFO Received uploads request method=POST path=/api/pending_closures4331worker1 # [ 60.800096] nix-grpc-daemon[907]: time=2026-09-20T13:33:34.206Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4332worker1 # [ 60.801698] nix-grpc-daemon[907]: time=2026-09-20T13:33:34.206Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4333lb # [ 60.811503] niks3-server[1180]: 2026/09/20 13:33:34 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4334lb # [ 60.827766] niks3-server[1180]: 2026/09/20 13:33:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4335worker1 # [ 60.848209] nix-grpc-daemon[907]: time=2026-09-20T13:33:34.254Z level=INFO msg="Uploading 1 narinfos"4336lb # [ 60.842820] niks3-server[1180]: 2026/09/20 13:33:34 INFO Signed narinfos id=3 count=14337lb # [ 60.849268] niks3-server[1180]: 2026/09/20 13:33:34 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4338lb # [ 60.857820] niks3-server[1180]: 2026/09/20 13:33:34 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4339lb # [ 60.868163] niks3-server[1180]: 2026/09/20 13:33:34 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4340lb # [ 60.876290] niks3-server[1180]: 2026/09/20 13:33:34 INFO Completed upload id=34341worker1 # [ 60.889239] nix-grpc-daemon[907]: time=2026-09-20T13:33:34.295Z level=INFO msg="Upload complete. (518ms)"4342worker1 # [ 60.900722] nix-grpc-daemon[834]: ts=2026-09-20T13:33:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_in=6244343worker1 # [ 60.916588] nix-grpc-daemon[834]: ts=2026-09-20T13:33:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444344worker1 # [ 60.925258] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4345worker1 # [ 60.930697] nix-daemon[898]: reaped child process 952, status = succeeded4346client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4347client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4348worker1 # [ 60.945358] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4349client # farm-b> LOG-farm-b-t14350worker1 # [ 61.996505] nix-daemon[898]: reaped child process 966, status = succeeded4351worker1 # [ 62.174444] nix-daemon[898]: accepted connection from pid 981, user nix-grpc-daemon (trusted)4352worker1 # [ 62.197991] nix-daemon[898]: reaped child process 984, status = succeeded4353lb # [ 62.198288] niks3-server[1180]: 2026/09/20 13:33:35 INFO Received uploads request method=POST path=/api/pending_closures4354worker1 # [ 62.222695] nix-grpc-daemon[907]: time=2026-09-20T13:33:35.628Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4355worker1 # [ 62.225578] nix-grpc-daemon[907]: time=2026-09-20T13:33:35.630Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4356lb # [ 62.245340] niks3-server[1180]: 2026/09/20 13:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4357worker1 # [ 62.268506] nix-grpc-daemon[907]: time=2026-09-20T13:33:35.674Z level=INFO msg="Uploading 1 narinfos"4358lb # [ 62.261483] niks3-server[1180]: 2026/09/20 13:33:35 INFO Signed narinfos id=4 count=14359lb # [ 62.270198] niks3-server[1180]: 2026/09/20 13:33:35 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4360worker1 # [ 62.302165] nix-grpc-daemon[907]: time=2026-09-20T13:33:35.708Z level=INFO msg="Upload complete. (344ms)"4361lb # [ 62.289551] niks3-server[1180]: 2026/09/20 13:33:35 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4362lb # [ 62.297307] niks3-server[1180]: 2026/09/20 13:33:35 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4363worker1 # [ 62.313217] nix-grpc-daemon[834]: ts=2026-09-20T13:33:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14364worker2 # [ 62.317806] nix-grpc-daemon[838]: ts=2026-09-20T13:33:35Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04365lb # [ 62.304149] niks3-server[1180]: 2026/09/20 13:33:35 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4366worker2 # [ 62.318027] nix-grpc-daemon[838]: ts=2026-09-20T13:33:35Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14367lb # [ 62.308662] niks3-server[1180]: 2026/09/20 13:33:35 INFO Completed upload id=44368lb # [ 62.311354] niks3-server[1180]: 2026/09/20 13:33:35 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4369worker1 # [ 62.331767] nix-daemon[898]: reaped child process 964, status = succeeded4370worker1 # [ 62.365189] nix-grpc-daemon[834]: ts=2026-09-20T13:33:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444371worker1 # [ 62.384100] nix-grpc-daemon[834]: ts=2026-09-20T13:33:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444372worker1 # [ 62.400662] nix-grpc-daemon[907]: time=2026-09-20T13:33:35.806Z level=INFO msg="All 2 paths already cached"4373client # copying 1 paths to 'lb:50051'...4374client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4375worker1 # [ 62.410391] nix-grpc-daemon[834]: ts=2026-09-20T13:33:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444376worker1 # [ 62.426075] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4377worker1 # [ 62.633365] nix-daemon[898]: accepted connection from pid 988, user nix-grpc-daemon (trusted)4378worker1 # [ 62.655373] nix-daemon[898]: reaped child process 991, status = succeeded4379worker1 # [ 62.839057] nix-daemon[898]: accepted connection from pid 993, user nix-grpc-daemon (trusted)4380worker1 # [ 62.858269] nix-daemon[898]: reaped child process 996, status = succeeded4381lb # [ 62.858967] niks3-server[1180]: 2026/09/20 13:33:36 INFO Received uploads request method=POST path=/api/pending_closures4382worker1 # [ 62.882182] nix-grpc-daemon[907]: time=2026-09-20T13:33:36.288Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4383worker1 # [ 62.883735] nix-grpc-daemon[907]: time=2026-09-20T13:33:36.288Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4384lb # [ 62.891164] niks3-server[1180]: 2026/09/20 13:33:36 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4385lb # [ 62.907295] niks3-server[1180]: 2026/09/20 13:33:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4386worker1 # [ 62.927566] nix-grpc-daemon[907]: time=2026-09-20T13:33:36.333Z level=INFO msg="Uploading 1 narinfos"4387lb # [ 62.921073] niks3-server[1180]: 2026/09/20 13:33:36 INFO Signed narinfos id=5 count=14388lb # [ 62.923678] niks3-server[1180]: 2026/09/20 13:33:36 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4389lb # [ 62.935134] niks3-server[1180]: 2026/09/20 13:33:36 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4390lb # [ 62.943982] niks3-server[1180]: 2026/09/20 13:33:36 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4391lb # [ 62.952562] niks3-server[1180]: 2026/09/20 13:33:36 INFO Completed upload id=54392worker1 # [ 62.965589] nix-grpc-daemon[907]: time=2026-09-20T13:33:36.371Z level=INFO msg="Upload complete. (520ms)"4393worker1 # [ 62.975398] nix-grpc-daemon[834]: ts=2026-09-20T13:33:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_in=7524394worker1 # [ 62.988447] nix-grpc-daemon[834]: ts=2026-09-20T13:33:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444395worker1 # [ 63.003072] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4396worker1 # [ 63.005826] nix-daemon[898]: reaped child process 986, status = succeeded4397client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4398client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4399worker1 # [ 63.022630] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4400lb # [ 63.267674] envoy[911]: conn peer=[2001:db8:1::1]:52812 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=27924401lb # [ 63.272805] envoy[911]: conn peer=[2001:db8:1::1]:52822 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=1204402lb # [ 63.277865] envoy[911]: conn peer=[2001:db8:1::1]:52836 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=164403lb # [ 63.283432] envoy[911]: conn peer=[2001:db8:1::1]:52850 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=184404client # farm-top> LOG-farm-top-t14405worker1 # [ 64.848996] nix-daemon[898]: reaped child process 1000, status = succeeded4406worker1 # [ 65.015445] nix-daemon[898]: accepted connection from pid 1015, user nix-grpc-daemon (trusted)4407worker1 # [ 65.036657] nix-daemon[898]: reaped child process 1018, status = succeeded4408lb # [ 65.038569] niks3-server[1180]: 2026/09/20 13:33:38 INFO Received uploads request method=POST path=/api/pending_closures4409worker1 # [ 65.064529] nix-grpc-daemon[907]: time=2026-09-20T13:33:38.469Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4410worker1 # [ 65.066143] nix-grpc-daemon[907]: time=2026-09-20T13:33:38.469Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4411lb # [ 65.078679] niks3-server[1180]: 2026/09/20 13:33:38 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4412worker1 # [ 65.111075] nix-grpc-daemon[907]: time=2026-09-20T13:33:38.517Z level=INFO msg="Uploading 1 narinfos"4413lb # [ 65.098941] niks3-server[1180]: 2026/09/20 13:33:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4414lb # [ 65.108314] niks3-server[1180]: 2026/09/20 13:33:38 INFO Signed narinfos id=6 count=14415lb # [ 65.114030] niks3-server[1180]: 2026/09/20 13:33:38 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4416lb # [ 65.120198] niks3-server[1180]: 2026/09/20 13:33:38 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4417lb # [ 65.127804] niks3-server[1180]: 2026/09/20 13:33:38 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4418worker1 # [ 65.143992] nix-grpc-daemon[907]: time=2026-09-20T13:33:38.550Z level=INFO msg="Upload complete. (308ms)"4419lb # [ 65.132050] niks3-server[1180]: 2026/09/20 13:33:38 INFO Completed upload id=64420lb # [ 65.135799] niks3-server[1180]: 2026/09/20 13:33:38 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4421worker1 # [ 65.152689] nix-grpc-daemon[834]: ts=2026-09-20T13:33:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14422worker1 # [ 65.159079] nix-daemon[898]: reaped child process 998, status = succeeded4423client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.86 seconds)4424client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4425client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.26 seconds)4426client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4427client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds)4428client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4429client # copying 3 paths...4430client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4431client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4432client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4433client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4434(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.52 seconds)4435subtest: repeat is answered Cached by the scheduler without building4436worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4437worker1 # finding garbage collector roots...4438worker1 # deleting specified paths...4439worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4440worker1 # deleting unused links...4441worker1 # note: hard linking is currently saving 0.0 KiB4442worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.28 seconds)4443worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4444worker2 # finding garbage collector roots...4445worker2 # deleting specified paths...4446worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4447worker2 # deleting unused links...4448worker2 # note: hard linking is currently saving 0.0 KiB4449worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.31 seconds)4450client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24451worker1 # [ 66.703408] nix-grpc-daemon[834]: ts=2026-09-20T13:33:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:494444452client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4453client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4454worker1 # [ 66.726141] nix-grpc-daemon[834]: ts=2026-09-20T13:33:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444455worker1 # [ 66.738501] nix-grpc-daemon[907]: time=2026-09-20T13:33:40.143Z level=INFO msg="All 2 paths already cached"4456worker2 # [ 66.759626] nix-grpc-daemon[838]: ts=2026-09-20T13:33:40Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14457client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.35 seconds)4458client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4459client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.22 seconds)4460lb: must succeed: curl -sf localhost:9901/clusters4461lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4462lb: must succeed: curl -sf localhost:9901/clusters4463lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4464worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4465worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4466(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.48 seconds)4467subtest: inputs already in the cache are not sent to the scheduler4468lb: must succeed: curl -sf localhost:9901/clusters4469lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4470lb: must succeed: curl -sf localhost:9901/clusters4471lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4472worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4473worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4474client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24475worker2 # [ 67.794133] nix-grpc-daemon[838]: ts=2026-09-20T13:33:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:453404476client # don't know how to build these paths:4477client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4478worker2 # [ 67.806269] nix-grpc-daemon[838]: ts=2026-09-20T13:33:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453404479worker2 # [ 67.839213] nix-grpc-daemon[838]: ts=2026-09-20T13:33:41Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04480worker2 # [ 67.845877] nix-grpc-daemon[838]: ts=2026-09-20T13:33:41Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14481worker1 # [ 67.887307] nix-grpc-daemon[834]: ts=2026-09-20T13:33:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444482worker1 # [ 67.905833] nix-grpc-daemon[834]: ts=2026-09-20T13:33:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444483worker1 # [ 67.923395] nix-grpc-daemon[907]: time=2026-09-20T13:33:41.328Z level=INFO msg="All 2 paths already cached"4484client # copying 1 paths to 'lb:50051'...4485client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4486worker1 # [ 67.936880] nix-grpc-daemon[834]: ts=2026-09-20T13:33:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444487worker1 # [ 67.953141] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4488worker1 # [ 68.156785] nix-daemon[898]: accepted connection from pid 1035, user nix-grpc-daemon (trusted)4489worker1 # [ 68.178986] nix-daemon[898]: reaped child process 1038, status = succeeded4490worker1 # [ 68.366291] nix-daemon[898]: accepted connection from pid 1040, user nix-grpc-daemon (trusted)4491worker1 # [ 68.386085] nix-daemon[898]: reaped child process 1043, status = succeeded4492lb # [ 68.390373] niks3-server[1180]: 2026/09/20 13:33:41 INFO Received uploads request method=POST path=/api/pending_closures4493worker1 # [ 68.411965] nix-grpc-daemon[907]: time=2026-09-20T13:33:41.817Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4494worker1 # [ 68.413836] nix-grpc-daemon[907]: time=2026-09-20T13:33:41.817Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4495lb # [ 68.418358] niks3-server[1180]: 2026/09/20 13:33:41 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4496lb # [ 68.434868] niks3-server[1180]: 2026/09/20 13:33:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4497worker1 # [ 68.454218] nix-grpc-daemon[907]: time=2026-09-20T13:33:41.860Z level=INFO msg="Uploading 1 narinfos"4498lb # [ 68.447688] niks3-server[1180]: 2026/09/20 13:33:41 INFO Signed narinfos id=7 count=14499lb # [ 68.455886] niks3-server[1180]: 2026/09/20 13:33:41 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4500lb # [ 68.466625] niks3-server[1180]: 2026/09/20 13:33:41 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4501worker1 # [ 68.486862] nix-grpc-daemon[907]: time=2026-09-20T13:33:41.893Z level=INFO msg="Upload complete. (515ms)"4502lb # [ 68.474724] niks3-server[1180]: 2026/09/20 13:33:41 INFO Completed upload id=74503worker1 # [ 68.491992] nix-grpc-daemon[834]: ts=2026-09-20T13:33:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_in=7524504lb # [ 68.479008] niks3-server[1180]: 2026/09/20 13:33:41 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4505worker1 # [ 68.497526] nix-grpc-daemon[834]: ts=2026-09-20T13:33:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444506worker1 # [ 68.508627] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4507worker1 # [ 68.510515] nix-daemon[898]: reaped child process 1033, status = succeeded4508client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4509worker1 # [ 68.521043] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4510client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4511worker1 # [ 69.744625] nix-daemon[898]: reaped child process 1047, status = succeeded4512client # farm-top> LOG-farm-top-t1b4513worker1 # [ 69.910388] nix-daemon[898]: accepted connection from pid 1062, user nix-grpc-daemon (trusted)4514worker1 # [ 69.931108] nix-daemon[898]: reaped child process 1065, status = succeeded4515lb # [ 69.937158] niks3-server[1180]: 2026/09/20 13:33:43 INFO Received uploads request method=POST path=/api/pending_closures4516worker1 # [ 69.964466] nix-grpc-daemon[907]: time=2026-09-20T13:33:43.369Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4517worker1 # [ 69.966037] nix-grpc-daemon[907]: time=2026-09-20T13:33:43.369Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4518lb # [ 69.987605] niks3-server[1180]: 2026/09/20 13:33:43 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4519lb # [ 70.000527] niks3-server[1180]: 2026/09/20 13:33:43 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4520lb # [ 70.011086] niks3-server[1180]: 2026/09/20 13:33:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4521worker1 # [ 70.030091] nix-grpc-daemon[907]: time=2026-09-20T13:33:43.436Z level=INFO msg="Uploading 1 narinfos"4522lb # [ 70.017603] niks3-server[1180]: 2026/09/20 13:33:43 INFO Signed narinfos id=8 count=14523lb # [ 70.023864] niks3-server[1180]: 2026/09/20 13:33:43 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4524lb # [ 70.035383] niks3-server[1180]: 2026/09/20 13:33:43 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4525worker1 # [ 70.053861] nix-grpc-daemon[907]: time=2026-09-20T13:33:43.460Z level=INFO msg="Upload complete. (324ms)"4526lb # [ 70.041683] niks3-server[1180]: 2026/09/20 13:33:43 INFO Completed upload id=84527lb # [ 70.045498] niks3-server[1180]: 2026/09/20 13:33:43 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4528worker1 # [ 70.060468] nix-grpc-daemon[834]: ts=2026-09-20T13:33:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14529worker1 # [ 70.068273] nix-daemon[898]: reaped child process 1045, status = succeeded4530client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.58 seconds)4531client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4532client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.24 seconds)4533lb: must succeed: curl -sf localhost:9901/clusters4534lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4535lb: must succeed: curl -sf localhost:9901/clusters4536lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4537worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4538worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4539(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.21 seconds)4540subtest: two clients wanting the same drv build it once4541client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4542client # [ 70.645034] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4543client # Running as unit: dedup1.service; invocation ID: 0da4e81bba3b484fa7d68fddf3bc91534544client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.11 seconds)4545client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4546client # [ 70.896797] nix[891]: don't know how to build these paths:4547client # [ 70.897919] nix[891]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4548worker1 # [ 70.894832] nix-grpc-daemon[834]: ts=2026-09-20T13:33:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:494444549worker2 # [ 70.914760] nix-grpc-daemon[838]: ts=2026-09-20T13:33:44Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04550worker2 # [ 70.915247] nix-grpc-daemon[838]: ts=2026-09-20T13:33:44Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14551worker2 # [ 70.957295] nix-grpc-daemon[838]: ts=2026-09-20T13:33:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453404552worker2 # [ 70.974555] nix-grpc-daemon[838]: ts=2026-09-20T13:33:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453404553client # [ 70.988434] nix[891]: copying 1 paths to 'lb:50051'...4554worker2 # [ 70.991355] nix-grpc-daemon[838]: ts=2026-09-20T13:33:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453404555client # [ 70.992402] nix[891]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4556worker2 # [ 71.012451] nix-daemon[913]: accepted connection from pid 838, user nix-grpc-daemon (trusted)4557worker2 # [ 71.045332] nix-grpc-daemon[838]: ts=2026-09-20T13:33:44Z level=info event=niks3_push_started pid=9604558worker2 # [ 71.325581] nix-grpc-daemon[960]: time=2026-09-20T13:33:44.698Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""4559worker2 # [ 71.534102] nix-daemon[913]: accepted connection from pid 973, user nix-grpc-daemon (trusted)4560worker2 # [ 71.555644] nix-daemon[913]: reaped child process 976, status = succeeded4561worker2 # [ 71.747044] nix-daemon[913]: accepted connection from pid 978, user nix-grpc-daemon (trusted)4562worker2 # [ 71.768493] nix-daemon[913]: reaped child process 981, status = succeeded4563client # [ 71.771879] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4564lb # [ 71.769448] niks3-server[1180]: 2026/09/20 13:33:45 INFO Received uploads request method=POST path=/api/pending_closures4565client # Running as unit: dedup2.service; invocation ID: 542dc1c3a7d04c5da43528424500fc514566worker2 # [ 71.794578] nix-grpc-daemon[960]: time=2026-09-20T13:33:45.169Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4567worker2 # [ 71.799002] nix-grpc-daemon[960]: time=2026-09-20T13:33:45.171Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4568client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.14 seconds)4569lb # [ 71.820174] niks3-server[1180]: 2026/09/20 13:33:45 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4570lb # [ 71.836160] niks3-server[1180]: 2026/09/20 13:33:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4571worker2 # [ 71.856954] nix-grpc-daemon[960]: time=2026-09-20T13:33:45.231Z level=INFO msg="Uploading 1 narinfos"4572lb # [ 71.849046] niks3-server[1180]: 2026/09/20 13:33:45 INFO Signed narinfos id=9 count=14573lb # [ 71.857125] niks3-server[1180]: 2026/09/20 13:33:45 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4574lb # [ 71.870311] niks3-server[1180]: 2026/09/20 13:33:45 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4575lb # [ 71.881117] niks3-server[1180]: 2026/09/20 13:33:45 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4576lb # [ 71.886306] niks3-server[1180]: 2026/09/20 13:33:45 INFO Completed upload id=94577worker2 # [ 71.900848] nix-grpc-daemon[960]: time=2026-09-20T13:33:45.275Z level=INFO msg="Upload complete. (566ms)"4578worker2 # [ 71.915062] nix-grpc-daemon[838]: ts=2026-09-20T13:33:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45340 duration_s=0 paths=1 nar_bytes_in=4324579worker2 # [ 71.934871] nix-grpc-daemon[838]: ts=2026-09-20T13:33:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453404580worker2 # [ 71.945610] nix-daemon[913]: accepted connection from pid 838, user nix-grpc-daemon (trusted)4581worker2 # [ 71.966686] nix-daemon[913]: reaped child process 958, status = succeeded4582client # [ 71.971846] nix[891]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4583worker2 # [ 71.980154] nix-daemon[913]: accepted connection from pid 838, user nix-grpc-daemon (trusted)4584worker2 # [ 72.090804] nix-grpc-daemon[838]: ts=2026-09-20T13:33:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:453404585client # [ 72.100669] nix[906]: this derivation will be built:4586worker1 # [ 72.107168] nix-grpc-daemon[834]: ts=2026-09-20T13:33:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:494444587worker1 # [ 72.119959] nix-grpc-daemon[834]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' from 'http://lb:5751'...4588client # [ 72.140795] nix[906]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4589worker2 # [ 72.155346] nix-grpc-daemon[838]: ts=2026-09-20T13:33:45Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04590worker2 # [ 72.163234] nix-grpc-daemon[838]: ts=2026-09-20T13:33:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453404591client # [ 72.170024] nix[906]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4592lb # [ 73.266280] envoy[911]: conn peer=[2001:db8:1::1]:52862 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9592 tx=3895 ms=75794593lb # [ 73.271498] envoy[911]: conn peer=[2001:db8:1::1]:58470 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=814594lb # [ 73.276389] envoy[911]: conn peer=[2001:db8:1::1]:58478 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4172 tx=1728 ms=22874595client: must succeed: sleep 24596client: (finished: must succeed: sleep 2, in 2.07 seconds)4597??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4598 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394599client: waiting for success: ! systemctl is-active dedup1 dedup24600??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4601 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394602worker2 # [ 76.611094] nix-daemon[913]: reaped child process 989, status = succeeded4603worker2 # [ 76.770291] nix-daemon[913]: accepted connection from pid 1020, user nix-grpc-daemon (trusted)4604worker2 # [ 76.789040] nix-daemon[913]: reaped child process 1023, status = succeeded4605lb # [ 76.788861] niks3-server[1180]: 2026/09/20 13:33:50 INFO Received uploads request method=POST path=/api/pending_closures4606worker2 # [ 76.814089] nix-grpc-daemon[960]: time=2026-09-20T13:33:50.187Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4607worker2 # [ 76.818017] nix-grpc-daemon[960]: time=2026-09-20T13:33:50.187Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4608lb # [ 76.830876] niks3-server[1180]: 2026/09/20 13:33:50 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4609lb # [ 76.845898] niks3-server[1180]: 2026/09/20 13:33:50 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4610lb # [ 76.860157] niks3-server[1180]: 2026/09/20 13:33:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4611lb # [ 76.866862] niks3-server[1180]: 2026/09/20 13:33:50 INFO Signed narinfos id=10 count=14612worker2 # [ 76.879831] nix-grpc-daemon[960]: time=2026-09-20T13:33:50.253Z level=INFO msg="Uploading 1 narinfos"4613lb # [ 76.870250] niks3-server[1180]: 2026/09/20 13:33:50 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4614lb # [ 76.884157] niks3-server[1180]: 2026/09/20 13:33:50 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4615lb # [ 76.891251] niks3-server[1180]: 2026/09/20 13:33:50 INFO Completed upload id=104616worker2 # [ 76.903594] nix-grpc-daemon[960]: time=2026-09-20T13:33:50.278Z level=INFO msg="Upload complete. (305ms)"4617lb # [ 76.894405] niks3-server[1180]: 2026/09/20 13:33:50 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4618worker2 # [ 76.916578] nix-grpc-daemon[838]: ts=2026-09-20T13:33:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45340 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14619worker2 # [ 76.932331] nix-grpc-daemon[838]: ts=2026-09-20T13:33:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45340 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14620worker2 # [ 76.938736] nix-daemon[913]: reaped child process 985, status = succeeded4621client # [ 76.963318] systemd[1]: dedup1.service: Deactivated successfully.4622client # [ 76.970144] systemd[1]: dedup1.service: Consumed 169ms CPU time over 6.317s wall clock time, 37.1M memory peak, 8.5K incoming IP traffic, 8.9K outgoing IP traffic.4623client # [ 77.009331] systemd[1]: dedup2.service: Deactivated successfully.4624client # [ 77.012116] systemd[1]: dedup2.service: Consumed 185ms CPU time over 5.238s wall clock time, 31M memory peak, 7.8K incoming IP traffic, 6.6K outgoing IP traffic.4625client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds)4626client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4627client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4628worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4629worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4630worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4631worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4632worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4633worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4634worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4635worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4636(finished: subtest: two clients wanting the same drv build it once, in 7.54 seconds)4637subtest: build hook: nix-daemon with builders = grpc://lb4638client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4639client # [ 78.324645] systemd[1]: Started Nix Daemon.4640client # [ 78.508298] nix-daemon[947]: accepted connection from pid 944, user root (trusted)4641worker2 # [ 78.702733] nix-grpc-daemon[838]: ts=2026-09-20T13:33:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:453404642worker1 # [ 78.709256] nix-grpc-daemon[834]: ts=2026-09-20T13:33:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444643worker2 # [ 78.723037] nix-grpc-daemon[838]: ts=2026-09-20T13:33:52Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04644worker2 # [ 78.729654] nix-grpc-daemon[838]: ts=2026-09-20T13:33:52Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14645worker1 # [ 78.770418] nix-grpc-daemon[834]: ts=2026-09-20T13:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444646worker1 # [ 78.788764] nix-grpc-daemon[834]: ts=2026-09-20T13:33:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444647worker1 # [ 78.807289] nix-grpc-daemon[834]: ts=2026-09-20T13:33:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444648worker1 # [ 78.827582] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4649worker1 # [ 79.043198] nix-daemon[898]: accepted connection from pid 1095, user nix-grpc-daemon (trusted)4650worker1 # [ 79.063401] nix-daemon[898]: reaped child process 1098, status = succeeded4651worker1 # [ 79.234252] nix-daemon[898]: accepted connection from pid 1100, user nix-grpc-daemon (trusted)4652worker1 # [ 79.255820] nix-daemon[898]: reaped child process 1103, status = succeeded4653lb # [ 79.255647] niks3-server[1180]: 2026/09/20 13:33:52 INFO Received uploads request method=POST path=/api/pending_closures4654worker1 # [ 79.278939] nix-grpc-daemon[907]: time=2026-09-20T13:33:52.684Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4655worker1 # [ 79.280731] nix-grpc-daemon[907]: time=2026-09-20T13:33:52.684Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4656lb # [ 79.284754] niks3-server[1180]: 2026/09/20 13:33:52 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4657lb # [ 79.303217] niks3-server[1180]: 2026/09/20 13:33:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4658worker1 # [ 79.322354] nix-grpc-daemon[907]: time=2026-09-20T13:33:52.728Z level=INFO msg="Uploading 1 narinfos"4659lb # [ 79.317193] niks3-server[1180]: 2026/09/20 13:33:52 INFO Signed narinfos id=11 count=14660lb # [ 79.325102] niks3-server[1180]: 2026/09/20 13:33:52 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4661lb # [ 79.337394] niks3-server[1180]: 2026/09/20 13:33:52 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4662lb # [ 79.342363] niks3-server[1180]: 2026/09/20 13:33:52 INFO Completed upload id=114663worker1 # [ 79.354683] nix-grpc-daemon[907]: time=2026-09-20T13:33:52.761Z level=INFO msg="Upload complete. (510ms)"4664lb # [ 79.345639] niks3-server[1180]: 2026/09/20 13:33:52 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4665worker1 # [ 79.364460] nix-grpc-daemon[834]: ts=2026-09-20T13:33:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_in=5124666worker1 # [ 79.379799] nix-grpc-daemon[834]: ts=2026-09-20T13:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444667worker1 # [ 79.387400] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4668worker1 # [ 79.393745] nix-daemon[898]: reaped child process 1093, status = succeeded4669worker1 # [ 79.410180] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4670worker1 # [ 80.643412] nix-daemon[898]: reaped child process 1107, status = succeeded4671worker1 # [ 80.819238] nix-daemon[898]: accepted connection from pid 1122, user nix-grpc-daemon (trusted)4672worker1 # [ 80.842426] nix-daemon[898]: reaped child process 1125, status = succeeded4673lb # [ 80.842449] niks3-server[1180]: 2026/09/20 13:33:54 INFO Received uploads request method=POST path=/api/pending_closures4674worker1 # [ 80.868499] nix-grpc-daemon[907]: time=2026-09-20T13:33:54.271Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4675worker1 # [ 80.870087] nix-grpc-daemon[907]: time=2026-09-20T13:33:54.272Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4676lb # [ 80.891583] niks3-server[1180]: 2026/09/20 13:33:54 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4677lb # [ 80.903630] niks3-server[1180]: 2026/09/20 13:33:54 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4678worker1 # [ 80.931521] nix-grpc-daemon[907]: time=2026-09-20T13:33:54.337Z level=INFO msg="Uploading 1 narinfos"4679lb # [ 80.918232] niks3-server[1180]: 2026/09/20 13:33:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4680lb # [ 80.926792] niks3-server[1180]: 2026/09/20 13:33:54 INFO Signed narinfos id=12 count=14681lb # [ 80.931549] niks3-server[1180]: 2026/09/20 13:33:54 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4682lb # [ 80.948322] niks3-server[1180]: 2026/09/20 13:33:54 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4683lb # [ 80.958183] niks3-server[1180]: 2026/09/20 13:33:54 INFO Completed upload id=124684worker1 # [ 80.971180] nix-grpc-daemon[907]: time=2026-09-20T13:33:54.377Z level=INFO msg="Upload complete. (348ms)"4685lb # [ 80.962901] niks3-server[1180]: 2026/09/20 13:33:54 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4686worker2 # [ 80.989869] nix-grpc-daemon[838]: ts=2026-09-20T13:33:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:453404687worker1 # [ 80.985472] nix-grpc-daemon[834]: ts=2026-09-20T13:33:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14688worker2 # [ 81.000694] nix-grpc-daemon[838]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4689worker1 # [ 81.004742] nix-daemon[898]: reaped child process 1105, status = succeeded4690worker1 # [ 81.044104] nix-grpc-daemon[834]: ts=2026-09-20T13:33:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:494444691worker1 # [ 81.052959] nix-grpc-daemon[834]: ts=2026-09-20T13:33:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_out=1284692worker2 # [ 81.311294] nix-grpc-daemon[838]: ts=2026-09-20T13:33:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:453404693worker1 # [ 81.315018] nix-grpc-daemon[834]: ts=2026-09-20T13:33:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444694worker1 # [ 81.326377] nix-grpc-daemon[907]: time=2026-09-20T13:33:54.732Z level=INFO msg="All 1 paths already cached"4695worker2 # [ 81.345943] nix-grpc-daemon[838]: ts=2026-09-20T13:33:54Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04696worker2 # [ 81.352508] nix-grpc-daemon[838]: ts=2026-09-20T13:33:54Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14697worker1 # [ 81.355749] nix-grpc-daemon[834]: ts=2026-09-20T13:33:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444698worker1 # [ 81.372818] nix-grpc-daemon[834]: ts=2026-09-20T13:33:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444699worker1 # [ 81.387672] nix-grpc-daemon[907]: time=2026-09-20T13:33:54.794Z level=INFO msg="All 1 paths already cached"4700worker1 # [ 81.398227] nix-grpc-daemon[834]: ts=2026-09-20T13:33:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444701worker1 # [ 81.414364] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4702worker1 # [ 81.633830] nix-daemon[898]: accepted connection from pid 1129, user nix-grpc-daemon (trusted)4703worker1 # [ 81.657744] nix-daemon[898]: reaped child process 1132, status = succeeded4704worker1 # [ 81.832453] nix-daemon[898]: accepted connection from pid 1134, user nix-grpc-daemon (trusted)4705worker1 # [ 81.853077] nix-daemon[898]: reaped child process 1137, status = succeeded4706lb # [ 81.849158] niks3-server[1180]: 2026/09/20 13:33:55 INFO Received uploads request method=POST path=/api/pending_closures4707worker1 # [ 81.873452] nix-grpc-daemon[907]: time=2026-09-20T13:33:55.279Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4708worker1 # [ 81.875040] nix-grpc-daemon[907]: time=2026-09-20T13:33:55.279Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4709lb # [ 81.880492] niks3-server[1180]: 2026/09/20 13:33:55 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4710worker1 # [ 81.907551] nix-grpc-daemon[907]: time=2026-09-20T13:33:55.313Z level=INFO msg="Uploading 1 narinfos"4711lb # [ 81.897586] niks3-server[1180]: 2026/09/20 13:33:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4712lb # [ 81.909317] niks3-server[1180]: 2026/09/20 13:33:55 INFO Signed narinfos id=13 count=14713lb # [ 81.917523] niks3-server[1180]: 2026/09/20 13:33:55 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4714worker1 # [ 81.942298] nix-grpc-daemon[907]: time=2026-09-20T13:33:55.348Z level=INFO msg="Upload complete. (508ms)"4715lb # [ 81.929896] niks3-server[1180]: 2026/09/20 13:33:55 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4716lb # [ 81.935762] niks3-server[1180]: 2026/09/20 13:33:55 INFO Completed upload id=134717lb # [ 81.939559] niks3-server[1180]: 2026/09/20 13:33:55 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4718worker1 # [ 81.953171] nix-grpc-daemon[834]: ts=2026-09-20T13:33:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_in=6244719worker1 # [ 81.964316] nix-grpc-daemon[834]: ts=2026-09-20T13:33:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444720worker1 # [ 81.971646] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4721worker1 # [ 81.977869] nix-daemon[898]: reaped child process 1127, status = succeeded4722worker1 # [ 81.993075] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4723lb # [ 83.268749] envoy[911]: conn peer=[2001:db8:1::1]:36660 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2975 tx=1357 ms=60374724worker1 # [ 83.421876] nix-daemon[898]: reaped child process 1141, status = succeeded4725worker1 # [ 83.602200] nix-daemon[898]: accepted connection from pid 1156, user nix-grpc-daemon (trusted)4726worker1 # [ 83.624348] nix-daemon[898]: reaped child process 1159, status = succeeded4727lb # [ 83.274485] envoy[911]: conn peer=[2001:db8:1::1]:36676 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1579 tx=1487 ms=48924728lb # [ 83.274654] envoy[911]: conn peer=[2001:db8:1::1]:39630 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=334 ms=354729lb # [ 83.274772] envoy[911]: conn peer=[2001:db8:1::1]:36678 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3418 tx=1775 ms=23704730lb # [ 83.626094] niks3-server[1180]: 2026/09/20 13:33:57 INFO Received uploads request method=POST path=/api/pending_closures4731worker1 # [ 83.668490] nix-grpc-daemon[907]: time=2026-09-20T13:33:57.072Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4732worker1 # [ 83.670225] nix-grpc-daemon[907]: time=2026-09-20T13:33:57.073Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4733lb # [ 83.689595] niks3-server[1180]: 2026/09/20 13:33:57 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4734lb # [ 83.702362] niks3-server[1180]: 2026/09/20 13:33:57 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4735lb # [ 83.714419] niks3-server[1180]: 2026/09/20 13:33:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4736worker1 # [ 83.732195] nix-grpc-daemon[907]: time=2026-09-20T13:33:57.138Z level=INFO msg="Uploading 1 narinfos"4737lb # [ 83.722040] niks3-server[1180]: 2026/09/20 13:33:57 INFO Signed narinfos id=14 count=14738lb # [ 83.726908] niks3-server[1180]: 2026/09/20 13:33:57 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4739lb # [ 83.734918] niks3-server[1180]: 2026/09/20 13:33:57 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4740worker1 # [ 83.755203] nix-grpc-daemon[907]: time=2026-09-20T13:33:57.161Z level=INFO msg="Upload complete. (350ms)"4741lb # [ 83.743882] niks3-server[1180]: 2026/09/20 13:33:57 INFO Completed upload id=144742lb # [ 83.748125] niks3-server[1180]: 2026/09/20 13:33:57 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4743worker1 # [ 83.763470] nix-grpc-daemon[834]: ts=2026-09-20T13:33:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14744worker2 # [ 83.770871] nix-grpc-daemon[838]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:453404745worker1 # [ 83.774470] nix-daemon[898]: reaped child process 1139, status = succeeded4746worker2 # [ 83.781382] nix-grpc-daemon[838]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4747worker2 # [ 83.837563] nix-grpc-daemon[838]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:453404748worker2 # [ 83.846041] nix-grpc-daemon[838]: ts=2026-09-20T13:33:57Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45340 duration_s=0 paths=1 nar_bytes_out=1764749worker1 # [ 84.164376] nix-grpc-daemon[834]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:494444750worker1 # [ 84.172169] nix-grpc-daemon[834]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444751worker1 # [ 84.182966] nix-grpc-daemon[907]: time=2026-09-20T13:33:57.589Z level=INFO msg="All 2 paths already cached"4752worker2 # [ 84.205088] nix-grpc-daemon[838]: ts=2026-09-20T13:33:57Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04753worker2 # [ 84.211998] nix-grpc-daemon[838]: ts=2026-09-20T13:33:57Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14754worker1 # [ 84.215633] nix-grpc-daemon[834]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444755worker1 # [ 84.232807] nix-grpc-daemon[834]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444756worker1 # [ 84.250179] nix-grpc-daemon[907]: time=2026-09-20T13:33:57.656Z level=INFO msg="All 2 paths already cached"4757worker1 # [ 84.261944] nix-grpc-daemon[834]: ts=2026-09-20T13:33:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444758worker1 # [ 84.277604] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4759worker1 # [ 84.492368] nix-daemon[898]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)4760worker1 # [ 84.511743] nix-daemon[898]: reaped child process 1166, status = succeeded4761worker1 # [ 84.702708] nix-daemon[898]: accepted connection from pid 1168, user nix-grpc-daemon (trusted)4762worker1 # [ 84.722068] nix-daemon[898]: reaped child process 1171, status = succeeded4763lb # [ 84.721971] niks3-server[1180]: 2026/09/20 13:33:58 INFO Received uploads request method=POST path=/api/pending_closures4764worker1 # [ 84.741341] nix-grpc-daemon[907]: time=2026-09-20T13:33:58.146Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4765worker1 # [ 84.742904] nix-grpc-daemon[907]: time=2026-09-20T13:33:58.147Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4766lb # [ 84.751327] niks3-server[1180]: 2026/09/20 13:33:58 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4767lb # [ 84.768573] niks3-server[1180]: 2026/09/20 13:33:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4768worker1 # [ 84.786870] nix-grpc-daemon[907]: time=2026-09-20T13:33:58.192Z level=INFO msg="Uploading 1 narinfos"4769lb # [ 84.781679] niks3-server[1180]: 2026/09/20 13:33:58 INFO Signed narinfos id=15 count=14770lb # [ 84.789385] niks3-server[1180]: 2026/09/20 13:33:58 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4771lb # [ 84.798470] niks3-server[1180]: 2026/09/20 13:33:58 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4772lb # [ 84.806760] niks3-server[1180]: 2026/09/20 13:33:58 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4773lb # [ 84.811551] niks3-server[1180]: 2026/09/20 13:33:58 INFO Completed upload id=154774worker1 # [ 84.824465] nix-grpc-daemon[907]: time=2026-09-20T13:33:58.230Z level=INFO msg="Upload complete. (527ms)"4775worker1 # [ 84.833015] nix-grpc-daemon[834]: ts=2026-09-20T13:33:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_in=7524776worker1 # [ 84.850464] nix-grpc-daemon[834]: ts=2026-09-20T13:33:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444777worker1 # [ 84.858971] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4778worker1 # [ 84.866207] nix-daemon[898]: reaped child process 1161, status = succeeded4779worker1 # [ 84.879133] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4780worker1 # [ 86.132349] nix-daemon[898]: reaped child process 1175, status = succeeded4781worker1 # [ 86.314539] nix-daemon[898]: accepted connection from pid 1190, user nix-grpc-daemon (trusted)4782worker1 # [ 86.334122] nix-daemon[898]: reaped child process 1193, status = succeeded4783lb # [ 86.338316] niks3-server[1180]: 2026/09/20 13:33:59 INFO Received uploads request method=POST path=/api/pending_closures4784worker1 # [ 86.360574] nix-grpc-daemon[907]: time=2026-09-20T13:33:59.765Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4785worker1 # [ 86.362409] nix-grpc-daemon[907]: time=2026-09-20T13:33:59.765Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4786lb # [ 86.379876] niks3-server[1180]: 2026/09/20 13:33:59 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4787worker1 # [ 86.406357] nix-grpc-daemon[907]: time=2026-09-20T13:33:59.812Z level=INFO msg="Uploading 1 narinfos"4788lb # [ 86.395626] niks3-server[1180]: 2026/09/20 13:33:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4789lb # [ 86.405559] niks3-server[1180]: 2026/09/20 13:33:59 INFO Signed narinfos id=16 count=14790lb # [ 86.407547] niks3-server[1180]: 2026/09/20 13:33:59 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4791lb # [ 86.414202] niks3-server[1180]: 2026/09/20 13:33:59 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4792lb # [ 86.421596] niks3-server[1180]: 2026/09/20 13:33:59 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4793worker1 # [ 86.438092] nix-grpc-daemon[907]: time=2026-09-20T13:33:59.844Z level=INFO msg="Upload complete. (344ms)"4794lb # [ 86.425652] niks3-server[1180]: 2026/09/20 13:33:59 INFO Completed upload id=164795lb # [ 86.428090] niks3-server[1180]: 2026/09/20 13:33:59 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4796worker1 # [ 86.449113] nix-grpc-daemon[834]: ts=2026-09-20T13:33:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14797worker1 # [ 86.465849] nix-daemon[898]: reaped child process 1173, status = succeeded4798worker2 # [ 86.495113] nix-grpc-daemon[838]: ts=2026-09-20T13:33:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:453404799worker2 # [ 86.511717] nix-grpc-daemon[838]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4800worker1 # [ 86.557015] nix-grpc-daemon[834]: ts=2026-09-20T13:33:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:494444801worker1 # [ 86.564837] nix-grpc-daemon[834]: ts=2026-09-20T13:33:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=0 paths=1 nar_bytes_out=2324802client # [ 86.652324] nix-daemon[947]: reaped child process 949, status = succeeded4803client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.57 seconds)4804client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4805client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.05 seconds)4806client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4807client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4808client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4809client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds)4810(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.71 seconds)4811subtest: an input only one worker has still reaches the builder4812client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4813client # this derivation will be built:4814client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4815client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4816client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.47 seconds)4817client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4818client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.21 seconds)4819worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4820worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.25 seconds)4821worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4822worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4823client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24824client # [ 88.003985] nix-daemon[947]: accepted connection from pid 1026, user root (trusted)4825client # this derivation will be built:4826client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4827worker2 # [ 88.214709] nix-grpc-daemon[838]: ts=2026-09-20T13:34:01Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:453404828client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4829client # copying 1 paths...4830client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4831worker2 # [ 88.222946] nix-grpc-daemon[838]: ts=2026-09-20T13:34:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453404832worker2 # [ 88.234395] nix-grpc-daemon[838]: ts=2026-09-20T13:34:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453404833worker2 # [ 88.265284] nix-daemon[913]: accepted connection from pid 838, user nix-grpc-daemon (trusted)4834worker2 # [ 88.480951] nix-daemon[913]: accepted connection from pid 1048, user nix-grpc-daemon (trusted)4835worker2 # [ 88.502428] nix-daemon[913]: reaped child process 1051, status = succeeded4836lb # [ 88.502690] niks3-server[1180]: 2026/09/20 13:34:01 INFO Received uploads request method=POST path=/api/pending_closures4837worker2 # [ 88.526713] nix-grpc-daemon[960]: time=2026-09-20T13:34:01.901Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4838worker2 # [ 88.528362] nix-grpc-daemon[960]: time=2026-09-20T13:34:01.901Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4839lb # [ 88.534013] niks3-server[1180]: 2026/09/20 13:34:01 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4840lb # [ 88.552806] niks3-server[1180]: 2026/09/20 13:34:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4841worker2 # [ 88.571390] nix-grpc-daemon[960]: time=2026-09-20T13:34:01.945Z level=INFO msg="Uploading 1 narinfos"4842lb # [ 88.565560] niks3-server[1180]: 2026/09/20 13:34:01 INFO Signed narinfos id=17 count=14843lb # [ 88.573095] niks3-server[1180]: 2026/09/20 13:34:01 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4844lb # [ 88.583865] niks3-server[1180]: 2026/09/20 13:34:01 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4845lb # [ 88.589630] niks3-server[1180]: 2026/09/20 13:34:01 INFO Completed upload id=174846worker2 # [ 88.602547] nix-grpc-daemon[960]: time=2026-09-20T13:34:01.977Z level=INFO msg="Upload complete. (330ms)"4847lb # [ 88.592164] niks3-server[1180]: 2026/09/20 13:34:01 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4848worker2 # [ 88.612406] nix-grpc-daemon[838]: ts=2026-09-20T13:34:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45340 duration_s=0 paths=1 nar_bytes_in=1204849worker2 # [ 88.629123] nix-grpc-daemon[838]: ts=2026-09-20T13:34:02Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04850worker2 # [ 88.633546] nix-grpc-daemon[838]: ts=2026-09-20T13:34:02Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14851worker2 # [ 88.638777] nix-daemon[913]: reaped child process 1046, status = succeeded4852worker2 # [ 88.677616] nix-grpc-daemon[838]: ts=2026-09-20T13:34:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453404853worker2 # [ 88.688992] nix-grpc-daemon[838]: ts=2026-09-20T13:34:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453404854client # copying 2 paths to 'lb:50051'...4855client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4856worker2 # [ 88.705184] nix-grpc-daemon[838]: ts=2026-09-20T13:34:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453404857client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4858worker2 # [ 88.719889] nix-daemon[913]: accepted connection from pid 838, user nix-grpc-daemon (trusted)4859worker2 # [ 88.934859] nix-daemon[913]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)4860worker2 # [ 88.958762] nix-daemon[913]: reaped child process 1060, status = succeeded4861worker2 # [ 89.139782] nix-daemon[913]: accepted connection from pid 1062, user nix-grpc-daemon (trusted)4862worker2 # [ 89.159489] nix-daemon[913]: reaped child process 1065, status = succeeded4863lb # [ 89.158591] niks3-server[1180]: 2026/09/20 13:34:02 INFO Received uploads request method=POST path=/api/pending_closures4864lb # [ 89.169334] niks3-server[1180]: 2026/09/20 13:34:02 INFO Received uploads request method=POST path=/api/pending_closures4865worker2 # [ 89.191522] nix-grpc-daemon[960]: time=2026-09-20T13:34:02.565Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4866worker2 # [ 89.193242] nix-grpc-daemon[960]: time=2026-09-20T13:34:02.565Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4867worker2 # [ 89.195106] nix-grpc-daemon[960]: time=2026-09-20T13:34:02.565Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4868lb # [ 89.190395] niks3-server[1180]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4869lb # [ 89.202271] niks3-server[1180]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4870lb # [ 89.207806] niks3-server[1180]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4871lb # [ 89.213663] niks3-server[1180]: 2026/09/20 13:34:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4872lb # [ 89.219732] niks3-server[1180]: 2026/09/20 13:34:02 INFO Signed narinfos id=18 count=14873worker2 # [ 89.232987] nix-grpc-daemon[960]: time=2026-09-20T13:34:02.607Z level=INFO msg="Uploading 3 narinfos"4874lb # [ 89.222870] niks3-server[1180]: 2026/09/20 13:34:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4875lb # [ 89.227725] niks3-server[1180]: 2026/09/20 13:34:02 INFO Signed narinfos id=19 count=24876lb # [ 89.230657] niks3-server[1180]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4877lb # [ 89.243247] niks3-server[1180]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4878lb # [ 89.248301] niks3-server[1180]: 2026/09/20 13:34:02 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4879lb # [ 89.265177] niks3-server[1180]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4880lb # [ 89.268156] niks3-server[1180]: 2026/09/20 13:34:02 INFO Completed upload id=184881lb # [ 89.271003] niks3-server[1180]: 2026/09/20 13:34:02 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4882lb # [ 89.273385] niks3-server[1180]: 2026/09/20 13:34:02 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4883lb # [ 89.275836] niks3-server[1180]: 2026/09/20 13:34:02 INFO Completed upload id=194884worker2 # [ 89.288742] nix-grpc-daemon[960]: time=2026-09-20T13:34:02.663Z level=INFO msg="Upload complete. (551ms)"4885worker2 # [ 89.301292] nix-grpc-daemon[838]: ts=2026-09-20T13:34:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45340 duration_s=0 paths=2 nar_bytes_in=9844886worker2 # [ 89.320730] nix-grpc-daemon[838]: ts=2026-09-20T13:34:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453404887worker2 # [ 89.324970] nix-daemon[913]: accepted connection from pid 838, user nix-grpc-daemon (trusted)4888worker2 # [ 89.335952] nix-daemon[913]: reaped child process 1055, status = succeeded4889client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4890worker2 # [ 89.344777] nix-daemon[913]: accepted connection from pid 838, user nix-grpc-daemon (trusted)4891client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4892worker2 # [ 90.767027] nix-daemon[913]: reaped child process 1069, status = succeeded4893worker2 # [ 90.929579] nix-daemon[913]: accepted connection from pid 1084, user nix-grpc-daemon (trusted)4894worker2 # [ 90.951088] nix-daemon[913]: reaped child process 1087, status = succeeded4895lb # [ 90.953013] niks3-server[1180]: 2026/09/20 13:34:04 INFO Received uploads request method=POST path=/api/pending_closures4896worker2 # [ 90.976500] nix-grpc-daemon[960]: time=2026-09-20T13:34:04.350Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4897lb # [ 90.986054] niks3-server[1180]: 2026/09/20 13:34:04 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4898lb # [ 90.998391] niks3-server[1180]: 2026/09/20 13:34:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4899worker2 # [ 91.018090] nix-grpc-daemon[960]: time=2026-09-20T13:34:04.392Z level=INFO msg="Uploading 1 narinfos"4900lb # [ 91.009966] niks3-server[1180]: 2026/09/20 13:34:04 INFO Signed narinfos id=20 count=14901lb # [ 91.016675] niks3-server[1180]: 2026/09/20 13:34:04 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4902lb # [ 91.026235] niks3-server[1180]: 2026/09/20 13:34:04 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4903worker2 # [ 91.048621] nix-grpc-daemon[960]: time=2026-09-20T13:34:04.423Z level=INFO msg="Upload complete. (292ms)"4904lb # [ 91.036491] niks3-server[1180]: 2026/09/20 13:34:04 INFO Completed upload id=204905lb # [ 91.039849] niks3-server[1180]: 2026/09/20 13:34:04 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4906worker2 # [ 91.059018] nix-grpc-daemon[838]: ts=2026-09-20T13:34:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45340 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14907worker2 # [ 91.074765] nix-daemon[913]: reaped child process 1067, status = succeeded4908client # copying 1 paths...4909worker2 # [ 91.107044] nix-grpc-daemon[838]: ts=2026-09-20T13:34:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:453404910client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4911worker2 # [ 91.132150] nix-grpc-daemon[838]: ts=2026-09-20T13:34:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:453404912worker2 # [ 91.138542] nix-grpc-daemon[838]: ts=2026-09-20T13:34:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45340 duration_s=0 paths=1 nar_bytes_out=1204913client # [ 91.269468] nix-daemon[947]: reaped child process 1029, status = succeeded4914client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.51 seconds)4915client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24916client # [ 91.503059] nix-daemon[947]: accepted connection from pid 1046, user root (trusted)4917client # this derivation will be built:4918client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4919worker2 # [ 91.712175] nix-grpc-daemon[838]: ts=2026-09-20T13:34:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:453404920client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4921worker2 # [ 91.726961] nix-grpc-daemon[838]: ts=2026-09-20T13:34:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453404922worker2 # [ 91.738006] nix-grpc-daemon[960]: time=2026-09-20T13:34:05.112Z level=INFO msg="All 1 paths already cached"4923worker2 # [ 91.756986] nix-grpc-daemon[838]: ts=2026-09-20T13:34:05Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04924worker2 # [ 91.763973] nix-grpc-daemon[838]: ts=2026-09-20T13:34:05Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14925worker1 # [ 91.769749] nix-grpc-daemon[834]: ts=2026-09-20T13:34:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444926worker1 # [ 91.788387] nix-grpc-daemon[834]: ts=2026-09-20T13:34:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:494444927client # copying 1 paths to 'lb:50051'...4928client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4929worker1 # [ 91.811984] nix-grpc-daemon[834]: ts=2026-09-20T13:34:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:494444930worker1 # [ 91.825417] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4931worker1 # [ 91.856167] nix-grpc-daemon[834]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 1/5)4932worker1 # [ 91.876402] nix-grpc-daemon[834]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 2/5)4933worker1 # [ 91.948623] nix-grpc-daemon[834]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 3/5)4934worker1 # [ 91.993004] nix-grpc-daemon[834]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 654 ms (attempt 4/5)4935worker1 # [ 92.655759] nix-grpc-daemon[834]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4936worker1 # [ 92.664517] nix-grpc-daemon[834]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4937worker1 # [ 92.914950] nix-daemon[898]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)4938worker1 # [ 92.937549] nix-daemon[898]: reaped child process 1209, status = succeeded4939worker1 # [ 93.126548] nix-daemon[898]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)4940worker1 # [ 93.145097] nix-daemon[898]: reaped child process 1214, status = succeeded4941lb # [ 93.149466] niks3-server[1180]: 2026/09/20 13:34:06 INFO Received uploads request method=POST path=/api/pending_closures4942worker1 # [ 93.173104] nix-grpc-daemon[907]: time=2026-09-20T13:34:06.578Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4943worker1 # [ 93.174632] nix-grpc-daemon[907]: time=2026-09-20T13:34:06.578Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4944lb # [ 93.196518] niks3-server[1180]: 2026/09/20 13:34:06 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4945worker1 # [ 93.219395] nix-grpc-daemon[907]: time=2026-09-20T13:34:06.625Z level=INFO msg="Uploading 1 narinfos"4946lb # [ 93.209950] niks3-server[1180]: 2026/09/20 13:34:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4947lb # [ 93.217972] niks3-server[1180]: 2026/09/20 13:34:06 INFO Signed narinfos id=21 count=14948lb # [ 93.222367] niks3-server[1180]: 2026/09/20 13:34:06 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4949lb # [ 93.229151] niks3-server[1180]: 2026/09/20 13:34:06 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4950worker1 # [ 93.246291] nix-grpc-daemon[907]: time=2026-09-20T13:34:06.652Z level=INFO msg="Upload complete. (552ms)"4951lb # [ 93.234291] niks3-server[1180]: 2026/09/20 13:34:06 INFO Completed upload id=214952lb # [ 93.237723] niks3-server[1180]: 2026/09/20 13:34:06 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4953worker1 # [ 93.257125] nix-grpc-daemon[834]: ts=2026-09-20T13:34:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 paths=1 nar_bytes_in=5684954worker1 # [ 93.268655] nix-grpc-daemon[834]: ts=2026-09-20T13:34:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:494444955worker1 # [ 93.277778] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4956lb # [ 93.268895] envoy[911]: conn peer=[2001:db8:1::1]:39636 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3901 tx=1989 ms=25954957lb # [ 93.275020] envoy[911]: conn peer=[2001:db8:1::1]:39646 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=834958worker1 # [ 93.291734] nix-daemon[898]: reaped child process 1200, status = succeeded4959lb # [ 93.280387] envoy[911]: conn peer=[2001:db8:1::1]:39652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4339 tx=2199 ms=24144960worker1 # [ 93.300135] nix-daemon[898]: accepted connection from pid 834, user nix-grpc-daemon (trusted)4961client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4962lb # [ 93.285934] envoy[911]: conn peer=[2001:db8:1::1]:39658 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=304963lb # [ 93.290782] envoy[911]: conn peer=[2001:db8:1::1]:39674 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4263 tx=1771 ms=29864964lb # [ 93.296821] envoy[911]: conn peer=[2001:db8:1::1]:43538 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=664965client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4966worker1 # [ 94.334151] nix-daemon[898]: reaped child process 1218, status = succeeded4967worker1 # [ 94.515025] nix-daemon[898]: accepted connection from pid 1233, user nix-grpc-daemon (trusted)4968worker1 # [ 94.536361] nix-daemon[898]: reaped child process 1236, status = succeeded4969lb # [ 94.537890] niks3-server[1180]: 2026/09/20 13:34:07 INFO Received uploads request method=POST path=/api/pending_closures4970worker1 # [ 94.560419] nix-grpc-daemon[907]: time=2026-09-20T13:34:07.965Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4971lb # [ 94.574061] niks3-server[1180]: 2026/09/20 13:34:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4972worker1 # [ 94.596284] nix-grpc-daemon[907]: time=2026-09-20T13:34:08.002Z level=INFO msg="Uploading 1 narinfos"4973lb # [ 94.589749] niks3-server[1180]: 2026/09/20 13:34:07 INFO Signed narinfos id=22 count=14974lb # [ 94.596283] niks3-server[1180]: 2026/09/20 13:34:07 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4975lb # [ 94.603744] niks3-server[1180]: 2026/09/20 13:34:07 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4976lb # [ 94.611306] niks3-server[1180]: 2026/09/20 13:34:08 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4977worker1 # [ 94.627746] nix-grpc-daemon[907]: time=2026-09-20T13:34:08.034Z level=INFO msg="Upload complete. (317ms)"4978lb # [ 94.615797] niks3-server[1180]: 2026/09/20 13:34:08 INFO Completed upload id=224979lb # [ 94.619351] niks3-server[1180]: 2026/09/20 13:34:08 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4980worker1 # [ 94.639053] nix-grpc-daemon[834]: ts=2026-09-20T13:34:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49444 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14981worker1 # [ 94.642433] nix-daemon[898]: reaped child process 1216, status = succeeded4982client # copying 1 paths...4983worker1 # [ 94.682862] nix-grpc-daemon[834]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:494444984client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4985worker2 # [ 94.719466] nix-grpc-daemon[838]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:453404986worker2 # [ 94.734765] nix-grpc-daemon[838]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4987worker2 # [ 94.754313] nix-grpc-daemon[838]: ts=2026-09-20T13:34:08Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45340 duration_s=0 paths=1 nar_bytes_out=1204988client # [ 94.883647] nix-daemon[947]: reaped child process 1049, status = succeeded4989client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.61 seconds)4990client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24991client # [ 95.137671] nix-daemon[947]: accepted connection from pid 1067, user root (trusted)4992client # this derivation will be built:4993client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4994worker1 # [ 95.368244] nix-grpc-daemon[834]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:494444995client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4996worker2 # [ 95.376267] nix-grpc-daemon[838]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453404997worker2 # [ 95.390303] nix-grpc-daemon[960]: time=2026-09-20T13:34:08.764Z level=INFO msg="All 1 paths already cached"4998worker2 # [ 95.409714] nix-grpc-daemon[838]: ts=2026-09-20T13:34:08Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04999worker2 # [ 95.418290] nix-grpc-daemon[838]: ts=2026-09-20T13:34:08Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15000worker2 # [ 95.428706] nix-grpc-daemon[838]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453405001worker2 # [ 95.445547] nix-grpc-daemon[838]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453405002worker2 # [ 95.465455] nix-grpc-daemon[960]: time=2026-09-20T13:34:08.840Z level=INFO msg="All 1 paths already cached"5003client # copying 1 paths to 'lb:50051'...5004client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5005worker2 # [ 95.480053] nix-grpc-daemon[838]: ts=2026-09-20T13:34:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453405006worker2 # [ 95.502481] nix-daemon[913]: accepted connection from pid 838, user nix-grpc-daemon (trusted)5007worker2 # [ 95.743768] nix-daemon[913]: accepted connection from pid 1096, user nix-grpc-daemon (trusted)5008worker2 # [ 95.766946] nix-daemon[913]: reaped child process 1099, status = succeeded5009worker2 # [ 95.963438] nix-daemon[913]: accepted connection from pid 1101, user nix-grpc-daemon (trusted)5010worker2 # [ 95.983939] nix-daemon[913]: reaped child process 1104, status = succeeded5011lb # [ 95.987224] niks3-server[1180]: 2026/09/20 13:34:09 INFO Received uploads request method=POST path=/api/pending_closures5012worker2 # [ 96.011064] nix-grpc-daemon[960]: time=2026-09-20T13:34:09.385Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5013worker2 # [ 96.012792] nix-grpc-daemon[960]: time=2026-09-20T13:34:09.385Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5014lb # [ 96.030874] niks3-server[1180]: 2026/09/20 13:34:09 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5015worker2 # [ 96.060290] nix-grpc-daemon[960]: time=2026-09-20T13:34:09.434Z level=INFO msg="Uploading 1 narinfos"5016lb # [ 96.051344] niks3-server[1180]: 2026/09/20 13:34:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5017lb # [ 96.059793] niks3-server[1180]: 2026/09/20 13:34:09 INFO Signed narinfos id=23 count=15018lb # [ 96.065470] niks3-server[1180]: 2026/09/20 13:34:09 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5019lb # [ 96.072830] niks3-server[1180]: 2026/09/20 13:34:09 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5020lb # [ 96.078998] niks3-server[1180]: 2026/09/20 13:34:09 INFO Completed upload id=235021worker2 # [ 96.091795] nix-grpc-daemon[960]: time=2026-09-20T13:34:09.466Z level=INFO msg="Upload complete. (567ms)"5022lb # [ 96.081504] niks3-server[1180]: 2026/09/20 13:34:09 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5023worker2 # [ 96.100930] nix-grpc-daemon[838]: ts=2026-09-20T13:34:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45340 duration_s=0 paths=1 nar_bytes_in=5685024worker2 # [ 96.110693] nix-grpc-daemon[838]: ts=2026-09-20T13:34:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453405025worker2 # [ 96.116727] nix-daemon[913]: accepted connection from pid 838, user nix-grpc-daemon (trusted)5026worker2 # [ 96.120929] nix-daemon[913]: reaped child process 1094, status = succeeded5027client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5028worker2 # [ 96.134234] nix-daemon[913]: accepted connection from pid 838, user nix-grpc-daemon (trusted)5029client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5030worker2 # [ 97.388830] nix-daemon[913]: reaped child process 1108, status = succeeded5031worker2 # [ 97.583848] nix-daemon[913]: accepted connection from pid 1123, user nix-grpc-daemon (trusted)5032worker2 # [ 97.605308] nix-daemon[913]: reaped child process 1126, status = succeeded5033lb # [ 97.605916] niks3-server[1180]: 2026/09/20 13:34:10 INFO Received uploads request method=POST path=/api/pending_closures5034worker2 # [ 97.630306] nix-grpc-daemon[960]: time=2026-09-20T13:34:11.002Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5035lb # [ 97.644579] niks3-server[1180]: 2026/09/20 13:34:11 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5036lb # [ 97.651024] niks3-server[1180]: 2026/09/20 13:34:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5037worker2 # [ 97.669145] nix-grpc-daemon[960]: time=2026-09-20T13:34:11.043Z level=INFO msg="Uploading 1 narinfos"5038lb # [ 97.660620] niks3-server[1180]: 2026/09/20 13:34:11 INFO Signed narinfos id=24 count=15039lb # [ 97.666636] niks3-server[1180]: 2026/09/20 13:34:11 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5040lb # [ 97.676265] niks3-server[1180]: 2026/09/20 13:34:11 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5041lb # [ 97.684834] niks3-server[1180]: 2026/09/20 13:34:11 INFO Completed upload id=245042worker2 # [ 97.697711] nix-grpc-daemon[960]: time=2026-09-20T13:34:11.072Z level=INFO msg="Upload complete. (352ms)"5043lb # [ 97.687210] niks3-server[1180]: 2026/09/20 13:34:11 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5044worker2 # [ 97.709206] nix-grpc-daemon[838]: ts=2026-09-20T13:34:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45340 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=15045worker2 # [ 97.724166] nix-daemon[913]: reaped child process 1106, status = succeeded5046client # copying 1 paths...5047worker1 # [ 97.763154] nix-grpc-daemon[834]: ts=2026-09-20T13:34:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:494445048worker1 # [ 97.783242] nix-grpc-daemon[834]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...5049client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5050worker2 # [ 97.840369] nix-grpc-daemon[838]: ts=2026-09-20T13:34:11Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:453405051worker2 # [ 97.847247] nix-grpc-daemon[838]: ts=2026-09-20T13:34:11Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45340 duration_s=0 paths=1 nar_bytes_out=1205052client # [ 97.941596] nix-daemon[947]: reaped child process 1070, status = succeeded5053client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 3.06 seconds)5054(finished: subtest: an input only one worker has still reaches the builder, in 11.15 seconds)5055subtest: upload whose reference the worker lacks is completed from the cache5056client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache5057client # this derivation will be built:5058client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5059client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5060client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.53 seconds)5061client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache5062client # this derivation will be built:5063client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5064client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5065client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.50 seconds)5066client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5067client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.24 seconds)5068worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5069worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.32 seconds)5070worker2 # [ 99.602957] systemd[1]: Stopping Nix worker-protocol over gRPC...5071worker2 # [ 99.809358] nix-grpc-daemon[838]: ts=2026-09-20T13:34:13Z level=info event=signal_exit5072worker2 # [ 100.270529] nix-grpc-daemon[838]: ts=2026-09-20T13:34:13Z level=info event=scheduler_yield5073worker2 # [ 100.277431] nix-grpc-daemon[838]: ts=2026-09-20T13:34:13Z level=info event=scheduler_restarting addr=lb:500515074lb # [ 100.270893] niks3-server[1180]: 2026/09/20 13:34:13 INFO lead: released remote=192.168.1.4:585325075worker2 # [ 100.284195] nix-grpc-daemon[838]: ts=2026-09-20T13:34:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:453245076worker1 # [ 100.290591] nix-grpc-daemon[834]: ts=2026-09-20T13:34:13Z level=info event=scheduler_restarting addr=lb:500515077worker2 # [ 100.292274] nix-grpc-daemon[838]: ts=2026-09-20T13:34:13Z level=info event=worker_gone addr=192.168.1.4:500515078worker2 # [ 100.296744] nix-grpc-daemon[838]: ts=2026-09-20T13:34:13Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:453245079worker2 # [ 100.301500] nix-grpc-daemon[838]: ts=2026-09-20T13:34:13Z level=info event=worker_gone addr=192.168.1.3:500515080worker2 # [ 100.479398] nix-grpc-daemon[838]: ts=2026-09-20T13:34:13Z level=info event=scheduler_connected addr=lb:500515081worker2 # [ 100.485526] nix-grpc-daemon[838]: ts=2026-09-20T13:34:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5082worker1 # [ 100.488210] nix-grpc-daemon[834]: ts=2026-09-20T13:34:13Z level=info event=scheduler_connected addr=lb:500515083worker1 # [ 100.494149] nix-grpc-daemon[834]: ts=2026-09-20T13:34:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5084worker2 # [ 100.687667] nix-grpc-daemon[838]: ts=2026-09-20T13:34:14Z level=info event=scheduler_connected addr=lb:500515085worker1 # [ 100.695540] nix-grpc-daemon[834]: ts=2026-09-20T13:34:14Z level=info event=scheduler_connected addr=lb:500515086worker2 # [ 100.694941] nix-grpc-daemon[838]: ts=2026-09-20T13:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5087worker1 # [ 100.702873] nix-grpc-daemon[834]: ts=2026-09-20T13:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5088worker2 # [ 100.900969] nix-grpc-daemon[838]: ts=2026-09-20T13:34:14Z level=info event=scheduler_connected addr=lb:500515089worker1 # [ 100.904210] nix-grpc-daemon[834]: ts=2026-09-20T13:34:14Z level=info event=scheduler_connected addr=lb:500515090worker2 # [ 100.906021] nix-grpc-daemon[838]: ts=2026-09-20T13:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5091worker1 # [ 100.910983] nix-grpc-daemon[834]: ts=2026-09-20T13:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5092worker2 # [ 101.101172] nix-grpc-daemon[838]: ts=2026-09-20T13:34:14Z level=info event=scheduler_connected addr=lb:500515093worker2 # [ 101.107999] nix-grpc-daemon[838]: ts=2026-09-20T13:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5094worker1 # [ 101.111641] nix-grpc-daemon[834]: ts=2026-09-20T13:34:14Z level=info event=scheduler_connected addr=lb:500515095worker1 # [ 101.117615] nix-grpc-daemon[834]: ts=2026-09-20T13:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5096worker1 # [ 101.252639] nix-grpc-daemon[834]: ts=2026-09-20T13:34:14Z level=info event=scheduler_take_over5097lb # [ 101.239519] niks3-server[1180]: 2026/09/20 13:34:14 INFO lead: acquired remote=192.168.1.3:524325098worker2 # [ 101.278711] nix-grpc-daemon[838]: ts=2026-09-20T13:34:14Z level=info event=scheduler_restarting5099worker1 # [ 101.321014] nix-grpc-daemon[834]: ts=2026-09-20T13:34:14Z level=info event=scheduler_connected addr=lb:500515100worker1 # [ 101.326258] nix-grpc-daemon[834]: ts=2026-09-20T13:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5101worker2 # [ 101.333051] nix-grpc-daemon[838]: ts=2026-09-20T13:34:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5102worker2 # [ 101.355039] nix-daemon[913]: reaped child process 915, status = succeeded5103worker1 # [ 101.527606] nix-grpc-daemon[834]: ts=2026-09-20T13:34:14Z level=info event=scheduler_connected addr=lb:500515104worker1 # [ 101.533763] nix-grpc-daemon[834]: ts=2026-09-20T13:34:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5105worker1 # [ 101.734751] nix-grpc-daemon[834]: ts=2026-09-20T13:34:15Z level=info event=scheduler_connected addr=lb:500515106worker1 # [ 101.740843] nix-grpc-daemon[834]: ts=2026-09-20T13:34:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5107worker1 # [ 101.942750] nix-grpc-daemon[834]: ts=2026-09-20T13:34:15Z level=info event=scheduler_connected addr=lb:500515108worker1 # [ 101.968170] nix-grpc-daemon[834]: ts=2026-09-20T13:34:15Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:505385109worker1 # [ 101.975578] nix-grpc-daemon[834]: ts=2026-09-20T13:34:15Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05110worker2 # [ 101.984598] nix-grpc-daemon[838]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5111worker2 # [ 101.989316] nix-grpc-daemon[838]: E0000 00:00:1789911255.357046 881 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5112worker2 # [ 101.996113] nix-grpc-daemon[838]: E0000 00:00:1789911255.358103 881 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5113worker2 # [ 102.178517] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5114worker2 # [ 102.189539] systemd[1]: Stopped Nix worker-protocol over gRPC.5115worker2 # [ 102.191895] systemd[1]: nix-grpc-daemon.service: Consumed 2.432s CPU time over 1min 18.736s wall clock time, 52.1M memory peak, 71.8K incoming IP traffic, 224K outgoing IP traffic.5116worker2 # [ 102.211548] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5117worker2 # [ 102.215132] systemd[1]: Closed Nix worker-protocol over gRPC.5118worker2 # [ 102.217663] systemd[1]: nix-grpc-daemon.socket: Consumed 778us CPU time over 1min 28.367s wall clock time, 256K memory peak, 84.4K incoming IP traffic, 68.6K outgoing IP traffic.5119lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5120lb: must succeed: curl -sf localhost:9901/clusters5121lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5122lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.11 seconds)5123client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25124worker1 # [ 102.627908] nix-grpc-daemon[834]: ts=2026-09-20T13:34:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:494445125worker1 # [ 102.640105] nix-grpc-daemon[834]: ts=2026-09-20T13:34:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:494445126client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5127client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.35 seconds)5128worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5129worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.05 seconds)5130worker2 # [ 102.806627] systemd[1]: Listening on Nix worker-protocol over gRPC.5131worker1 # [ 102.876933] systemd[1]: Stopping Nix worker-protocol over gRPC...5132worker2 # [ 103.162252] systemd[1]: Starting scheduler-token.service...5133lb # [ 103.267213] envoy[911]: conn peer=[2001:db8:1::1]:43548 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3726 tx=1799 ms=31025134worker2 # [ 103.282959] systemd[1]: scheduler-token.service: Deactivated successfully.5135lb # [ 103.272790] envoy[911]: conn peer=[2001:db8:1::1]:43564 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1005136lb # [ 103.277835] envoy[911]: conn peer=[2001:db8:1::1]:43582 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=295137worker2 # [ 103.292992] systemd[1]: Finished scheduler-token.service.5138lb # [ 103.282867] envoy[911]: conn peer=[2001:db8:1::1]:43574 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3713 tx=1918 ms=24995139worker2 # [ 103.296559] systemd[1]: scheduler-token.service: Consumed 62ms CPU time over 124ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5140worker2 # [ 103.302966] systemd[1]: Starting Nix worker-protocol over gRPC...5141lb # [ 103.288173] envoy[911]: conn peer=[2001:db8:1::4]:56202 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=109043 tx=5149 ms=775155142lb # [ 103.293518] envoy[911]: conn peer=[2001:db8:1::1]:32982 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=523 ms=375143worker2 # [ 103.508350] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5144worker2 # [ 103.511191] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5145worker2 # [ 103.515785] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15146worker2 # [ 103.525111] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5147worker2 # [ 103.527004] systemd[1]: Started Nix worker-protocol over gRPC.5148worker2 # [ 103.559066] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:16Z level=info event=scheduler_connected addr=lb:500515149worker1 # [ 103.563687] nix-grpc-daemon[834]: ts=2026-09-20T13:34:16Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:505385150worker1 # [ 103.570170] nix-grpc-daemon[834]: ts=2026-09-20T13:34:16Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05151worker1 # [ 103.683996] nix-grpc-daemon[834]: ts=2026-09-20T13:34:17Z level=info event=signal_exit5152worker1 # [ 104.148195] nix-grpc-daemon[834]: ts=2026-09-20T13:34:17Z level=info event=scheduler_yield5153worker2 # [ 104.156700] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:17Z level=info event=scheduler_restarting addr=lb:500515154lb # [ 104.145611] niks3-server[1180]: 2026/09/20 13:34:17 INFO lead: released remote=192.168.1.3:524325155worker1 # [ 104.158376] nix-grpc-daemon[834]: ts=2026-09-20T13:34:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:505385156worker1 # [ 104.165527] nix-grpc-daemon[834]: ts=2026-09-20T13:34:17Z level=info event=worker_gone addr=192.168.1.4:500515157worker1 # [ 104.172294] nix-grpc-daemon[834]: ts=2026-09-20T13:34:17Z level=info event=scheduler_restarting addr=lb:500515158worker1 # [ 104.176553] nix-grpc-daemon[834]: ts=2026-09-20T13:34:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:505385159worker1 # [ 104.181155] nix-grpc-daemon[834]: ts=2026-09-20T13:34:17Z level=info event=worker_gone addr=192.168.1.3:500515160worker2 # [ 104.358680] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:17Z level=info event=scheduler_connected addr=lb:500515161worker2 # [ 104.364828] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5162worker1 # [ 104.366714] nix-grpc-daemon[834]: ts=2026-09-20T13:34:17Z level=info event=scheduler_connected addr=lb:500515163worker1 # [ 104.372146] nix-grpc-daemon[834]: ts=2026-09-20T13:34:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5164worker2 # [ 104.568301] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:17Z level=info event=scheduler_connected addr=lb:500515165worker2 # [ 104.572704] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5166worker1 # [ 104.573825] nix-grpc-daemon[834]: ts=2026-09-20T13:34:17Z level=info event=scheduler_connected addr=lb:500515167worker1 # [ 104.579744] nix-grpc-daemon[834]: ts=2026-09-20T13:34:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5168worker2 # [ 104.770045] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:18Z level=info event=scheduler_connected addr=lb:500515169worker2 # [ 104.775814] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5170worker1 # [ 104.783866] nix-grpc-daemon[834]: ts=2026-09-20T13:34:18Z level=info event=scheduler_connected addr=lb:500515171worker1 # [ 104.788430] nix-grpc-daemon[834]: ts=2026-09-20T13:34:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5172worker2 # [ 104.977389] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:18Z level=info event=scheduler_connected addr=lb:500515173worker2 # [ 104.983324] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5174worker1 # [ 104.984158] nix-grpc-daemon[834]: ts=2026-09-20T13:34:18Z level=info event=scheduler_connected addr=lb:500515175worker1 # [ 104.990113] nix-grpc-daemon[834]: ts=2026-09-20T13:34:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5176worker1 # [ 105.154691] nix-grpc-daemon[834]: ts=2026-09-20T13:34:18Z level=info event=scheduler_restarting5177worker2 # [ 105.185517] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:18Z level=info event=scheduler_connected addr=lb:500515178worker2 # [ 105.191452] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5179worker1 # [ 105.224927] nix-grpc-daemon[834]: ts=2026-09-20T13:34:18Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5180worker1 # [ 105.230069] nix-daemon[898]: reaped child process 900, status = succeeded5181worker2 # [ 105.393442] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:18Z level=info event=scheduler_connected addr=lb:500515182worker2 # [ 105.399453] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5183worker2 # [ 105.603127] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:18Z level=info event=scheduler_connected addr=lb:500515184worker2 # [ 105.606914] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5185worker2 # [ 105.804657] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:19Z level=info event=scheduler_connected addr=lb:500515186worker1 # [ 105.810486] nix-grpc-daemon[834]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5187worker2 # [ 105.810542] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5188worker1 # [ 105.815341] nix-grpc-daemon[834]: E0000 00:00:1789911259.214946 877 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5189worker1 # [ 105.887344] nix-grpc-daemon[834]: E0000 00:00:1789911259.293406 877 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5190worker2 # [ 106.011900] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:19Z level=info event=scheduler_connected addr=lb:500515191worker2 # [ 106.017864] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5192worker1 # [ 106.106008] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5193worker1 # [ 106.115735] systemd[1]: Stopped Nix worker-protocol over gRPC.5194worker1 # [ 106.125577] systemd[1]: nix-grpc-daemon.service: Consumed 4.050s CPU time over 1min 22.828s wall clock time, 59.2M memory peak, 102.4K incoming IP traffic, 189.6K outgoing IP traffic.5195worker1 # [ 106.137924] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5196worker1 # [ 106.145389] systemd[1]: Closed Nix worker-protocol over gRPC.5197worker1 # [ 106.147478] systemd[1]: nix-grpc-daemon.socket: Consumed 783us CPU time over 1min 32.478s wall clock time, 256K memory peak, 93.2K incoming IP traffic, 76.8K outgoing IP traffic.5198lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5199lb: must succeed: curl -sf localhost:9901/clusters5200worker2 # [ 106.219728] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:19Z level=info event=scheduler_connected addr=lb:500515201worker2 # [ 106.225477] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5202lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5203lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.11 seconds)5204client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25205worker2 # [ 106.426192] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:19Z level=info event=scheduler_connected addr=lb:500515206worker2 # [ 106.428720] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5207worker2 # [ 106.563867] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:361445208worker2 # [ 106.587968] nix-daemon[913]: accepted connection from pid 1150, user nix-grpc-daemon (trusted)5209worker2 # [ 106.624875] nix-grpc-daemon[1150]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 1/5)5210worker2 # [ 106.632920] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:20Z level=info event=scheduler_connected addr=lb:500515211worker2 # [ 106.637810] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5212worker2 # [ 106.686216] nix-grpc-daemon[1150]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 65 ms (attempt 2/5)5213worker2 # [ 106.759182] nix-grpc-daemon[1150]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 3/5)5214worker2 # [ 106.839627] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:20Z level=info event=scheduler_connected addr=lb:500515215worker2 # [ 106.845458] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5216worker2 # [ 106.910335] nix-grpc-daemon[1150]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 571 ms (attempt 4/5)5217worker2 # [ 107.046753] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:20Z level=info event=scheduler_connected addr=lb:500515218worker2 # [ 107.052933] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5219worker2 # [ 107.254965] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:20Z level=info event=scheduler_connected addr=lb:500515220worker2 # [ 107.260782] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5221worker2 # [ 107.462327] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:20Z level=info event=scheduler_connected addr=lb:500515222worker2 # [ 107.467990] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5223worker2 # [ 107.490397] nix-grpc-daemon[1150]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5224client # copying 2 paths...5225client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5226client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5227worker2 # [ 107.529030] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:361445228worker2 # [ 107.547640] nix-daemon[913]: accepted connection from pid 1150, user nix-grpc-daemon (trusted)5229worker2 # [ 107.596110] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:20Z level=info event=niks3_push_started pid=12065230worker2 # [ 107.670375] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:21Z level=info event=scheduler_connected addr=lb:500515231worker2 # [ 107.674186] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5232worker2 # [ 107.870964] nix-grpc-daemon[1206]: time=2026-09-20T13:34:21.244Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""5233worker2 # [ 107.877933] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:21Z level=info event=scheduler_connected addr=lb:500515234worker2 # [ 107.879602] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5235worker2 # [ 108.080737] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:21Z level=info event=scheduler_connected addr=lb:500515236worker2 # [ 108.086896] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5237worker2 # [ 108.089111] nix-daemon[913]: accepted connection from pid 1219, user nix-grpc-daemon (trusted)5238worker2 # [ 108.110581] nix-daemon[913]: reaped child process 1222, status = succeeded5239lb # [ 108.115731] niks3-server[1180]: 2026/09/20 13:34:21 INFO Received uploads request method=POST path=/api/pending_closures5240lb # [ 108.130023] niks3-server[1180]: 2026/09/20 13:34:21 INFO Received uploads request method=POST path=/api/pending_closures5241worker2 # [ 108.156198] nix-grpc-daemon[1206]: time=2026-09-20T13:34:21.529Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5242worker2 # [ 108.158005] nix-grpc-daemon[1206]: time=2026-09-20T13:34:21.529Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5243worker2 # [ 108.163951] nix-grpc-daemon[1206]: time=2026-09-20T13:34:21.530Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5244lb # [ 108.221500] niks3-server[1180]: 2026/09/20 13:34:21 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5245lb # [ 108.232119] niks3-server[1180]: 2026/09/20 13:34:21 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5246worker2 # [ 108.256210] nix-grpc-daemon[1206]: time=2026-09-20T13:34:21.628Z level=INFO msg="Uploading 3 narinfos"5247lb # [ 108.245066] niks3-server[1180]: 2026/09/20 13:34:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5248lb # [ 108.247712] niks3-server[1180]: 2026/09/20 13:34:21 INFO Signed narinfos id=25 count=15249lb # [ 108.259212] niks3-server[1180]: 2026/09/20 13:34:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5250lb # [ 108.269000] niks3-server[1180]: 2026/09/20 13:34:21 INFO Signed narinfos id=26 count=25251lb # [ 108.270638] niks3-server[1180]: 2026/09/20 13:34:21 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5252worker2 # [ 108.284734] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:21Z level=info event=scheduler_connected addr=lb:500515253worker2 # [ 108.290549] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5254lb # [ 108.303979] niks3-server[1180]: 2026/09/20 13:34:21 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5255lb # [ 108.310695] niks3-server[1180]: 2026/09/20 13:34:21 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5256lb # [ 108.316305] niks3-server[1180]: 2026/09/20 13:34:21 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5257lb # [ 108.322335] niks3-server[1180]: 2026/09/20 13:34:21 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5258lb # [ 108.328470] niks3-server[1180]: 2026/09/20 13:34:21 INFO Completed upload id=255259lb # [ 108.332190] niks3-server[1180]: 2026/09/20 13:34:21 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5260lb # [ 108.334817] niks3-server[1180]: 2026/09/20 13:34:21 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5261worker2 # [ 108.347718] nix-grpc-daemon[1206]: time=2026-09-20T13:34:21.722Z level=INFO msg="Upload complete. (472ms)"5262lb # [ 108.337594] niks3-server[1180]: 2026/09/20 13:34:21 INFO Completed upload id=265263worker2 # [ 108.356345] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36144 duration_s=0 paths=2 nar_bytes_in=3125264worker2 # [ 108.367783] nix-daemon[913]: reaped child process 1204, status = succeeded5265client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 2.14 seconds)5266worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5267worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds)5268worker2 # [ 108.492380] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:21Z level=info event=scheduler_connected addr=lb:500515269worker2 # [ 108.498364] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5270lb # [ 108.529026] niks3-server[1180]: 2026/09/20 13:34:21 INFO lead: acquired remote=192.168.1.4:429185271worker2 # [ 108.542029] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:21Z level=info event=scheduler_take_over5272worker1 # [ 108.553355] systemd[1]: Listening on Nix worker-protocol over gRPC.5273worker1 # [ 108.564257] systemd[1]: Starting Nix worker-protocol over gRPC...5274worker2 # [ 108.701586] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:22Z level=info event=scheduler_connected addr=lb:500515275worker2 # [ 108.728242] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:22Z level=info event=oidc_jwks provider=mock keys=15276worker2 # [ 108.733335] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:22Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:361605277worker2 # [ 108.738993] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:22Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05278worker1 # [ 108.781812] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:22Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5279worker1 # [ 108.783748] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:22Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5280worker1 # [ 108.788722] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:22Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15281worker1 # [ 108.798464] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:22Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5282worker1 # [ 108.808402] systemd[1]: Started Nix worker-protocol over gRPC.5283lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5284lb: must succeed: curl -sf localhost:9901/clusters5285worker1 # [ 108.830702] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:22Z level=info event=scheduler_connected addr=lb:500515286worker2 # [ 108.836303] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:22Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:361605287worker2 # [ 108.842508] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:22Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05288lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.12 seconds)5289lb: must succeed: curl -sf localhost:9901/clusters5290lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5291lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.22 seconds)5292(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.07 seconds)5293subtest: scheduler down: the other node takes the lock and keeps it5294lb: must succeed: curl -sf localhost:9901/clusters5295lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5296lb: must succeed: curl -sf localhost:9901/clusters5297lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5298lb: must succeed: curl -sf localhost:9901/clusters5299lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5300lb: must succeed: curl -sf localhost:9901/clusters5301lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5302worker2 # [ 110.349991] systemd[1]: Stopping Nix worker-protocol over gRPC...5303worker2 # [ 110.540881] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:23Z level=info event=signal_exit5304worker2 # [ 110.550046] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:23Z level=info event=scheduler_yield5305lb # [ 110.541199] niks3-server[1180]: 2026/09/20 13:34:23 INFO lead: released remote=192.168.1.4:429185306worker1 # [ 110.559106] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:23Z level=info event=scheduler_restarting addr=lb:500515307worker2 # [ 110.559219] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:361605308worker2 # [ 110.567010] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:23Z level=info event=worker_gone addr=192.168.1.3:500515309worker2 # [ 110.573513] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:23Z level=info event=scheduler_restarting addr=lb:500515310worker2 # [ 110.579763] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:361605311worker2 # [ 110.584604] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:23Z level=info event=worker_gone addr=192.168.1.4:500515312worker1 # [ 110.758483] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:24Z level=info event=scheduler_connected addr=lb:500515313worker1 # [ 110.764876] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5314worker2 # [ 110.768267] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:24Z level=info event=scheduler_connected addr=lb:500515315worker2 # [ 110.773953] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5316worker1 # [ 110.965757] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:24Z level=info event=scheduler_connected addr=lb:500515317worker1 # [ 110.971868] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5318worker2 # [ 110.975274] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:24Z level=info event=scheduler_connected addr=lb:500515319worker2 # [ 110.978976] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5320worker1 # [ 111.173428] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:24Z level=info event=scheduler_connected addr=lb:500515321worker1 # [ 111.179133] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5322worker2 # [ 111.180695] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:24Z level=info event=scheduler_connected addr=lb:500515323worker2 # [ 111.186491] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5324worker1 # [ 111.380047] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:24Z level=info event=scheduler_connected addr=lb:500515325worker1 # [ 111.385690] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5326worker2 # [ 111.388303] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:24Z level=info event=scheduler_connected addr=lb:500515327worker2 # [ 111.394019] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5328worker2 # [ 111.563107] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:24Z level=info event=scheduler_restarting5329worker2 # [ 111.576534] nix-grpc-daemon[1150]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5330worker2 # [ 111.581987] nix-grpc-daemon[1150]: E0000 00:00:1789911264.951222 1193 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5331worker1 # [ 111.589382] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:24Z level=info event=scheduler_connected addr=lb:500515332worker1 # [ 111.593743] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5333worker2 # [ 111.600566] nix-grpc-daemon[1150]: E0000 00:00:1789911264.974617 1193 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5334worker2 # [ 111.636582] nix-grpc-daemon[1150]: ts=2026-09-20T13:34:25Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5335worker2 # [ 111.643591] nix-daemon[913]: reaped child process 1199, status = succeeded5336worker1 # [ 111.790690] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:25Z level=info event=scheduler_connected addr=lb:500515337worker1 # [ 111.797106] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5338worker1 # [ 111.998019] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:25Z level=info event=scheduler_connected addr=lb:500515339worker1 # [ 112.004307] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5340worker1 # [ 112.204708] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:25Z level=info event=scheduler_connected addr=lb:500515341worker1 # [ 112.207202] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5342worker1 # [ 112.408606] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:25Z level=info event=scheduler_connected addr=lb:500515343worker1 # [ 112.414405] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5344worker1 # [ 112.618011] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:26Z level=info event=scheduler_connected addr=lb:500515345worker1 # [ 112.623174] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5346worker1 # [ 112.824487] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:26Z level=info event=scheduler_connected addr=lb:500515347worker1 # [ 112.831548] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5348worker1 # [ 113.032571] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:26Z level=info event=scheduler_connected addr=lb:500515349worker1 # [ 113.038596] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5350worker1 # [ 113.239554] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:26Z level=info event=scheduler_connected addr=lb:500515351worker1 # [ 113.245786] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5352lb # [ 113.263344] envoy[911]: conn peer=[2001:db8:1::3]:33128 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=21413 tx=5626 ms=806735353lb # [ 113.269596] envoy[911]: conn peer=[2001:db8:1::1]:32990 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1010 tx=304 ms=18175354lb # [ 113.277258] envoy[911]: conn peer=[2001:db8:1::4]:50836 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=20956 tx=1101 ms=80505355worker1 # [ 113.446467] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:26Z level=info event=scheduler_connected addr=lb:500515356worker1 # [ 113.452530] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5357worker2 # [ 113.573658] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5358worker2 # [ 113.581181] systemd[1]: Stopped Nix worker-protocol over gRPC.5359worker2 # [ 113.583431] systemd[1]: nix-grpc-daemon.service: Consumed 632ms CPU time over 10.265s wall clock time, 30.8M memory peak, 23.3K incoming IP traffic, 47.8K outgoing IP traffic.5360worker2 # [ 113.596619] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5361worker2 # [ 113.605074] systemd[1]: Closed Nix worker-protocol over gRPC.5362worker2 # [ 113.607234] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 10.805s wall clock time, 256K memory peak, 19.9K incoming IP traffic, 14.7K outgoing IP traffic.5363worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5364worker1 # [ 113.659090] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:27Z level=info event=scheduler_connected addr=lb:500515365worker1 # [ 113.666680] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5366worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.11 seconds)5367worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5368worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5369worker1 # [ 113.813449] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:27Z level=info event=scheduler_take_over5370lb # [ 113.802264] niks3-server[1180]: 2026/09/20 13:34:27 INFO lead: acquired remote=192.168.1.3:604265371worker1 # [ 113.859300] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:27Z level=info event=scheduler_connected addr=lb:500515372worker1 # [ 113.864847] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5373worker1 # [ 114.065950] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:27Z level=info event=scheduler_connected addr=lb:500515374worker1 # [ 114.071959] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5375worker1 # [ 114.273183] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:27Z level=info event=scheduler_connected addr=lb:500515376worker1 # [ 114.285442] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:27Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:359445377worker1 # [ 114.291212] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:27Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05378worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5379worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)5380client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25381worker1 # [ 115.193786] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:359585382worker1 # [ 115.212387] nix-daemon[898]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5383client # don't know how to build these paths:5384client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5385worker1 # [ 115.238431] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359585386worker1 # [ 115.261421] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 80 ms (attempt 1/5)5387worker1 # [ 115.341843] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 2/5)5388worker1 # [ 115.490087] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 214 ms (attempt 3/5)5389worker1 # [ 115.712237] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 602 ms (attempt 4/5)5390worker1 # [ 116.325167] nix-grpc-daemon[1260]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5391worker1 # [ 116.372262] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:29Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05392worker1 # [ 116.380171] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:29Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15393worker1 # [ 116.420765] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359585394worker1 # [ 116.441325] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359585395client # copying 1 paths to 'lb:50051'...5396worker1 # [ 116.459588] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:359585397client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5398worker1 # [ 116.482225] nix-daemon[898]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5399worker1 # [ 116.515496] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:29Z level=info event=niks3_push_started pid=13305400worker1 # [ 116.750859] nix-grpc-daemon[1330]: time=2026-09-20T13:34:30.156Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5401worker1 # [ 116.933769] nix-daemon[898]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5402worker1 # [ 116.953500] nix-daemon[898]: reaped child process 1346, status = succeeded5403worker1 # [ 117.132607] nix-daemon[898]: accepted connection from pid 1348, user nix-grpc-daemon (trusted)5404worker1 # [ 117.155100] nix-daemon[898]: reaped child process 1351, status = succeeded5405lb # [ 117.155622] niks3-server[1180]: 2026/09/20 13:34:30 INFO Received uploads request method=POST path=/api/pending_closures5406worker1 # [ 117.182189] nix-grpc-daemon[1330]: time=2026-09-20T13:34:30.587Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5407worker1 # [ 117.183809] nix-grpc-daemon[1330]: time=2026-09-20T13:34:30.587Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5408lb # [ 117.196430] niks3-server[1180]: 2026/09/20 13:34:30 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5409lb # [ 117.213995] niks3-server[1180]: 2026/09/20 13:34:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5410worker1 # [ 117.236161] nix-grpc-daemon[1330]: time=2026-09-20T13:34:30.642Z level=INFO msg="Uploading 1 narinfos"5411lb # [ 117.228674] niks3-server[1180]: 2026/09/20 13:34:30 INFO Signed narinfos id=27 count=15412lb # [ 117.251967] niks3-server[1180]: 2026/09/20 13:34:30 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5413worker1 # [ 117.272180] nix-grpc-daemon[1330]: time=2026-09-20T13:34:30.678Z level=INFO msg="Upload complete. (517ms)"5414lb # [ 117.263982] niks3-server[1180]: 2026/09/20 13:34:30 INFO Completed upload id=275415worker1 # [ 117.283749] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35958 duration_s=0 paths=1 nar_bytes_in=5205416worker1 # [ 117.300603] nix-grpc-daemon[1330]: time=2026-09-20T13:34:30.689Z level=WARN msg="Failed to register uploaded object" key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls error="server returned 404: object is not pending upload\n"5417worker1 # [ 117.307395] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359585418worker1 # [ 117.317696] nix-grpc-daemon[1330]: time=2026-09-20T13:34:30.699Z level=WARN msg="Failed to register uploaded object" key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo error="server returned 404: object is not pending upload\n"5419worker1 # [ 117.328262] nix-daemon[898]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5420worker1 # [ 117.330566] nix-daemon[898]: reaped child process 1328, status = succeeded5421client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5422client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5423worker1 # [ 117.341721] nix-daemon[898]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5424client # farm-a> LOG-farm-a-on-standby5425worker1 # [ 118.580850] nix-daemon[898]: reaped child process 1355, status = succeeded5426worker1 # [ 118.749568] nix-daemon[898]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)5427worker1 # [ 118.770052] nix-daemon[898]: reaped child process 1373, status = succeeded5428lb # [ 118.765306] niks3-server[1180]: 2026/09/20 13:34:32 INFO Received uploads request method=POST path=/api/pending_closures5429worker1 # [ 118.796570] nix-grpc-daemon[1330]: time=2026-09-20T13:34:32.199Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5430worker1 # [ 118.798145] nix-grpc-daemon[1330]: time=2026-09-20T13:34:32.199Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5431lb # [ 118.830990] niks3-server[1180]: 2026/09/20 13:34:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5432worker1 # [ 118.851916] nix-grpc-daemon[1330]: time=2026-09-20T13:34:32.256Z level=INFO msg="Uploading 1 narinfos"5433lb # [ 118.842898] niks3-server[1180]: 2026/09/20 13:34:32 INFO Signed narinfos id=28 count=15434lb # [ 118.853364] niks3-server[1180]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5435lb # [ 118.862413] niks3-server[1180]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5436lb # [ 118.872858] niks3-server[1180]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5437lb # [ 118.876445] niks3-server[1180]: 2026/09/20 13:34:32 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5438lb # [ 118.881134] niks3-server[1180]: 2026/09/20 13:34:32 INFO Completed upload id=285439worker1 # [ 118.893299] nix-grpc-daemon[1330]: time=2026-09-20T13:34:32.299Z level=INFO msg="Upload complete. (339ms)"5440lb # [ 118.884067] niks3-server[1180]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5441worker1 # [ 118.920481] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35958 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15442worker1 # [ 118.932534] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:32Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05443worker1 # [ 118.935834] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:32Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15444worker1 # [ 118.940467] nix-daemon[898]: reaped child process 1353, status = succeeded5445worker1 # [ 118.964152] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359585446worker1 # [ 118.981981] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359585447worker1 # [ 118.999314] nix-grpc-daemon[1330]: time=2026-09-20T13:34:32.405Z level=INFO msg="All 1 paths already cached"5448client # copying 1 paths to 'lb:50051'...5449worker1 # [ 119.008555] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:359585450client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5451worker1 # [ 119.023193] nix-daemon[898]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5452worker1 # [ 119.211581] nix-daemon[898]: accepted connection from pid 1379, user nix-grpc-daemon (trusted)5453worker1 # [ 119.232145] nix-daemon[898]: reaped child process 1382, status = succeeded5454worker1 # [ 119.414119] nix-daemon[898]: accepted connection from pid 1384, user nix-grpc-daemon (trusted)5455worker1 # [ 119.434668] nix-daemon[898]: reaped child process 1387, status = succeeded5456lb # [ 119.428738] niks3-server[1180]: 2026/09/20 13:34:32 INFO Received uploads request method=POST path=/api/pending_closures5457worker1 # [ 119.455172] nix-grpc-daemon[1330]: time=2026-09-20T13:34:32.861Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5458worker1 # [ 119.457057] nix-grpc-daemon[1330]: time=2026-09-20T13:34:32.861Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5459lb # [ 119.465162] niks3-server[1180]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5460worker1 # [ 119.494905] nix-grpc-daemon[1330]: time=2026-09-20T13:34:32.900Z level=INFO msg="Uploading 1 narinfos"5461lb # [ 119.482234] niks3-server[1180]: 2026/09/20 13:34:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5462lb # [ 119.490207] niks3-server[1180]: 2026/09/20 13:34:32 INFO Signed narinfos id=29 count=15463lb # [ 119.496342] niks3-server[1180]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5464lb # [ 119.502238] niks3-server[1180]: 2026/09/20 13:34:32 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5465lb # [ 119.513626] niks3-server[1180]: 2026/09/20 13:34:32 INFO Completed upload id=295466worker1 # [ 119.526429] nix-grpc-daemon[1330]: time=2026-09-20T13:34:32.932Z level=INFO msg="Upload complete. (492ms)"5467lb # [ 119.516493] niks3-server[1180]: 2026/09/20 13:34:32 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5468worker1 # [ 119.537232] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35958 duration_s=0 paths=1 nar_bytes_in=6405469worker1 # [ 119.548001] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359585470worker1 # [ 119.555810] nix-daemon[898]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5471worker1 # [ 119.561598] nix-daemon[898]: reaped child process 1377, status = succeeded5472client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5473client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5474worker1 # [ 119.577036] nix-daemon[898]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5475client # farm-b> LOG-farm-b-on-standby5476worker1 # [ 121.026335] nix-daemon[898]: reaped child process 1391, status = succeeded5477worker1 # [ 121.202297] nix-daemon[898]: accepted connection from pid 1406, user nix-grpc-daemon (trusted)5478worker1 # [ 121.223813] nix-daemon[898]: reaped child process 1409, status = succeeded5479lb # [ 121.223316] niks3-server[1180]: 2026/09/20 13:34:34 INFO Received uploads request method=POST path=/api/pending_closures5480worker1 # [ 121.247515] nix-grpc-daemon[1330]: time=2026-09-20T13:34:34.652Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5481worker1 # [ 121.250097] nix-grpc-daemon[1330]: time=2026-09-20T13:34:34.652Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5482lb # [ 121.256736] niks3-server[1180]: 2026/09/20 13:34:34 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5483lb # [ 121.270059] niks3-server[1180]: 2026/09/20 13:34:34 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5484lb # [ 121.287219] niks3-server[1180]: 2026/09/20 13:34:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5485worker1 # [ 121.307006] nix-grpc-daemon[1330]: time=2026-09-20T13:34:34.712Z level=INFO msg="Uploading 1 narinfos"5486lb # [ 121.298994] niks3-server[1180]: 2026/09/20 13:34:34 INFO Signed narinfos id=30 count=15487lb # [ 121.305363] niks3-server[1180]: 2026/09/20 13:34:34 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5488lb # [ 121.312214] niks3-server[1180]: 2026/09/20 13:34:34 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5489lb # [ 121.322375] niks3-server[1180]: 2026/09/20 13:34:34 INFO Completed upload id=305490worker1 # [ 121.334670] nix-grpc-daemon[1330]: time=2026-09-20T13:34:34.741Z level=INFO msg="Upload complete. (345ms)"5491lb # [ 121.324819] niks3-server[1180]: 2026/09/20 13:34:34 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5492worker1 # [ 121.357391] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35958 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15493worker1 # [ 121.374058] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:34Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05494worker1 # [ 121.379634] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:34Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15495worker1 # [ 121.385592] nix-daemon[898]: reaped child process 1389, status = succeeded5496worker1 # [ 121.399826] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359585497worker1 # [ 121.417421] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359585498worker1 # [ 121.433066] nix-grpc-daemon[1330]: time=2026-09-20T13:34:34.839Z level=INFO msg="All 2 paths already cached"5499client # copying 1 paths to 'lb:50051'...5500client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5501worker1 # [ 121.443933] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:359585502worker1 # [ 121.460296] nix-daemon[898]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5503worker1 # [ 121.660499] nix-daemon[898]: accepted connection from pid 1415, user nix-grpc-daemon (trusted)5504worker1 # [ 121.678717] nix-daemon[898]: reaped child process 1418, status = succeeded5505worker1 # [ 121.857925] nix-daemon[898]: accepted connection from pid 1420, user nix-grpc-daemon (trusted)5506worker1 # [ 121.876416] nix-daemon[898]: reaped child process 1423, status = succeeded5507lb # [ 121.874714] niks3-server[1180]: 2026/09/20 13:34:35 INFO Received uploads request method=POST path=/api/pending_closures5508worker1 # [ 121.897506] nix-grpc-daemon[1330]: time=2026-09-20T13:34:35.303Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5509worker1 # [ 121.899058] nix-grpc-daemon[1330]: time=2026-09-20T13:34:35.303Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5510lb # [ 121.903147] niks3-server[1180]: 2026/09/20 13:34:35 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5511lb # [ 121.916978] niks3-server[1180]: 2026/09/20 13:34:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5512worker1 # [ 121.937660] nix-grpc-daemon[1330]: time=2026-09-20T13:34:35.343Z level=INFO msg="Uploading 1 narinfos"5513lb # [ 121.929711] niks3-server[1180]: 2026/09/20 13:34:35 INFO Signed narinfos id=31 count=15514lb # [ 121.935680] niks3-server[1180]: 2026/09/20 13:34:35 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5515lb # [ 121.943103] niks3-server[1180]: 2026/09/20 13:34:35 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5516lb # [ 121.949500] niks3-server[1180]: 2026/09/20 13:34:35 INFO Completed upload id=315517worker1 # [ 121.962056] nix-grpc-daemon[1330]: time=2026-09-20T13:34:35.368Z level=INFO msg="Upload complete. (484ms)"5518lb # [ 121.951862] niks3-server[1180]: 2026/09/20 13:34:35 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5519worker1 # [ 121.971562] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35958 duration_s=0 paths=1 nar_bytes_in=7685520worker1 # [ 121.983321] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359585521worker1 # [ 121.989433] nix-daemon[898]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5522worker1 # [ 121.994327] nix-daemon[898]: reaped child process 1413, status = succeeded5523client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5524client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5525worker1 # [ 122.009212] nix-daemon[898]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5526client # farm-top> LOG-farm-top-on-standby5527worker1 # [ 123.049954] nix-daemon[898]: reaped child process 1427, status = succeeded5528worker1 # [ 123.213224] nix-daemon[898]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5529worker1 # [ 123.233588] nix-daemon[898]: reaped child process 1445, status = succeeded5530lb # [ 123.236212] niks3-server[1180]: 2026/09/20 13:34:36 INFO Received uploads request method=POST path=/api/pending_closures5531worker1 # [ 123.258522] nix-grpc-daemon[1330]: time=2026-09-20T13:34:36.664Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5532worker1 # [ 123.262453] nix-grpc-daemon[1330]: time=2026-09-20T13:34:36.666Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5533lb # [ 123.270984] niks3-server[1180]: 2026/09/20 13:34:36 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5534lb # [ 123.287007] niks3-server[1180]: 2026/09/20 13:34:36 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5535worker1 # [ 123.310904] nix-grpc-daemon[1330]: time=2026-09-20T13:34:36.716Z level=INFO msg="Uploading 1 narinfos"5536lb # [ 123.299993] niks3-server[1180]: 2026/09/20 13:34:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5537lb # [ 123.303655] niks3-server[1180]: 2026/09/20 13:34:36 INFO Signed narinfos id=32 count=15538lb # [ 123.314758] niks3-server[1180]: 2026/09/20 13:34:36 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5539lb # [ 123.317626] niks3-server[1180]: 2026/09/20 13:34:36 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5540lb # [ 123.321549] niks3-server[1180]: 2026/09/20 13:34:36 INFO Completed upload id=325541lb # [ 123.323643] niks3-server[1180]: 2026/09/20 13:34:36 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5542worker1 # [ 123.333318] nix-grpc-daemon[1330]: time=2026-09-20T13:34:36.739Z level=INFO msg="Upload complete. (313ms)"5543worker1 # [ 123.349415] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35958 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15544worker1 # [ 123.364759] nix-daemon[898]: reaped child process 1425, status = succeeded5545client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 8.50 seconds)5546client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5547client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.23 seconds)5548worker2 # [ 123.722832] systemd[1]: Listening on Nix worker-protocol over gRPC.5549worker2 # [ 123.731976] systemd[1]: Starting scheduler-token.service...5550worker2 # [ 123.833892] systemd[1]: scheduler-token.service: Deactivated successfully.5551worker2 # [ 123.840607] systemd[1]: Finished scheduler-token.service.5552worker2 # [ 123.841937] systemd[1]: scheduler-token.service: Consumed 42ms CPU time over 104ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5553worker2 # [ 123.848534] systemd[1]: Starting Nix worker-protocol over gRPC...5554worker2 # [ 124.032579] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5555worker2 # [ 124.035617] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5556worker2 # [ 124.040317] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15557worker2 # [ 124.048639] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5558worker2 # [ 124.050426] systemd[1]: Started Nix worker-protocol over gRPC.5559lb: must succeed: curl -sf localhost:9901/clusters5560worker2 # [ 124.072509] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:37Z level=info event=scheduler_connected addr=lb:500515561worker1 # [ 124.084190] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:37Z level=info event=oidc_jwks provider=mock keys=15562worker1 # [ 124.090173] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:37Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:359445563worker1 # [ 124.095537] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:37Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05564lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5565worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5566worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5567lb: must succeed: curl -sf localhost:9901/clusters5568lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5569lb: must succeed: curl -sf localhost:9901/clusters5570lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5571worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5572worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5573(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 14.40 seconds)5574subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5575client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5576client # [ 124.509965] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5577client # Running as unit: rs.service; invocation ID: bdcd8899a5d24fd7adbb328cb16ae05e5578client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.11 seconds)5579worker1 # [ 124.763809] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:359585580client # [ 124.770471] nix[1169]: don't know how to build these paths:5581client # [ 124.771634] nix[1169]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5582worker1 # [ 124.784181] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:38Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05583worker1 # [ 124.791216] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:38Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15584worker1 # [ 124.829173] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359585585worker1 # [ 124.845281] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359585586client # [ 124.854831] nix[1169]: copying 1 paths to 'lb:50051'...5587worker1 # [ 124.861454] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:359585588client # [ 124.861655] nix[1169]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5589worker1 # [ 124.878465] nix-daemon[898]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5590worker1 # [ 125.080108] nix-daemon[898]: accepted connection from pid 1458, user nix-grpc-daemon (trusted)5591worker1 # [ 125.100471] nix-daemon[898]: reaped child process 1461, status = succeeded5592worker1 # [ 125.280147] nix-daemon[898]: accepted connection from pid 1463, user nix-grpc-daemon (trusted)5593worker1 # [ 125.298728] nix-daemon[898]: reaped child process 1466, status = succeeded5594lb # [ 125.298630] niks3-server[1180]: 2026/09/20 13:34:38 INFO Received uploads request method=POST path=/api/pending_closures5595worker1 # [ 125.321763] nix-grpc-daemon[1330]: time=2026-09-20T13:34:38.727Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5596worker1 # [ 125.323239] nix-grpc-daemon[1330]: time=2026-09-20T13:34:38.727Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5597lb # [ 125.327764] niks3-server[1180]: 2026/09/20 13:34:38 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5598lb # [ 125.342276] niks3-server[1180]: 2026/09/20 13:34:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5599worker1 # [ 125.361360] nix-grpc-daemon[1330]: time=2026-09-20T13:34:38.767Z level=INFO msg="Uploading 1 narinfos"5600lb # [ 125.354394] niks3-server[1180]: 2026/09/20 13:34:38 INFO Signed narinfos id=33 count=15601lb # [ 125.360299] niks3-server[1180]: 2026/09/20 13:34:38 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5602lb # [ 125.366916] niks3-server[1180]: 2026/09/20 13:34:38 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5603lb # [ 125.374188] niks3-server[1180]: 2026/09/20 13:34:38 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5604lb # [ 125.378030] niks3-server[1180]: 2026/09/20 13:34:38 INFO Completed upload id=335605worker1 # [ 125.390821] nix-grpc-daemon[1330]: time=2026-09-20T13:34:38.797Z level=INFO msg="Upload complete. (496ms)"5606worker1 # [ 125.399549] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35958 duration_s=0 paths=1 nar_bytes_in=4165607worker1 # [ 125.414032] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359585608worker1 # [ 125.420947] nix-daemon[898]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5609worker1 # [ 125.426168] nix-daemon[898]: reaped child process 1456, status = succeeded5610client # [ 125.434601] nix[1169]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5611worker1 # [ 125.439976] nix-daemon[898]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5612lb: must succeed: curl -sf localhost:9901/clusters5613lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5614lb: must succeed: curl -sf localhost:9901/clusters5615lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5616lb: must succeed: curl -sf localhost:9901/clusters5617lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5618lb: must succeed: curl -sf localhost:9901/clusters5619lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5620worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5621worker2 # [ 127.492195] systemd-journald[276]: Received client request to rotate journal, rotating.5622worker2 # [ 127.513562] systemd-journald[276]: Vacuuming done, freed 0B of archived journals from /var/log/journal/67d7290bb28f490eb2a8de352531cbaf.5623worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)5624worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5625worker1 # [ 127.566331] systemd[1]: Stopping Nix worker-protocol over gRPC...5626worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5627??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5628 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735629client: must succeed: sleep 5.05630worker1 # [ 127.812266] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:41Z level=info event=draining5631client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5632worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5633worker1 # [ 133.088685] nix-daemon[898]: reaped child process 1470, status = succeeded5634lb # [ 133.272100] envoy[911]: conn peer=[2001:db8:1::1]:58818 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9593 tx=3883 ms=81755635worker1 # [ 133.293713] nix-daemon[898]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)5636worker1 # [ 133.315790] nix-daemon[898]: reaped child process 1513, status = succeeded5637lb # [ 133.317494] niks3-server[1180]: 2026/09/20 13:34:46 INFO Received uploads request method=POST path=/api/pending_closures5638worker1 # [ 133.348625] nix-grpc-daemon[1330]: time=2026-09-20T13:34:46.752Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5639worker1 # [ 133.350600] nix-grpc-daemon[1330]: time=2026-09-20T13:34:46.752Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5640lb # [ 133.379123] niks3-server[1180]: 2026/09/20 13:34:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5641worker1 # [ 133.399346] nix-grpc-daemon[1330]: time=2026-09-20T13:34:46.805Z level=INFO msg="Uploading 1 narinfos"5642lb # [ 133.390649] niks3-server[1180]: 2026/09/20 13:34:46 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5643lb # [ 133.402413] niks3-server[1180]: 2026/09/20 13:34:46 INFO Signed narinfos id=34 count=15644lb # [ 133.410454] niks3-server[1180]: 2026/09/20 13:34:46 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5645lb # [ 133.417779] niks3-server[1180]: 2026/09/20 13:34:46 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5646lb # [ 133.421218] niks3-server[1180]: 2026/09/20 13:34:46 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5647lb # [ 133.427581] niks3-server[1180]: 2026/09/20 13:34:46 INFO Completed upload id=345648worker1 # [ 133.440615] nix-grpc-daemon[1330]: time=2026-09-20T13:34:46.846Z level=INFO msg="Upload complete. (384ms)"5649lb # [ 133.433863] niks3-server[1180]: 2026/09/20 13:34:46 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5650worker1 # [ 133.454970] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35958 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15651worker1 # [ 133.475857] nix-daemon[898]: reaped child process 1468, status = succeeded5652client # [ 133.497630] systemd[1]: rs.service: Deactivated successfully.5653client # [ 133.502068] systemd[1]: rs.service: Consumed 166ms CPU time over 8.991s wall clock time, 37M memory peak, 8.4K incoming IP traffic, 9K outgoing IP traffic.5654worker1 # [ 133.813476] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:47Z level=info event=signal_exit5655worker1 # [ 134.046837] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:47Z level=info event=scheduler_yield5656lb # [ 134.037903] niks3-server[1180]: 2026/09/20 13:34:47 INFO lead: released remote=192.168.1.3:604265657worker1 # [ 134.051470] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:47Z level=info event=scheduler_restarting addr=lb:500515658worker2 # [ 134.053511] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:47Z level=info event=scheduler_restarting addr=lb:500515659worker1 # [ 134.058605] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:359445660worker2 # [ 134.060576] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:47Z level=info event=scheduler_take_over5661worker1 # [ 134.063497] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:47Z level=info event=worker_gone addr=192.168.1.4:500515662lb # [ 134.053445] niks3-server[1180]: 2026/09/20 13:34:47 INFO lead: acquired remote=192.168.1.4:406405663worker1 # [ 134.067643] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:359445664worker1 # [ 134.072263] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:47Z level=info event=worker_gone addr=192.168.1.3:500515665worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 1.15 seconds)5666client: waiting for success: ! systemctl is-active rs5667client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5668client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5669client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5670client: must fail: journalctl -u rs | grep -q 'reconnecting'5671worker2 # [ 134.253653] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:47Z level=info event=scheduler_connected addr=lb:500515672worker1 # [ 134.252787] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:47Z level=info event=scheduler_connected addr=lb:500515673client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.04 seconds)5674lb: must succeed: curl -sf localhost:9901/clusters5675worker1 # [ 134.258929] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5676worker2 # [ 134.259157] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5677lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5678worker2 # [ 134.460422] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:47Z level=info event=scheduler_connected addr=lb:500515679worker1 # [ 134.461534] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:47Z level=info event=scheduler_connected addr=lb:500515680worker2 # [ 134.465699] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5681worker1 # [ 134.466065] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5682worker1 # [ 134.662775] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:48Z level=info event=scheduler_connected addr=lb:500515683worker2 # [ 134.666842] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:48Z level=info event=scheduler_connected addr=lb:500515684worker1 # [ 134.668556] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5685worker2 # [ 134.672158] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5686worker1 # [ 134.869456] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:48Z level=info event=scheduler_connected addr=lb:500515687worker2 # [ 134.873364] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:48Z level=info event=scheduler_connected addr=lb:500515688worker1 # [ 134.875040] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5689worker2 # [ 134.878521] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5690worker1 # [ 135.050488] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:48Z level=info event=scheduler_restarting5691worker2 # [ 135.081743] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:48Z level=info event=scheduler_connected addr=lb:500515692worker2 # [ 135.085993] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5693worker1 # [ 135.095866] nix-grpc-daemon[1260]: ts=2026-09-20T13:34:48Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5694worker1 # [ 135.112475] nix-daemon[898]: reaped child process 1321, status = succeeded5695worker1 # [ 135.177934] nix-grpc-daemon[1260]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5696worker1 # [ 135.182746] nix-grpc-daemon[1260]: E0000 00:00:1789911288.583966 1303 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5697worker1 # [ 135.189428] nix-grpc-daemon[1260]: E0000 00:00:1789911288.584247 1303 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5698worker2 # [ 135.282727] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:48Z level=info event=scheduler_connected addr=lb:500515699worker2 # [ 135.288291] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5700lb: must succeed: curl -sf localhost:9901/clusters5701lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5702worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5703worker2 # [ 135.489400] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:48Z level=info event=scheduler_connected addr=lb:500515704worker2 # [ 135.495011] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5705worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)5706worker2 # [ 135.695591] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:49Z level=info event=scheduler_connected addr=lb:500515707worker2 # [ 135.714316] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:49Z level=info event=oidc_jwks provider=mock keys=15708worker2 # [ 135.718887] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:49Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:460605709worker2 # [ 135.724287] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:49Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05710worker1 # [ 136.276967] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5711worker1 # [ 136.282371] systemd[1]: Stopped Nix worker-protocol over gRPC.5712worker1 # [ 136.288643] systemd[1]: nix-grpc-daemon.service: Consumed 1.933s CPU time over 27.713s wall clock time, 34.5M memory peak, 49.4K incoming IP traffic, 89.5K outgoing IP traffic.5713worker1 # [ 136.307998] systemd[1]: Starting Nix worker-protocol over gRPC...5714worker1 # [ 136.503991] nix-grpc-daemon[1520]: ts=2026-09-20T13:34:49Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5715worker1 # [ 136.506702] nix-grpc-daemon[1520]: ts=2026-09-20T13:34:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5716worker1 # [ 136.510844] nix-grpc-daemon[1520]: ts=2026-09-20T13:34:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15717worker1 # [ 136.519173] nix-grpc-daemon[1520]: ts=2026-09-20T13:34:49Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5718worker1 # [ 136.523147] systemd[1]: Started Nix worker-protocol over gRPC.5719lb: must succeed: curl -sf localhost:9901/clusters5720worker1 # [ 136.541279] nix-grpc-daemon[1520]: ts=2026-09-20T13:34:49Z level=info event=scheduler_connected addr=lb:500515721worker2 # [ 136.546492] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:49Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:460605722worker2 # [ 136.546790] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:49Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05723lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5724worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5725worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5726(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.26 seconds)5727subtest: scheduler restart mid-build: build finishes, no double build5728client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5729client # [ 136.761089] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5730client # Running as unit: mid.service; invocation ID: 72b817a289324bebb36ed22c13c757655731client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)5732worker2 # [ 137.000118] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:460685733worker2 # [ 137.015058] nix-daemon[913]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5734client # [ 137.032061] nix[1204]: don't know how to build these paths:5735client # [ 137.034218] nix[1204]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5736worker2 # [ 137.044360] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:50Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05737worker2 # [ 137.051194] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:50Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15738worker2 # [ 137.097447] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460685739worker2 # [ 137.115971] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 1/5)5740worker2 # [ 137.152644] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 155 ms (attempt 2/5)5741worker2 # [ 137.315351] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 348 ms (attempt 3/5)5742worker2 # [ 137.664898] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 369 ms (attempt 4/5)5743worker2 # [ 138.034094] nix-grpc-daemon[1246]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5744worker2 # [ 138.058569] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:460685745client # [ 138.073015] nix[1204]: copying 1 paths to 'lb:50051'...5746worker2 # [ 138.074719] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:460685747client # [ 138.073265] nix[1204]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5748worker2 # [ 138.091008] nix-daemon[913]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5749worker2 # [ 138.127185] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:51Z level=info event=niks3_push_started pid=13625750worker2 # [ 138.346960] nix-grpc-daemon[1362]: time=2026-09-20T13:34:51.721Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""5751worker2 # [ 138.526675] nix-daemon[913]: accepted connection from pid 1375, user nix-grpc-daemon (trusted)5752worker2 # [ 138.546086] nix-daemon[913]: reaped child process 1378, status = succeeded5753worker2 # [ 138.725087] nix-daemon[913]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5754worker2 # [ 138.743491] nix-daemon[913]: reaped child process 1383, status = succeeded5755lb # [ 138.743481] niks3-server[1180]: 2026/09/20 13:34:52 INFO Received uploads request method=POST path=/api/pending_closures5756worker2 # [ 138.768079] nix-grpc-daemon[1362]: time=2026-09-20T13:34:52.142Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5757worker2 # [ 138.769811] nix-grpc-daemon[1362]: time=2026-09-20T13:34:52.142Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5758lb # [ 138.781033] niks3-server[1180]: 2026/09/20 13:34:52 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5759lb # [ 138.796661] niks3-server[1180]: 2026/09/20 13:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5760worker2 # [ 138.816230] nix-grpc-daemon[1362]: time=2026-09-20T13:34:52.190Z level=INFO msg="Uploading 1 narinfos"5761lb # [ 138.809444] niks3-server[1180]: 2026/09/20 13:34:52 INFO Signed narinfos id=35 count=15762lb # [ 138.816725] niks3-server[1180]: 2026/09/20 13:34:52 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5763lb # [ 138.828111] niks3-server[1180]: 2026/09/20 13:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5764lb # [ 138.837079] niks3-server[1180]: 2026/09/20 13:34:52 INFO Completed upload id=355765worker2 # [ 138.849357] nix-grpc-daemon[1362]: time=2026-09-20T13:34:52.224Z level=INFO msg="Upload complete. (497ms)"5766lb # [ 138.841935] niks3-server[1180]: 2026/09/20 13:34:52 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5767worker2 # [ 138.862066] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46068 duration_s=0 paths=1 nar_bytes_in=4245768worker2 # [ 138.876399] nix-grpc-daemon[1246]: ts=2026-09-20T13:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:460685769worker2 # [ 138.883111] nix-daemon[913]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5770worker2 # [ 138.888828] nix-daemon[913]: reaped child process 1359, status = succeeded5771client # [ 138.898308] nix[1204]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5772worker2 # [ 138.904460] nix-daemon[913]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5773lb: must succeed: curl -sf localhost:9901/clusters5774lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5775lb: must succeed: curl -sf localhost:9901/clusters5776lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5777worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5778worker2 # [ 140.884482] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1246 (nix-grpc-daemon) on client request.5779worker2 # [ 140.886311] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5780lb # [ 140.882732] niks3-server[1180]: 2026/09/20 13:34:54 INFO lead: released remote=192.168.1.4:406405781worker2 # [ 140.909635] systemd[1]: Starting scheduler-token.service...5782client # [ 140.908512] nix[1204]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5783worker1 # [ 140.909718] nix-grpc-daemon[1520]: ts=2026-09-20T13:34:54Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5784client # [ 140.908816] nix[1204]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5785worker2 # [ 140.927123] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5786worker2 # [ 140.933838] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5787worker2 # [ 140.935165] systemd[1]: nix-grpc-daemon.service: Consumed 577ms CPU time over 17.077s wall clock time, 31.6M memory peak, 16.4K incoming IP traffic, 27.8K outgoing IP traffic.5788worker2 # [ 140.959103] nix-daemon[913]: reaped child process 1349, status = succeeded5789worker2 # [ 140.962329] nix-daemon[913]: reaped child process 1385, status = succeeded5790worker2 # [ 140.968530] nix-daemon[1387]: unexpected Nix daemon error: error: interrupted by the user5791worker2 # [ 140.977201] nix-daemon[913]: reaped child process 1387, status = failed with exit code 15792worker2 # [ 141.018234] systemd[1]: scheduler-token.service: Deactivated successfully.5793worker2 # [ 141.022763] systemd[1]: Finished scheduler-token.service.5794worker2 # [ 141.023711] systemd[1]: scheduler-token.service: Consumed 26ms CPU time over 108ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5795worker2 # [ 141.026292] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5796worker2 # [ 141.029533] systemd[1]: Starting scheduler-token.service...5797worker2 # [ 141.099850] systemd[1]: scheduler-token.service: Deactivated successfully.5798worker2 # [ 141.102424] systemd[1]: Finished scheduler-token.service.5799worker2 # [ 141.103237] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 72ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5800worker2 # [ 141.109686] systemd[1]: Starting Nix worker-protocol over gRPC...5801worker1 # [ 141.109974] nix-grpc-daemon[1520]: ts=2026-09-20T13:34:54Z level=info event=scheduler_connected addr=lb:500515802worker2 # [ 141.281611] nix-grpc-daemon[1429]: ts=2026-09-20T13:34:54Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5803worker2 # [ 141.284278] nix-grpc-daemon[1429]: ts=2026-09-20T13:34:54Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5804worker2 # [ 141.287328] nix-grpc-daemon[1429]: ts=2026-09-20T13:34:54Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15805worker2 # [ 141.295733] nix-grpc-daemon[1429]: ts=2026-09-20T13:34:54Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5806worker2 # [ 141.306606] systemd[1]: Started Nix worker-protocol over gRPC.5807worker2 # [ 141.310491] nix-grpc-daemon[1429]: ts=2026-09-20T13:34:54Z level=info event=scheduler_take_over5808worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.48 seconds)5809lb # [ 141.296665] niks3-server[1180]: 2026/09/20 13:34:54 INFO lead: acquired remote=192.168.1.4:399665810client: must succeed: sleep 5.05811worker2 # [ 141.322609] nix-grpc-daemon[1429]: ts=2026-09-20T13:34:54Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:517365812worker2 # [ 141.325556] nix-grpc-daemon[1429]: ts=2026-09-20T13:34:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05813worker2 # [ 141.328255] nix-grpc-daemon[1429]: ts=2026-09-20T13:34:54Z level=info event=scheduler_connected addr=lb:500515814worker2 # [ 141.333566] nix-grpc-daemon[1429]: ts=2026-09-20T13:34:54Z level=info event=oidc_jwks provider=mock keys=15815worker2 # [ 141.334865] nix-grpc-daemon[1429]: ts=2026-09-20T13:34:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:517365816worker2 # [ 141.336864] nix-grpc-daemon[1429]: ts=2026-09-20T13:34:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05817worker2 # [ 141.420166] nix-grpc-daemon[1429]: ts=2026-09-20T13:34:54Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05818worker2 # [ 141.424586] nix-grpc-daemon[1429]: ts=2026-09-20T13:34:54Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15819worker1 # [ 141.475026] nix-grpc-daemon[1520]: ts=2026-09-20T13:34:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594625820worker1 # [ 141.490378] nix-daemon[898]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)5821worker1 # [ 141.516439] nix-grpc-daemon[1520]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 1/5)5822worker1 # [ 141.524636] nix-grpc-daemon[1520]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 2/5)5823worker1 # [ 141.638719] nix-grpc-daemon[1520]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 122 ms (attempt 3/5)5824worker1 # [ 141.768541] nix-grpc-daemon[1520]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 770 ms (attempt 4/5)5825worker1 # [ 142.547851] nix-grpc-daemon[1520]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5826worker1 # [ 142.568354] nix-grpc-daemon[1520]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5827worker1 # [ 142.600351] nix-daemon[898]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)5828client # [ 142.617752] nix[1204]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5829worker1 # [ 142.623218] nix-daemon[898]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)5830lb # [ 143.270454] envoy[911]: conn peer=[2001:db8:1::1]:57218 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2940 tx=1349 ms=87055831lb # [ 143.275688] envoy[911]: conn peer=[2001:db8:1::3]:54386 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4295 tx=1320 ms=262625832lb # [ 143.280908] envoy[911]: conn peer=[2001:db8:1::4]:60074 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7245 tx=567 ms=168445833client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5834worker1 # [ 146.649601] nix-grpc-daemon[1520]: ts=2026-09-20T13:35:00Z level=info event=niks3_push_started pid=16215835worker1 # [ 146.669686] nix-daemon[898]: reaped child process 1597, status = succeeded5836client: waiting for success: ! systemctl is-active mid5837worker1 # [ 146.849945] nix-grpc-daemon[1621]: time=2026-09-20T13:35:00.255Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5838worker1 # [ 147.027022] nix-daemon[898]: accepted connection from pid 1634, user nix-grpc-daemon (trusted)5839worker1 # [ 147.045360] nix-daemon[898]: reaped child process 1637, status = succeeded5840lb # [ 147.046094] niks3-server[1180]: 2026/09/20 13:35:00 INFO Received uploads request method=POST path=/api/pending_closures5841worker1 # [ 147.070635] nix-grpc-daemon[1621]: time=2026-09-20T13:35:00.475Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5842worker1 # [ 147.073526] nix-grpc-daemon[1621]: time=2026-09-20T13:35:00.475Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5843lb # [ 147.097960] niks3-server[1180]: 2026/09/20 13:35:00 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5844worker1 # [ 147.136153] nix-grpc-daemon[1621]: time=2026-09-20T13:35:00.541Z level=INFO msg="Uploading 1 narinfos"5845lb # [ 147.121786] niks3-server[1180]: 2026/09/20 13:35:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5846lb # [ 147.136621] niks3-server[1180]: 2026/09/20 13:35:00 INFO Signed narinfos id=36 count=15847lb # [ 147.139121] niks3-server[1180]: 2026/09/20 13:35:00 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5848lb # [ 147.153060] niks3-server[1180]: 2026/09/20 13:35:00 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5849worker1 # [ 147.172590] nix-grpc-daemon[1621]: time=2026-09-20T13:35:00.578Z level=INFO msg="Upload complete. (318ms)"5850lb # [ 147.160854] niks3-server[1180]: 2026/09/20 13:35:00 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5851lb # [ 147.164375] niks3-server[1180]: 2026/09/20 13:35:00 INFO Completed upload id=365852lb # [ 147.171176] niks3-server[1180]: 2026/09/20 13:35:00 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5853worker1 # [ 147.184283] nix-grpc-daemon[1520]: ts=2026-09-20T13:35:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59462 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15854worker1 # [ 147.200337] nix-daemon[898]: reaped child process 1595, status = succeeded5855client # [ 147.229628] systemd[1]: mid.service: Deactivated successfully.5856client # [ 147.233946] systemd[1]: mid.service: Consumed 162ms CPU time over 10.472s wall clock time, 36.9M memory peak, 9.6K incoming IP traffic, 10.5K outgoing IP traffic.5857client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5858client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5859client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)5860(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 11.19 seconds)5861subtest: interrupting the client stops the build on the worker5862client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5863client # [ 147.921696] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5864client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5865client # Running as unit: intr.service; invocation ID: bcf983822cb0465382e85e51d31e20b65866client # [ 148.154587] nix[1239]: don't know how to build these paths:5867client # [ 148.156623] nix[1239]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5868worker1 # [ 148.154788] nix-grpc-daemon[1520]: ts=2026-09-20T13:35:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:594625869worker2 # [ 148.173398] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:01Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05870worker2 # [ 148.173730] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:01Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15871worker2 # [ 148.221872] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517705872worker2 # [ 148.236973] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5873worker2 # [ 148.260600] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5)5874worker2 # [ 148.271961] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 2/5)5875worker2 # [ 148.307999] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 157 ms (attempt 3/5)5876worker2 # [ 148.473002] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 4/5)5877worker2 # [ 148.483607] nix-grpc-daemon[1429]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5878worker2 # [ 148.505022] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517705879client # [ 148.519842] nix[1239]: copying 1 paths to 'lb:50051'...5880worker2 # [ 148.521192] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517705881client # [ 148.522614] nix[1239]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5882worker2 # [ 148.535886] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5883worker2 # [ 148.564126] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:01Z level=info event=niks3_push_started pid=15015884worker2 # [ 148.782747] nix-grpc-daemon[1501]: time=2026-09-20T13:35:02.157Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""5885worker2 # [ 148.959723] nix-daemon[913]: accepted connection from pid 1514, user nix-grpc-daemon (trusted)5886worker2 # [ 148.980389] nix-daemon[913]: reaped child process 1517, status = succeeded5887worker2 # [ 149.169309] nix-daemon[913]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)5888worker2 # [ 149.190014] nix-daemon[913]: reaped child process 1522, status = succeeded5889lb # [ 149.192977] niks3-server[1180]: 2026/09/20 13:35:02 INFO Received uploads request method=POST path=/api/pending_closures5890worker2 # [ 149.467410] nix-grpc-daemon[1501]: time=2026-09-20T13:35:02.838Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5891worker2 # [ 149.473029] nix-grpc-daemon[1501]: time=2026-09-20T13:35:02.839Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5892lb # [ 149.494990] niks3-server[1180]: 2026/09/20 13:35:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5893worker2 # [ 149.518407] nix-grpc-daemon[1501]: time=2026-09-20T13:35:02.892Z level=INFO msg="Uploading 1 narinfos"5894lb # [ 149.508492] niks3-server[1180]: 2026/09/20 13:35:02 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5895lb # [ 149.525969] niks3-server[1180]: 2026/09/20 13:35:02 INFO Signed narinfos id=37 count=15896lb # [ 149.537993] niks3-server[1180]: 2026/09/20 13:35:02 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5897lb # [ 149.548292] niks3-server[1180]: 2026/09/20 13:35:02 INFO Completed upload id=375898lb # [ 149.551837] niks3-server[1180]: 2026/09/20 13:35:02 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5899worker2 # [ 149.561312] nix-grpc-daemon[1501]: time=2026-09-20T13:35:02.936Z level=INFO msg="Upload complete. (774ms)"5900lb # [ 149.562233] niks3-server[1180]: 2026/09/20 13:35:02 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5901worker2 # [ 149.574299] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51770 duration_s=1 paths=1 nar_bytes_in=4245902worker2 # [ 149.595628] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517705903worker2 # [ 149.605828] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5904worker2 # [ 149.615024] nix-daemon[913]: reaped child process 1499, status = succeeded5905worker2 # [ 149.629766] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5906client # [ 149.627111] nix[1239]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5907client: must succeed: systemctl kill -s INT intr5908client # [ 150.467882] systemd[1]: intr.service: Sent signal SIGINT to main process 1239 (nix) on client request.5909client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)5910client # [ 150.473754] nix[1239]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5911client # [ 150.477259] nix[1239]: error: interrupted by the user5912client # [ 150.521143] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5913client # [ 150.526294] systemd[1]: intr.service: Failed with result 'exit-code'.5914client # [ 150.528702] systemd[1]: intr.service: Consumed 149ms CPU time over 2.602s wall clock time, 37.2M memory peak, 7.8K incoming IP traffic, 8.9K outgoing IP traffic.5915worker2 # [ 150.668475] nix-daemon[1530]: unexpected Nix daemon error: error: interrupted by the user5916worker2 # [ 150.672719] nix-daemon[913]: reaped child process 1528, status = succeeded5917worker2 # [ 150.678664] nix-daemon[913]: reaped child process 1530, status = failed with exit code 15918(finished: subtest: interrupting the client stops the build on the worker, in 3.93 seconds)5919subtest: a deploy mid-build drains: the build finishes, the new generation takes over5920client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5921client # [ 151.878066] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5922client # Running as unit: sw.service; invocation ID: 7332cbf1b18941a691afcfe3e498423c5923client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5924client # [ 152.124476] nix[1259]: don't know how to build these paths:5925client # [ 152.125621] nix[1259]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5926worker2 # [ 152.125721] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:517705927worker2 # [ 152.141890] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:05Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05928worker2 # [ 152.148241] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:05Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15929worker1 # [ 152.190946] nix-grpc-daemon[1520]: ts=2026-09-20T13:35:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594625930worker1 # [ 152.208894] nix-grpc-daemon[1520]: ts=2026-09-20T13:35:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:594625931client # [ 152.224086] nix[1259]: copying 1 paths to 'lb:50051'...5932worker1 # [ 152.225228] nix-grpc-daemon[1520]: ts=2026-09-20T13:35:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:594625933client # [ 152.226729] nix[1259]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5934worker1 # [ 152.241291] nix-daemon[898]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)5935worker1 # [ 152.453771] nix-daemon[898]: accepted connection from pid 1665, user nix-grpc-daemon (trusted)5936worker1 # [ 152.473685] nix-daemon[898]: reaped child process 1668, status = succeeded5937worker1 # [ 152.650907] nix-daemon[898]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)5938worker1 # [ 152.670388] nix-daemon[898]: reaped child process 1673, status = succeeded5939lb # [ 152.671761] niks3-server[1180]: 2026/09/20 13:35:06 INFO Received uploads request method=POST path=/api/pending_closures5940worker1 # [ 152.701955] nix-grpc-daemon[1621]: time=2026-09-20T13:35:06.107Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5941worker1 # [ 152.703498] nix-grpc-daemon[1621]: time=2026-09-20T13:35:06.107Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5942lb # [ 152.710521] niks3-server[1180]: 2026/09/20 13:35:06 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5943lb # [ 152.726964] niks3-server[1180]: 2026/09/20 13:35:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5944worker1 # [ 152.746864] nix-grpc-daemon[1621]: time=2026-09-20T13:35:06.152Z level=INFO msg="Uploading 1 narinfos"5945lb # [ 152.742107] niks3-server[1180]: 2026/09/20 13:35:06 INFO Signed narinfos id=38 count=15946lb # [ 152.748325] niks3-server[1180]: 2026/09/20 13:35:06 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5947lb # [ 152.754795] niks3-server[1180]: 2026/09/20 13:35:06 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5948lb # [ 152.768473] niks3-server[1180]: 2026/09/20 13:35:06 INFO Completed upload id=385949worker1 # [ 152.781307] nix-grpc-daemon[1621]: time=2026-09-20T13:35:06.187Z level=INFO msg="Upload complete. (522ms)"5950lb # [ 152.771513] niks3-server[1180]: 2026/09/20 13:35:06 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5951worker1 # [ 152.792889] nix-grpc-daemon[1520]: ts=2026-09-20T13:35:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59462 duration_s=0 paths=1 nar_bytes_in=4165952worker1 # [ 152.804966] nix-grpc-daemon[1520]: ts=2026-09-20T13:35:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:594625953worker1 # [ 152.817273] nix-daemon[898]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)5954worker1 # [ 152.828410] nix-daemon[898]: reaped child process 1663, status = succeeded5955client # [ 152.834420] nix[1259]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5956worker1 # [ 152.838859] nix-daemon[898]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)5957lb # [ 153.271616] envoy[911]: conn peer=[2001:db8:1::1]:43878 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3854 tx=1661 ms=102055958lb # [ 153.277411] envoy[911]: conn peer=[2001:db8:1::1]:47586 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2957 tx=732 ms=23465959worker1: must succeed: systemctl reload nix-grpc-daemon.service5960worker1 # [ 154.610098] systemd[1]: Reloading Nix worker-protocol over gRPC...5961worker1 # [ 154.663640] systemd[1]: Reloaded Nix worker-protocol over gRPC.5962worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.11 seconds)5963worker1: must succeed: pgrep -f 'read -t [9]0 x'5964worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5965client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25966worker2 # [ 154.983612] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:517705967client # don't know how to build these paths:5968client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5969worker2 # [ 154.994783] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517705970worker2 # [ 155.023797] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:08Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05971worker2 # [ 155.030960] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:08Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=15972worker2 # [ 155.074649] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517705973worker2 # [ 155.090454] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517705974client # copying 1 paths to 'lb:50051'...5975worker2 # [ 155.107122] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517705976client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5977worker2 # [ 155.131140] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5978worker2 # [ 155.341753] nix-daemon[913]: accepted connection from pid 1580, user nix-grpc-daemon (trusted)5979worker2 # [ 155.362411] nix-daemon[913]: reaped child process 1583, status = succeeded5980worker1 # [ 155.532143] nix-grpc-daemon[1520]: ts=2026-09-20T13:35:08Z level=info event=draining5981worker2 # [ 155.537829] nix-daemon[913]: accepted connection from pid 1585, user nix-grpc-daemon (trusted)5982worker2 # [ 155.555854] nix-daemon[913]: reaped child process 1588, status = succeeded5983lb # [ 155.555811] niks3-server[1180]: 2026/09/20 13:35:08 INFO Received uploads request method=POST path=/api/pending_closures5984worker2 # [ 155.578750] nix-grpc-daemon[1501]: time=2026-09-20T13:35:08.953Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5985worker2 # [ 155.580309] nix-grpc-daemon[1501]: time=2026-09-20T13:35:08.953Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5986lb # [ 155.586602] niks3-server[1180]: 2026/09/20 13:35:08 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5987worker2 # [ 155.613994] nix-grpc-daemon[1501]: time=2026-09-20T13:35:08.988Z level=INFO msg="Uploading 1 narinfos"5988lb # [ 155.602886] niks3-server[1180]: 2026/09/20 13:35:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5989lb # [ 155.614941] niks3-server[1180]: 2026/09/20 13:35:08 INFO Signed narinfos id=39 count=15990lb # [ 155.622610] niks3-server[1180]: 2026/09/20 13:35:09 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5991lb # [ 155.637135] niks3-server[1180]: 2026/09/20 13:35:09 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5992lb # [ 155.642687] niks3-server[1180]: 2026/09/20 13:35:09 INFO Completed upload id=395993worker2 # [ 155.655199] nix-grpc-daemon[1501]: time=2026-09-20T13:35:09.030Z level=INFO msg="Upload complete. (506ms)"5994lb # [ 155.645298] niks3-server[1180]: 2026/09/20 13:35:09 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5995worker2 # [ 155.664306] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51770 duration_s=0 paths=1 nar_bytes_in=5285996worker2 # [ 155.679989] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517705997worker2 # [ 155.687046] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)5998worker2 # [ 155.692868] nix-daemon[913]: reaped child process 1578, status = succeeded5999client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6000client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6001worker2 # [ 155.708168] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6002client # farm-a> LOG-farm-a-during-drain6003worker2 # [ 156.959136] nix-daemon[913]: reaped child process 1592, status = succeeded6004worker2 # [ 157.137556] nix-daemon[913]: accepted connection from pid 1607, user nix-grpc-daemon (trusted)6005worker2 # [ 157.155253] nix-daemon[913]: reaped child process 1610, status = succeeded6006lb # [ 157.157271] niks3-server[1180]: 2026/09/20 13:35:10 INFO Received uploads request method=POST path=/api/pending_closures6007worker2 # [ 157.183278] nix-grpc-daemon[1501]: time=2026-09-20T13:35:10.555Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6008worker2 # [ 157.185517] nix-grpc-daemon[1501]: time=2026-09-20T13:35:10.555Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6009lb # [ 157.198129] niks3-server[1180]: 2026/09/20 13:35:10 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6010lb # [ 157.210880] niks3-server[1180]: 2026/09/20 13:35:10 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6011lb # [ 157.224143] niks3-server[1180]: 2026/09/20 13:35:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6012lb # [ 157.230965] niks3-server[1180]: 2026/09/20 13:35:10 INFO Signed narinfos id=40 count=16013worker2 # [ 157.247607] nix-grpc-daemon[1501]: time=2026-09-20T13:35:10.621Z level=INFO msg="Uploading 1 narinfos"6014lb # [ 157.237706] niks3-server[1180]: 2026/09/20 13:35:10 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6015lb # [ 157.250981] niks3-server[1180]: 2026/09/20 13:35:10 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6016lb # [ 157.258889] niks3-server[1180]: 2026/09/20 13:35:10 INFO Completed upload id=406017worker2 # [ 157.271458] nix-grpc-daemon[1501]: time=2026-09-20T13:35:10.646Z level=INFO msg="Upload complete. (351ms)"6018lb # [ 157.261391] niks3-server[1180]: 2026/09/20 13:35:10 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6019worker2 # [ 157.289151] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51770 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16020worker2 # [ 157.304353] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:10Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06021worker2 # [ 157.311223] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:10Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16022worker2 # [ 157.315481] nix-daemon[913]: reaped child process 1590, status = succeeded6023worker2 # [ 157.339415] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517706024worker2 # [ 157.355298] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517706025worker2 # [ 157.370328] nix-grpc-daemon[1501]: time=2026-09-20T13:35:10.745Z level=INFO msg="All 1 paths already cached"6026client # copying 1 paths to 'lb:50051'...6027worker2 # [ 157.380906] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517706028client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6029worker2 # [ 157.404511] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6030worker2 # [ 157.608111] nix-daemon[913]: accepted connection from pid 1616, user nix-grpc-daemon (trusted)6031worker2 # [ 157.626705] nix-daemon[913]: reaped child process 1619, status = succeeded6032worker2 # [ 157.799361] nix-daemon[913]: accepted connection from pid 1621, user nix-grpc-daemon (trusted)6033worker2 # [ 157.818564] nix-daemon[913]: reaped child process 1624, status = succeeded6034lb # [ 157.816912] niks3-server[1180]: 2026/09/20 13:35:11 INFO Received uploads request method=POST path=/api/pending_closures6035worker2 # [ 157.840062] nix-grpc-daemon[1501]: time=2026-09-20T13:35:11.214Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6036worker2 # [ 157.841518] nix-grpc-daemon[1501]: time=2026-09-20T13:35:11.214Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6037lb # [ 157.848949] niks3-server[1180]: 2026/09/20 13:35:11 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6038worker2 # [ 157.875458] nix-grpc-daemon[1501]: time=2026-09-20T13:35:11.250Z level=INFO msg="Uploading 1 narinfos"6039lb # [ 157.863577] niks3-server[1180]: 2026/09/20 13:35:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6040lb # [ 157.876093] niks3-server[1180]: 2026/09/20 13:35:11 INFO Signed narinfos id=41 count=16041lb # [ 157.878938] niks3-server[1180]: 2026/09/20 13:35:11 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6042lb # [ 157.887786] niks3-server[1180]: 2026/09/20 13:35:11 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6043lb # [ 157.894517] niks3-server[1180]: 2026/09/20 13:35:11 INFO Completed upload id=416044worker2 # [ 157.907346] nix-grpc-daemon[1501]: time=2026-09-20T13:35:11.282Z level=INFO msg="Upload complete. (485ms)"6045lb # [ 157.897513] niks3-server[1180]: 2026/09/20 13:35:11 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6046worker2 # [ 157.915423] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51770 duration_s=0 paths=1 nar_bytes_in=6406047worker2 # [ 157.927360] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517706048worker2 # [ 157.934500] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6049worker2 # [ 157.939605] nix-daemon[913]: reaped child process 1614, status = succeeded6050client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6051worker2 # [ 157.951488] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6052client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6053client # farm-b> LOG-farm-b-during-drain6054worker2 # [ 158.782361] nix-daemon[913]: reaped child process 1628, status = succeeded6055worker2 # [ 158.950619] nix-daemon[913]: accepted connection from pid 1643, user nix-grpc-daemon (trusted)6056worker2 # [ 158.968618] nix-daemon[913]: reaped child process 1646, status = succeeded6057lb # [ 158.970980] niks3-server[1180]: 2026/09/20 13:35:12 INFO Received uploads request method=POST path=/api/pending_closures6058worker2 # [ 158.992266] nix-grpc-daemon[1501]: time=2026-09-20T13:35:12.366Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6059worker2 # [ 158.993817] nix-grpc-daemon[1501]: time=2026-09-20T13:35:12.366Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6060lb # [ 159.012439] niks3-server[1180]: 2026/09/20 13:35:12 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6061worker2 # [ 159.039525] nix-grpc-daemon[1501]: time=2026-09-20T13:35:12.413Z level=INFO msg="Uploading 1 narinfos"6062lb # [ 159.026625] niks3-server[1180]: 2026/09/20 13:35:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6063lb # [ 159.031374] niks3-server[1180]: 2026/09/20 13:35:12 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6064lb # [ 159.039578] niks3-server[1180]: 2026/09/20 13:35:12 INFO Signed narinfos id=42 count=16065lb # [ 159.043139] niks3-server[1180]: 2026/09/20 13:35:12 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6066lb # [ 159.050133] niks3-server[1180]: 2026/09/20 13:35:12 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6067lb # [ 159.057070] niks3-server[1180]: 2026/09/20 13:35:12 INFO Completed upload id=426068worker2 # [ 159.069453] nix-grpc-daemon[1501]: time=2026-09-20T13:35:12.444Z level=INFO msg="Upload complete. (306ms)"6069lb # [ 159.060117] niks3-server[1180]: 2026/09/20 13:35:12 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6070worker2 # [ 159.088370] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51770 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16071worker2 # [ 159.104327] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:12Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06072worker2 # [ 159.108454] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:12Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16073worker2 # [ 159.112651] nix-daemon[913]: reaped child process 1626, status = succeeded6074worker2 # [ 159.133487] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517706075worker2 # [ 159.155966] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517706076client # copying 1 paths to 'lb:50051'...6077worker2 # [ 159.170582] nix-grpc-daemon[1501]: time=2026-09-20T13:35:12.545Z level=INFO msg="All 2 paths already cached"6078worker2 # [ 159.177535] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517706079client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6080worker2 # [ 159.201361] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6081worker2 # [ 159.401396] nix-daemon[913]: accepted connection from pid 1652, user nix-grpc-daemon (trusted)6082worker2 # [ 159.421350] nix-daemon[913]: reaped child process 1655, status = succeeded6083worker2 # [ 159.595533] nix-daemon[913]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)6084worker2 # [ 159.614675] nix-daemon[913]: reaped child process 1660, status = succeeded6085lb # [ 159.613079] niks3-server[1180]: 2026/09/20 13:35:13 INFO Received uploads request method=POST path=/api/pending_closures6086worker2 # [ 159.634715] nix-grpc-daemon[1501]: time=2026-09-20T13:35:13.009Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6087worker2 # [ 159.636269] nix-grpc-daemon[1501]: time=2026-09-20T13:35:13.009Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6088lb # [ 159.642071] niks3-server[1180]: 2026/09/20 13:35:13 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6089worker2 # [ 159.669479] nix-grpc-daemon[1501]: time=2026-09-20T13:35:13.044Z level=INFO msg="Uploading 1 narinfos"6090lb # [ 159.658779] niks3-server[1180]: 2026/09/20 13:35:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6091lb # [ 159.669991] niks3-server[1180]: 2026/09/20 13:35:13 INFO Signed narinfos id=43 count=16092lb # [ 159.682442] niks3-server[1180]: 2026/09/20 13:35:13 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6093lb # [ 159.686503] niks3-server[1180]: 2026/09/20 13:35:13 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6094worker2 # [ 159.703646] nix-grpc-daemon[1501]: time=2026-09-20T13:35:13.078Z level=INFO msg="Upload complete. (483ms)"6095lb # [ 159.691717] niks3-server[1180]: 2026/09/20 13:35:13 INFO Completed upload id=436096lb # [ 159.695638] niks3-server[1180]: 2026/09/20 13:35:13 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6097worker2 # [ 159.711010] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51770 duration_s=0 paths=1 nar_bytes_in=7686098worker2 # [ 159.724318] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517706099worker2 # [ 159.731158] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6100worker2 # [ 159.736250] nix-daemon[913]: reaped child process 1650, status = succeeded6101client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6102client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6103worker2 # [ 159.749930] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6104client # farm-top> LOG-farm-top-during-drain6105worker2 # [ 160.997873] nix-daemon[913]: reaped child process 1664, status = succeeded6106worker2 # [ 161.176787] nix-daemon[913]: accepted connection from pid 1679, user nix-grpc-daemon (trusted)6107worker2 # [ 161.196154] nix-daemon[913]: reaped child process 1682, status = succeeded6108lb # [ 161.197611] niks3-server[1180]: 2026/09/20 13:35:14 INFO Received uploads request method=POST path=/api/pending_closures6109worker2 # [ 161.220414] nix-grpc-daemon[1501]: time=2026-09-20T13:35:14.594Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6110worker2 # [ 161.221908] nix-grpc-daemon[1501]: time=2026-09-20T13:35:14.594Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6111lb # [ 161.236430] niks3-server[1180]: 2026/09/20 13:35:14 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6112lb # [ 161.254193] niks3-server[1180]: 2026/09/20 13:35:14 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6113worker2 # [ 161.273906] nix-grpc-daemon[1501]: time=2026-09-20T13:35:14.648Z level=INFO msg="Uploading 1 narinfos"6114lb # [ 161.264929] niks3-server[1180]: 2026/09/20 13:35:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6115lb # [ 161.271721] niks3-server[1180]: 2026/09/20 13:35:14 INFO Signed narinfos id=44 count=16116lb # [ 161.274465] niks3-server[1180]: 2026/09/20 13:35:14 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6117lb # [ 161.282044] niks3-server[1180]: 2026/09/20 13:35:14 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6118lb # [ 161.286391] niks3-server[1180]: 2026/09/20 13:35:14 INFO Completed upload id=446119worker2 # [ 161.299286] nix-grpc-daemon[1501]: time=2026-09-20T13:35:14.674Z level=INFO msg="Upload complete. (338ms)"6120lb # [ 161.288807] niks3-server[1180]: 2026/09/20 13:35:14 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6121worker2 # [ 161.313125] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51770 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16122worker2 # [ 161.327408] nix-daemon[913]: reaped child process 1662, status = succeeded6123client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.63 seconds)6124client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath6125client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)6126worker1: must succeed: pkill -f 'read -t [9]0 x'6127worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)6128client: waiting for success: ! systemctl is-active sw6129worker1 # [ 161.898797] nix-daemon[898]: reaped child process 1677, status = succeeded6130worker1 # [ 162.082190] nix-daemon[898]: accepted connection from pid 1717, user nix-grpc-daemon (trusted)6131worker1 # [ 162.101816] nix-daemon[898]: reaped child process 1720, status = succeeded6132lb # [ 162.102338] niks3-server[1180]: 2026/09/20 13:35:15 INFO Received uploads request method=POST path=/api/pending_closures6133worker1 # [ 162.125953] nix-grpc-daemon[1621]: time=2026-09-20T13:35:15.531Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6134worker1 # [ 162.127469] nix-grpc-daemon[1621]: time=2026-09-20T13:35:15.531Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6135lb # [ 162.139967] niks3-server[1180]: 2026/09/20 13:35:15 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6136lb # [ 162.153564] niks3-server[1180]: 2026/09/20 13:35:15 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6137worker1 # [ 162.174641] nix-grpc-daemon[1621]: time=2026-09-20T13:35:15.579Z level=INFO msg="Uploading 1 narinfos"6138lb # [ 162.163787] niks3-server[1180]: 2026/09/20 13:35:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6139lb # [ 162.170876] niks3-server[1180]: 2026/09/20 13:35:15 INFO Signed narinfos id=45 count=16140lb # [ 162.178061] niks3-server[1180]: 2026/09/20 13:35:15 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6141lb # [ 162.181933] niks3-server[1180]: 2026/09/20 13:35:15 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6142lb # [ 162.185940] niks3-server[1180]: 2026/09/20 13:35:15 INFO Completed upload id=456143worker1 # [ 162.198438] nix-grpc-daemon[1621]: time=2026-09-20T13:35:15.604Z level=INFO msg="Upload complete. (337ms)"6144lb # [ 162.188193] niks3-server[1180]: 2026/09/20 13:35:15 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6145worker1 # [ 162.210406] nix-grpc-daemon[1520]: ts=2026-09-20T13:35:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59462 duration_s=9 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16146worker1 # [ 162.226386] nix-daemon[898]: reaped child process 1675, status = succeeded6147client # [ 162.249230] systemd[1]: sw.service: Deactivated successfully.6148client # [ 162.254675] systemd[1]: sw.service: Consumed 163ms CPU time over 10.374s wall clock time, 37M memory peak, 8.7K incoming IP traffic, 9.1K outgoing IP traffic.6149worker1 # [ 162.534968] nix-grpc-daemon[1520]: ts=2026-09-20T13:35:15Z level=info event=signal_exit6150client: (finished: waiting for success: ! systemctl is-active sw, in 1.13 seconds)6151client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6152client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)6153worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6154lb # [ 163.269787] envoy[911]: conn peer=[2001:db8:1::1]:34008 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9655 tx=3860 ms=63466155lb # [ 163.274768] envoy[911]: conn peer=[2001:db8:1::1]:33998 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=1326 ms=100986156worker1 # [ 163.761877] nix-grpc-daemon[1520]: ts=2026-09-20T13:35:17Z level=info event=scheduler_restarting6157worker2 # [ 163.761705] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:517366158worker2 # [ 163.766518] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:17Z level=info event=worker_gone addr=192.168.1.3:500516159worker1 # [ 163.783478] nix-grpc-daemon[1520]: ts=2026-09-20T13:35:17Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6160worker1 # [ 163.810221] nix-daemon[898]: reaped child process 1589, status = succeeded6161worker1 # [ 164.345309] nix-grpc-daemon[1520]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6162worker1 # [ 164.350048] nix-grpc-daemon[1520]: E0000 00:00:1789911317.751342 1563 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6163worker1 # [ 164.356726] nix-grpc-daemon[1520]: E0000 00:00:1789911317.751630 1563 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6164worker1 # [ 164.573021] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6165worker1 # [ 164.580242] systemd[1]: nix-grpc-daemon.service: Consumed 860ms CPU time over 28.264s wall clock time, 30.6M memory peak, 20.7K incoming IP traffic, 27.1K outgoing IP traffic.6166worker1 # [ 164.602672] systemd[1]: Starting Nix worker-protocol over gRPC...6167worker1 # [ 164.803081] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6168worker1 # [ 164.805660] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6169worker1 # [ 164.809815] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16170worker1 # [ 164.817866] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6171worker1 # [ 164.819672] systemd[1]: Started Nix worker-protocol over gRPC.6172worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.99 seconds)6173lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6174lb: must succeed: curl -sf localhost:9901/clusters6175worker1 # [ 164.843205] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:18Z level=info event=scheduler_connected addr=lb:500516176worker2 # [ 164.848504] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:18Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:517366177worker2 # [ 164.848915] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:18Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06178lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6179lb: must succeed: curl -sf localhost:9901/clusters6180lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6181lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)6182lb: must succeed: curl -sf localhost:9901/clusters6183lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6184worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6185worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6186(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.33 seconds)6187subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6188client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16189worker1 # [ 166.393644] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:419966190worker1 # [ 166.407911] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6191worker2 # [ 166.431179] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517706192worker2 # [ 166.479143] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:19Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06193worker2 # [ 166.485308] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:19Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16194worker2 # [ 166.491297] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:19Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06195worker2 # [ 166.496539] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:19Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06196worker1 # [ 166.525142] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966197worker1 # [ 166.543034] nix-grpc-daemon[1734]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 1/5)6198worker1 # [ 166.604062] nix-grpc-daemon[1734]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 2/5)6199worker1 # [ 166.631653] nix-grpc-daemon[1734]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 111 ms (attempt 3/5)6200worker1 # [ 166.750522] nix-grpc-daemon[1734]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 267 ms (attempt 4/5)6201worker1 # [ 167.025911] nix-grpc-daemon[1734]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6202worker1 # [ 167.047425] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419966203worker1 # [ 167.063068] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419966204worker1 # [ 167.078209] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6205worker1 # [ 167.108620] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:20Z level=info event=niks3_push_started pid=17906206worker1 # [ 167.327130] nix-grpc-daemon[1790]: time=2026-09-20T13:35:20.732Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""6207worker1 # [ 167.501950] nix-daemon[898]: accepted connection from pid 1803, user nix-grpc-daemon (trusted)6208worker1 # [ 167.522475] nix-daemon[898]: reaped child process 1806, status = succeeded6209worker1 # [ 167.703939] nix-daemon[898]: accepted connection from pid 1808, user nix-grpc-daemon (trusted)6210worker1 # [ 167.721561] nix-daemon[898]: reaped child process 1811, status = succeeded6211lb # [ 167.725039] niks3-server[1180]: 2026/09/20 13:35:21 INFO Received uploads request method=POST path=/api/pending_closures6212worker1 # [ 167.751008] nix-grpc-daemon[1790]: time=2026-09-20T13:35:21.156Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6213worker1 # [ 167.752937] nix-grpc-daemon[1790]: time=2026-09-20T13:35:21.156Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6214lb # [ 167.763326] niks3-server[1180]: 2026/09/20 13:35:21 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6215worker1 # [ 167.789210] nix-grpc-daemon[1790]: time=2026-09-20T13:35:21.194Z level=INFO msg="Uploading 1 narinfos"6216lb # [ 167.776239] niks3-server[1180]: 2026/09/20 13:35:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6217lb # [ 167.783786] niks3-server[1180]: 2026/09/20 13:35:21 INFO Signed narinfos id=46 count=16218lb # [ 167.798646] niks3-server[1180]: 2026/09/20 13:35:21 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6219lb # [ 167.811319] niks3-server[1180]: 2026/09/20 13:35:21 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6220worker1 # [ 167.822608] nix-grpc-daemon[1790]: time=2026-09-20T13:35:21.228Z level=INFO msg="Upload complete. (490ms)"6221lb # [ 167.817298] niks3-server[1180]: 2026/09/20 13:35:21 INFO Completed upload id=466222lb # [ 167.820943] niks3-server[1180]: 2026/09/20 13:35:21 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6223worker1 # [ 167.832945] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=0 paths=1 nar_bytes_in=5126224worker1 # [ 167.846039] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966225worker1 # [ 167.851661] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6226worker1 # [ 167.856906] nix-daemon[898]: reaped child process 1788, status = succeeded6227worker1 # [ 167.869674] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6228worker1 # [ 168.694878] nix-daemon[898]: reaped child process 1815, status = succeeded6229worker1 # [ 168.852237] nix-daemon[898]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6230worker1 # [ 168.871022] nix-daemon[898]: reaped child process 1833, status = succeeded6231lb # [ 168.871677] niks3-server[1180]: 2026/09/20 13:35:22 INFO Received uploads request method=POST path=/api/pending_closures6232worker1 # [ 168.895851] nix-grpc-daemon[1790]: time=2026-09-20T13:35:22.299Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6233worker1 # [ 168.898083] nix-grpc-daemon[1790]: time=2026-09-20T13:35:22.299Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6234lb # [ 168.909210] niks3-server[1180]: 2026/09/20 13:35:22 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6235lb # [ 168.922316] niks3-server[1180]: 2026/09/20 13:35:22 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6236worker1 # [ 168.947711] nix-grpc-daemon[1790]: time=2026-09-20T13:35:22.353Z level=INFO msg="Uploading 1 narinfos"6237lb # [ 168.935521] niks3-server[1180]: 2026/09/20 13:35:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6238lb # [ 168.946540] niks3-server[1180]: 2026/09/20 13:35:22 INFO Signed narinfos id=47 count=16239lb # [ 168.952991] niks3-server[1180]: 2026/09/20 13:35:22 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6240worker1 # [ 168.973036] nix-grpc-daemon[1790]: time=2026-09-20T13:35:22.379Z level=INFO msg="Upload complete. (294ms)"6241lb # [ 168.961723] niks3-server[1180]: 2026/09/20 13:35:22 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6242lb # [ 168.967356] niks3-server[1180]: 2026/09/20 13:35:22 INFO Completed upload id=476243lb # [ 168.970443] niks3-server[1180]: 2026/09/20 13:35:22 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6244worker2 # [ 168.981788] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:22Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16245worker1 # [ 168.984114] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=16246worker2 # [ 168.994714] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:22Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06247worker1 # [ 169.000316] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966248worker1 # [ 169.004666] nix-daemon[898]: reaped child process 1813, status = succeeded6249worker1 # [ 169.011529] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419966250worker1 # [ 169.022698] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419966251worker1 # [ 169.033530] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6252worker1 # [ 169.225666] nix-daemon[898]: accepted connection from pid 1837, user nix-grpc-daemon (trusted)6253worker1 # [ 169.243375] nix-daemon[898]: reaped child process 1840, status = succeeded6254worker1 # [ 169.421778] nix-daemon[898]: accepted connection from pid 1842, user nix-grpc-daemon (trusted)6255worker1 # [ 169.439259] nix-daemon[898]: reaped child process 1845, status = succeeded6256lb # [ 169.437975] niks3-server[1180]: 2026/09/20 13:35:22 INFO Received uploads request method=POST path=/api/pending_closures6257worker1 # [ 169.463833] nix-grpc-daemon[1790]: time=2026-09-20T13:35:22.869Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6258worker1 # [ 169.465529] nix-grpc-daemon[1790]: time=2026-09-20T13:35:22.869Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6259lb # [ 169.468856] niks3-server[1180]: 2026/09/20 13:35:22 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6260lb # [ 169.483549] niks3-server[1180]: 2026/09/20 13:35:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6261worker1 # [ 169.503522] nix-grpc-daemon[1790]: time=2026-09-20T13:35:22.909Z level=INFO msg="Uploading 1 narinfos"6262lb # [ 169.494911] niks3-server[1180]: 2026/09/20 13:35:22 INFO Signed narinfos id=48 count=16263lb # [ 169.503184] niks3-server[1180]: 2026/09/20 13:35:22 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6264lb # [ 169.510605] niks3-server[1180]: 2026/09/20 13:35:22 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6265lb # [ 169.520305] niks3-server[1180]: 2026/09/20 13:35:22 INFO Completed upload id=486266worker1 # [ 169.533062] nix-grpc-daemon[1790]: time=2026-09-20T13:35:22.939Z level=INFO msg="Upload complete. (486ms)"6267lb # [ 169.522934] niks3-server[1180]: 2026/09/20 13:35:22 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6268worker1 # [ 169.540298] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=0 paths=1 nar_bytes_in=5126269worker1 # [ 169.555357] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966270worker1 # [ 169.565619] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6271worker1 # [ 169.568769] nix-daemon[898]: reaped child process 1835, status = succeeded6272worker1 # [ 169.583706] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6273worker1 # [ 171.031682] nix-daemon[898]: reaped child process 1849, status = succeeded6274worker1 # [ 171.216460] nix-daemon[898]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)6275worker1 # [ 171.234577] nix-daemon[898]: reaped child process 1867, status = succeeded6276lb # [ 171.235982] niks3-server[1180]: 2026/09/20 13:35:24 INFO Received uploads request method=POST path=/api/pending_closures6277worker1 # [ 171.258917] nix-grpc-daemon[1790]: time=2026-09-20T13:35:24.664Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6278worker1 # [ 171.260597] nix-grpc-daemon[1790]: time=2026-09-20T13:35:24.664Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6279lb # [ 171.272527] niks3-server[1180]: 2026/09/20 13:35:24 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6280lb # [ 171.286047] niks3-server[1180]: 2026/09/20 13:35:24 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6281lb # [ 171.292158] niks3-server[1180]: 2026/09/20 13:35:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6282worker1 # [ 171.310051] nix-grpc-daemon[1790]: time=2026-09-20T13:35:24.715Z level=INFO msg="Uploading 1 narinfos"6283lb # [ 171.303575] niks3-server[1180]: 2026/09/20 13:35:24 INFO Signed narinfos id=49 count=16284lb # [ 171.309453] niks3-server[1180]: 2026/09/20 13:35:24 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6285lb # [ 171.316649] niks3-server[1180]: 2026/09/20 13:35:24 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6286lb # [ 171.324097] niks3-server[1180]: 2026/09/20 13:35:24 INFO Completed upload id=496287worker1 # [ 171.336094] nix-grpc-daemon[1790]: time=2026-09-20T13:35:24.742Z level=INFO msg="Upload complete. (340ms)"6288lb # [ 171.326854] niks3-server[1180]: 2026/09/20 13:35:24 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6289worker2 # [ 171.346215] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:24Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16290worker1 # [ 171.349236] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=16291worker2 # [ 171.359539] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:24Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06292worker1 # [ 171.368216] nix-daemon[898]: reaped child process 1847, status = succeeded6293worker1 # [ 171.401264] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966294worker1 # [ 171.417637] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419966295worker1 # [ 171.432601] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419966296worker1 # [ 171.446399] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6297worker1 # [ 171.651657] nix-daemon[898]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6298worker1 # [ 171.670197] nix-daemon[898]: reaped child process 1874, status = succeeded6299worker1 # [ 171.846463] nix-daemon[898]: accepted connection from pid 1876, user nix-grpc-daemon (trusted)6300worker1 # [ 171.863481] nix-daemon[898]: reaped child process 1879, status = succeeded6301lb # [ 171.862771] niks3-server[1180]: 2026/09/20 13:35:25 INFO Received uploads request method=POST path=/api/pending_closures6302worker1 # [ 171.884672] nix-grpc-daemon[1790]: time=2026-09-20T13:35:25.290Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6303worker1 # [ 171.886191] nix-grpc-daemon[1790]: time=2026-09-20T13:35:25.290Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6304lb # [ 171.890568] niks3-server[1180]: 2026/09/20 13:35:25 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6305worker1 # [ 171.916137] nix-grpc-daemon[1790]: time=2026-09-20T13:35:25.322Z level=INFO msg="Uploading 1 narinfos"6306lb # [ 171.908239] niks3-server[1180]: 2026/09/20 13:35:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6307lb # [ 171.918794] niks3-server[1180]: 2026/09/20 13:35:25 INFO Signed narinfos id=50 count=16308lb # [ 171.924920] niks3-server[1180]: 2026/09/20 13:35:25 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6309lb # [ 171.935660] niks3-server[1180]: 2026/09/20 13:35:25 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6310worker1 # [ 171.953898] nix-grpc-daemon[1790]: time=2026-09-20T13:35:25.360Z level=INFO msg="Upload complete. (491ms)"6311lb # [ 171.942005] niks3-server[1180]: 2026/09/20 13:35:25 INFO Completed upload id=506312lb # [ 171.946454] niks3-server[1180]: 2026/09/20 13:35:25 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6313worker1 # [ 171.964385] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=0 paths=1 nar_bytes_in=5126314worker1 # [ 171.975364] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966315worker1 # [ 171.982339] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6316worker1 # [ 171.987066] nix-daemon[898]: reaped child process 1869, status = succeeded6317worker1 # [ 172.000402] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6318worker1 # [ 172.847324] nix-daemon[898]: reaped child process 1883, status = succeeded6319worker1 # [ 173.026109] nix-daemon[898]: accepted connection from pid 1898, user nix-grpc-daemon (trusted)6320worker1 # [ 173.045184] nix-daemon[898]: reaped child process 1901, status = succeeded6321lb # [ 173.044312] niks3-server[1180]: 2026/09/20 13:35:26 INFO Received uploads request method=POST path=/api/pending_closures6322worker1 # [ 173.064321] nix-grpc-daemon[1790]: time=2026-09-20T13:35:26.468Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6323worker1 # [ 173.065880] nix-grpc-daemon[1790]: time=2026-09-20T13:35:26.468Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6324lb # [ 173.062816] niks3-server[1180]: 2026/09/20 13:35:26 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6325lb # [ 173.070509] niks3-server[1180]: 2026/09/20 13:35:26 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6326lb # [ 173.077752] niks3-server[1180]: 2026/09/20 13:35:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6327worker1 # [ 173.093691] nix-grpc-daemon[1790]: time=2026-09-20T13:35:26.499Z level=INFO msg="Uploading 1 narinfos"6328lb # [ 173.083067] niks3-server[1180]: 2026/09/20 13:35:26 INFO Signed narinfos id=51 count=16329lb # [ 173.086380] niks3-server[1180]: 2026/09/20 13:35:26 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6330lb # [ 173.091889] niks3-server[1180]: 2026/09/20 13:35:26 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6331lb # [ 173.097629] niks3-server[1180]: 2026/09/20 13:35:26 INFO Completed upload id=516332worker1 # [ 173.110367] nix-grpc-daemon[1790]: time=2026-09-20T13:35:26.516Z level=INFO msg="Upload complete. (298ms)"6333lb # [ 173.099444] niks3-server[1180]: 2026/09/20 13:35:26 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6334worker1 # [ 173.117511] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=16335worker2 # [ 173.117805] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:26Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16336worker1 # [ 173.128237] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966337worker2 # [ 173.130868] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:26Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06338worker1 # [ 173.134352] nix-daemon[898]: reaped child process 1881, status = succeeded6339worker1 # [ 173.137297] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419966340worker1 # [ 173.143112] nix-grpc-daemon[1790]: time=2026-09-20T13:35:26.549Z level=INFO msg="All 1 paths already cached"6341worker1 # [ 173.148694] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419966342worker1 # [ 173.156093] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6343lb # [ 173.270987] envoy[911]: conn peer=[2001:db8:1::3]:36772 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=986 tx=448 ms=272376344worker1 # [ 173.338112] nix-daemon[898]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6345worker1 # [ 173.357881] nix-daemon[898]: reaped child process 1908, status = succeeded6346worker1 # [ 173.538387] nix-daemon[898]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6347worker1 # [ 173.556673] nix-daemon[898]: reaped child process 1913, status = succeeded6348lb # [ 173.555108] niks3-server[1180]: 2026/09/20 13:35:26 INFO Received uploads request method=POST path=/api/pending_closures6349worker1 # [ 173.578173] nix-grpc-daemon[1790]: time=2026-09-20T13:35:26.984Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6350worker1 # [ 173.579667] nix-grpc-daemon[1790]: time=2026-09-20T13:35:26.984Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6351lb # [ 173.588310] niks3-server[1180]: 2026/09/20 13:35:26 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6352worker1 # [ 173.613612] nix-grpc-daemon[1790]: time=2026-09-20T13:35:27.019Z level=INFO msg="Uploading 1 narinfos"6353lb # [ 173.605453] niks3-server[1180]: 2026/09/20 13:35:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6354lb # [ 173.615090] niks3-server[1180]: 2026/09/20 13:35:26 INFO Signed narinfos id=52 count=16355lb # [ 173.620738] niks3-server[1180]: 2026/09/20 13:35:27 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6356lb # [ 173.628928] niks3-server[1180]: 2026/09/20 13:35:27 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6357lb # [ 173.633820] niks3-server[1180]: 2026/09/20 13:35:27 INFO Completed upload id=526358worker1 # [ 173.646557] nix-grpc-daemon[1790]: time=2026-09-20T13:35:27.052Z level=INFO msg="Upload complete. (482ms)"6359lb # [ 173.636695] niks3-server[1180]: 2026/09/20 13:35:27 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6360worker1 # [ 173.656699] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=0 paths=1 nar_bytes_in=6246361worker1 # [ 173.668002] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966362worker1 # [ 173.674909] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6363worker1 # [ 173.679615] nix-daemon[898]: reaped child process 1903, status = succeeded6364worker1 # [ 173.690595] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6365worker1 # [ 174.916770] nix-daemon[898]: reaped child process 1917, status = succeeded6366worker1 # [ 175.082292] nix-daemon[898]: accepted connection from pid 1932, user nix-grpc-daemon (trusted)6367worker1 # [ 175.101051] nix-daemon[898]: reaped child process 1935, status = succeeded6368lb # [ 175.102006] niks3-server[1180]: 2026/09/20 13:35:28 INFO Received uploads request method=POST path=/api/pending_closures6369worker1 # [ 175.124432] nix-grpc-daemon[1790]: time=2026-09-20T13:35:28.529Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6370worker1 # [ 175.125976] nix-grpc-daemon[1790]: time=2026-09-20T13:35:28.529Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6371lb # [ 175.141138] niks3-server[1180]: 2026/09/20 13:35:28 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6372lb # [ 175.154959] niks3-server[1180]: 2026/09/20 13:35:28 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6373lb # [ 175.164572] niks3-server[1180]: 2026/09/20 13:35:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6374worker1 # [ 175.183418] nix-grpc-daemon[1790]: time=2026-09-20T13:35:28.589Z level=INFO msg="Uploading 1 narinfos"6375lb # [ 175.174251] niks3-server[1180]: 2026/09/20 13:35:28 INFO Signed narinfos id=53 count=16376lb # [ 175.180616] niks3-server[1180]: 2026/09/20 13:35:28 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6377lb # [ 175.190232] niks3-server[1180]: 2026/09/20 13:35:28 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6378worker1 # [ 175.208605] nix-grpc-daemon[1790]: time=2026-09-20T13:35:28.614Z level=INFO msg="Upload complete. (305ms)"6379lb # [ 175.197138] niks3-server[1180]: 2026/09/20 13:35:28 INFO Completed upload id=536380lb # [ 175.199477] niks3-server[1180]: 2026/09/20 13:35:28 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6381worker2 # [ 175.219382] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:28Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16382worker1 # [ 175.221619] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16383worker2 # [ 175.232250] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:28Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06384worker1 # [ 175.235121] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966385worker1 # [ 175.240995] nix-daemon[898]: reaped child process 1915, status = succeeded6386worker1 # [ 175.248146] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419966387worker1 # [ 175.259629] nix-grpc-daemon[1790]: time=2026-09-20T13:35:28.666Z level=INFO msg="All 1 paths already cached"6388worker1 # [ 175.267099] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419966389worker1 # [ 175.277915] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6390worker1 # [ 175.470007] nix-daemon[898]: accepted connection from pid 1939, user nix-grpc-daemon (trusted)6391worker1 # [ 175.489548] nix-daemon[898]: reaped child process 1942, status = succeeded6392worker1 # [ 175.665842] nix-daemon[898]: accepted connection from pid 1944, user nix-grpc-daemon (trusted)6393worker1 # [ 175.683446] nix-daemon[898]: reaped child process 1947, status = succeeded6394lb # [ 175.682051] niks3-server[1180]: 2026/09/20 13:35:29 INFO Received uploads request method=POST path=/api/pending_closures6395worker1 # [ 175.704781] nix-grpc-daemon[1790]: time=2026-09-20T13:35:29.110Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6396worker1 # [ 175.706307] nix-grpc-daemon[1790]: time=2026-09-20T13:35:29.110Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6397lb # [ 175.719066] niks3-server[1180]: 2026/09/20 13:35:29 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6398lb # [ 175.731646] niks3-server[1180]: 2026/09/20 13:35:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6399worker1 # [ 175.751939] nix-grpc-daemon[1790]: time=2026-09-20T13:35:29.158Z level=INFO msg="Uploading 1 narinfos"6400lb # [ 175.739315] niks3-server[1180]: 2026/09/20 13:35:29 INFO Signed narinfos id=54 count=16401lb # [ 175.747286] niks3-server[1180]: 2026/09/20 13:35:29 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6402lb # [ 175.758427] niks3-server[1180]: 2026/09/20 13:35:29 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6403worker1 # [ 175.776691] nix-grpc-daemon[1790]: time=2026-09-20T13:35:29.183Z level=INFO msg="Upload complete. (486ms)"6404lb # [ 175.766228] niks3-server[1180]: 2026/09/20 13:35:29 INFO Completed upload id=546405lb # [ 175.769341] niks3-server[1180]: 2026/09/20 13:35:29 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6406worker1 # [ 175.784158] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=0 paths=1 nar_bytes_in=6246407worker1 # [ 175.798307] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966408worker1 # [ 175.808251] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6409worker1 # [ 175.810910] nix-daemon[898]: reaped child process 1937, status = succeeded6410worker1 # [ 175.824875] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6411worker1 # [ 177.469952] nix-daemon[898]: reaped child process 1951, status = succeeded6412worker1 # [ 177.659895] nix-daemon[898]: accepted connection from pid 1966, user nix-grpc-daemon (trusted)6413worker1 # [ 177.679994] nix-daemon[898]: reaped child process 1969, status = succeeded6414lb # [ 177.682118] niks3-server[1180]: 2026/09/20 13:35:31 INFO Received uploads request method=POST path=/api/pending_closures6415worker1 # [ 177.702948] nix-grpc-daemon[1790]: time=2026-09-20T13:35:31.108Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6416worker1 # [ 177.709662] nix-grpc-daemon[1790]: time=2026-09-20T13:35:31.111Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6417lb # [ 177.731559] niks3-server[1180]: 2026/09/20 13:35:31 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6418lb # [ 177.739424] niks3-server[1180]: 2026/09/20 13:35:31 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6419worker1 # [ 177.758522] nix-grpc-daemon[1790]: time=2026-09-20T13:35:31.164Z level=INFO msg="Uploading 1 narinfos"6420lb # [ 177.749427] niks3-server[1180]: 2026/09/20 13:35:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6421lb # [ 177.758090] niks3-server[1180]: 2026/09/20 13:35:31 INFO Signed narinfos id=55 count=16422lb # [ 177.763358] niks3-server[1180]: 2026/09/20 13:35:31 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6423lb # [ 177.773108] niks3-server[1180]: 2026/09/20 13:35:31 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6424lb # [ 177.844641] niks3-server[1180]: 2026/09/20 13:35:31 INFO Completed upload id=556425lb # [ 177.848491] niks3-server[1180]: 2026/09/20 13:35:31 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6426worker1 # [ 177.857521] nix-grpc-daemon[1790]: time=2026-09-20T13:35:31.263Z level=INFO msg="Upload complete. (421ms)"6427worker2 # [ 177.869759] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:31Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16428worker1 # [ 177.869394] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=2 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16429worker2 # [ 177.883698] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:31Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06430worker1 # [ 177.890057] nix-daemon[898]: reaped child process 1949, status = succeeded6431worker1 # [ 177.921650] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966432worker1 # [ 177.937876] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419966433worker1 # [ 177.954249] nix-grpc-daemon[1790]: time=2026-09-20T13:35:31.360Z level=INFO msg="All 1 paths already cached"6434worker1 # [ 177.963102] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419966435worker1 # [ 177.977524] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6436worker1 # [ 178.184235] nix-daemon[898]: accepted connection from pid 1973, user nix-grpc-daemon (trusted)6437worker1 # [ 178.201393] nix-daemon[898]: reaped child process 1976, status = succeeded6438worker1 # [ 178.377906] nix-daemon[898]: accepted connection from pid 1978, user nix-grpc-daemon (trusted)6439worker1 # [ 178.395105] nix-daemon[898]: reaped child process 1981, status = succeeded6440lb # [ 178.394012] niks3-server[1180]: 2026/09/20 13:35:31 INFO Received uploads request method=POST path=/api/pending_closures6441worker1 # [ 178.416057] nix-grpc-daemon[1790]: time=2026-09-20T13:35:31.822Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6442worker1 # [ 178.417556] nix-grpc-daemon[1790]: time=2026-09-20T13:35:31.822Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6443lb # [ 178.427507] niks3-server[1180]: 2026/09/20 13:35:31 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6444worker1 # [ 178.453220] nix-grpc-daemon[1790]: time=2026-09-20T13:35:31.859Z level=INFO msg="Uploading 1 narinfos"6445lb # [ 178.443276] niks3-server[1180]: 2026/09/20 13:35:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6446lb # [ 178.453934] niks3-server[1180]: 2026/09/20 13:35:31 INFO Signed narinfos id=56 count=16447worker1 # [ 178.474754] nix-grpc-daemon[1790]: time=2026-09-20T13:35:31.881Z level=INFO msg="Upload complete. (479ms)"6448lb # [ 178.462730] niks3-server[1180]: 2026/09/20 13:35:31 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6449lb # [ 178.467923] niks3-server[1180]: 2026/09/20 13:35:31 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6450lb # [ 178.472537] niks3-server[1180]: 2026/09/20 13:35:31 INFO Completed upload id=566451worker1 # [ 178.484266] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=0 paths=1 nar_bytes_in=6246452lb # [ 178.474743] niks3-server[1180]: 2026/09/20 13:35:31 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6453worker1 # [ 178.495300] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966454worker1 # [ 178.502927] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6455worker1 # [ 178.508270] nix-daemon[898]: reaped child process 1971, status = succeeded6456worker1 # [ 178.521629] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6457worker1 # [ 179.967939] nix-daemon[898]: reaped child process 1985, status = succeeded6458worker1 # [ 180.146395] nix-daemon[898]: accepted connection from pid 2000, user nix-grpc-daemon (trusted)6459worker1 # [ 180.165204] nix-daemon[898]: reaped child process 2003, status = succeeded6460lb # [ 180.166328] niks3-server[1180]: 2026/09/20 13:35:33 INFO Received uploads request method=POST path=/api/pending_closures6461worker1 # [ 180.190989] nix-grpc-daemon[1790]: time=2026-09-20T13:35:33.594Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6462worker1 # [ 180.192689] nix-grpc-daemon[1790]: time=2026-09-20T13:35:33.594Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6463lb # [ 180.203948] niks3-server[1180]: 2026/09/20 13:35:33 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6464lb # [ 180.222410] niks3-server[1180]: 2026/09/20 13:35:33 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6465worker1 # [ 180.246446] nix-grpc-daemon[1790]: time=2026-09-20T13:35:33.652Z level=INFO msg="Uploading 1 narinfos"6466lb # [ 180.240327] niks3-server[1180]: 2026/09/20 13:35:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6467lb # [ 180.246551] niks3-server[1180]: 2026/09/20 13:35:33 INFO Signed narinfos id=57 count=16468lb # [ 180.253164] niks3-server[1180]: 2026/09/20 13:35:33 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6469lb # [ 180.261741] niks3-server[1180]: 2026/09/20 13:35:33 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6470worker1 # [ 180.277599] nix-grpc-daemon[1790]: time=2026-09-20T13:35:33.683Z level=INFO msg="Upload complete. (344ms)"6471lb # [ 180.267657] niks3-server[1180]: 2026/09/20 13:35:33 INFO Completed upload id=576472lb # [ 180.269907] niks3-server[1180]: 2026/09/20 13:35:33 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6473worker2 # [ 180.291857] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:33Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16474worker2 # [ 180.301117] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:33Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06475worker1 # [ 180.289126] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16476worker1 # [ 180.308517] nix-daemon[898]: reaped child process 1983, status = succeeded6477worker1 # [ 180.337997] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966478worker1 # [ 180.354441] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419966479worker1 # [ 180.369288] nix-grpc-daemon[1790]: time=2026-09-20T13:35:33.775Z level=INFO msg="All 2 paths already cached"6480worker1 # [ 180.378497] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419966481worker1 # [ 180.392689] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6482worker1 # [ 180.587455] nix-daemon[898]: accepted connection from pid 2007, user nix-grpc-daemon (trusted)6483worker1 # [ 180.608599] nix-daemon[898]: reaped child process 2010, status = succeeded6484worker1 # [ 180.782758] nix-daemon[898]: accepted connection from pid 2012, user nix-grpc-daemon (trusted)6485worker1 # [ 180.803233] nix-daemon[898]: reaped child process 2015, status = succeeded6486lb # [ 180.802821] niks3-server[1180]: 2026/09/20 13:35:34 INFO Received uploads request method=POST path=/api/pending_closures6487worker1 # [ 181.001524] nix-grpc-daemon[1790]: time=2026-09-20T13:35:34.407Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6488worker1 # [ 181.006283] nix-grpc-daemon[1790]: time=2026-09-20T13:35:34.407Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6489lb # [ 181.012514] niks3-server[1180]: 2026/09/20 13:35:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6490worker1 # [ 181.034683] nix-grpc-daemon[1790]: time=2026-09-20T13:35:34.440Z level=INFO msg="Uploading 1 narinfos"6491lb # [ 181.025404] niks3-server[1180]: 2026/09/20 13:35:34 INFO Signed narinfos id=58 count=16492lb # [ 181.033305] niks3-server[1180]: 2026/09/20 13:35:34 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6493lb # [ 181.047230] niks3-server[1180]: 2026/09/20 13:35:34 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6494worker1 # [ 181.065582] nix-grpc-daemon[1790]: time=2026-09-20T13:35:34.471Z level=INFO msg="Upload complete. (657ms)"6495lb # [ 181.053925] niks3-server[1180]: 2026/09/20 13:35:34 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6496lb # [ 181.060220] niks3-server[1180]: 2026/09/20 13:35:34 INFO Completed upload id=586497lb # [ 181.065085] niks3-server[1180]: 2026/09/20 13:35:34 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6498worker1 # [ 181.076373] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=0 paths=1 nar_bytes_in=7526499worker1 # [ 181.095444] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966500worker1 # [ 181.110445] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6501worker1 # [ 181.116164] nix-daemon[898]: reaped child process 2005, status = succeeded6502worker1 # [ 181.136194] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6503worker1 # [ 182.386018] nix-daemon[898]: reaped child process 2019, status = succeeded6504worker1 # [ 182.566581] nix-daemon[898]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)6505worker1 # [ 182.584648] nix-daemon[898]: reaped child process 2037, status = succeeded6506lb # [ 182.589486] niks3-server[1180]: 2026/09/20 13:35:35 INFO Received uploads request method=POST path=/api/pending_closures6507worker1 # [ 182.616525] nix-grpc-daemon[1790]: time=2026-09-20T13:35:36.019Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6508worker1 # [ 182.618586] nix-grpc-daemon[1790]: time=2026-09-20T13:35:36.019Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6509lb # [ 182.635033] niks3-server[1180]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6510lb # [ 182.648825] niks3-server[1180]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6511lb # [ 182.657775] niks3-server[1180]: 2026/09/20 13:35:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6512worker1 # [ 182.679174] nix-grpc-daemon[1790]: time=2026-09-20T13:35:36.085Z level=INFO msg="Uploading 1 narinfos"6513lb # [ 182.672363] niks3-server[1180]: 2026/09/20 13:35:36 INFO Signed narinfos id=59 count=16514lb # [ 182.675562] niks3-server[1180]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6515lb # [ 182.686897] niks3-server[1180]: 2026/09/20 13:35:36 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6516lb # [ 182.694218] niks3-server[1180]: 2026/09/20 13:35:36 INFO Completed upload id=596517worker1 # [ 182.707053] nix-grpc-daemon[1790]: time=2026-09-20T13:35:36.113Z level=INFO msg="Upload complete. (357ms)"6518lb # [ 182.697831] niks3-server[1180]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6519worker2 # [ 182.718065] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:36Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16520worker1 # [ 182.721247] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16521worker1 # [ 182.740466] nix-daemon[898]: reaped child process 2017, status = succeeded6522worker1 # [ 182.773018] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966523worker1 # [ 182.788472] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419966524worker1 # [ 182.803092] nix-grpc-daemon[1790]: time=2026-09-20T13:35:36.209Z level=INFO msg="All 2 paths already cached"6525worker1 # [ 182.812357] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419966526worker1 # [ 182.828785] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6527worker1 # [ 183.029615] nix-daemon[898]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6528worker1 # [ 183.050108] nix-daemon[898]: reaped child process 2044, status = succeeded6529worker1 # [ 183.225026] nix-daemon[898]: accepted connection from pid 2046, user nix-grpc-daemon (trusted)6530worker1 # [ 183.244908] nix-daemon[898]: reaped child process 2049, status = succeeded6531lb # [ 183.246933] niks3-server[1180]: 2026/09/20 13:35:36 INFO Received uploads request method=POST path=/api/pending_closures6532worker1 # [ 183.269170] nix-grpc-daemon[1790]: time=2026-09-20T13:35:36.675Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6533worker1 # [ 183.270717] nix-grpc-daemon[1790]: time=2026-09-20T13:35:36.675Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6534lb # [ 183.276643] niks3-server[1180]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6535lb # [ 183.291351] niks3-server[1180]: 2026/09/20 13:35:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6536worker1 # [ 183.310279] nix-grpc-daemon[1790]: time=2026-09-20T13:35:36.716Z level=INFO msg="Uploading 1 narinfos"6537lb # [ 183.303293] niks3-server[1180]: 2026/09/20 13:35:36 INFO Signed narinfos id=60 count=16538lb # [ 183.309170] niks3-server[1180]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6539lb # [ 183.315697] niks3-server[1180]: 2026/09/20 13:35:36 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6540lb # [ 183.322942] niks3-server[1180]: 2026/09/20 13:35:36 INFO Completed upload id=606541worker1 # [ 183.334986] nix-grpc-daemon[1790]: time=2026-09-20T13:35:36.741Z level=INFO msg="Upload complete. (489ms)"6542lb # [ 183.325748] niks3-server[1180]: 2026/09/20 13:35:36 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6543worker1 # [ 183.344488] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=0 paths=1 nar_bytes_in=7526544worker1 # [ 183.356384] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966545worker1 # [ 183.362973] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6546worker1 # [ 183.367849] nix-daemon[898]: reaped child process 2039, status = succeeded6547worker1 # [ 183.379359] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6548worker1 # [ 184.217603] nix-daemon[898]: reaped child process 2053, status = succeeded6549worker1 # [ 184.392665] nix-daemon[898]: accepted connection from pid 2068, user nix-grpc-daemon (trusted)6550worker1 # [ 184.413257] nix-daemon[898]: reaped child process 2071, status = succeeded6551lb # [ 184.413090] niks3-server[1180]: 2026/09/20 13:35:37 INFO Received uploads request method=POST path=/api/pending_closures6552worker1 # [ 184.436967] nix-grpc-daemon[1790]: time=2026-09-20T13:35:37.842Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6553worker1 # [ 184.438613] nix-grpc-daemon[1790]: time=2026-09-20T13:35:37.842Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6554lb # [ 184.448398] niks3-server[1180]: 2026/09/20 13:35:37 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6555lb # [ 184.461257] niks3-server[1180]: 2026/09/20 13:35:37 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6556worker1 # [ 184.485047] nix-grpc-daemon[1790]: time=2026-09-20T13:35:37.891Z level=INFO msg="Uploading 1 narinfos"6557lb # [ 184.474197] niks3-server[1180]: 2026/09/20 13:35:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6558lb # [ 184.481858] niks3-server[1180]: 2026/09/20 13:35:37 INFO Signed narinfos id=61 count=16559lb # [ 184.485775] niks3-server[1180]: 2026/09/20 13:35:37 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6560lb # [ 184.492270] niks3-server[1180]: 2026/09/20 13:35:37 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6561lb # [ 184.496529] niks3-server[1180]: 2026/09/20 13:35:37 INFO Completed upload id=616562worker1 # [ 184.509164] nix-grpc-daemon[1790]: time=2026-09-20T13:35:37.915Z level=INFO msg="Upload complete. (317ms)"6563lb # [ 184.498807] niks3-server[1180]: 2026/09/20 13:35:37 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6564worker2 # [ 184.517421] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:37Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16565worker1 # [ 184.521211] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16566worker1 # [ 184.543963] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966567worker1 # [ 184.550385] nix-daemon[898]: reaped child process 2051, status = succeeded6568worker1 # [ 184.576631] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419966569worker1 # [ 184.590962] nix-grpc-daemon[1790]: time=2026-09-20T13:35:37.997Z level=INFO msg="All 2 paths already cached"6570worker1 # [ 184.599929] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:419966571worker1 # [ 184.615701] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6572worker1 # [ 184.817613] nix-daemon[898]: accepted connection from pid 2075, user nix-grpc-daemon (trusted)6573worker1 # [ 184.836804] nix-daemon[898]: reaped child process 2078, status = succeeded6574worker1 # [ 185.008102] nix-daemon[898]: accepted connection from pid 2080, user nix-grpc-daemon (trusted)6575worker1 # [ 185.024810] nix-daemon[898]: reaped child process 2083, status = succeeded6576lb # [ 185.025757] niks3-server[1180]: 2026/09/20 13:35:38 INFO Received uploads request method=POST path=/api/pending_closures6577worker1 # [ 185.046841] nix-grpc-daemon[1790]: time=2026-09-20T13:35:38.452Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6578worker1 # [ 185.048512] nix-grpc-daemon[1790]: time=2026-09-20T13:35:38.452Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6579lb # [ 185.054241] niks3-server[1180]: 2026/09/20 13:35:38 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6580worker1 # [ 185.078599] nix-grpc-daemon[1790]: time=2026-09-20T13:35:38.484Z level=INFO msg="Uploading 1 narinfos"6581lb # [ 185.070732] niks3-server[1180]: 2026/09/20 13:35:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6582lb # [ 185.081408] niks3-server[1180]: 2026/09/20 13:35:38 INFO Signed narinfos id=62 count=16583lb # [ 185.085454] niks3-server[1180]: 2026/09/20 13:35:38 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6584worker1 # [ 185.106675] nix-grpc-daemon[1790]: time=2026-09-20T13:35:38.513Z level=INFO msg="Upload complete. (475ms)"6585lb # [ 185.094237] niks3-server[1180]: 2026/09/20 13:35:38 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6586lb # [ 185.099576] niks3-server[1180]: 2026/09/20 13:35:38 INFO Completed upload id=626587worker1 # [ 185.112316] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=0 paths=1 nar_bytes_in=7526588lb # [ 185.103078] niks3-server[1180]: 2026/09/20 13:35:38 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6589worker1 # [ 185.125784] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:419966590worker1 # [ 185.131580] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6591worker1 # [ 185.136153] nix-daemon[898]: reaped child process 2073, status = succeeded6592worker1 # [ 185.149489] nix-daemon[898]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6593worker1 # [ 186.798191] nix-daemon[898]: reaped child process 2087, status = succeeded6594worker1 # [ 186.980098] nix-daemon[898]: accepted connection from pid 2102, user nix-grpc-daemon (trusted)6595worker1 # [ 186.998365] nix-daemon[898]: reaped child process 2105, status = succeeded6596lb # [ 187.002312] niks3-server[1180]: 2026/09/20 13:35:40 INFO Received uploads request method=POST path=/api/pending_closures6597worker1 # [ 187.024507] nix-grpc-daemon[1790]: time=2026-09-20T13:35:40.428Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6598worker1 # [ 187.026174] nix-grpc-daemon[1790]: time=2026-09-20T13:35:40.428Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6599lb # [ 187.037052] niks3-server[1180]: 2026/09/20 13:35:40 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6600worker1 # [ 187.062935] nix-grpc-daemon[1790]: time=2026-09-20T13:35:40.469Z level=INFO msg="Uploading 1 narinfos"6601lb # [ 187.052792] niks3-server[1180]: 2026/09/20 13:35:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6602lb # [ 187.063109] niks3-server[1180]: 2026/09/20 13:35:40 INFO Signed narinfos id=63 count=16603lb # [ 187.068907] niks3-server[1180]: 2026/09/20 13:35:40 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6604worker1 # [ 187.087288] nix-grpc-daemon[1790]: time=2026-09-20T13:35:40.493Z level=INFO msg="Upload complete. (326ms)"6605lb # [ 187.076652] niks3-server[1180]: 2026/09/20 13:35:40 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6606lb # [ 187.082373] niks3-server[1180]: 2026/09/20 13:35:40 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6607lb # [ 187.086180] niks3-server[1180]: 2026/09/20 13:35:40 INFO Completed upload id=636608worker1 # [ 187.096988] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41996 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16609lb # [ 187.088701] niks3-server[1180]: 2026/09/20 13:35:40 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6610worker1 # [ 187.113215] nix-daemon[898]: reaped child process 2085, status = succeeded6611client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 21.04 seconds)6612worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6613worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6614client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16615worker2 # [ 187.488999] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:517706616worker1 # [ 187.491254] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419966617worker2 # [ 187.518062] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:40Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06618client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 0.31 seconds)6619(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 21.43 seconds)6620subtest: low disk drains a worker and builds go to the other6621worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6622worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.12 seconds)6623worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6624worker1 # [ 188.850921] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:42Z level=info event=unhealthy reason=min_free6625worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.19 seconds)6626client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26627client # don't know how to build these paths:6628client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6629worker1 # [ 190.134419] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:419966630worker1 # [ 190.142678] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:419966631worker2 # [ 190.179285] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:43Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06632worker2 # [ 190.179694] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:43Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16633worker2 # [ 190.222258] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517706634worker2 # [ 190.239966] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517706635client # copying 1 paths to 'lb:50051'...6636client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6637worker2 # [ 190.257248] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517706638worker2 # [ 190.273337] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6639worker2 # [ 190.486665] nix-daemon[913]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6640worker2 # [ 190.505680] nix-daemon[913]: reaped child process 1710, status = succeeded6641worker2 # [ 190.690947] nix-daemon[913]: accepted connection from pid 1712, user nix-grpc-daemon (trusted)6642worker2 # [ 190.708806] nix-daemon[913]: reaped child process 1715, status = succeeded6643lb # [ 190.715452] niks3-server[1180]: 2026/09/20 13:35:44 INFO Received uploads request method=POST path=/api/pending_closures6644worker2 # [ 190.736884] nix-grpc-daemon[1501]: time=2026-09-20T13:35:44.111Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6645worker2 # [ 190.738438] nix-grpc-daemon[1501]: time=2026-09-20T13:35:44.111Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6646lb # [ 190.748528] niks3-server[1180]: 2026/09/20 13:35:44 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6647worker2 # [ 190.775886] nix-grpc-daemon[1501]: time=2026-09-20T13:35:44.150Z level=INFO msg="Uploading 1 narinfos"6648lb # [ 190.764422] niks3-server[1180]: 2026/09/20 13:35:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6649lb # [ 190.773996] niks3-server[1180]: 2026/09/20 13:35:44 INFO Signed narinfos id=64 count=16650lb # [ 190.780084] niks3-server[1180]: 2026/09/20 13:35:44 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6651lb # [ 190.789934] niks3-server[1180]: 2026/09/20 13:35:44 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6652lb # [ 190.795145] niks3-server[1180]: 2026/09/20 13:35:44 INFO Completed upload id=646653worker2 # [ 190.807629] nix-grpc-daemon[1501]: time=2026-09-20T13:35:44.182Z level=INFO msg="Upload complete. (516ms)"6654lb # [ 190.797954] niks3-server[1180]: 2026/09/20 13:35:44 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6655worker2 # [ 190.816358] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51770 duration_s=0 paths=1 nar_bytes_in=5126656worker2 # [ 190.831325] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517706657worker2 # [ 190.838378] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6658worker2 # [ 190.843611] nix-daemon[913]: reaped child process 1705, status = succeeded6659client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6660client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6661worker2 # [ 190.859045] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6662client # farm-a> LOG-farm-a-drain6663worker2 # [ 191.902787] nix-daemon[913]: reaped child process 1719, status = succeeded6664worker2 # [ 192.089526] nix-daemon[913]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6665worker2 # [ 192.108483] nix-daemon[913]: reaped child process 1737, status = succeeded6666lb # [ 192.110968] niks3-server[1180]: 2026/09/20 13:35:45 INFO Received uploads request method=POST path=/api/pending_closures6667worker2 # [ 192.135808] nix-grpc-daemon[1501]: time=2026-09-20T13:35:45.508Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6668worker2 # [ 192.137533] nix-grpc-daemon[1501]: time=2026-09-20T13:35:45.508Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6669lb # [ 192.149314] niks3-server[1180]: 2026/09/20 13:35:45 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6670lb # [ 192.161355] niks3-server[1180]: 2026/09/20 13:35:45 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6671lb # [ 192.171952] niks3-server[1180]: 2026/09/20 13:35:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6672worker2 # [ 192.191899] nix-grpc-daemon[1501]: time=2026-09-20T13:35:45.566Z level=INFO msg="Uploading 1 narinfos"6673lb # [ 192.183003] niks3-server[1180]: 2026/09/20 13:35:45 INFO Signed narinfos id=65 count=16674lb # [ 192.189710] niks3-server[1180]: 2026/09/20 13:35:45 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6675lb # [ 192.200529] niks3-server[1180]: 2026/09/20 13:35:45 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6676lb # [ 192.207642] niks3-server[1180]: 2026/09/20 13:35:45 INFO Completed upload id=656677worker2 # [ 192.220619] nix-grpc-daemon[1501]: time=2026-09-20T13:35:45.595Z level=INFO msg="Upload complete. (350ms)"6678lb # [ 192.210989] niks3-server[1180]: 2026/09/20 13:35:45 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6679worker2 # [ 192.238776] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51770 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=16680worker2 # [ 192.255596] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:45Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06681worker2 # [ 192.259584] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:45Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16682worker2 # [ 192.263582] nix-daemon[913]: reaped child process 1717, status = succeeded6683worker2 # [ 192.285838] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517706684worker2 # [ 192.301369] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517706685worker2 # [ 192.316478] nix-grpc-daemon[1501]: time=2026-09-20T13:35:45.691Z level=INFO msg="All 1 paths already cached"6686client # copying 1 paths to 'lb:50051'...6687client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6688worker2 # [ 192.327376] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517706689worker2 # [ 192.341540] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6690worker2 # [ 192.552168] nix-daemon[913]: accepted connection from pid 1743, user nix-grpc-daemon (trusted)6691worker2 # [ 192.571810] nix-daemon[913]: reaped child process 1746, status = succeeded6692worker2 # [ 192.753276] nix-daemon[913]: accepted connection from pid 1748, user nix-grpc-daemon (trusted)6693worker2 # [ 192.771083] nix-daemon[913]: reaped child process 1751, status = succeeded6694lb # [ 192.770337] niks3-server[1180]: 2026/09/20 13:35:46 INFO Received uploads request method=POST path=/api/pending_closures6695worker2 # [ 192.792061] nix-grpc-daemon[1501]: time=2026-09-20T13:35:46.166Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6696worker2 # [ 192.793568] nix-grpc-daemon[1501]: time=2026-09-20T13:35:46.166Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6697lb # [ 192.801869] niks3-server[1180]: 2026/09/20 13:35:46 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6698lb # [ 192.815282] niks3-server[1180]: 2026/09/20 13:35:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6699worker2 # [ 192.835075] nix-grpc-daemon[1501]: time=2026-09-20T13:35:46.209Z level=INFO msg="Uploading 1 narinfos"6700lb # [ 192.827920] niks3-server[1180]: 2026/09/20 13:35:46 INFO Signed narinfos id=66 count=16701lb # [ 192.834242] niks3-server[1180]: 2026/09/20 13:35:46 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6702worker2 # [ 192.854073] nix-grpc-daemon[1501]: time=2026-09-20T13:35:46.229Z level=INFO msg="Upload complete. (495ms)"6703lb # [ 192.842965] niks3-server[1180]: 2026/09/20 13:35:46 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6704lb # [ 192.848417] niks3-server[1180]: 2026/09/20 13:35:46 INFO Completed upload id=666705worker2 # [ 192.860324] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51770 duration_s=0 paths=1 nar_bytes_in=6326706lb # [ 192.851697] niks3-server[1180]: 2026/09/20 13:35:46 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6707worker2 # [ 192.874589] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517706708worker2 # [ 192.880546] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6709worker2 # [ 192.885159] nix-daemon[913]: reaped child process 1741, status = succeeded6710client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6711worker2 # [ 192.898008] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6712client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6713lb # [ 193.271697] envoy[911]: conn peer=[2001:db8:1::1]:35964 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27161 tx=10736 ms=207336714lb # [ 193.277725] envoy[911]: conn peer=[2001:db8:1::1]:46906 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=880 tx=482 ms=526715client # farm-b> LOG-farm-b-drain6716worker2 # [ 194.325616] nix-daemon[913]: reaped child process 1755, status = succeeded6717worker2 # [ 194.491216] nix-daemon[913]: accepted connection from pid 1770, user nix-grpc-daemon (trusted)6718worker2 # [ 194.510476] nix-daemon[913]: reaped child process 1773, status = succeeded6719lb # [ 194.510638] niks3-server[1180]: 2026/09/20 13:35:47 INFO Received uploads request method=POST path=/api/pending_closures6720worker2 # [ 194.538842] nix-grpc-daemon[1501]: time=2026-09-20T13:35:47.911Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6721worker2 # [ 194.542234] nix-grpc-daemon[1501]: time=2026-09-20T13:35:47.911Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6722lb # [ 194.559046] niks3-server[1180]: 2026/09/20 13:35:47 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6723lb # [ 194.572590] niks3-server[1180]: 2026/09/20 13:35:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6724worker2 # [ 194.589258] nix-grpc-daemon[1501]: time=2026-09-20T13:35:47.964Z level=INFO msg="Uploading 1 narinfos"6725lb # [ 194.579600] niks3-server[1180]: 2026/09/20 13:35:47 INFO Signed narinfos id=67 count=16726lb # [ 194.584351] niks3-server[1180]: 2026/09/20 13:35:47 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6727lb # [ 194.591092] niks3-server[1180]: 2026/09/20 13:35:47 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6728lb # [ 194.599765] niks3-server[1180]: 2026/09/20 13:35:47 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6729lb # [ 194.606001] niks3-server[1180]: 2026/09/20 13:35:47 INFO Completed upload id=676730worker2 # [ 194.616987] nix-grpc-daemon[1501]: time=2026-09-20T13:35:47.991Z level=INFO msg="Upload complete. (308ms)"6731lb # [ 194.608594] niks3-server[1180]: 2026/09/20 13:35:47 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6732worker2 # [ 194.633698] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51770 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=16733worker2 # [ 194.648950] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:48Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06734worker2 # [ 194.653457] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:48Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16735worker2 # [ 194.657945] nix-daemon[913]: reaped child process 1753, status = succeeded6736worker2 # [ 194.680689] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517706737worker2 # [ 194.697840] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517706738worker2 # [ 194.711703] nix-grpc-daemon[1501]: time=2026-09-20T13:35:48.086Z level=INFO msg="All 2 paths already cached"6739client # copying 1 paths to 'lb:50051'...6740client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6741worker2 # [ 194.722379] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517706742worker2 # [ 194.737882] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6743worker2 # [ 194.945910] nix-daemon[913]: accepted connection from pid 1779, user nix-grpc-daemon (trusted)6744worker2 # [ 194.964475] nix-daemon[913]: reaped child process 1782, status = succeeded6745worker2 # [ 195.138482] nix-daemon[913]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)6746worker2 # [ 195.155497] nix-daemon[913]: reaped child process 1787, status = succeeded6747lb # [ 195.153829] niks3-server[1180]: 2026/09/20 13:35:48 INFO Received uploads request method=POST path=/api/pending_closures6748worker2 # [ 195.176061] nix-grpc-daemon[1501]: time=2026-09-20T13:35:48.550Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6749worker2 # [ 195.177554] nix-grpc-daemon[1501]: time=2026-09-20T13:35:48.551Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6750lb # [ 195.183137] niks3-server[1180]: 2026/09/20 13:35:48 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6751lb # [ 195.197187] niks3-server[1180]: 2026/09/20 13:35:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6752worker2 # [ 195.215753] nix-grpc-daemon[1501]: time=2026-09-20T13:35:48.590Z level=INFO msg="Uploading 1 narinfos"6753lb # [ 195.209165] niks3-server[1180]: 2026/09/20 13:35:48 INFO Signed narinfos id=68 count=16754lb # [ 195.216664] niks3-server[1180]: 2026/09/20 13:35:48 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6755worker2 # [ 195.239271] nix-grpc-daemon[1501]: time=2026-09-20T13:35:48.614Z level=INFO msg="Upload complete. (482ms)"6756lb # [ 195.226907] niks3-server[1180]: 2026/09/20 13:35:48 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6757lb # [ 195.232747] niks3-server[1180]: 2026/09/20 13:35:48 INFO Completed upload id=686758lb # [ 195.236329] niks3-server[1180]: 2026/09/20 13:35:48 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6759worker2 # [ 195.248223] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51770 duration_s=0 paths=1 nar_bytes_in=7606760worker2 # [ 195.256576] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517706761worker2 # [ 195.269280] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6762worker2 # [ 195.271966] nix-daemon[913]: reaped child process 1777, status = succeeded6763client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6764client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6765worker2 # [ 195.286149] nix-daemon[913]: accepted connection from pid 1429, user nix-grpc-daemon (trusted)6766client # farm-top> LOG-farm-top-drain6767worker2 # [ 196.318958] nix-daemon[913]: reaped child process 1791, status = succeeded6768worker2 # [ 196.484226] nix-daemon[913]: accepted connection from pid 1806, user nix-grpc-daemon (trusted)6769worker2 # [ 196.502967] nix-daemon[913]: reaped child process 1809, status = succeeded6770lb # [ 196.506417] niks3-server[1180]: 2026/09/20 13:35:49 INFO Received uploads request method=POST path=/api/pending_closures6771worker2 # [ 196.526125] nix-grpc-daemon[1501]: time=2026-09-20T13:35:49.900Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6772worker2 # [ 196.528794] nix-grpc-daemon[1501]: time=2026-09-20T13:35:49.902Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6773lb # [ 196.542976] niks3-server[1180]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6774lb # [ 196.553614] niks3-server[1180]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6775worker2 # [ 196.578813] nix-grpc-daemon[1501]: time=2026-09-20T13:35:49.953Z level=INFO msg="Uploading 1 narinfos"6776lb # [ 196.565932] niks3-server[1180]: 2026/09/20 13:35:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6777lb # [ 196.572509] niks3-server[1180]: 2026/09/20 13:35:49 INFO Signed narinfos id=69 count=16778lb # [ 196.574903] niks3-server[1180]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6779lb # [ 196.585038] niks3-server[1180]: 2026/09/20 13:35:49 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6780lb # [ 196.591525] niks3-server[1180]: 2026/09/20 13:35:49 INFO Completed upload id=696781worker2 # [ 196.604323] nix-grpc-daemon[1501]: time=2026-09-20T13:35:49.979Z level=INFO msg="Upload complete. (307ms)"6782lb # [ 196.594489] niks3-server[1180]: 2026/09/20 13:35:49 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6783worker2 # [ 196.616911] nix-grpc-daemon[1429]: ts=2026-09-20T13:35:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51770 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=16784worker2 # [ 196.631741] nix-daemon[913]: reaped child process 1789, status = succeeded6785client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 6.79 seconds)6786client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6787client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.24 seconds)6788worker1: must succeed: rm /nix/.rw-store/fill6789worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)6790worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6791worker1 # [ 197.862798] nix-grpc-daemon[1734]: ts=2026-09-20T13:35:51Z level=info event=healthy reason=min_free6792worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)6793(finished: subtest: low disk drains a worker and builds go to the other, in 10.72 seconds)6794(finished: run the VM test script, in 199.27 seconds)6795test script finished in 199.56s6796cleanup6797kill QemuMachine (pid 49)6798client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6799kill QemuMachine (pid 50)6800lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6801kill QemuMachine (pid 51)6802worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6803kill QemuMachine (pid 52)6804worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6805(finished: cleanup, in 1.19 seconds)6806additionally exposed symbols:6807 client, lb, worker1, worker2,6808 vlan1,6809 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh