vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #468
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13client # Disk image does not exist, creating the virtualisation disk image...14worker2 # Formatting '/build/vm-state-worker2/tmp.Ij7Ogrs4oQ', fmt=raw size=107374182415client: QEMU running (pid 49)16lb # Disk image does not exist, creating the virtualisation disk image...17worker2: QEMU running (pid 52)18worker2 # mke2fs 1.47.4 (6-Mar-2025)19client # Formatting '/build/vm-state-client/tmp.hyBPmEVUUE', fmt=raw size=107374182420lb # Formatting '/build/vm-state-lb/tmp.TCzldPiA6s', fmt=raw size=107374182421worker1: QEMU running (pid 51)22worker1 # Disk image does not exist, creating the virtualisation disk image...23lb: QEMU running (pid 50)24worker2 # Discarding device blocks: 0/262144 done25worker1 # Formatting '/build/vm-state-worker1/tmp.Niy1cJ03Pc', fmt=raw size=107374182426client # mke2fs 1.47.4 (6-Mar-2025)27lb # mke2fs 1.47.4 (6-Mar-2025)28worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes29client # Discarding device blocks: 0/262144 done30lb # Discarding device blocks: 0/262144 done31worker2 # Filesystem UUID: c505e0dd-6cd1-4a4d-987e-6473a7a5488a32client # Creating filesystem with 262144 4k blocks and 65536 inodes33lb # Creating filesystem with 262144 4k blocks and 65536 inodes34worker2 # Superblock backups stored on blocks:35client # Filesystem UUID: ccba5fde-d28b-41ba-821e-f865ab10dd7436lb # Filesystem UUID: d93edd27-a566-4a7b-b6c6-a3b0f7d759a037worker2 # 32768, 98304, 163840, 22937638client # Superblock backups stored on blocks:39lb # Superblock backups stored on blocks:40worker2 # 41client # 32768, 98304, 163840, 22937642lb # 32768, 98304, 163840, 22937643worker2 # Allocating group tables: 0/8 done44client # 45lb # 46worker2 # Writing inode tables: 0/8 done47client # Allocating group tables: 0/8 done48lb # Allocating group tables: 0/8 done49worker2 # Creating journal (8192 blocks): done50client # Writing inode tables: 0/8 done51lb # Writing inode tables: 0/8 done52worker2 # Writing superblocks and filesystem accounting information: 0/8 done53client # Creating journal (8192 blocks): done54lb # Creating journal (8192 blocks): done55worker2 # 56client # Writing superblocks and filesystem accounting information: 0/8 done57lb # Writing superblocks and filesystem accounting information: 0/8 done58worker2 # Virtualisation disk image created.59client # 60lb # 61worker2 # Starting virtiofs daemons...62client # Virtualisation disk image created.63lb # Virtualisation disk image created.64worker2 # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)65client # Starting virtiofs daemons...66lb # Starting virtiofs daemons...67worker2 # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether68client # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)69lb # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)70worker2 # [2026-09-21T09:52:00Z INFO virtiofsd] Waiting for vhost-user socket connection...71client # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether72lb # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether73worker2 # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)74client # [2026-09-21T09:52:00Z INFO virtiofsd] Waiting for vhost-user socket connection...75lb # [2026-09-21T09:52:00Z INFO virtiofsd] Waiting for vhost-user socket connection...76worker2 # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether77client # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)78lb # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)79worker2 # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)80client # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether81lb # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether82worker2 # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether83client # [2026-09-21T09:52:00Z INFO virtiofsd] Waiting for vhost-user socket connection...84lb # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)85worker2 # [2026-09-21T09:52:00Z INFO virtiofsd] Waiting for vhost-user socket connection...86client # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)87lb # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether88worker2 # [2026-09-21T09:52:00Z INFO virtiofsd] Waiting for vhost-user socket connection...89client # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether90lb # [2026-09-21T09:52:00Z INFO virtiofsd] Waiting for vhost-user socket connection...91worker2 # [2026-09-21T09:52:00Z INFO virtiofsd] Client connected, servicing requests92client # [2026-09-21T09:52:00Z INFO virtiofsd] Waiting for vhost-user socket connection...93lb # [2026-09-21T09:52:00Z INFO virtiofsd] Waiting for vhost-user socket connection...94worker2 # [2026-09-21T09:52:00Z INFO virtiofsd] Client connected, servicing requests95client # [2026-09-21T09:52:00Z INFO virtiofsd] Client connected, servicing requests96lb # [2026-09-21T09:52:00Z INFO virtiofsd] Client connected, servicing requests97worker2 # [2026-09-21T09:52:00Z INFO virtiofsd] Client connected, servicing requests98client # [2026-09-21T09:52:00Z INFO virtiofsd] Client connected, servicing requests99lb # [2026-09-21T09:52:00Z INFO virtiofsd] Client connected, servicing requests100(finished: start all VMs, in 0.51 seconds)101worker1 # mke2fs 1.47.4 (6-Mar-2025)102client # [2026-09-21T09:52:00Z INFO virtiofsd] Client connected, servicing requests103lb # [2026-09-21T09:52:00Z INFO virtiofsd] Client connected, servicing requests104client: waiting for unit network-addresses-eth1.service105worker1 # Discarding device blocks: 0/262144 done106client: waiting for the VM to finish booting107worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes108worker1 # Filesystem UUID: b0f48c2a-a2e1-47a5-9470-a3528a75f8de109worker1 # Superblock backups stored on blocks:110worker1 # 32768, 98304, 163840, 229376111worker1 # 112worker1 # Allocating group tables: 0/8 done113worker1 # Writing inode tables: 0/8 done114worker1 # Creating journal (8192 blocks): done115worker1 # Writing superblocks and filesystem accounting information: 0/8 done116worker1 # 117worker1 # Virtualisation disk image created.118worker1 # Starting virtiofs daemons...119worker1 # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker1 # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121worker1 # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)122worker1 # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether123worker1 # [2026-09-21T09:52:00Z INFO virtiofsd] Waiting for vhost-user socket connection...124worker1 # [2026-09-21T09:52:00Z INFO virtiofsd] Waiting for vhost-user socket connection...125worker1 # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126worker1 # [2026-09-21T09:52:00Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127worker1 # [2026-09-21T09:52:00Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker1 # [2026-09-21T09:52:00Z INFO virtiofsd] Client connected, servicing requests129worker1 # [2026-09-21T09:52:00Z INFO virtiofsd] Client connected, servicing requests130worker1 # [2026-09-21T09:52:00Z INFO virtiofsd] Client connected, servicing requests131client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133client # [ 0.000000] KASLR enabled134client # [ 0.000000] random: crng init done135client # [ 0.000000] Machine model: linux,dummy-virt136client # [ 0.000000] efi: UEFI not found.137client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140client # [ 0.000000] Zone ranges:141client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142client # [ 0.000000] DMA32 empty143client # [ 0.000000] Normal empty144client # [ 0.000000] Device empty145client # [ 0.000000] Movable zone start for each node146client # [ 0.000000] Early memory node ranges147client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150client # [ 0.000000] psci: probing for conduit method from DT.151client # [ 0.000000] psci: PSCIv1.3 detected in firmware.152client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153client # [ 0.000000] psci: Trusted OS migration not required154client # [ 0.000000] psci: SMC Calling Convention v1.1155worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]156client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)157client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296158worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026159client # [ 0.000000] Detected PIPT I-cache on CPU0160worker2 # [ 0.000000] KASLR enabled161worker2 # [ 0.000000] random: crng init done162worker2 # [ 0.000000] Machine model: linux,dummy-virt163client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)164worker2 # [ 0.000000] efi: UEFI not found.165client # [ 0.000000] CPU features: detected: GICv3 CPU interface166client # [ 0.000000] CPU features: detected: Spectre-v4167worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT168client # [ 0.000000] CPU features: detected: Spectre-BHB169worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]170client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38171worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]172worker2 # [ 0.000000] Zone ranges:173client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23174client # [ 0.000000] alternatives: applying boot alternatives175worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]176worker2 # [ 0.000000] DMA32 empty177worker2 # [ 0.000000] Normal empty178worker2 # [ 0.000000] Device empty179worker2 # [ 0.000000] Movable zone start for each node180worker2 # [ 0.000000] Early memory node ranges181worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]182worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]183client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rna2s44q968xsxpa8a3qx3wdj7ybhqp0-nixos-system-client-test/init regInfo=/nix/store/3jvf2nb4n1wk5a5jlywbhn9j8cbi1dwk-closure-info/registration console=ttyAMA0,115200n8 console=tty0184worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000185worker2 # [ 0.000000] psci: probing for conduit method from DT.186worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.187client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/3jvf2nb4n1wk5a5jlywbhn9j8cbi1dwk-closure-info/registration", will be passed to user space.188worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]189worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs190client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes191worker2 # [ 0.000000] psci: Trusted OS migration not required192worker2 # [ 0.000000] psci: SMC Calling Convention v1.1193client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)194worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026195worker1 # [ 0.000000] KASLR enabled196worker1 # [ 0.000000] random: crng init done197client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)198worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)199worker1 # [ 0.000000] Machine model: linux,dummy-virt200client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB201worker1 # [ 0.000000] efi: UEFI not found.202worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296203client # [ 0.000000] software IO TLB: area num 1.204worker2 # [ 0.000000] Detected PIPT I-cache on CPU0205worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT206client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB207worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)208worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]209client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)210worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface211worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]212client # [ 0.000000] Fallback order for Node 0: 0213worker2 # [ 0.000000] CPU features: detected: Spectre-v4214worker1 # [ 0.000000] Zone ranges:215worker2 # [ 0.000000] CPU features: detected: Spectre-BHB216client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216217worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]218worker1 # [ 0.000000] DMA32 empty219client # [ 0.000000] Policy zone: DMA220worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38221worker1 # [ 0.000000] Normal empty222worker1 # [ 0.000000] Device empty223client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off224worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23225worker1 # [ 0.000000] Movable zone start for each node226worker2 # [ 0.000000] alternatives: applying boot alternatives227client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1228worker1 # [ 0.000000] Early memory node ranges229client # [ 0.000000] allocated 3145728 bytes of page_ext230worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]231client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages232worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]233client # [ 0.000000] ftrace: allocated 294 pages with 4 groups234worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000235client # [ 0.000000] rcu: Hierarchical RCU implementation.236worker1 # [ 0.000000] psci: probing for conduit method from DT.237client # [ 0.000000] rcu: RCU event tracing is enabled.238worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.239client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.240worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6ghap5spp9qmqf61q44rs8qmaswirq62-nixos-system-worker2-test/init regInfo=/nix/store/c9v153r0s71yl1wa1wj82wqf2w8m5ibb-closure-info/registration console=ttyAMA0,115200n8 console=tty0241worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs242client # [ 0.000000] Trampoline variant of Tasks RCU enabled.243worker1 # [ 0.000000] psci: Trusted OS migration not required244client # [ 0.000000] Rude variant of Tasks RCU enabled.245worker1 # [ 0.000000] psci: SMC Calling Convention v1.1246worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/c9v153r0s71yl1wa1wj82wqf2w8m5ibb-closure-info/registration", will be passed to user space.247client # [ 0.000000] Tracing variant of Tasks RCU enabled.248worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)249worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes250client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.251worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296252client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1253worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)254worker1 # [ 0.000000] Detected PIPT I-cache on CPU0255worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)256client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.257worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)258worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB259worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface260client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.261worker2 # [ 0.000000] software IO TLB: area num 1.262worker1 # [ 0.000000] CPU features: detected: Spectre-v4263worker1 # [ 0.000000] CPU features: detected: Spectre-BHB264worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB265client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.266worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38267client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0268worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)269client # [ 0.000000] GICv3: 256 SPIs implemented270worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23271worker2 # [ 0.000000] Fallback order for Node 0: 0272client # [ 0.000000] GICv3: 0 Extended SPIs implemented273worker1 # [ 0.000000] alternatives: applying boot alternatives274worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216275client # [ 0.000000] Root IRQ handler: gic_handle_irq276worker2 # [ 0.000000] Policy zone: DMA277client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI278worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off279client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0280worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1281client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000282worker2 # [ 0.000000] allocated 3145728 bytes of page_ext283client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]284worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages285worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/n8pcdav0rzsa2im3lhsnir3k47y9xwrf-nixos-system-worker1-test/init regInfo=/nix/store/kqjy7kdri00320ry3yixvhbiq5x2sdxy-closure-info/registration console=ttyAMA0,115200n8 console=tty0286worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups287client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)288worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.289worker2 # [ 0.000000] rcu: RCU event tracing is enabled.290client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)291worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/kqjy7kdri00320ry3yixvhbiq5x2sdxy-closure-info/registration", will be passed to user space.292worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.293client # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000294worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.295client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000296worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.297worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes298worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.299client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.300worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)301worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.302client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns303worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)304worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1305client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).306worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB307worker1 # [ 0.000000] software IO TLB: area num 1.308worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.309client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns310worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB311worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.312client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns313worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)314client # [ 0.000037] arm-pv: using stolen time PV315worker1 # [ 0.000000] Fallback order for Node 0: 0316worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.317worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0318worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216319client # [ 0.000654] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)320worker1 # [ 0.000000] Policy zone: DMA321worker2 # [ 0.000000] GICv3: 256 SPIs implemented322client # [ 0.000833] Console: colour dummy device 80x25323worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented324worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off325client # [ 0.000841] printk: legacy console [tty0] enabled326worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq327worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1328worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI329worker1 # [ 0.000000] allocated 3145728 bytes of page_ext330client # [ 0.001041] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)331worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0332worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages333client # [ 0.001047] pid_max: default: 32768 minimum: 301334worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000335worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups336client # [ 0.001121] LSM: initializing lsm=capability,landlock,yama,bpf,ima337worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]338worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.339client # [ 0.001278] landlock: Up and running.340lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]341client # [ 0.001280] Yama: becoming mindful.342worker1 # [ 0.000000] rcu: RCU event tracing is enabled.343worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)344client # [ 0.001953] LSM support for eBPF active345worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.346client # [ 0.002115] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)347worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.348lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026349worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)350lb # [ 0.000000] KASLR enabled351worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.352lb # [ 0.000000] random: crng init done353client # [ 0.002148] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)354worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000355worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.356lb # [ 0.000000] Machine model: linux,dummy-virt357client # [ 0.004077] rcu: Hierarchical SRCU implementation.358lb # [ 0.000000] efi: UEFI not found.359worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000360worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.361client # [ 0.004083] rcu: Max phase no-delay instances is 1000.362worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.363lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT364client # [ 0.005324] fsl-mc MSI: its@8080000 domain created365worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1366client # [ 0.005419] EFI services will not be available.367lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]368client # [ 0.005490] smp: Bringing up secondary CPUs ...369worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns370worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.371lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]372client # [ 0.005500] smp: Brought up 1 node, 1 CPU373lb # [ 0.000000] Zone ranges:374worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).375client # [ 0.005502] SMP: Total of 1 processors activated.376worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.377lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]378client # [ 0.005508] CPU: All CPU(s) started at EL1379lb # [ 0.000000] DMA32 empty380worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns381lb # [ 0.000000] Normal empty382client # [ 0.005519] CPU features: detected: Branch Target Identification383worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.384lb # [ 0.000000] Device empty385worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0386lb # [ 0.000000] Movable zone start for each node387client # [ 0.005524] CPU features: detected: ARMv8.4 Translation Table Level388worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns389worker1 # [ 0.000000] GICv3: 256 SPIs implemented390lb # [ 0.000000] Early memory node ranges391worker2 # [ 0.000033] arm-pv: using stolen time PV392worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented393client # [ 0.005527] CPU features: detected: Instruction cache invalidation not required for I/D coherence394lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]395worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq396worker2 # [ 0.000565] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)397worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI398lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]399client # [ 0.005531] CPU features: detected: Data cache clean to the PoU not required for I/D coherence400worker2 # [ 0.000749] Console: colour dummy device 80x25401worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0402lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000403worker2 # [ 0.000756] printk: legacy console [tty0] enabled404client # [ 0.005534] CPU features: detected: Common not Private translations405lb # [ 0.000000] psci: probing for conduit method from DT.406worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000407client # [ 0.005538] CPU features: detected: CRC32 instructions408lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.409worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]410worker2 # [ 0.000944] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)411client # [ 0.005541] CPU features: detected: Data cache clean to Point of Deep Persistence412lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs413worker2 # [ 0.000951] pid_max: default: 32768 minimum: 301414worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)415lb # [ 0.000000] psci: Trusted OS migration not required416client # [ 0.005544] CPU features: detected: Data cache clean to Point of Persistence417worker2 # [ 0.001052] LSM: initializing lsm=capability,landlock,yama,bpf,ima418lb # [ 0.000000] psci: SMC Calling Convention v1.1419worker2 # [ 0.001244] landlock: Up and running.420client # [ 0.005547] CPU features: detected: Data independent timing control (DIT)421worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)422worker2 # [ 0.001247] Yama: becoming mindful.423client # [ 0.005550] CPU features: detected: E0PD424lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)425worker2 # [ 0.001918] LSM support for eBPF active426worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000427client # [ 0.005552] CPU features: detected: Enhanced Counter Virtualization428lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296429worker2 # [ 0.002070] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)430worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000431lb # [ 0.000000] Detected PIPT I-cache on CPU0432client # [ 0.005555] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)433worker2 # [ 0.002108] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)434worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.435client # [ 0.005559] CPU features: detected: Enhanced Virtualization Traps436lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)437worker2 # [ 0.004053] rcu: Hierarchical SRCU implementation.438client # [ 0.005562] CPU features: detected: Fine Grained Traps439lb # [ 0.000000] CPU features: detected: GICv3 CPU interface440worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns441worker2 # [ 0.004058] rcu: Max phase no-delay instances is 1000.442lb # [ 0.000000] CPU features: detected: Spectre-v4443worker2 # [ 0.005356] fsl-mc MSI: its@8080000 domain created444worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).445client # [ 0.005565] CPU features: detected: Generic authentication (architected QARMA5 algorithm)446lb # [ 0.000000] CPU features: detected: Spectre-BHB447worker2 # [ 0.005446] EFI services will not be available.448client # [ 0.005570] CPU features: detected: RCpc load-acquire (LDAPR)449lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38450worker2 # [ 0.005533] smp: Bringing up secondary CPUs ...451worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns452client # [ 0.005573] CPU features: detected: LSE atomic instructions453worker2 # [ 0.005542] smp: Brought up 1 node, 1 CPU454lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23455worker2 # [ 0.005545] SMP: Total of 1 processors activated.456client # [ 0.005576] CPU features: detected: Privileged Access Never457worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns458lb # [ 0.000000] alternatives: applying boot alternatives459worker2 # [ 0.005551] CPU: All CPU(s) started at EL1460client # [ 0.005578] CPU features: detected: PMUv3461worker1 # [ 0.000031] arm-pv: using stolen time PV462client # [ 0.005581] CPU features: detected: RAS Extension Support463worker2 # [ 0.005561] CPU features: detected: Branch Target Identification464worker1 # [ 0.000447] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)465client # [ 0.005584] CPU features: detected: RASv1p1 Extension Support466worker2 # [ 0.005565] CPU features: detected: ARMv8.4 Translation Table Level467worker1 # [ 0.000615] Console: colour dummy device 80x25468client # [ 0.005586] CPU features: detected: Random Number Generator469worker1 # [ 0.000622] printk: legacy console [tty0] enabled470worker2 # [ 0.005568] CPU features: detected: Instruction cache invalidation not required for I/D coherence471client # [ 0.005589] CPU features: detected: Speculation barrier (SB)472worker2 # [ 0.005571] CPU features: detected: Data cache clean to the PoU not required for I/D coherence473lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/c705lh1dgv9xj52a1nrv40gqcrq7spkz-nixos-system-lb-test/init regInfo=/nix/store/73w9grnhn0qslazn8y04bivqvw1y0kh9-closure-info/registration console=ttyAMA0,115200n8 console=tty0474worker1 # [ 0.000810] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)475client # [ 0.005592] CPU features: detected: Stage-2 Force Write-Back476worker1 # [ 0.000816] pid_max: default: 32768 minimum: 301477worker2 # [ 0.005575] CPU features: detected: Common not Private translations478worker2 # [ 0.005578] CPU features: detected: CRC32 instructions479client # [ 0.005595] CPU features: detected: TLB range maintenance instructions480worker1 # [ 0.000888] LSM: initializing lsm=capability,landlock,yama,bpf,ima481lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/73w9grnhn0qslazn8y04bivqvw1y0kh9-closure-info/registration", will be passed to user space.482worker1 # [ 0.001046] landlock: Up and running.483client # [ 0.005600] CPU features: detected: Speculative Store Bypassing Safe (SSBS)484worker2 # [ 0.005581] CPU features: detected: Data cache clean to Point of Deep Persistence485worker1 # [ 0.001049] Yama: becoming mindful.486lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes487worker1 # [ 0.001648] LSM support for eBPF active488client # [ 0.005649] alternatives: applying system-wide alternatives489worker2 # [ 0.005585] CPU features: detected: Data cache clean to Point of Persistence490lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)491worker1 # [ 0.001794] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)492client # [ 0.008577] CPU features: detected: BBM Level 2 without TLB conflict abort493worker2 # [ 0.005588] CPU features: detected: Data independent timing control (DIT)494worker2 # [ 0.005591] CPU features: detected: E0PD495lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)496worker1 # [ 0.001828] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)497worker1 # [ 0.003619] rcu: Hierarchical SRCU implementation.498worker2 # [ 0.005594] CPU features: detected: Enhanced Counter Virtualization499client # [ 0.008785] Memory: 1405664K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)500worker1 # [ 0.003625] rcu: Max phase no-delay instances is 1000.501client # [ 0.009172] devtmpfs: initialized502lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB503worker2 # [ 0.005597] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)504worker1 # [ 0.004868] fsl-mc MSI: its@8080000 domain created505lb # [ 0.000000] software IO TLB: area num 1.506client # [ 0.011015] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)507worker2 # [ 0.005600] CPU features: detected: Enhanced Virtualization Traps508worker1 # [ 0.004959] EFI services will not be available.509lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB510worker2 # [ 0.005603] CPU features: detected: Fine Grained Traps511worker1 # [ 0.005041] smp: Bringing up secondary CPUs ...512client # [ 0.011038] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).513worker1 # [ 0.005048] smp: Brought up 1 node, 1 CPU514lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)515worker2 # [ 0.005607] CPU features: detected: Generic authentication (architected QARMA5 algorithm)516client # [ 0.011240] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL517worker1 # [ 0.005051] SMP: Total of 1 processors activated.518lb # [ 0.000000] Fallback order for Node 0: 0519client # [ 0.011245] 0 pages in range for non-PLT usage520worker1 # [ 0.005057] CPU: All CPU(s) started at EL1521worker2 # [ 0.005611] CPU features: detected: RCpc load-acquire (LDAPR)522lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216523client # [ 0.011246] 508272 pages in range for PLT usage524worker2 # [ 0.005614] CPU features: detected: LSE atomic instructions525worker1 # [ 0.005067] CPU features: detected: Branch Target Identification526lb # [ 0.000000] Policy zone: DMA527client # [ 0.011357] pinctrl core: initialized pinctrl subsystem528worker2 # [ 0.005617] CPU features: detected: Privileged Access Never529worker1 # [ 0.005073] CPU features: detected: ARMv8.4 Translation Table Level530client # [ 0.012154] DMI not present or invalid.531lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off532worker2 # [ 0.005620] CPU features: detected: PMUv3533client # [ 0.015794] NET: Registered PF_NETLINK/PF_ROUTE protocol family534lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1535worker2 # [ 0.005623] CPU features: detected: RAS Extension Support536worker1 # [ 0.005076] CPU features: detected: Instruction cache invalidation not required for I/D coherence537lb # [ 0.000000] allocated 3145728 bytes of page_ext538worker2 # [ 0.005626] CPU features: detected: RASv1p1 Extension Support539client # [ 0.018028] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations540worker1 # [ 0.005080] CPU features: detected: Data cache clean to the PoU not required for I/D coherence541lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages542worker2 # [ 0.005628] CPU features: detected: Random Number Generator543client # [ 0.018223] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations544lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups545worker1 # [ 0.005083] CPU features: detected: Common not Private translations546worker2 # [ 0.005631] CPU features: detected: Speculation barrier (SB)547lb # [ 0.000000] rcu: Hierarchical RCU implementation.548worker1 # [ 0.005087] CPU features: detected: CRC32 instructions549client # [ 0.018420] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations550worker2 # [ 0.005634] CPU features: detected: Stage-2 Force Write-Back551lb # [ 0.000000] rcu: RCU event tracing is enabled.552client # [ 0.018441] audit: initializing netlink subsys (disabled)553worker1 # [ 0.005090] CPU features: detected: Data cache clean to Point of Deep Persistence554worker2 # [ 0.005637] CPU features: detected: TLB range maintenance instructions555lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.556client # [ 0.019075] thermal_sys: Registered thermal governor 'fair_share'557worker1 # [ 0.005093] CPU features: detected: Data cache clean to Point of Persistence558lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.559worker2 # [ 0.005641] CPU features: detected: Speculative Store Bypassing Safe (SSBS)560client # [ 0.019077] thermal_sys: Registered thermal governor 'bang_bang'561lb # [ 0.000000] Rude variant of Tasks RCU enabled.562worker1 # [ 0.005096] CPU features: detected: Data independent timing control (DIT)563worker2 # [ 0.005681] alternatives: applying system-wide alternatives564lb # [ 0.000000] Tracing variant of Tasks RCU enabled.565worker1 # [ 0.005099] CPU features: detected: E0PD566client # [ 0.019081] thermal_sys: Registered thermal governor 'step_wise'567worker2 # [ 0.008653] CPU features: detected: BBM Level 2 without TLB conflict abort568client # [ 0.019084] thermal_sys: Registered thermal governor 'user_space'569lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.570worker1 # [ 0.005102] CPU features: detected: Enhanced Counter Virtualization571client # [ 0.019090] thermal_sys: Registered thermal governor 'power_allocator'572lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1573worker1 # [ 0.005105] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)574worker2 # [ 0.008884] Memory: 1405660K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)575worker1 # [ 0.005109] CPU features: detected: Enhanced Virtualization Traps576client # [ 0.019126] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1577worker2 # [ 0.009267] devtmpfs: initialized578lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.579client # [ 0.019134] cpuidle: using governor ladder580worker1 # [ 0.005112] CPU features: detected: Fine Grained Traps581worker2 # [ 0.011119] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)582client # [ 0.019140] cpuidle: using governor menu583lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.584worker1 # [ 0.005115] CPU features: detected: Generic authentication (architected QARMA5 algorithm)585client # [ 0.019326] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.586worker2 # [ 0.011140] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).587worker1 # [ 0.005120] CPU features: detected: RCpc load-acquire (LDAPR)588lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.589client # [ 0.019343] ASID allocator initialised with 65536 entries590worker2 # [ 0.011333] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL591worker1 # [ 0.005123] CPU features: detected: LSE atomic instructions592client # [ 0.020578] Serial: AMBA PL011 UART driver593lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0594lb # [ 0.000000] GICv3: 256 SPIs implemented595worker1 # [ 0.005126] CPU features: detected: Privileged Access Never596client # [ 0.025823] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1597worker2 # [ 0.011337] 0 pages in range for non-PLT usage598worker1 # [ 0.005128] CPU features: detected: PMUv3599lb # [ 0.000000] GICv3: 0 Extended SPIs implemented600client # [ 0.025984] printk: console [ttyAMA0] enabled601worker2 # [ 0.011338] 508272 pages in range for PLT usage602lb # [ 0.000000] Root IRQ handler: gic_handle_irq603worker1 # [ 0.005131] CPU features: detected: RAS Extension Support604worker2 # [ 0.011463] pinctrl core: initialized pinctrl subsystem605lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI606worker1 # [ 0.005134] CPU features: detected: RASv1p1 Extension Support607worker2 # [ 0.012256] DMI not present or invalid.608lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0609worker1 # [ 0.005136] CPU features: detected: Random Number Generator610worker2 # [ 0.015551] NET: Registered PF_NETLINK/PF_ROUTE protocol family611lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000612worker1 # [ 0.005139] CPU features: detected: Speculation barrier (SB)613lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]614worker2 # [ 0.017732] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations615worker1 # [ 0.005142] CPU features: detected: Stage-2 Force Write-Back616worker2 # [ 0.017906] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations617lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)618worker1 # [ 0.005145] CPU features: detected: TLB range maintenance instructions619worker2 # [ 0.018105] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations620worker1 # [ 0.005150] CPU features: detected: Speculative Store Bypassing Safe (SSBS)621lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)622worker2 # [ 0.018124] audit: initializing netlink subsys (disabled)623worker1 # [ 0.005187] alternatives: applying system-wide alternatives624lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000625worker2 # [ 0.018864] thermal_sys: Registered thermal governor 'fair_share'626worker1 # [ 0.008144] CPU features: detected: BBM Level 2 without TLB conflict abort627lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000628worker2 # [ 0.018866] thermal_sys: Registered thermal governor 'bang_bang'629lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.630worker2 # [ 0.018869] thermal_sys: Registered thermal governor 'step_wise'631client # [ 0.144183] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages632worker1 # [ 0.008355] Memory: 1405608K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)633worker2 # [ 0.018872] thermal_sys: Registered thermal governor 'user_space'634worker1 # [ 0.008728] devtmpfs: initialized635client # [ 0.144200] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page636lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns637worker2 # [ 0.018877] thermal_sys: Registered thermal governor 'power_allocator'638worker1 # [ 0.010578] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)639client # [ 0.144205] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages640lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).641worker2 # [ 0.018903] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1642client # [ 0.144209] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page643worker1 # [ 0.010600] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).644worker2 # [ 0.018911] cpuidle: using governor ladder645lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns646client # [ 0.144214] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages647worker2 # [ 0.018916] cpuidle: using governor menu648worker1 # [ 0.010775] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL649worker1 # [ 0.010780] 0 pages in range for non-PLT usage650lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns651client # [ 0.144218] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page652worker2 # [ 0.019104] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.653lb # [ 0.000029] arm-pv: using stolen time PV654worker1 # [ 0.010781] 508272 pages in range for PLT usage655worker2 # [ 0.019118] ASID allocator initialised with 65536 entries656client # [ 0.144222] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages657worker1 # [ 0.010894] pinctrl core: initialized pinctrl subsystem658worker2 # [ 0.020427] Serial: AMBA PL011 UART driver659lb # [ 0.000407] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)660client # [ 0.144226] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page661lb # [ 0.000576] Console: colour dummy device 80x25662worker2 # [ 0.025871] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1663lb # [ 0.000583] printk: legacy console [tty0] enabled664worker2 # [ 0.026002] printk: console [ttyAMA0] enabled665client # [ 0.151498] fbcon: Taking over console666lb # [ 0.000770] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)667client # [ 0.151512] ACPI: Interpreter disabled.668lb # [ 0.000776] pid_max: default: 32768 minimum: 301669lb # [ 0.000855] LSM: initializing lsm=capability,landlock,yama,bpf,ima670lb # [ 0.000988] landlock: Up and running.671lb # [ 0.000990] Yama: becoming mindful.672lb # [ 0.001446] LSM support for eBPF active673lb # [ 0.001575] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)674lb # [ 0.001607] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)675lb # [ 0.003415] rcu: Hierarchical SRCU implementation.676worker1 # [ 0.011652] DMI not present or invalid.677lb # [ 0.003420] rcu: Max phase no-delay instances is 1000.678worker2 # [ 0.143195] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages679lb # [ 0.004698] fsl-mc MSI: its@8080000 domain created680worker1 # [ 0.014999] NET: Registered PF_NETLINK/PF_ROUTE protocol family681worker2 # [ 0.143211] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page682lb # [ 0.004793] EFI services will not be available.683worker1 # [ 0.017187] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations684worker2 # [ 0.143216] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages685lb # [ 0.004872] smp: Bringing up secondary CPUs ...686lb # [ 0.004880] smp: Brought up 1 node, 1 CPU687worker2 # [ 0.143220] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page688worker1 # [ 0.017364] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations689lb # [ 0.004883] SMP: Total of 1 processors activated.690worker2 # [ 0.143225] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages691lb # [ 0.004888] CPU: All CPU(s) started at EL1692client # [ 0.160501] iommu: Default domain type: Translated693worker1 # [ 0.017564] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations694client # [ 0.160511] iommu: DMA domain TLB invalidation policy: strict mode695lb # [ 0.004897] CPU features: detected: Branch Target Identification696worker1 # [ 0.017586] audit: initializing netlink subsys (disabled)697worker2 # [ 0.143229] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page698client # [ 0.160872] SCSI subsystem initialized699lb # [ 0.004902] CPU features: detected: ARMv8.4 Translation Table Level700worker2 # [ 0.143233] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages701worker1 # [ 0.018118] thermal_sys: Registered thermal governor 'fair_share'702worker2 # [ 0.143238] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page703worker1 # [ 0.018120] thermal_sys: Registered thermal governor 'bang_bang'704client # [ 0.162803] usbcore: registered new interface driver usbfs705lb # [ 0.004905] CPU features: detected: Instruction cache invalidation not required for I/D coherence706worker1 # [ 0.018124] thermal_sys: Registered thermal governor 'step_wise'707client # [ 0.162834] usbcore: registered new interface driver hub708lb # [ 0.004909] CPU features: detected: Data cache clean to the PoU not required for I/D coherence709worker1 # [ 0.018126] thermal_sys: Registered thermal governor 'user_space'710client # [ 0.162849] usbcore: registered new device driver usb711client # [ 0.163128] pps_core: LinuxPPS API ver. 1 registered712lb # [ 0.004913] CPU features: detected: Common not Private translations713worker2 # [ 0.150498] fbcon: Taking over console714worker1 # [ 0.018132] thermal_sys: Registered thermal governor 'power_allocator'715lb # [ 0.004916] CPU features: detected: CRC32 instructions716worker2 # [ 0.150513] ACPI: Interpreter disabled.717worker1 # [ 0.018158] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1718client # [ 0.163134] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>719lb # [ 0.004919] CPU features: detected: Data cache clean to Point of Deep Persistence720client # [ 0.163143] PTP clock support registered721worker1 # [ 0.018166] cpuidle: using governor ladder722client # [ 0.163190] EDAC MC: Ver: 3.0.0723worker1 # [ 0.018172] cpuidle: using governor menu724lb # [ 0.004922] CPU features: detected: Data cache clean to Point of Persistence725client # [ 0.167666] scmi_core: SCMI protocol bus registered726worker1 # [ 0.018370] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.727lb # [ 0.004925] CPU features: detected: Data independent timing control (DIT)728worker1 # [ 0.018385] ASID allocator initialised with 65536 entries729lb # [ 0.004928] CPU features: detected: E0PD730client # [ 0.168621] FPGA manager framework731worker1 # [ 0.019639] Serial: AMBA PL011 UART driver732client # [ 0.169543] vgaarb: loaded733lb # [ 0.004931] CPU features: detected: Enhanced Counter Virtualization734worker1 # [ 0.024948] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1735lb # [ 0.004934] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)736client # [ 0.170178] clocksource: Switched to clocksource arch_sys_counter737worker1 # [ 0.025069] printk: console [ttyAMA0] enabled738lb # [ 0.004938] CPU features: detected: Enhanced Virtualization Traps739lb # [ 0.004941] CPU features: detected: Fine Grained Traps740worker1 # [ 0.143622] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages741worker1 # [ 0.143639] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page742lb # [ 0.004944] CPU features: detected: Generic authentication (architected QARMA5 algorithm)743worker1 # [ 0.143644] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages744lb # [ 0.004949] CPU features: detected: RCpc load-acquire (LDAPR)745client # [ 0.173654] VFS: Disk quotas dquot_6.6.0746worker2 # [ 0.159516] iommu: Default domain type: Translated747worker1 # [ 0.143649] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page748lb # [ 0.004952] CPU features: detected: LSE atomic instructions749client # [ 0.173689] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)750lb # [ 0.004954] CPU features: detected: Privileged Access Never751worker2 # [ 0.159526] iommu: DMA domain TLB invalidation policy: strict mode752worker1 # [ 0.143653] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages753worker2 # [ 0.159878] SCSI subsystem initialized754lb # [ 0.004957] CPU features: detected: PMUv3755worker1 # [ 0.143658] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page756lb # [ 0.004960] CPU features: detected: RAS Extension Support757worker1 # [ 0.143662] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages758lb # [ 0.004963] CPU features: detected: RASv1p1 Extension Support759worker2 # [ 0.161803] usbcore: registered new interface driver usbfs760lb # [ 0.004965] CPU features: detected: Random Number Generator761worker1 # [ 0.143666] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page762client # [ 0.177269] netfs: FS-Cache loaded763worker2 # [ 0.161835] usbcore: registered new interface driver hub764client # [ 0.177403] pnp: PnP ACPI: disabled765worker2 # [ 0.161851] usbcore: registered new device driver usb766lb # [ 0.004968] CPU features: detected: Speculation barrier (SB)767worker2 # [ 0.162152] pps_core: LinuxPPS API ver. 1 registered768lb # [ 0.004971] CPU features: detected: Stage-2 Force Write-Back769lb # [ 0.004973] CPU features: detected: TLB range maintenance instructions770worker1 # [ 0.151001] fbcon: Taking over console771worker2 # [ 0.162158] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>772worker1 # [ 0.151016] ACPI: Interpreter disabled.773worker2 # [ 0.162168] PTP clock support registered774lb # [ 0.004978] CPU features: detected: Speculative Store Bypassing Safe (SSBS)775worker2 # [ 0.162230] EDAC MC: Ver: 3.0.0776worker1 # [ 0.152890] iommu: Default domain type: Translated777lb # [ 0.005015] alternatives: applying system-wide alternatives778worker2 # [ 0.166811] scmi_core: SCMI protocol bus registered779worker1 # [ 0.152900] iommu: DMA domain TLB invalidation policy: strict mode780lb # [ 0.007895] CPU features: detected: BBM Level 2 without TLB conflict abort781client # [ 0.181324] NET: Registered PF_INET protocol family782worker2 # [ 0.167766] FPGA manager framework783worker1 # [ 0.154627] SCSI subsystem initialized784worker2 # [ 0.168729] vgaarb: loaded785client # [ 0.181590] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)786lb # [ 0.008092] Memory: 1405696K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)787worker2 # [ 0.169341] clocksource: Switched to clocksource arch_sys_counter788lb # [ 0.008470] devtmpfs: initialized789lb # [ 0.010257] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)790lb # [ 0.010279] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).791lb # [ 0.010485] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL792lb # [ 0.010489] 0 pages in range for non-PLT usage793lb # [ 0.010490] 508272 pages in range for PLT usage794lb # [ 0.010594] pinctrl core: initialized pinctrl subsystem795worker1 # [ 0.159974] usbcore: registered new interface driver usbfs796lb # [ 0.011328] DMI not present or invalid.797worker1 # [ 0.160016] usbcore: registered new interface driver hub798lb # [ 0.014512] NET: Registered PF_NETLINK/PF_ROUTE protocol family799worker1 # [ 0.160033] usbcore: registered new device driver usb800worker2 # [ 0.175656] VFS: Disk quotas dquot_6.6.0801lb # [ 0.016700] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations802worker1 # [ 0.160301] pps_core: LinuxPPS API ver. 1 registered803worker2 # [ 0.175704] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)804worker1 # [ 0.160307] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>805worker2 # [ 0.177327] netfs: FS-Cache loaded806worker1 # [ 0.160317] PTP clock support registered807worker2 # [ 0.177883] pnp: PnP ACPI: disabled808worker1 # [ 0.160365] EDAC MC: Ver: 3.0.0809worker1 # [ 0.165047] scmi_core: SCMI protocol bus registered810worker1 # [ 0.166033] FPGA manager framework811worker1 # [ 0.166969] vgaarb: loaded812worker2 # [ 0.181304] NET: Registered PF_INET protocol family813worker1 # [ 0.167606] clocksource: Switched to clocksource arch_sys_counter814lb # [ 0.016877] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations815worker1 # [ 0.171517] VFS: Disk quotas dquot_6.6.0816worker2 # [ 0.182181] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)817lb # [ 0.017069] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations818worker1 # [ 0.171545] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)819lb # [ 0.017090] audit: initializing netlink subsys (disabled)820lb # [ 0.017603] thermal_sys: Registered thermal governor 'fair_share'821lb # [ 0.017605] thermal_sys: Registered thermal governor 'bang_bang'822lb # [ 0.017608] thermal_sys: Registered thermal governor 'step_wise'823worker1 # [ 0.175232] netfs: FS-Cache loaded824worker1 # [ 0.175365] pnp: PnP ACPI: disabled825lb # [ 0.017611] thermal_sys: Registered thermal governor 'user_space'826lb # [ 0.017616] thermal_sys: Registered thermal governor 'power_allocator'827lb # [ 0.017645] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1828lb # [ 0.017656] cpuidle: using governor ladder829lb # [ 0.017661] cpuidle: using governor menu830lb # [ 0.017844] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.831worker1 # [ 0.179391] NET: Registered PF_INET protocol family832lb # [ 0.017859] ASID allocator initialised with 65536 entries833lb # [ 0.019045] Serial: AMBA PL011 UART driver834lb # [ 0.024265] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1835lb # [ 0.024412] printk: console [ttyAMA0] enabled836lb # [ 0.144796] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages837worker1 # [ 0.180294] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)838lb # [ 0.144817] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page839lb # [ 0.144823] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages840lb # [ 0.144828] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page841lb # [ 0.144832] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages842lb # [ 0.144837] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page843lb # [ 0.144841] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages844lb # [ 0.144845] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page845client # [ 0.211927] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)846lb # [ 0.152405] fbcon: Taking over console847client # [ 0.211982] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)848lb # [ 0.152420] ACPI: Interpreter disabled.849lb # [ 0.154311] iommu: Default domain type: Translated850client # [ 0.212013] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)851lb # [ 0.154321] iommu: DMA domain TLB invalidation policy: strict mode852client # [ 0.212090] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)853lb # [ 0.156090] SCSI subsystem initialized854client # [ 0.212234] TCP: Hash tables configured (established 16384 bind 16384)855client # [ 0.212320] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)856client # [ 0.212357] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)857client # [ 0.212392] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)858client # [ 0.212493] NET: Registered PF_UNIX/PF_LOCAL protocol family859client # [ 0.212514] NET: Registered PF_XDP protocol family860client # [ 0.212533] PCI: CLS 0 bytes, default 64861client # [ 0.212786] Trying to unpack rootfs image as initramfs...862lb # [ 0.163191] usbcore: registered new interface driver usbfs863lb # [ 0.163223] usbcore: registered new interface driver hub864lb # [ 0.163239] usbcore: registered new device driver usb865lb # [ 0.163529] pps_core: LinuxPPS API ver. 1 registered866worker2 # [ 0.210873] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)867lb # [ 0.163538] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>868worker2 # [ 0.210928] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)869lb # [ 0.163559] PTP clock support registered870lb # [ 0.163609] EDAC MC: Ver: 3.0.0871worker2 # [ 0.210959] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)872client # [ 0.227860] kvm [1]: HYP mode not available873lb # [ 0.168274] scmi_core: SCMI protocol bus registered874worker2 # [ 0.211035] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)875lb # [ 0.169255] FPGA manager framework876worker2 # [ 0.211178] TCP: Hash tables configured (established 16384 bind 16384)877lb # [ 0.170220] vgaarb: loaded878worker2 # [ 0.211261] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)879lb # [ 0.170848] clocksource: Switched to clocksource arch_sys_counter880worker2 # [ 0.211297] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)881worker2 # [ 0.211332] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)882worker2 # [ 0.211448] NET: Registered PF_UNIX/PF_LOCAL protocol family883lb # [ 0.172881] VFS: Disk quotas dquot_6.6.0884worker2 # [ 0.211499] NET: Registered PF_XDP protocol family885lb # [ 0.172910] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)886worker2 # [ 0.211517] PCI: CLS 0 bytes, default 64887worker2 # [ 0.211772] Trying to unpack rootfs image as initramfs...888lb # [ 0.176612] netfs: FS-Cache loaded889lb # [ 0.176736] pnp: PnP ACPI: disabled890worker1 # [ 0.209346] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)891worker1 # [ 0.209391] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)892worker1 # [ 0.209419] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)893worker1 # [ 0.209490] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)894worker1 # [ 0.209632] TCP: Hash tables configured (established 16384 bind 16384)895worker2 # [ 0.227131] kvm [1]: HYP mode not available896worker1 # [ 0.209718] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)897lb # [ 0.180582] NET: Registered PF_INET protocol family898worker1 # [ 0.209754] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)899lb # [ 0.180835] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)900worker1 # [ 0.209810] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)901worker1 # [ 0.209902] NET: Registered PF_UNIX/PF_LOCAL protocol family902worker1 # [ 0.209923] NET: Registered PF_XDP protocol family903worker1 # [ 0.209941] PCI: CLS 0 bytes, default 64904worker1 # [ 0.210193] Trying to unpack rootfs image as initramfs...905worker1 # [ 0.225398] kvm [1]: HYP mode not available906lb # [ 0.210611] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)907lb # [ 0.210666] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)908lb # [ 0.210695] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)909lb # [ 0.210767] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)910lb # [ 0.214797] TCP: Hash tables configured (established 16384 bind 16384)911lb # [ 0.215685] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)912lb # [ 0.215728] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)913lb # [ 0.215766] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)914lb # [ 0.215868] NET: Registered PF_UNIX/PF_LOCAL protocol family915lb # [ 0.215913] NET: Registered PF_XDP protocol family916lb # [ 0.215933] PCI: CLS 0 bytes, default 64917lb # [ 0.216183] Trying to unpack rootfs image as initramfs...918lb # [ 0.234886] kvm [1]: HYP mode not available919worker2 # [ 0.321933] Initialise system trusted keyrings920worker2 # [ 0.322793] workingset: timestamp_bits=42 max_order=19 bucket_order=0921worker2 # [ 0.324523] squashfs: version 4.0 (2009/01/31) Phillip Lougher922client # [ 0.338847] Initialise system trusted keyrings923worker2 # [ 0.325290] 9p: Installing v9fs 9p2000 file system support924client # [ 0.339625] workingset: timestamp_bits=42 max_order=19 bucket_order=0925client # [ 0.340956] squashfs: version 4.0 (2009/01/31) Phillip Lougher926client # [ 0.341743] 9p: Installing v9fs 9p2000 file system support927worker1 # [ 0.321367] Initialise system trusted keyrings928worker1 # [ 0.322131] workingset: timestamp_bits=42 max_order=19 bucket_order=0929worker1 # [ 0.323376] squashfs: version 4.0 (2009/01/31) Phillip Lougher930worker1 # [ 0.324195] 9p: Installing v9fs 9p2000 file system support931worker2 # [ 0.353988] Key type asymmetric registered932worker2 # [ 0.354020] Asymmetric key parser 'x509' registered933worker2 # [ 0.354110] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)934worker2 # [ 0.356247] io scheduler mq-deadline registered935client # [ 0.370530] Key type asymmetric registered936worker2 # [ 0.356259] io scheduler kyber registered937client # [ 0.370556] Asymmetric key parser 'x509' registered938client # [ 0.370641] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)939worker1 # [ 0.344209] Key type asymmetric registered940worker1 # [ 0.344232] Asymmetric key parser 'x509' registered941client # [ 0.372764] io scheduler mq-deadline registered942client # [ 0.372776] io scheduler kyber registered943worker1 # [ 0.344303] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)944worker1 # [ 0.351652] io scheduler mq-deadline registered945worker1 # [ 0.351671] io scheduler kyber registered946client # [ 0.382328] pl061_gpio 9030000.pl061: PL061 GPIO chip registered947client # [ 0.383758] ledtrig-cpu: registered to indicate activity on CPUs948client # [ 0.384160] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:949client # [ 0.384178] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000950client # [ 0.384191] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000951client # [ 0.384199] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000952client # [ 0.384233] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits953worker2 # [ 0.369511] pl061_gpio 9030000.pl061: PL061 GPIO chip registered954worker2 # [ 0.370183] ledtrig-cpu: registered to indicate activity on CPUs955client # [ 0.384259] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]956worker2 # [ 0.370581] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:957lb # [ 0.327401] Initialise system trusted keyrings958client # [ 0.384342] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00959client # [ 0.384352] pci_bus 0000:00: root bus resource [bus 00-ff]960worker2 # [ 0.370598] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000961lb # [ 0.328167] workingset: timestamp_bits=42 max_order=19 bucket_order=0962client # [ 0.384359] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]963worker2 # [ 0.370611] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000964lb # [ 0.329379] squashfs: version 4.0 (2009/01/31) Phillip Lougher965client # [ 0.384364] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]966worker2 # [ 0.370619] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000967client # [ 0.384369] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]968lb # [ 0.330129] 9p: Installing v9fs 9p2000 file system support969worker2 # [ 0.370639] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits970client # [ 0.384456] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint971worker2 # [ 0.370677] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]972client # [ 0.384901] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint973client # [ 0.385084] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]974worker2 # [ 0.370752] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00975worker1 # [ 0.363771] pl061_gpio 9030000.pl061: PL061 GPIO chip registered976client # [ 0.385101] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]977worker2 # [ 0.370762] pci_bus 0000:00: root bus resource [bus 00-ff]978worker1 # [ 0.364425] ledtrig-cpu: registered to indicate activity on CPUs979client # [ 0.385130] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]980worker2 # [ 0.370768] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]981worker1 # [ 0.364806] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:982client # [ 0.385146] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]983worker2 # [ 0.370773] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]984worker1 # [ 0.364823] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000985worker2 # [ 0.370778] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]986client # [ 0.385599] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint987client # [ 0.385792] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]988worker1 # [ 0.364836] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000989worker2 # [ 0.370851] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint990client # [ 0.385808] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]991worker1 # [ 0.364845] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000992worker2 # [ 0.371289] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint993client # [ 0.385837] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]994worker2 # [ 0.371486] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]995worker1 # [ 0.364865] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits996worker2 # [ 0.371504] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]997worker1 # [ 0.364892] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]998worker2 # [ 0.371533] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]999client # [ 0.405241] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1000worker1 # [ 0.364967] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001001worker2 # [ 0.371549] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1002client # [ 0.405426] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1003worker1 # [ 0.364977] pci_bus 0000:00: root bus resource [bus 00-ff]1004client # [ 0.405442] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1005worker2 # [ 0.372003] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1006worker1 # [ 0.364983] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1007client # [ 0.405471] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1008worker2 # [ 0.372189] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1009worker1 # [ 0.364988] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1010worker2 # [ 0.372205] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1011client # [ 0.405936] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1012worker1 # [ 0.364992] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1013worker2 # [ 0.372234] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1014client # [ 0.406119] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1015client # [ 0.406134] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1016worker1 # [ 0.365066] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1017worker2 # [ 0.372683] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1018client # [ 0.406163] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1019worker2 # [ 0.372865] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1020worker1 # [ 0.365507] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1021worker2 # [ 0.372881] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1022worker1 # [ 0.365694] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1023worker1 # [ 0.365713] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1024worker2 # [ 0.372910] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025worker1 # [ 0.365745] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1026worker2 # [ 0.373379] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1027worker1 # [ 0.365761] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1028worker2 # [ 0.373562] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1029worker2 # [ 0.373577] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1030worker1 # [ 0.366228] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1031worker1 # [ 0.366412] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1032worker2 # [ 0.373606] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1033worker1 # [ 0.366428] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1034worker2 # [ 0.373625] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1035worker1 # [ 0.366457] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1036worker2 # [ 0.374086] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1037worker2 # [ 0.374273] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1038worker1 # [ 0.366910] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1039worker1 # [ 0.367091] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1040worker2 # [ 0.374303] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1041worker1 # [ 0.367107] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1042worker2 # [ 0.374756] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1043worker1 # [ 0.367137] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1044worker2 # [ 0.374942] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1045worker1 # [ 0.367637] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1046worker2 # [ 0.374971] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1047worker1 # [ 0.367821] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1048worker2 # [ 0.375384] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1049worker1 # [ 0.367837] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1050worker2 # [ 0.375579] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1051worker1 # [ 0.367867] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1052worker1 # [ 0.367885] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1053worker2 # [ 0.375825] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1054lb # [ 0.358977] Key type asymmetric registered1055lb # [ 0.359002] Asymmetric key parser 'x509' registered1056worker2 # [ 0.376011] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1057worker1 # [ 0.368336] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1058worker2 # [ 0.376040] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1059lb # [ 0.359065] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1060worker1 # [ 0.368521] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1061client # [ 0.418236] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1062lb # [ 0.361130] io scheduler mq-deadline registered1063worker1 # [ 0.368550] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1064client # [ 0.418779] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1065worker2 # [ 0.376488] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1066lb # [ 0.361140] io scheduler kyber registered1067client # [ 0.418972] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1068worker2 # [ 0.376674] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1069worker1 # [ 0.369006] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1070client # [ 0.419001] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1071worker2 # [ 0.376704] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1072worker1 # [ 0.369194] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1073client # [ 0.419453] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1074worker1 # [ 0.369223] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1075worker2 # [ 0.377167] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1076client # [ 0.419639] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1077worker1 # [ 0.369607] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1078client # [ 0.419667] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1079worker1 # [ 0.369793] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1080client # [ 0.420053] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1081worker1 # [ 0.370039] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1082client # [ 0.420234] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1083worker1 # [ 0.370224] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1084worker1 # [ 0.370254] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1085client # [ 0.420512] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1086client # [ 0.420700] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1087worker1 # [ 0.370713] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1088client # [ 0.420729] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1089worker1 # [ 0.370898] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1090worker1 # [ 0.370928] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1091client # [ 0.421172] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1092client # [ 0.421356] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1093worker1 # [ 0.371373] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1094lb # [ 0.371004] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1095client # [ 0.421386] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1096worker1 # [ 0.371558] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1097worker1 # [ 0.371587] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1098client # [ 0.421843] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1099worker2 # [ 0.421410] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1100client # [ 0.422028] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1101worker2 # [ 0.421454] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1102client # [ 0.422058] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1103lb # [ 0.372442] ledtrig-cpu: registered to indicate activity on CPUs1104worker2 # [ 0.421975] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1105client # [ 0.422523] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1106lb # [ 0.372836] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1107worker2 # [ 0.422244] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1108client # [ 0.422814] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1109worker2 # [ 0.422261] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1110lb # [ 0.372852] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001111client # [ 0.422831] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1112worker2 # [ 0.422291] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1113client # [ 0.422860] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1114lb # [ 0.372865] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001115worker2 # [ 0.422745] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1116client # [ 0.423318] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1117lb # [ 0.372874] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001118worker2 # [ 0.422927] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1119client # [ 0.423499] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1120lb # [ 0.372894] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1121worker2 # [ 0.422942] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1122client # [ 0.423514] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1123worker2 # [ 0.422972] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1124client # [ 0.423544] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1125lb # [ 0.372921] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1126worker2 # [ 0.423567] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1127client # [ 0.424108] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1128lb # [ 0.372995] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001129worker2 # [ 0.423578] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1130client # [ 0.424120] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1131lb # [ 0.373005] pci_bus 0000:00: root bus resource [bus 00-ff]1132worker2 # [ 0.423584] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1133lb # [ 0.373011] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1134client # [ 0.424125] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1135lb # [ 0.373017] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1136worker2 # [ 0.423629] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1137client # [ 0.424170] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1138lb # [ 0.373022] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1139worker2 # [ 0.423677] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1140client # [ 0.424216] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1141worker1 # [ 0.420221] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1142lb # [ 0.373073] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1143worker2 # [ 0.423724] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1144client # [ 0.424263] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1145worker1 # [ 0.420552] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1146lb # [ 0.373516] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1147worker1 # [ 0.420571] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1148worker2 # [ 0.423772] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1149client # [ 0.424309] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1150lb # [ 0.373700] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1151worker1 # [ 0.420602] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1152worker2 # [ 0.423819] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1153lb # [ 0.373717] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1154client # [ 0.424354] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1155worker1 # [ 0.421128] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1156lb # [ 0.373747] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1157worker2 # [ 0.423867] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1158client # [ 0.424399] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1159worker1 # [ 0.421323] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1160lb # [ 0.373763] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1161worker2 # [ 0.423915] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1162worker1 # [ 0.421339] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1163client # [ 0.424444] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1164lb # [ 0.374225] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1165worker1 # [ 0.421371] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1166worker2 # [ 0.423961] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1167client # [ 0.424490] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1168lb # [ 0.374407] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1169worker1 # [ 0.421999] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1170worker2 # [ 0.424009] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1171lb # [ 0.374423] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1172client # [ 0.424535] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1173worker1 # [ 0.422011] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1174worker2 # [ 0.424077] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1175lb # [ 0.374452] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1176client # [ 0.424605] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1177worker1 # [ 0.422016] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1178worker2 # [ 0.424123] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1179client # [ 0.424650] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1180worker2 # [ 0.424145] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1181worker1 # [ 0.422067] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1182client # [ 0.424672] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1183lb # [ 0.393705] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1184worker2 # [ 0.424166] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1185worker1 # [ 0.422117] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1186client # [ 0.424693] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1187lb # [ 0.393892] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1188worker2 # [ 0.424187] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1189client # [ 0.424714] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1190lb # [ 0.393908] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1191worker1 # [ 0.422169] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1192worker2 # [ 0.424208] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1193client # [ 0.424735] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1194lb # [ 0.393937] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1195worker1 # [ 0.422220] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1196worker2 # [ 0.424230] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1197client # [ 0.424756] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1198lb # [ 0.394402] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1199worker1 # [ 0.422272] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1200worker2 # [ 0.424251] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1201client # [ 0.424778] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1202lb # [ 0.394586] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1203worker2 # [ 0.424277] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1204worker1 # [ 0.422321] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1205client # [ 0.424802] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1206lb # [ 0.394601] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1207worker2 # [ 0.424299] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1208worker1 # [ 0.422372] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1209lb # [ 0.394631] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1210client # [ 0.424824] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1211worker2 # [ 0.424321] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1212lb # [ 0.394649] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1213client # [ 0.424846] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1214worker1 # [ 0.422422] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1215worker2 # [ 0.424343] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1216client # [ 0.424868] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1217worker1 # [ 0.422473] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1218worker2 # [ 0.424365] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1219client # [ 0.424890] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1220worker1 # [ 0.422563] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1221worker2 # [ 0.424387] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1222client # [ 0.424912] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1223worker1 # [ 0.422621] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1224worker2 # [ 0.424407] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1225client # [ 0.424932] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1226worker1 # [ 0.422644] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1227worker2 # [ 0.424428] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1228client # [ 0.424953] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1229worker2 # [ 0.424449] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1230worker1 # [ 0.422668] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1231client # [ 0.424974] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1232worker2 # [ 0.424471] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1233worker1 # [ 0.422692] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1234client # [ 0.424995] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1235worker2 # [ 0.424492] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1236worker1 # [ 0.422714] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1237client # [ 0.425016] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1238worker2 # [ 0.424517] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1239client # [ 0.425041] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1240worker1 # [ 0.422738] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1241worker2 # [ 0.424526] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1242client # [ 0.425050] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1243worker1 # [ 0.422760] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1244worker2 # [ 0.424530] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1245client # [ 0.425055] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1246worker1 # [ 0.422790] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1247worker2 # [ 0.425338] pci 0000:00:07.0: enabling device (0000 -> 0002)1248client # [ 0.425874] pci 0000:00:07.0: enabling device (0000 -> 0002)1249worker1 # [ 0.422815] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1250worker1 # [ 0.422839] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1251worker1 # [ 0.422864] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1252worker1 # [ 0.422887] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1253worker1 # [ 0.422911] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1254worker1 # [ 0.422932] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1255worker1 # [ 0.422956] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1256lb # [ 0.407223] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1257worker1 # [ 0.422977] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1258lb # [ 0.407432] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1259worker2 # [ 0.466037] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 39750 usecs1260worker1 # [ 0.423001] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1261lb # [ 0.407464] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1262worker1 # [ 0.423023] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1263lb # [ 0.407947] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1264worker1 # [ 0.423055] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1265client # [ 0.482334] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 55141 usecs1266lb # [ 0.408134] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1267worker1 # [ 0.423064] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1268worker1 # [ 0.423069] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1269lb # [ 0.408164] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1270lb # [ 0.408593] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1271lb # [ 0.408775] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1272worker1 # [ 0.458432] pci 0000:00:07.0: enabling device (0000 -> 0002)1273lb # [ 0.409026] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1274lb # [ 0.409213] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1275lb # [ 0.409243] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1276lb # [ 0.409710] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1277lb # [ 0.409902] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1278lb # [ 0.409932] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1279lb # [ 0.410384] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1280lb # [ 0.410570] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1281lb # [ 0.410600] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1282lb # [ 0.411071] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1283lb # [ 0.411355] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1284lb # [ 0.411372] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1285lb # [ 0.411402] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1286lb # [ 0.411875] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1287lb # [ 0.412066] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1288lb # [ 0.412082] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1289lb # [ 0.412111] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1290lb # [ 0.412681] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1291lb # [ 0.412692] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1292lb # [ 0.412698] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1293worker2 # [ 0.486858] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1294lb # [ 0.412743] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1295worker2 # [ 0.489056] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1296lb # [ 0.412790] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1297client # [ 0.503239] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1298lb # [ 0.412837] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1299client # [ 0.505364] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1300lb # [ 0.412883] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1301lb # [ 0.412930] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1302lb # [ 0.412976] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1303lb # [ 0.413022] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1304lb # [ 0.413068] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1305worker1 # [ 0.483013] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1306lb # [ 0.413114] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1307lb # [ 0.413183] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1308lb # [ 0.413229] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1309lb # [ 0.413250] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1310lb # [ 0.413272] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1311worker2 # [ 0.499320] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1312lb # [ 0.413293] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1313worker2 # [ 0.501305] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1314lb # [ 0.413315] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1315client # [ 0.516530] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1316lb # [ 0.413336] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1317lb # [ 0.413358] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1318lb # [ 0.413383] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1319lb # [ 0.413406] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1320lb # [ 0.413428] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1321lb # [ 0.413451] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1322lb # [ 0.413473] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1323worker1 # [ 0.492951] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1324lb # [ 0.413494] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1325lb # [ 0.413515] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1326lb # [ 0.413536] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1327lb # [ 0.413558] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1328lb # [ 0.413581] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1329worker2 # [ 0.510978] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1330worker1 # [ 0.497346] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1331lb # [ 0.413602] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1332worker2 # [ 0.512970] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1333lb # [ 0.413628] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1334worker1 # [ 0.499265] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1335lb # [ 0.413637] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1336lb # [ 0.413642] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1337lb # [ 0.414468] pci 0000:00:07.0: enabling device (0000 -> 0002)1338client # [ 0.526395] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1339client # [ 0.530542] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1340client # [ 0.532590] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1341lb # [ 0.470996] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 55206 usecs1342worker2 # [ 0.522741] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1343worker2 # [ 0.524553] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1344worker1 # [ 0.513008] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1345worker2 # [ 0.528012] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1346worker1 # [ 0.515105] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1347client # [ 0.542363] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1348client # [ 0.544682] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1349worker1 # [ 0.518720] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1350worker2 # [ 0.533878] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1351worker2 # [ 0.536214] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1352worker1 # [ 0.521493] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1353lb # [ 0.492213] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1354client # [ 0.554410] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1355lb # [ 0.494706] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1356client # [ 0.556943] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1357worker1 # [ 0.532208] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1358client # [ 0.559311] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1359worker1 # [ 0.534169] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1360worker2 # [ 0.554615] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1361worker2 # [ 0.557295] msm_serial: driver initialized1362lb # [ 0.505594] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1363client # [ 0.572550] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1364worker2 # [ 0.558060] SuperH (H)SCI(F) driver initialized1365worker1 # [ 0.545514] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1366worker2 # [ 0.558119] STM32 USART driver initialized1367client # [ 0.579495] msm_serial: driver initialized1368client # [ 0.579635] SuperH (H)SCI(F) driver initialized1369client # [ 0.579689] STM32 USART driver initialized1370worker1 # [ 0.558568] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1371lb # [ 0.520153] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1372lb # [ 0.522179] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1373worker1 # [ 0.561254] msm_serial: driver initialized1374worker1 # [ 0.561423] SuperH (H)SCI(F) driver initialized1375worker1 # [ 0.561477] STM32 USART driver initialized1376lb # [ 0.526089] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1377lb # [ 0.528735] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1378worker2 # [ 0.592208] loop: module loaded1379worker2 # [ 0.592407] virtio_blk virtio2: 1/0/0 default/read/poll queues1380worker2 # [ 0.593190] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1381lb # [ 0.538914] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1382lb # [ 0.540902] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1383lb # [ 0.542768] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1384client # [ 0.611106] loop: module loaded1385client # [ 0.611301] virtio_blk virtio2: 1/0/0 default/read/poll queues1386client # [ 0.612098] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1387worker2 # [ 0.597993] megasas: 07.734.00.00-rc11388worker2 # [ 0.598694] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1389worker2 # [ 0.601273] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001390worker2 # [ 0.601297] Intel/Sharp Extended Query Table at 0x00311391worker1 # [ 0.592808] loop: module loaded1392worker1 # [ 0.592990] virtio_blk virtio2: 1/0/0 default/read/poll queues1393worker1 # [ 0.593754] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1394lb # [ 0.556483] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1395client # [ 0.622928] megasas: 07.734.00.00-rc11396client # [ 0.623642] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1397client # [ 0.625891] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001398lb # [ 0.562375] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1399client # [ 0.625921] Intel/Sharp Extended Query Table at 0x00311400worker1 # [ 0.604331] megasas: 07.734.00.00-rc11401worker2 # [ 0.618510] Using buffer write method1402worker1 # [ 0.605118] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1403lb # [ 0.568098] msm_serial: driver initialized1404worker2 # [ 0.618603] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1405lb # [ 0.568243] SuperH (H)SCI(F) driver initialized1406lb # [ 0.568300] STM32 USART driver initialized1407worker1 # [ 0.607166] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001408worker2 # [ 0.620479] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001409client # [ 0.635269] Using buffer write method1410worker1 # [ 0.607194] Intel/Sharp Extended Query Table at 0x00311411worker2 # [ 0.620506] Intel/Sharp Extended Query Table at 0x00311412client # [ 0.635357] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1413worker2 # [ 0.625451] Using buffer write method1414worker2 # [ 0.625477] Concatenating MTD devices:1415worker2 # [ 0.625482] (0): "0.flash"1416worker2 # [ 0.625495] (1): "0.flash"1417worker2 # [ 0.625499] into device "0.flash"1418worker1 # [ 0.616863] Using buffer write method1419worker1 # [ 0.616952] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1420client # [ 0.646739] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001421client # [ 0.646808] Intel/Sharp Extended Query Table at 0x00311422worker1 # [ 0.618727] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001423client # [ 0.648245] Using buffer write method1424worker1 # [ 0.618753] Intel/Sharp Extended Query Table at 0x00311425client # [ 0.648278] Concatenating MTD devices:1426client # [ 0.648282] (0): "0.flash"1427client # [ 0.648293] (1): "0.flash"1428client # [ 0.648297] into device "0.flash"1429worker1 # [ 0.632476] Using buffer write method1430worker1 # [ 0.632517] Concatenating MTD devices:1431worker1 # [ 0.632522] (0): "0.flash"1432worker1 # [ 0.632527] (1): "0.flash"1433worker1 # [ 0.632536] into device "0.flash"1434lb # [ 0.603131] loop: module loaded1435lb # [ 0.603336] virtio_blk virtio2: 1/0/0 default/read/poll queues1436lb # [ 0.604194] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1437lb # [ 0.611575] megasas: 07.734.00.00-rc11438lb # [ 0.612323] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1439lb # [ 0.614366] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001440lb # [ 0.614394] Intel/Sharp Extended Query Table at 0x00311441lb # [ 0.616157] Using buffer write method1442lb # [ 0.616225] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1443lb # [ 0.618079] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001444lb # [ 0.618103] Intel/Sharp Extended Query Table at 0x00311445lb # [ 0.638340] Using buffer write method1446lb # [ 0.638379] Concatenating MTD devices:1447lb # [ 0.638383] (0): "0.flash"1448lb # [ 0.638391] (1): "0.flash"1449lb # [ 0.638394] into device "0.flash"1450client # [ 0.905444] Freeing initrd memory: 26972K1451worker2 # [ 0.894433] Freeing initrd memory: 26944K1452client # [ 0.911526] tun: Universal TUN/TAP device driver, 1.61453worker2 # [ 0.900574] tun: Universal TUN/TAP device driver, 1.61454client # [ 0.915304] thunder_xcv, ver 1.01455client # [ 0.915350] thunder_bgx, ver 1.01456worker2 # [ 0.904355] thunder_xcv, ver 1.01457client # [ 0.915372] nicpf, ver 1.01458worker2 # [ 0.904403] thunder_bgx, ver 1.01459worker1 # [ 0.892141] Freeing initrd memory: 26976K1460worker2 # [ 0.904427] nicpf, ver 1.01461client # [ 0.915938] e1000: Intel(R) PRO/1000 Network Driver1462worker2 # [ 0.905026] e1000: Intel(R) PRO/1000 Network Driver1463client # [ 0.915945] e1000: Copyright (c) 1999-2006 Intel Corporation.1464worker2 # [ 0.905033] e1000: Copyright (c) 1999-2006 Intel Corporation.1465client # [ 0.915972] e1000e: Intel(R) PRO/1000 Network Driver1466worker2 # [ 0.905058] e1000e: Intel(R) PRO/1000 Network Driver1467client # [ 0.915980] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1468worker2 # [ 0.905066] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1469client # [ 0.916008] igb: Intel(R) Gigabit Ethernet Network Driver1470worker2 # [ 0.905095] igb: Intel(R) Gigabit Ethernet Network Driver1471client # [ 0.916014] igb: Copyright (c) 2007-2014 Intel Corporation.1472worker2 # [ 0.905101] igb: Copyright (c) 2007-2014 Intel Corporation.1473client # [ 0.916036] igbvf: Intel(R) Gigabit Virtual Function Network Driver1474worker2 # [ 0.905124] igbvf: Intel(R) Gigabit Virtual Function Network Driver1475client # [ 0.916042] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1476client # [ 0.916176] sky2: driver version 1.301477worker2 # [ 0.905130] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1478worker2 # [ 0.905270] sky2: driver version 1.301479worker1 # [ 0.898204] tun: Universal TUN/TAP device driver, 1.61480client # [ 0.917860] usbcore: registered new interface driver usb-storage1481client # [ 0.917914] usbcore: registered new interface driver usbserial_generic1482client # [ 0.917928] usbserial: USB Serial support registered for generic1483client # [ 0.918857] ehci-pci 0000:00:07.0: EHCI Host Controller1484client # [ 0.918889] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11485client # [ 0.919052] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001486worker2 # [ 0.914090] usbcore: registered new interface driver usb-storage1487worker1 # [ 0.901951] thunder_xcv, ver 1.01488client # [ 0.930775] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001489worker1 # [ 0.901999] thunder_bgx, ver 1.01490worker2 # [ 0.914147] usbcore: registered new interface driver usbserial_generic1491client # [ 0.931076] hub 1-0:1.0: USB hub found1492worker1 # [ 0.902022] nicpf, ver 1.01493worker2 # [ 0.914161] usbserial: USB Serial support registered for generic1494client # [ 0.931091] hub 1-0:1.0: 6 ports detected1495worker1 # [ 0.902594] e1000: Intel(R) PRO/1000 Network Driver1496worker2 # [ 0.914764] hv_vmbus: registering driver hyperv_keyboard1497worker1 # [ 0.902600] e1000: Copyright (c) 1999-2006 Intel Corporation.1498worker2 # [ 0.916231] ehci-pci 0000:00:07.0: EHCI Host Controller1499worker1 # [ 0.902624] e1000e: Intel(R) PRO/1000 Network Driver1500client # [ 0.933472] hv_vmbus: registering driver hyperv_keyboard1501worker1 # [ 0.902632] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1502worker2 # [ 0.916257] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11503worker1 # [ 0.902661] igb: Intel(R) Gigabit Ethernet Network Driver1504worker2 # [ 0.916543] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001505worker1 # [ 0.902667] igb: Copyright (c) 2007-2014 Intel Corporation.1506client # [ 0.935084] rtc-pl031 9010000.pl031: registered as rtc01507worker1 # [ 0.902691] igbvf: Intel(R) Gigabit Virtual Function Network Driver1508client # [ 0.935109] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:52:01 UTC (1789984321)1509worker2 # [ 0.921948] rtc-pl031 9010000.pl031: registered as rtc01510worker1 # [ 0.902696] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1511client # [ 0.935417] i2c_dev: i2c /dev entries driver1512worker1 # [ 0.902829] sky2: driver version 1.301513worker2 # [ 0.921975] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:52:01 UTC (1789984321)1514worker2 # [ 0.922323] i2c_dev: i2c /dev entries driver1515worker1 # [ 0.911474] usbcore: registered new interface driver usb-storage1516worker1 # [ 0.911590] usbcore: registered new interface driver usbserial_generic1517client # [ 0.940699] sdhci: Secure Digital Host Controller Interface driver1518worker1 # [ 0.913147] usbserial: USB Serial support registered for generic1519client # [ 0.940712] sdhci: Copyright(c) Pierre Ossman1520worker1 # [ 0.913803] hv_vmbus: registering driver hyperv_keyboard1521client # [ 0.940979] Synopsys Designware Multimedia Card Interface Driver1522worker2 # [ 0.927659] sdhci: Secure Digital Host Controller Interface driver1523worker2 # [ 0.927670] sdhci: Copyright(c) Pierre Ossman1524client # [ 0.941353] sdhci-pltfm: SDHCI platform and OF driver helper1525worker1 # [ 0.916506] ehci-pci 0000:00:07.0: EHCI Host Controller1526worker2 # [ 0.927935] Synopsys Designware Multimedia Card Interface Driver1527worker2 # [ 0.928303] sdhci-pltfm: SDHCI platform and OF driver helper1528worker1 # [ 0.916536] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11529client # [ 0.945653] hid: raw HID events driver (C) Jiri Kosina1530worker1 # [ 0.916795] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001531client # [ 0.945915] usbcore: registered new interface driver usbhid1532worker2 # [ 0.932051] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001533client # [ 0.945921] usbhid: USB HID core driver1534worker2 # [ 0.932355] hub 1-0:1.0: USB hub found1535worker1 # [ 0.919375] rtc-pl031 9010000.pl031: registered as rtc01536worker2 # [ 0.932374] hub 1-0:1.0: 6 ports detected1537worker1 # [ 0.919405] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:52:01 UTC (1789984321)1538worker1 # [ 0.921503] i2c_dev: i2c /dev entries driver1539worker2 # [ 0.934950] hid: raw HID events driver (C) Jiri Kosina1540client # [ 0.948819] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1541worker2 # [ 0.935207] usbcore: registered new interface driver usbhid1542worker2 # [ 0.935213] usbhid: USB HID core driver1543client # [ 0.951459] drop_monitor: Initializing network drop monitor service1544client # [ 0.951594] NET: Registered PF_INET6 protocol family1545client # [ 0.953492] Segment Routing with IPv61546client # [ 0.953509] In-situ OAM (IOAM) with IPv61547client # [ 0.953573] NET: Registered PF_PACKET protocol family1548worker2 # [ 0.938145] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1549worker2 # [ 0.939692] drop_monitor: Initializing network drop monitor service1550worker2 # [ 0.939853] NET: Registered PF_INET6 protocol family1551worker1 # [ 0.925199] sdhci: Secure Digital Host Controller Interface driver1552worker1 # [ 0.925211] sdhci: Copyright(c) Pierre Ossman1553worker2 # [ 0.942883] Segment Routing with IPv61554worker1 # [ 0.925479] Synopsys Designware Multimedia Card Interface Driver1555worker2 # [ 0.942902] In-situ OAM (IOAM) with IPv61556worker1 # [ 0.925857] sdhci-pltfm: SDHCI platform and OF driver helper1557client # [ 0.955444] 9pnet: Installing 9P2000 support1558worker2 # [ 0.942930] NET: Registered PF_PACKET protocol family1559client # [ 0.955494] Key type dns_resolver registered1560worker1 # [ 0.927358] hid: raw HID events driver (C) Jiri Kosina1561lb # [ 0.892786] Freeing initrd memory: 26948K1562worker1 # [ 0.927599] usbcore: registered new interface driver usbhid1563worker2 # [ 0.944592] 9pnet: Installing 9P2000 support1564worker2 # [ 0.944639] Key type dns_resolver registered1565worker1 # [ 0.927640] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001566worker1 # [ 0.927968] hub 1-0:1.0: USB hub found1567worker1 # [ 0.928002] hub 1-0:1.0: 6 ports detected1568worker1 # [ 0.933941] usbhid: USB HID core driver1569worker1 # [ 0.935568] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1570lb # [ 0.898775] tun: Universal TUN/TAP device driver, 1.61571client # [ 0.962250] registered taskstats version 11572worker1 # [ 0.938181] drop_monitor: Initializing network drop monitor service1573client # [ 0.962410] Loading compiled-in X.509 certificates1574worker1 # [ 0.938358] NET: Registered PF_INET6 protocol family1575worker2 # [ 0.951526] registered taskstats version 11576worker1 # [ 0.940307] Segment Routing with IPv61577worker2 # [ 0.951697] Loading compiled-in X.509 certificates1578worker1 # [ 0.940324] In-situ OAM (IOAM) with IPv61579lb # [ 0.902499] thunder_xcv, ver 1.01580lb # [ 0.902541] thunder_bgx, ver 1.01581worker1 # [ 0.940353] NET: Registered PF_PACKET protocol family1582lb # [ 0.902563] nicpf, ver 1.01583client # [ 0.970948] Demotion targets for Node 0: null1584worker1 # [ 0.941980] 9pnet: Installing 9P2000 support1585client # [ 0.971065] Key type .fscrypt registered1586worker1 # [ 0.942033] Key type dns_resolver registered1587client # [ 0.971072] Key type fscrypt-provisioning registered1588client # [ 0.971166] ima: No TPM chip found, activating TPM-bypass!1589client # [ 0.971185] ima: Allocated hash algorithm: sha11590lb # [ 0.904344] e1000: Intel(R) PRO/1000 Network Driver1591client # [ 0.971213] ima: No architecture policies found1592lb # [ 0.904355] e1000: Copyright (c) 1999-2006 Intel Corporation.1593lb # [ 0.904380] e1000e: Intel(R) PRO/1000 Network Driver1594worker2 # [ 0.960760] Demotion targets for Node 0: null1595lb # [ 0.904388] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1596client # [ 0.975574] input: gpio-keys as /devices/platform/gpio-keys/input/input01597worker2 # [ 0.960868] Key type .fscrypt registered1598worker2 # [ 0.960875] Key type fscrypt-provisioning registered1599lb # [ 0.904417] igb: Intel(R) Gigabit Ethernet Network Driver1600worker2 # [ 0.960969] ima: No TPM chip found, activating TPM-bypass!1601lb # [ 0.904423] igb: Copyright (c) 2007-2014 Intel Corporation.1602worker2 # [ 0.960988] ima: Allocated hash algorithm: sha11603lb # [ 0.904446] igbvf: Intel(R) Gigabit Virtual Function Network Driver1604worker2 # [ 0.961008] ima: No architecture policies found1605lb # [ 0.904452] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1606lb # [ 0.904587] sky2: driver version 1.301607worker1 # [ 0.948805] registered taskstats version 11608worker2 # [ 0.965061] input: gpio-keys as /devices/platform/gpio-keys/input/input01609lb # [ 0.906215] usbcore: registered new interface driver usb-storage1610worker1 # [ 0.948960] Loading compiled-in X.509 certificates1611lb # [ 0.906264] usbcore: registered new interface driver usbserial_generic1612lb # [ 0.906278] usbserial: USB Serial support registered for generic1613lb # [ 0.907503] ehci-pci 0000:00:07.0: EHCI Host Controller1614lb # [ 0.907527] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11615lb # [ 0.907786] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001616lb # [ 0.918309] hv_vmbus: registering driver hyperv_keyboard1617lb # [ 0.918903] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001618worker1 # [ 0.957762] Demotion targets for Node 0: null1619lb # [ 0.919254] hub 1-0:1.0: USB hub found1620worker1 # [ 0.957889] Key type .fscrypt registered1621lb # [ 0.921072] hub 1-0:1.0: 6 ports detected1622worker1 # [ 0.957896] Key type fscrypt-provisioning registered1623worker1 # [ 0.957993] ima: No TPM chip found, activating TPM-bypass!1624worker1 # [ 0.958012] ima: Allocated hash algorithm: sha11625worker1 # [ 0.958034] ima: No architecture policies found1626lb # [ 0.922314] rtc-pl031 9010000.pl031: registered as rtc01627lb # [ 0.922344] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T09:52:01 UTC (1789984321)1628worker1 # [ 0.962076] input: gpio-keys as /devices/platform/gpio-keys/input/input01629lb # [ 0.922664] i2c_dev: i2c /dev entries driver1630lb # [ 0.928112] sdhci: Secure Digital Host Controller Interface driver1631lb # [ 0.928122] sdhci: Copyright(c) Pierre Ossman1632lb # [ 0.928392] Synopsys Designware Multimedia Card Interface Driver1633lb # [ 0.928768] sdhci-pltfm: SDHCI platform and OF driver helper1634worker2 # [ 0.983028] clk: Disabling unused clocks1635lb # [ 0.930336] hid: raw HID events driver (C) Jiri Kosina1636worker2 # [ 0.983058] PM: genpd: Disabling unused power domains1637lb # [ 0.930570] usbcore: registered new interface driver usbhid1638lb # [ 0.930576] usbhid: USB HID core driver1639worker2 # [ 0.987245] Freeing unused kernel memory: 4736K1640worker2 # [ 0.987462] Run /init as init process1641lb # [ 0.936199] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1642lb # [ 0.937720] drop_monitor: Initializing network drop monitor service1643lb # [ 0.937852] NET: Registered PF_INET6 protocol family1644client # [ 1.004876] clk: Disabling unused clocks1645client # [ 1.004914] PM: genpd: Disabling unused power domains1646lb # [ 0.940847] Segment Routing with IPv61647lb # [ 0.940877] In-situ OAM (IOAM) with IPv61648lb # [ 0.940906] NET: Registered PF_PACKET protocol family1649lb # [ 0.942542] 9pnet: Installing 9P2000 support1650lb # [ 0.942588] Key type dns_resolver registered1651client # [ 1.009099] Freeing unused kernel memory: 4736K1652worker1 # [ 0.980407] clk: Disabling unused clocks1653client # [ 1.009294] Run /init as init process1654worker1 # [ 0.980439] PM: genpd: Disabling unused power domains1655worker1 # [ 0.984574] Freeing unused kernel memory: 4736K1656worker1 # [ 0.984790] Run /init as init process1657lb # [ 0.949579] registered taskstats version 11658lb # [ 0.949729] Loading compiled-in X.509 certificates1659worker2 # [ 1.003616] systemd[1]: Successfully made /usr/ read-only.1660client # [ 1.024403] systemd[1]: Successfully made /usr/ read-only.1661lb # [ 0.958363] Demotion targets for Node 0: null1662lb # [ 0.958475] Key type .fscrypt registered1663lb # [ 0.958482] Key type fscrypt-provisioning registered1664lb # [ 0.958576] ima: No TPM chip found, activating TPM-bypass!1665lb # [ 0.958596] ima: Allocated hash algorithm: sha11666lb # [ 0.958617] ima: No architecture policies found1667lb # [ 0.962835] input: gpio-keys as /devices/platform/gpio-keys/input/input01668worker1 # [ 1.001936] systemd[1]: Successfully made /usr/ read-only.1669lb # [ 0.980641] clk: Disabling unused clocks1670lb # [ 0.980666] PM: genpd: Disabling unused power domains1671lb # [ 0.984838] Freeing unused kernel memory: 4736K1672lb # [ 0.985030] Run /init as init process1673lb # [ 1.001074] systemd[1]: Successfully made /usr/ read-only.1674client # [ 1.178242] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker2 # [ 1.177419] usb 1-1: new high-speed USB device number 2 using ehci-pci1676worker1 # [ 1.179692] usb 1-1: new high-speed USB device number 2 using ehci-pci1677lb # [ 1.166952] usb 1-1: new high-speed USB device number 2 using ehci-pci1678client # [ 1.330727] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11679worker2 # [ 1.329851] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11680worker2 # [ 1.338411] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1681worker1 # [ 1.335992] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11682worker2 # [ 1.350845] systemd[1]: Detected virtualization qemu.1683worker2 # [ 1.350940] systemd[1]: Detected architecture arm64.1684client # [ 1.359249] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1685worker2 # [ 1.350966] systemd[1]: Running in initrd.1686client # [ 1.371489] systemd[1]: Detected virtualization qemu.1687worker1 # [ 1.339285] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1688worker2 # [ 1.351939] systemd[1]: Initializing machine ID from random generator.1689worker1 # [ 1.339320] systemd[1]: Detected virtualization qemu.1690client # [ 1.373560] systemd[1]: Detected architecture arm64.1691worker1 # [ 1.339371] systemd[1]: Detected architecture arm64.1692worker2 # [ 1.352258] systemd[1]: Hostname set to <worker2>.1693worker1 # [ 1.339384] systemd[1]: Running in initrd.1694client # [ 1.375550] systemd[1]: Running in initrd.1695worker1 # [ 1.339998] systemd[1]: Initializing machine ID from random generator.1696worker1 # [ 1.340174] systemd[1]: Hostname set to <worker1>.1697client # [ 1.378165] systemd[1]: Initializing machine ID from random generator.1698client # [ 1.381070] systemd[1]: Hostname set to <client>.1699lb # [ 1.319459] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11700lb # [ 1.335734] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1701lb # [ 1.347850] systemd[1]: Detected virtualization qemu.1702lb # [ 1.349904] systemd[1]: Detected architecture arm64.1703lb # [ 1.351876] systemd[1]: Running in initrd.1704lb # [ 1.354548] systemd[1]: Initializing machine ID from random generator.1705lb # [ 1.357612] systemd[1]: Hostname set to <lb>.1706client # [ 1.426563] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker2 # [ 1.417602] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01708worker1 # [ 1.419844] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01709lb # [ 1.411141] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01710client # [ 1.550265] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker2 # [ 1.541434] usb 1-2: new high-speed USB device number 3 using ehci-pci1712worker1 # [ 1.539661] usb 1-2: new high-speed USB device number 3 using ehci-pci1713lb # [ 1.534902] usb 1-2: new high-speed USB device number 3 using ehci-pci1714worker1 # [ 1.632027] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715worker2 # [ 1.673123] systemd[1]: bpf-restrict-fs: LSM BPF program attached1716client # [ 1.694958] systemd[1]: bpf-restrict-fs: LSM BPF program attached1717client # [ 1.712268] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21718worker2 # [ 1.700404] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21719lb # [ 1.657956] systemd[1]: bpf-restrict-fs: LSM BPF program attached1720worker2 # [ 1.706185] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01721worker1 # [ 1.694826] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21722client # [ 1.722323] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01723worker1 # [ 1.699690] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01724worker1 # [ 1.720701] systemd[1]: Queued start job for default target Initrd Default Target.1725worker1 # [ 1.730211] systemd[1]: Created slice Slice /system/modprobe.1726lb # [ 1.692133] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21727worker1 # [ 1.731468] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1728lb # [ 1.694080] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01729worker1 # [ 1.732953] systemd[1]: Expecting device /dev/disk/by-label/nixos...1730worker1 # [ 1.734155] systemd[1]: Reached target Path Units.1731worker1 # [ 1.734996] systemd[1]: Reached target Slice Units.1732worker1 # [ 1.735928] systemd[1]: Reached target Swaps.1733worker1 # [ 1.736697] systemd[1]: Reached target Timer Units.1734worker1 # [ 1.737747] systemd[1]: Listening on D-Bus System Message Bus Socket.1735worker1 # [ 1.739037] systemd[1]: Listening on Journal Socket (/dev/log).1736worker1 # [ 1.740371] systemd[1]: Listening on Journal Sockets.1737worker1 # [ 1.741398] systemd[1]: Listening on udev Control Socket.1738worker1 # [ 1.742475] systemd[1]: Listening on udev Kernel Socket.1739worker1 # [ 1.743404] systemd[1]: Reached target Socket Units.1740worker1 # [ 1.746033] systemd[1]: Starting Create List of Static Device Nodes...1741worker1 # [ 1.746118] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1742worker1 # [ 1.755772] systemd[1]: Mounting Kernel Configuration File System...1743lb # [ 1.721152] systemd[1]: Queued start job for default target Initrd Default Target.1744worker1 # [ 1.763876] systemd[1]: Starting Journal Service...1745lb # [ 1.728726] systemd[1]: Created slice Slice /system/modprobe.1746lb # [ 1.729647] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1747lb # [ 1.730665] systemd[1]: Expecting device /dev/disk/by-label/nixos...1748lb # [ 1.731589] systemd[1]: Reached target Path Units.1749lb # [ 1.732221] systemd[1]: Reached target Slice Units.1750lb # [ 1.732865] systemd[1]: Reached target Swaps.1751lb # [ 1.733437] systemd[1]: Reached target Timer Units.1752lb # [ 1.734209] systemd[1]: Listening on D-Bus System Message Bus Socket.1753lb # [ 1.735187] systemd[1]: Listening on Journal Socket (/dev/log).1754lb # [ 1.736058] systemd[1]: Listening on Journal Sockets.1755lb # [ 1.736814] systemd[1]: Listening on udev Control Socket.1756lb # [ 1.737607] systemd[1]: Listening on udev Kernel Socket.1757lb # [ 1.738395] systemd[1]: Reached target Socket Units.1758worker2 # [ 1.791154] systemd[1]: Queued start job for default target Initrd Default Target.1759client # [ 1.806313] systemd[1]: Queued start job for default target Initrd Default Target.1760lb # [ 1.740431] systemd[1]: Starting Create List of Static Device Nodes...1761lb # [ 1.740508] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1762worker2 # [ 1.799571] systemd[1]: Created slice Slice /system/modprobe.1763worker2 # [ 1.800638] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1764worker1 # [ 1.787763] systemd[1]: Starting Load Kernel Modules...1765worker2 # [ 1.801845] systemd[1]: Expecting device /dev/disk/by-label/nixos...1766lb # [ 1.750654] systemd[1]: Mounting Kernel Configuration File System...1767worker2 # [ 1.802764] systemd[1]: Reached target Path Units.1768worker1 # [ 1.788685] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1769worker2 # [ 1.803466] systemd[1]: Reached target Slice Units.1770client # [ 1.817894] systemd[1]: Created slice Slice /system/modprobe.1771worker2 # [ 1.804188] systemd[1]: Reached target Swaps.1772worker2 # [ 1.804834] systemd[1]: Reached target Timer Units.1773client # [ 1.819235] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1774worker2 # [ 1.805762] systemd[1]: Listening on D-Bus System Message Bus Socket.1775client # [ 1.820636] systemd[1]: Expecting device /dev/disk/by-label/nixos...1776worker2 # [ 1.806820] systemd[1]: Listening on Journal Socket (/dev/log).1777client # [ 1.821723] systemd[1]: Reached target Path Units.1778worker2 # [ 1.807908] systemd[1]: Listening on Journal Sockets.1779client # [ 1.822600] systemd[1]: Reached target Slice Units.1780worker2 # [ 1.808765] systemd[1]: Listening on udev Control Socket.1781client # [ 1.823470] systemd[1]: Reached target Swaps.1782client # [ 1.824233] systemd[1]: Reached target Timer Units.1783worker2 # [ 1.809789] systemd[1]: Listening on udev Kernel Socket.1784client # [ 1.825265] systemd[1]: Listening on D-Bus System Message Bus Socket.1785worker2 # [ 1.809815] systemd[1]: Reached target Socket Units.1786lb # [ 1.759297] systemd[1]: Starting Journal Service...1787worker2 # [ 1.812744] systemd[1]: Starting Create List of Static Device Nodes...1788client # [ 1.826564] systemd[1]: Listening on Journal Socket (/dev/log).1789worker2 # [ 1.813780] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1790client # [ 1.826712] systemd[1]: Listening on Journal Sockets.1791client # [ 1.826844] systemd[1]: Listening on udev Control Socket.1792client # [ 1.826961] systemd[1]: Listening on udev Kernel Socket.1793client # [ 1.826984] systemd[1]: Reached target Socket Units.1794client # [ 1.833001] systemd[1]: Starting Create List of Static Device Nodes...1795worker1 # [ 1.807899] systemd[1]: Starting Coldplug All udev Devices...1796client # [ 1.834247] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1797worker2 # [ 1.824251] systemd[1]: Mounting Kernel Configuration File System...1798lb # [ 1.778114] systemd[1]: Starting Load Kernel Modules...1799client # [ 1.845406] systemd[1]: Mounting Kernel Configuration File System...1800lb # [ 1.778243] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1801worker1 # [ 1.820808] systemd-journald[72]: Collecting audit messages is disabled.1802lb # [ 1.788499] systemd[1]: Starting Coldplug All udev Devices...1803worker1 # [ 1.831911] systemd[1]: Finished Create List of Static Device Nodes.1804worker1 # [ 1.832602] systemd[1]: Mounted Kernel Configuration File System.1805worker2 # [ 1.849544] systemd[1]: Starting Journal Service...1806worker2 # [ 1.852044] systemd[1]: Starting Load Kernel Modules...1807worker2 # [ 1.852949] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1808worker1 # [ 1.844084] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1809client # [ 1.874472] systemd[1]: Starting Journal Service...1810worker2 # [ 1.863037] systemd[1]: Starting Coldplug All udev Devices...1811client # [ 1.877227] systemd[1]: Starting Load Kernel Modules...1812lb # [ 1.810974] systemd[1]: Finished Create List of Static Device Nodes.1813client # [ 1.878386] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1814lb # [ 1.811951] systemd[1]: Mounted Kernel Configuration File System.1815lb # [ 1.815015] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1816client # [ 1.895193] systemd[1]: Starting Coldplug All udev Devices...1817worker2 # [ 1.885591] systemd[1]: Finished Create List of Static Device Nodes.1818lb # [ 1.836962] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1819worker2 # [ 1.886298] systemd[1]: Mounted Kernel Configuration File System.1820worker2 # [ 1.891526] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1821worker1 # [ 1.880297] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1822worker1 # [ 1.884921] systemd[1]: Starting Create Static Device Nodes in /dev...1823lb # [ 1.842163] systemd-journald[72]: Collecting audit messages is disabled.1824client # [ 1.914775] systemd[1]: Finished Create List of Static Device Nodes.1825client # [ 1.918594] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1826client # [ 1.919844] systemd[1]: Mounted Kernel Configuration File System.1827worker1 # [ 1.892966] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1828worker1 # [ 1.897570] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1829lb # [ 1.861635] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1830worker2 # [ 1.920719] systemd-journald[72]: Collecting audit messages is disabled.1831lb # [ 1.868672] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1832worker1 # [ 1.908857] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01833worker1 # [ 1.909097] [drm] features: -virgl +edid -resource_blob -host_visible1834worker1 # [ 1.909107] [drm] features: -context_init1835worker1 # [ 1.909826] [drm] number of scanouts: 11836worker1 # [ 1.909844] [drm] number of cap sets: 01837client # [ 1.944782] systemd-journald[72]: Collecting audit messages is disabled.1838lb # [ 1.879347] systemd[1]: Starting Create Static Device Nodes in /dev...1839lb # [ 1.884789] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01840lb # [ 1.885032] [drm] features: -virgl +edid -resource_blob -host_visible1841lb # [ 1.885042] [drm] features: -context_init1842worker2 # [ 1.937856] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1843lb # [ 1.885771] [drm] number of scanouts: 11844lb # [ 1.885790] [drm] number of cap sets: 01845worker2 # [ 1.940577] systemd[1]: Starting Create Static Device Nodes in /dev...1846client # [ 1.963827] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1847worker1 # [ 1.936239] systemd[1]: Finished Create Static Device Nodes in /dev.1848worker1 # [ 1.936599] systemd[1]: Reached target Preparation for Local File Systems.1849worker1 # [ 1.936630] systemd[1]: Reached target Local File Systems.1850worker2 # [ 1.954437] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1851lb # [ 1.899171] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1852worker1 # [ 1.940562] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1853lb # [ 1.899187] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01854client # [ 1.970793] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1855worker1 # [ 1.940576] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01856worker1 # [ 1.945729] systemd[1]: Starting Rule-based Manager for Device Events and Files...1857client # [ 1.973177] systemd[1]: Starting Create Static Device Nodes in /dev...1858client # [ 1.986242] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1859worker2 # [ 1.969632] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1860lb # [ 1.919193] Console: switching to colour frame buffer device 160x501861worker2 # [ 1.975806] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01862worker2 # [ 1.976031] [drm] features: -virgl +edid -resource_blob -host_visible1863worker2 # [ 1.976040] [drm] features: -context_init1864worker2 # [ 1.976742] [drm] number of scanouts: 11865worker2 # [ 1.976760] [drm] number of cap sets: 01866worker1 # [ 1.962973] Console: switching to colour frame buffer device 160x501867client # [ 1.999736] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01868client # [ 1.999968] [drm] features: -virgl +edid -resource_blob -host_visible1869client # [ 1.999978] [drm] features: -context_init1870client # [ 2.000680] [drm] number of scanouts: 11871client # [ 2.000697] [drm] number of cap sets: 01872lb # [ 1.939537] systemd[1]: Finished Create Static Device Nodes in /dev.1873lb # [ 1.939782] systemd[1]: Reached target Preparation for Local File Systems.1874lb # [ 1.939807] systemd[1]: Reached target Local File Systems.1875worker2 # [ 1.993928] systemd[1]: Finished Create Static Device Nodes in /dev.1876worker2 # [ 1.994287] systemd[1]: Reached target Preparation for Local File Systems.1877worker2 # [ 1.994317] systemd[1]: Reached target Local File Systems.1878lb # [ 1.943788] systemd[1]: Starting Rule-based Manager for Device Events and Files...1879worker2 # [ 1.998194] systemd[1]: Starting Rule-based Manager for Device Events and Files...1880lb # [ 1.955516] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1881worker2 # [ 2.005827] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1882worker1 # [ 1.996213] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1883client # [ 2.023013] systemd[1]: Finished Create Static Device Nodes in /dev.1884worker2 # [ 2.005842] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01885client # [ 2.023210] systemd[1]: Reached target Preparation for Local File Systems.1886client # [ 2.023234] systemd[1]: Reached target Local File Systems.1887client # [ 2.029935] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1888client # [ 2.029949] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01889worker1 # [ 2.009088] systemd[1]: Finished Load Kernel Modules.1890worker1 # [ 1.996355] systemd-modules-load[74]: Inserted module 'dm_mod'1891worker1 # [ 2.013622] systemd[1]: Starting Apply Kernel Variables...1892client # [ 2.038109] systemd[1]: Starting Rule-based Manager for Device Events and Files...1893lb # [ 1.975135] systemd[1]: Finished Load Kernel Modules.1894worker1 # [ 2.000097] systemd-modules-load[74]: Module 'virtio_balloon' is built in1895worker1 # [ 2.016792] systemd[1]: Started Journal Service.1896lb # [ 1.979293] systemd[1]: Starting Apply Kernel Variables...1897worker1 # [ 2.008328] systemd-modules-load[74]: Module 'virtio_console' is built in1898worker2 # [ 2.033711] Console: switching to colour frame buffer device 160x501899worker1 # [ 2.012278] systemd-modules-load[74]: Inserted module 'virtio_gpu'1900worker1 # [ 2.016428] systemd-modules-load[74]: Module 'virtio_rng' is built in1901client # [ 2.055451] Console: switching to colour frame buffer device 160x501902client # [ 2.062062] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1903worker1 # [ 2.031157] systemd-udevd[79]: Using default interface naming scheme 'v261'.1904worker2 # [ 2.064576] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1905worker2 # [ 2.074014] systemd[1]: Finished Load Kernel Modules.1906client # [ 2.090816] systemd[1]: Finished Load Kernel Modules.1907lb # [ 2.006915] systemd-modules-load[73]: Inserted module 'dm_mod'1908worker1 # [ 2.050358] systemd[1]: Starting Create System Files and Directories...1909client # [ 2.094176] systemd[1]: Starting Apply Kernel Variables...1910lb # [ 2.031099] systemd[1]: Started Journal Service.1911worker2 # [ 2.085750] systemd[1]: Starting Apply Kernel Variables...1912lb # [ 2.019884] systemd-modules-load[73]: Module 'virtio_balloon' is built in1913lb # [ 2.024331] systemd-modules-load[73]: Module 'virtio_console' is built in1914lb # [ 2.025566] systemd-modules-load[73]: Inserted module 'virtio_gpu'1915lb # [ 2.026593] systemd-modules-load[73]: Module 'virtio_rng' is built in1916lb # [ 2.027578] systemd[1]: Starting Create System Files and Directories...1917worker1 # [ 2.080541] systemd[1]: Finished Apply Kernel Variables.1918worker1 # [ 2.085925] systemd[1]: Finished Create System Files and Directories.1919lb # [ 2.046002] systemd-udevd[78]: Using default interface naming scheme 'v261'.1920lb # [ 2.047146] systemd[1]: Finished Apply Kernel Variables.1921worker2 # [ 2.108379] systemd-modules-load[73]: Inserted module 'dm_mod'1922worker1 # [ 2.094225] systemd[1]: Started Rule-based Manager for Device Events and Files.1923client # [ 2.135997] systemd[1]: Started Journal Service.1924worker2 # [ 2.122791] systemd[1]: Started Journal Service.1925client # [ 2.125197] systemd-modules-load[73]: Inserted module 'dm_mod'1926client # [ 2.126383] systemd-modules-load[73]: Module 'virtio_balloon' is built in1927worker2 # [ 2.116382] systemd-modules-load[73]: Module 'virtio_balloon' is built in1928worker2 # [ 2.117673] systemd-modules-load[73]: Module 'virtio_console' is built in1929lb # [ 2.065392] systemd[1]: Finished Create System Files and Directories.1930worker2 # [ 2.124187] systemd-modules-load[73]: Inserted module 'virtio_gpu'1931client # [ 2.139832] systemd-modules-load[73]: Module 'virtio_console' is built in1932worker2 # [ 2.129571] systemd-modules-load[73]: Module 'virtio_rng' is built in1933client # [ 2.144348] systemd-modules-load[73]: Inserted module 'virtio_gpu'1934client # [ 2.145325] systemd-modules-load[73]: Module 'virtio_rng' is built in1935worker2 # [ 2.132127] systemd-udevd[79]: Using default interface naming scheme 'v261'.1936client # [ 2.146618] systemd-udevd[79]: Using default interface naming scheme 'v261'.1937client # [ 2.147745] systemd[1]: Starting Create System Files and Directories...1938worker2 # [ 2.136293] systemd[1]: Starting Create System Files and Directories...1939lb # [ 2.083510] systemd[1]: Started Rule-based Manager for Device Events and Files.1940worker2 # [ 2.144416] systemd[1]: Finished Apply Kernel Variables.1941client # [ 2.161575] systemd[1]: Finished Apply Kernel Variables.1942client # [ 2.174383] systemd[1]: Finished Create System Files and Directories.1943worker1 # [ 2.145787] systemd[1]: Starting Virtual Console Setup...1944worker2 # [ 2.169886] systemd[1]: Finished Create System Files and Directories.1945worker2 # [ 2.172273] systemd[1]: Started Rule-based Manager for Device Events and Files.1946client # [ 2.197154] systemd[1]: Started Rule-based Manager for Device Events and Files.1947lb # [ 2.140845] systemd[1]: Starting Virtual Console Setup...1948worker1 # [ 2.200514] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1949worker1 # [ 2.204151] systemd[1]: Finished Virtual Console Setup.1950worker2 # [ 2.232787] systemd[1]: Starting Virtual Console Setup...1951client # [ 2.256109] systemd[1]: Starting Virtual Console Setup...1952lb # [ 2.192475] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1953lb # [ 2.196068] systemd[1]: Finished Virtual Console Setup.1954worker2 # [ 2.284598] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1955client # [ 2.300463] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1956worker2 # [ 2.288039] systemd[1]: Finished Virtual Console Setup.1957client # [ 2.303959] systemd[1]: Finished Virtual Console Setup.1958worker1 # [ 2.804713] systemd[1]: Finished Coldplug All udev Devices.1959worker1 # [ 2.806792] systemd[1]: Reached target System Initialization.1960worker1 # [ 2.813136] systemd[1]: Reached target Basic System.1961lb # [ 2.808815] systemd[1]: Finished Coldplug All udev Devices.1962lb # [ 2.809750] systemd[1]: Reached target System Initialization.1963lb # [ 2.810560] systemd[1]: Reached target Basic System.1964client # [ 2.896700] systemd[1]: Finished Coldplug All udev Devices.1965client # [ 2.897632] systemd[1]: Reached target System Initialization.1966client # [ 2.900106] systemd[1]: Reached target Basic System.1967worker2 # [ 2.903845] systemd[1]: Finished Coldplug All udev Devices.1968worker2 # [ 2.904891] systemd[1]: Reached target System Initialization.1969worker2 # [ 2.905714] systemd[1]: Reached target Basic System.1970worker1 # [ 2.983779] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1971worker1 # [ 2.987903] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1972lb # [ 2.959290] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1973worker1 # [ 3.004108] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1974client # [ 3.041743] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1975lb # [ 2.980951] (udev-worker)[100]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1976lb # [ 2.985098] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1977client # [ 3.058828] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1978client # [ 3.064189] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1979worker2 # [ 3.059432] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1980worker1 # [ 3.060934] systemd[1]: Found device /dev/disk/by-label/nixos.1981worker1 # [ 3.064821] systemd[1]: Reached target Initrd Root Device.1982worker2 # [ 3.080081] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1983worker2 # [ 3.084190] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1984worker1 # [ 3.069018] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1985lb # [ 3.062534] systemd[1]: Found device /dev/disk/by-label/nixos.1986lb # [ 3.067253] systemd[1]: Reached target Initrd Root Device.1987client # [ 3.142059] systemd[1]: Found device /dev/disk/by-label/nixos.1988lb # [ 3.072168] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1989client # [ 3.144298] systemd[1]: Reached target Initrd Root Device.1990client # [ 3.147906] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1991worker1 # [ 3.124315] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1992worker1 # [ 3.131649] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1993worker1 # [ 3.137966] systemd[1]: Mounting /sysroot...1994worker2 # [ 3.167908] systemd[1]: Found device /dev/disk/by-label/nixos.1995worker2 # [ 3.171009] systemd[1]: Reached target Initrd Root Device.1996worker2 # [ 3.176155] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1997lb # [ 3.120932] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1998lb # [ 3.128190] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1999lb # [ 3.132172] systemd[1]: Mounting /sysroot...2000client # [ 3.204879] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks
loading 1,447 hidden lines…
3448lb # [ 18.139594] mock-oidc-server[901]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json3449lb # [ 18.144158] mock-oidc-server[901]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration3450lb # [ 18.145472] mock-oidc-server[901]: Issue tokens: http://0.0.0.0:8081/issue?sub=...3451worker1 # [ 18.210376] dhcpcd[638]: eth0: leased 10.0.2.15 for 86400 seconds3452worker1 # [ 18.213657] dhcpcd[638]: eth0: adding route to 10.0.2.0/243453worker1 # [ 18.220164] dhcpcd[638]: eth0: adding default route via 10.0.2.23454worker1 # [ 18.354354] systemd[1]: Started DHCP Client.3455worker1 # [ 18.356502] systemd[1]: Reached target Multi-User System.3456worker1 # [ 18.357482] systemd[1]: Startup finished in 973ms (kernel) + 4.911s (initrd) + 12.471s (userspace) = 18.356s.3457lb # [ 18.358786] envoy[900]: [2026-09-21 09:52:18.954][900][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120)3458lb # [ 18.361626] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:469] statically linked extensions:3459lb # [ 18.363058] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr3460lb # [ 18.365101] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd3461lb # [ 18.370456] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router3462lb # [ 18.374486] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query3463lb # [ 18.380155] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json3464lb # [ 18.383724] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options3465lb # [ 18.388768] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit3466lb # [ 18.392457] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router3467lb # [ 18.441268] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory3468lb # [ 18.443836] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name3469lb # [ 18.447387] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default3470lb # [ 18.449807] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter3471lb # [ 18.452655] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log3472lb # [ 18.459860] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config3473lb # [ 18.464140] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action3474lb # [ 18.466117] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal3475lb # [ 18.467739] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response3476lb # [ 18.470347] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based3477lb # [ 18.472343] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name3478lb # [ 18.474590] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso3479lb # [ 18.476738] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls3480lb # [ 18.482899] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re23481lb # [ 18.484865] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value3482lb # [ 18.488255] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator3483lb # [ 18.490590] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift3484lb # [ 18.494021] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input3485lb # [ 18.495824] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy3486lb # [ 18.500431] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3487lb # [ 18.502446] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert3488lb # [ 18.504315] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service3489lb # [ 18.509107] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override3490lb # [ 18.511947] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection3491lb # [ 18.517399] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor3492lb # [ 18.519927] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian23493lb # [ 18.521682] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind3494lb # [ 18.523302] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http13495lb # [ 18.525555] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state3496lb # [ 18.528548] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource3497lb # [ 18.531835] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher3498lb # [ 18.534030] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default3499lb # [ 18.535914] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default3500lb # [ 18.537921] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata3501lb # [ 18.540176] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port3502lb # [ 18.541906] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory3503lb # [ 18.545088] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities3504lb # [ 18.546893] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction3505lb # [ 18.551543] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme3506lb # [ 18.555044] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator3507lb # [ 18.558003] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default3508lb # [ 18.559808] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter3509lb # [ 18.562570] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy3510lb # [ 18.576454] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality3511client # [ 18.653703] dhcpcd[603]: eth0: leased 10.0.2.15 for 86400 seconds3512lb # [ 18.584131] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule3513lb # [ 18.586833] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic3514client # [ 18.657670] dhcpcd[603]: eth0: adding route to 10.0.2.0/243515lb # [ 18.588887] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple3516lb # [ 18.591104] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain3517client # [ 18.661887] dhcpcd[603]: eth0: adding default route via 10.0.2.23518lb # [ 18.593418] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle3519lb # [ 18.595423] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector3520lb # [ 18.602323] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo3521lb # [ 18.604957] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san3522lb # [ 18.612674] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls3523lb # [ 18.617570] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3524lb # [ 18.620863] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input3525lb # [ 18.625203] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header3526worker2 # [ 18.685912] systemd[1]: Reached target Network is Online.3527lb # [ 18.627770] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm3528worker1 # [ 18.676633] systemd[1]: Starting Nix worker-protocol over gRPC...3529lb # [ 18.635042] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder3530worker2 # [ 18.695989] systemd[1]: Starting scheduler-token.service...3531lb # [ 18.639595] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest3532lb # [ 18.646837] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3533lb # [ 18.649432] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based3534lb # [ 18.653586] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy3535lb # [ 18.656217] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default3536lb # [ 18.657809] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params3537lb # [ 18.668424] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3538lb # [ 18.670221] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua3539lb # [ 18.671799] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3540lb # [ 18.673703] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3541lb # [ 18.676543] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3542lb # [ 18.678206] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable3543lb # [ 18.680114] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override3544lb # [ 18.682627] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple3545lb # [ 18.684914] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche3546lb # [ 18.686706] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3547lb # [ 18.688528] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth23548lb # [ 18.690807] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed3549lb # [ 18.692611] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3550lb # [ 18.694916] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3551lb # [ 18.697058] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin3552lb # [ 18.700245] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router3553lb # [ 18.701981] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default3554lb # [ 18.703706] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher3555lb # [ 18.706319] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor3556lb # [ 18.708905] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case3557lb # [ 18.711098] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection3558lb # [ 18.713156] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats3559lb # [ 18.715568] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff3560lb # [ 18.718284] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v83561lb # [ 18.720792] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation3562client # [ 18.793216] systemd[1]: Started DHCP Client.3563client # [ 18.794824] systemd[1]: Reached target Multi-User System.3564client # [ 18.795996] systemd[1]: Startup finished in 999ms (kernel) + 4.146s (initrd) + 13.649s (userspace) = 18.795s.3565lb # [ 18.723391] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3566lb # [ 18.730916] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe3567lb # [ 18.733850] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default3568lb # [ 18.735864] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid3569lb # [ 18.737536] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3570worker2 # [ 18.796856] systemd[1]: scheduler-token.service: Deactivated successfully.3571worker2 # [ 18.802234] systemd[1]: Finished scheduler-token.service.3572lb # [ 18.740082] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port3573lb # [ 18.748151] envoy[900]: [2026-09-21 09:52:18.957][900][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated3574lb # [ 18.749935] envoy[900]: [2026-09-21 09:52:18.979][900][info][main] [source/server/server.cc:531] HTTP header map info:3575worker2 # [ 18.809009] systemd[1]: scheduler-token.service: Consumed 64ms CPU time over 104ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3576worker2 # [ 18.816653] systemd[1]: Starting Nix worker-protocol over gRPC...3577lb # [ 18.751265] envoy[900]: [2026-09-21 09:52:18.980][900][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id3578lb # [ 18.766581] envoy[900]: [2026-09-21 09:52:18.980][900][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:3579lb # [ 18.767992] envoy[900]: [2026-09-21 09:52:18.980][900][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id3580lb # [ 18.776576] envoy[900]: [2026-09-21 09:52:18.980][900][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status3581lb # [ 18.778292] envoy[900]: [2026-09-21 09:52:19.035][900][info][main] [source/server/server.cc:968] runtime: {}3582lb # [ 18.779514] envoy[900]: [2026-09-21 09:52:19.043][900][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.3583lb # [ 18.781241] envoy[900]: [2026-09-21 09:52:19.043][900][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99013584lb # [ 18.782797] envoy[900]: [2026-09-21 09:52:19.043][900][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99013585lb # [ 18.784295] envoy[900]: [2026-09-21 09:52:19.046][900][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration3586lb # [ 18.785850] envoy[900]: [2026-09-21 09:52:19.046][900][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)3587lb # [ 18.787365] envoy[900]: [2026-09-21 09:52:19.046][900][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)3588lb # [ 18.788931] envoy[900]: [2026-09-21 09:52:19.220][900][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)3589lb # [ 18.790394] envoy[900]: [2026-09-21 09:52:19.242][900][info][config] [source/server/configuration_impl.cc:164] loading stats configuration3590lb # [ 18.791913] envoy[900]: [2026-09-21 09:52:19.242][900][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.3591lb # [ 18.795163] envoy[900]: [2026-09-21 09:52:19.242][900][info][main] [source/server/server.cc:1077] starting main dispatch loop3592worker1 # [ 18.858573] nix-grpc-daemon[765]: ts=2026-09-21T09:52:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3593worker1 # [ 18.866889] nix-grpc-daemon[765]: ts=2026-09-21T09:52:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3594worker1 # [ 18.868676] nix-grpc-daemon[765]: ts=2026-09-21T09:52:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=13595worker1 # [ 18.878426] nix-grpc-daemon[765]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR3596worker1 # [ 18.881288] nix-grpc-daemon[765]: E0000 00:00:1789984339.474314 765 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3597worker1 # [ 18.886283] nix-grpc-daemon[765]: E0000 00:00:1789984339.482166 765 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3598worker1 # [ 18.907316] nix-grpc-daemon[765]: ts=2026-09-21T09:52:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3599worker1 # [ 18.912717] systemd[1]: Started Nix worker-protocol over gRPC.3600lb # [ 18.873004] envoy[900]: [2026-09-21 09:52:19.469][900][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization3601lb # [ 18.877452] envoy[900]: [2026-09-21 09:52:19.469][900][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized3602lb # [ 18.881739] envoy[900]: [2026-09-21 09:52:19.469][900][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager3603worker1 # [ 18.915705] nix-grpc-daemon[765]: E0000 00:00:1789984339.510578 806 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3604lb # [ 18.888520] envoy[900]: [2026-09-21 09:52:19.483][900][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers3605worker1 # [ 18.933754] nix-grpc-daemon[765]: E0000 00:00:1789984339.529646 806 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3606worker1 # [ 18.940062] nix-grpc-daemon[765]: E0000 00:00:1789984339.532465 806 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3607worker2 # [ 18.965666] nix-grpc-daemon[769]: ts=2026-09-21T09:52:19Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3608worker1 # [ 18.954968] nix-grpc-daemon[765]: E0000 00:00:1789984339.550400 806 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3609worker2 # [ 18.973519] nix-grpc-daemon[769]: ts=2026-09-21T09:52:19Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3610worker2 # [ 18.975615] nix-grpc-daemon[769]: ts=2026-09-21T09:52:19Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=13611worker1 # [ 18.967832] nix-grpc-daemon[765]: ts=2026-09-21T09:52:19Z level=info event=scheduler_connected addr=lb:500513612worker1 # [ 18.970293] nix-grpc-daemon[765]: ts=2026-09-21T09:52:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3613worker2 # [ 18.988046] nix-grpc-daemon[769]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR3614worker2 # [ 18.989551] nix-grpc-daemon[769]: E0000 00:00:1789984339.579150 769 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3615worker2 # [ 18.996634] nix-grpc-daemon[769]: E0000 00:00:1789984339.587692 769 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3616worker2 # [ 19.014838] nix-grpc-daemon[769]: ts=2026-09-21T09:52:19Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3617worker2 # [ 19.018758] systemd[1]: Started Nix worker-protocol over gRPC.3618worker2 # [ 19.024073] nix-grpc-daemon[769]: E0000 00:00:1789984339.614934 811 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3619worker2 # [ 19.042442] nix-grpc-daemon[769]: E0000 00:00:1789984339.633711 811 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3620worker2 # [ 19.048049] nix-grpc-daemon[769]: E0000 00:00:1789984339.636528 811 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.3621worker2 # [ 19.064024] nix-grpc-daemon[769]: E0000 00:00:1789984339.654635 811 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.3622worker2 # [ 19.077046] nix-grpc-daemon[769]: ts=2026-09-21T09:52:19Z level=info event=scheduler_connected addr=lb:500513623worker2 # [ 19.079083] nix-grpc-daemon[769]: ts=2026-09-21T09:52:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3624worker1 # [ 19.171409] nix-grpc-daemon[765]: ts=2026-09-21T09:52:19Z level=info event=scheduler_connected addr=lb:500513625worker1 # [ 19.176637] nix-grpc-daemon[765]: ts=2026-09-21T09:52:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3626worker2 # [ 19.280649] nix-grpc-daemon[769]: ts=2026-09-21T09:52:19Z level=info event=scheduler_connected addr=lb:500513627worker2 # [ 19.285708] nix-grpc-daemon[769]: ts=2026-09-21T09:52:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3628worker1 # [ 19.377566] nix-grpc-daemon[765]: ts=2026-09-21T09:52:19Z level=info event=scheduler_connected addr=lb:500513629worker1 # [ 19.382715] nix-grpc-daemon[765]: ts=2026-09-21T09:52:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3630worker2 # [ 19.486972] nix-grpc-daemon[769]: ts=2026-09-21T09:52:20Z level=info event=scheduler_connected addr=lb:500513631worker2 # [ 19.492315] nix-grpc-daemon[769]: ts=2026-09-21T09:52:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3632worker1 # [ 19.583631] nix-grpc-daemon[765]: ts=2026-09-21T09:52:20Z level=info event=scheduler_connected addr=lb:500513633worker1 # [ 19.588921] nix-grpc-daemon[765]: ts=2026-09-21T09:52:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3634worker2 # [ 19.693140] nix-grpc-daemon[769]: ts=2026-09-21T09:52:20Z level=info event=scheduler_connected addr=lb:500513635worker2 # [ 19.696280] nix-grpc-daemon[769]: ts=2026-09-21T09:52:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3636worker1 # [ 19.789804] nix-grpc-daemon[765]: ts=2026-09-21T09:52:20Z level=info event=scheduler_connected addr=lb:500513637worker1 # [ 19.795367] nix-grpc-daemon[765]: ts=2026-09-21T09:52:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3638worker2 # [ 19.897689] nix-grpc-daemon[769]: ts=2026-09-21T09:52:20Z level=info event=scheduler_connected addr=lb:500513639worker2 # [ 19.902849] nix-grpc-daemon[769]: ts=2026-09-21T09:52:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3640worker1 # [ 19.998229] nix-grpc-daemon[765]: ts=2026-09-21T09:52:20Z level=info event=scheduler_connected addr=lb:500513641worker1 # [ 20.002439] nix-grpc-daemon[765]: ts=2026-09-21T09:52:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3642worker2 # [ 20.105758] nix-grpc-daemon[769]: ts=2026-09-21T09:52:20Z level=info event=scheduler_connected addr=lb:500513643worker2 # [ 20.109996] nix-grpc-daemon[769]: ts=2026-09-21T09:52:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3644worker1 # [ 20.198843] nix-grpc-daemon[765]: ts=2026-09-21T09:52:20Z level=info event=scheduler_connected addr=lb:500513645worker1 # [ 20.204484] nix-grpc-daemon[765]: ts=2026-09-21T09:52:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3646worker2 # [ 20.306436] nix-grpc-daemon[769]: ts=2026-09-21T09:52:20Z level=info event=scheduler_connected addr=lb:500513647worker2 # [ 20.311514] nix-grpc-daemon[769]: ts=2026-09-21T09:52:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3648worker1 # [ 20.405435] nix-grpc-daemon[765]: ts=2026-09-21T09:52:21Z level=info event=scheduler_connected addr=lb:500513649worker1 # [ 20.410953] nix-grpc-daemon[765]: ts=2026-09-21T09:52:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3650worker2 # [ 20.512892] nix-grpc-daemon[769]: ts=2026-09-21T09:52:21Z level=info event=scheduler_connected addr=lb:500513651worker2 # [ 20.518017] nix-grpc-daemon[769]: ts=2026-09-21T09:52:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3652worker1 # [ 20.611752] nix-grpc-daemon[765]: ts=2026-09-21T09:52:21Z level=info event=scheduler_connected addr=lb:500513653worker1 # [ 20.617291] nix-grpc-daemon[765]: ts=2026-09-21T09:52:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3654worker2 # [ 20.719268] nix-grpc-daemon[769]: ts=2026-09-21T09:52:21Z level=info event=scheduler_connected addr=lb:500513655worker2 # [ 20.724528] nix-grpc-daemon[769]: ts=2026-09-21T09:52:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3656worker1 # [ 20.818101] nix-grpc-daemon[765]: ts=2026-09-21T09:52:21Z level=info event=scheduler_connected addr=lb:500513657worker1 # [ 20.823491] nix-grpc-daemon[765]: ts=2026-09-21T09:52:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3658worker2 # [ 20.925811] nix-grpc-daemon[769]: ts=2026-09-21T09:52:21Z level=info event=scheduler_connected addr=lb:500513659worker2 # [ 20.931095] nix-grpc-daemon[769]: ts=2026-09-21T09:52:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3660worker1 # [ 21.026315] nix-grpc-daemon[765]: ts=2026-09-21T09:52:21Z level=info event=scheduler_connected addr=lb:500513661worker1 # [ 21.031232] nix-grpc-daemon[765]: ts=2026-09-21T09:52:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3662worker2 # [ 21.134112] nix-grpc-daemon[769]: ts=2026-09-21T09:52:21Z level=info event=scheduler_connected addr=lb:500513663worker2 # [ 21.138208] nix-grpc-daemon[769]: ts=2026-09-21T09:52:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3664worker1 # [ 21.226588] nix-grpc-daemon[765]: ts=2026-09-21T09:52:21Z level=info event=scheduler_connected addr=lb:500513665worker1 # [ 21.231895] nix-grpc-daemon[765]: ts=2026-09-21T09:52:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3666worker2 # [ 21.334955] nix-grpc-daemon[769]: ts=2026-09-21T09:52:21Z level=info event=scheduler_connected addr=lb:500513667worker2 # [ 21.340719] nix-grpc-daemon[769]: ts=2026-09-21T09:52:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3668worker1 # [ 21.432838] nix-grpc-daemon[765]: ts=2026-09-21T09:52:22Z level=info event=scheduler_connected addr=lb:500513669worker1 # [ 21.438130] nix-grpc-daemon[765]: ts=2026-09-21T09:52:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3670worker2 # [ 21.541952] nix-grpc-daemon[769]: ts=2026-09-21T09:52:22Z level=info event=scheduler_connected addr=lb:500513671worker2 # [ 21.547074] nix-grpc-daemon[769]: ts=2026-09-21T09:52:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3672worker1 # [ 21.638881] nix-grpc-daemon[765]: ts=2026-09-21T09:52:22Z level=info event=scheduler_connected addr=lb:500513673worker1 # [ 21.644119] nix-grpc-daemon[765]: ts=2026-09-21T09:52:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3674worker2 # [ 21.748582] nix-grpc-daemon[769]: ts=2026-09-21T09:52:22Z level=info event=scheduler_connected addr=lb:500513675worker2 # [ 21.753790] nix-grpc-daemon[769]: ts=2026-09-21T09:52:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3676worker1 # [ 21.845011] nix-grpc-daemon[765]: ts=2026-09-21T09:52:22Z level=info event=scheduler_connected addr=lb:500513677worker1 # [ 21.850120] nix-grpc-daemon[765]: ts=2026-09-21T09:52:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3678worker2 # [ 21.954975] nix-grpc-daemon[769]: ts=2026-09-21T09:52:22Z level=info event=scheduler_connected addr=lb:500513679worker2 # [ 21.960260] nix-grpc-daemon[769]: ts=2026-09-21T09:52:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3680worker1 # [ 22.052542] nix-grpc-daemon[765]: ts=2026-09-21T09:52:22Z level=info event=scheduler_connected addr=lb:500513681worker1 # [ 22.056584] nix-grpc-daemon[765]: ts=2026-09-21T09:52:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3682worker2 # [ 22.162996] nix-grpc-daemon[769]: ts=2026-09-21T09:52:22Z level=info event=scheduler_connected addr=lb:500513683worker2 # [ 22.167073] nix-grpc-daemon[769]: ts=2026-09-21T09:52:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3684worker1 # [ 22.253037] nix-grpc-daemon[765]: ts=2026-09-21T09:52:22Z level=info event=scheduler_connected addr=lb:500513685worker1 # [ 22.258130] nix-grpc-daemon[765]: ts=2026-09-21T09:52:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3686worker2 # [ 22.363686] nix-grpc-daemon[769]: ts=2026-09-21T09:52:22Z level=info event=scheduler_connected addr=lb:500513687worker2 # [ 22.368815] nix-grpc-daemon[769]: ts=2026-09-21T09:52:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3688worker1 # [ 22.458907] nix-grpc-daemon[765]: ts=2026-09-21T09:52:23Z level=info event=scheduler_connected addr=lb:500513689worker1 # [ 22.464179] nix-grpc-daemon[765]: ts=2026-09-21T09:52:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3690worker2 # [ 22.570005] nix-grpc-daemon[769]: ts=2026-09-21T09:52:23Z level=info event=scheduler_connected addr=lb:500513691worker2 # [ 22.574840] nix-grpc-daemon[769]: ts=2026-09-21T09:52:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3692worker1 # [ 22.665015] nix-grpc-daemon[765]: ts=2026-09-21T09:52:23Z level=info event=scheduler_connected addr=lb:500513693worker1 # [ 22.670086] nix-grpc-daemon[765]: ts=2026-09-21T09:52:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3694lb # [ 22.682144] rustfs-bucket-start[954]: mb s3://farm3695lb # [ 22.697543] systemd[1]: Finished rustfs-bucket.service.3696lb # [ 22.713313] systemd[1]: Starting niks3 server...3697worker2 # [ 22.776040] nix-grpc-daemon[769]: ts=2026-09-21T09:52:23Z level=info event=scheduler_connected addr=lb:500513698worker2 # [ 22.781257] nix-grpc-daemon[769]: ts=2026-09-21T09:52:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3699worker1 # [ 22.870929] nix-grpc-daemon[765]: ts=2026-09-21T09:52:23Z level=info event=scheduler_connected addr=lb:500513700worker1 # [ 22.876216] nix-grpc-daemon[765]: ts=2026-09-21T09:52:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3701lb # [ 22.849818] postgres[970]: [970] ERROR: relation "goose_db_version" does not exist at character 363702lb # [ 22.851058] postgres[970]: [970] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3703lb # [ 22.876458] niks3-server[964]: 2026/09/21 09:52:23 OK 20241026095416_initial_model.sql (15.57ms)3704lb # [ 22.879803] niks3-server[964]: 2026/09/21 09:52:23 OK 20251210153512_drop_unused_gin_index.sql (1.8ms)3705lb # [ 22.886426] niks3-server[964]: 2026/09/21 09:52:23 OK 20251218171726_add_pins.sql (6.57ms)3706lb # [ 22.891177] niks3-server[964]: 2026/09/21 09:52:23 OK 20260628120000_add_object_size_and_stats.sql (4.72ms)3707lb # [ 22.895958] niks3-server[964]: 2026/09/21 09:52:23 OK 20260905000000_add_claims.sql (4.75ms)3708lb # [ 22.899481] niks3-server[964]: 2026/09/21 09:52:23 OK 20260920000000_drop_claims.sql (3.42ms)3709lb # [ 22.901404] niks3-server[964]: 2026/09/21 09:52:23 goose: successfully migrated database to version: 202609200000003710lb # [ 22.904754] niks3-server[964]: 2026/09/21 09:52:23 OK 1_commit_pending_closure.sql (5.24ms)3711lb # [ 22.907387] niks3-server[964]: 2026/09/21 09:52:23 OK 2_object_stats_trigger.sql (2.49ms)3712lb # [ 22.909345] niks3-server[964]: 2026/09/21 09:52:23 goose: up to current file version: 23713lb # [ 22.910902] niks3-server[964]: 2026/09/21 09:52:23 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key3714worker2 # [ 22.982621] nix-grpc-daemon[769]: ts=2026-09-21T09:52:23Z level=info event=scheduler_connected addr=lb:500513715worker2 # [ 22.987774] nix-grpc-daemon[769]: ts=2026-09-21T09:52:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3716lb # [ 22.934051] niks3-server[964]: 2026/09/21 09:52:23 INFO Read proxy enabled — serving cache objects from S33717lb # [ 22.938881] niks3-server[964]: 2026/09/21 09:52:23 INFO Using socket-activated listener address=0.0.0.0:57513718lb # [ 22.945710] niks3-server[964]: 2026/09/21 09:52:23 INFO systemd watchdog enabled interval=15s3719worker1 # [ 22.987646] nix-grpc-daemon[765]: ts=2026-09-21T09:52:23Z level=info event=scheduler_take_over3720lb # [ 22.950391] niks3-server[964]: 2026/09/21 09:52:23 INFO Starting HTTP server address=0.0.0.0:57513721lb # [ 22.955501] systemd[1]: Started niks3 server.3722lb # [ 22.956256] niks3-server[964]: 2026/09/21 09:52:23 INFO lead: acquired remote=192.168.1.3:482303723lb # [ 22.957397] systemd[1]: Reached target Multi-User System.3724lb # [ 22.958114] systemd[1]: Startup finished in 970ms (kernel) + 4.878s (initrd) + 17.094s (userspace) = 22.943s.3725worker1 # [ 23.078627] nix-grpc-daemon[765]: ts=2026-09-21T09:52:23Z level=info event=scheduler_connected addr=lb:500513726worker1 # [ 23.082732] nix-grpc-daemon[765]: ts=2026-09-21T09:52:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3727worker2 # [ 23.190604] nix-grpc-daemon[769]: ts=2026-09-21T09:52:23Z level=info event=scheduler_connected addr=lb:500513728worker2 # [ 23.194748] nix-grpc-daemon[769]: ts=2026-09-21T09:52:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3729worker1 # [ 23.279156] nix-grpc-daemon[765]: ts=2026-09-21T09:52:23Z level=info event=scheduler_connected addr=lb:500513730worker1 # [ 23.284374] nix-grpc-daemon[765]: ts=2026-09-21T09:52:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3731worker2 # [ 23.391609] nix-grpc-daemon[769]: ts=2026-09-21T09:52:23Z level=info event=scheduler_connected addr=lb:500513732worker2 # [ 23.396869] nix-grpc-daemon[769]: ts=2026-09-21T09:52:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3733worker1 # [ 23.485167] nix-grpc-daemon[765]: ts=2026-09-21T09:52:24Z level=info event=scheduler_connected addr=lb:500513734worker1 # [ 23.490578] nix-grpc-daemon[765]: ts=2026-09-21T09:52:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3735worker2 # [ 23.597845] nix-grpc-daemon[769]: ts=2026-09-21T09:52:24Z level=info event=scheduler_connected addr=lb:500513736worker2 # [ 23.603332] nix-grpc-daemon[769]: ts=2026-09-21T09:52:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3737lb: (finished: waiting for unit niks3.service, in 11.23 seconds)3738lb: waiting for TCP port 5751 on localhost3739lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3740lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds)3741lb: waiting for unit envoy.service3742worker1 # [ 23.691729] nix-grpc-daemon[765]: ts=2026-09-21T09:52:24Z level=info event=scheduler_connected addr=lb:500513743worker1 # [ 23.697185] nix-grpc-daemon[765]: ts=2026-09-21T09:52:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3744lb: (finished: waiting for unit envoy.service, in 0.04 seconds)3745lb: waiting for TCP port 50051 on localhost3746lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3747lb # [ 23.702500] envoy[900]: conn peer=[::1]:35210 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=33748lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds)3749worker2 # [ 23.804477] nix-grpc-daemon[769]: ts=2026-09-21T09:52:24Z level=info event=scheduler_connected addr=lb:500513750worker2 # [ 23.809698] nix-grpc-daemon[769]: ts=2026-09-21T09:52:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3751worker1 # [ 23.898041] nix-grpc-daemon[765]: ts=2026-09-21T09:52:24Z level=info event=scheduler_connected addr=lb:500513752worker1 # [ 23.903403] nix-grpc-daemon[765]: ts=2026-09-21T09:52:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3753worker2 # [ 23.925247] systemd[1]: Starting scheduler-token.service...3754worker2 # [ 24.002420] systemd[1]: scheduler-token.service: Deactivated successfully.3755worker2 # [ 24.008696] systemd[1]: Finished scheduler-token.service.3756worker2 # [ 24.011935] systemd[1]: scheduler-token.service: Consumed 44ms CPU time over 80ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3757worker2 # [ 24.016298] nix-grpc-daemon[769]: ts=2026-09-21T09:52:24Z level=info event=scheduler_connected addr=lb:500513758lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3759lb: must succeed: curl -sf localhost:9901/clusters3760worker1 # [ 24.020251] nix-grpc-daemon[765]: ts=2026-09-21T09:52:24Z level=info event=oidc_jwks provider=mock keys=13761worker1 # [ 24.025570] nix-grpc-daemon[765]: ts=2026-09-21T09:52:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:397783762worker1 # [ 24.030777] nix-grpc-daemon[765]: ts=2026-09-21T09:52:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03763lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3764lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.07 seconds)3765subtest: one node schedules and both builders hold a WorkerSession on it3766lb: must succeed: curl -sf localhost:9901/clusters3767worker1 # [ 24.106188] nix-grpc-daemon[765]: ts=2026-09-21T09:52:24Z level=info event=scheduler_connected addr=lb:500513768worker1 # [ 24.110148] nix-grpc-daemon[765]: ts=2026-09-21T09:52:24Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:397783769worker1 # [ 24.115055] nix-grpc-daemon[765]: ts=2026-09-21T09:52:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03770lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)3771worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3772worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)3773lb: must succeed: curl -sf localhost:9901/clusters3774lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3775lb: must succeed: curl -sf localhost:9901/clusters3776lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)3777lb: must succeed: curl -sf localhost:9901/clusters3778lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)3779lb: must succeed: curl -sf localhost:9901/clusters3780lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)3781worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'3782worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.04 seconds)3783worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3784worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)3785lb: must succeed: curl -sf localhost:9901/clusters3786lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3787lb: must succeed: curl -sf localhost:9901/clusters3788lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)3789worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3790worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)3791worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3792worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)3793worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3794worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)3795worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics3796worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)3797(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.65 seconds)3798subtest: balancer auth: client cert, bearer token, nothing, unknown CN3799worker2 # [ 24.997506] nix-grpc-daemon[769]: ts=2026-09-21T09:52:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:562483800worker2 # [ 25.014539] systemd[1]: Started Nix Daemon.3801worker2 # [ 25.190646] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)3802worker2 # [ 25.210157] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 1/5)3803worker2 # [ 25.243678] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 2/5)3804worker2 # [ 25.343794] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 3/5)3805worker2 # [ 25.443539] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 766 ms (attempt 4/5)3806worker2 # [ 26.213289] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds3807worker2 # [ 26.218125] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3808worker1 # [ 26.237328] nix-grpc-daemon[765]: ts=2026-09-21T09:52:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:397923809worker1 # [ 26.253366] systemd[1]: Started Nix Daemon.3810worker1 # [ 26.481563] nix-daemon[844]: accepted connection from pid 765, user nix-grpc-daemon (trusted)3811worker1 # [ 26.501527] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5)3812worker1 # [ 26.579208] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 69 ms (attempt 2/5)3813worker1 # [ 26.651667] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 3/5)3814worker1 # [ 26.757693] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 301 ms (attempt 4/5)3815worker1 # [ 27.062956] nix-grpc-daemon[765]: disabling binary cache 'https://cache.nixos.org' for 60 seconds3816worker1 # [ 27.069611] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3817worker1 # [ 27.082590] nix-grpc-daemon[765]: ts=2026-09-21T09:52:27Z level=debug event=path_info_miss path=rna2s44q968xsxpa8a3qx3wdj7ybhqp0-nixos-system-client-test error="error: path '/nix/store/rna2s44q968xsxpa8a3qx3wdj7ybhqp0-nixos-system-client-test' is required, but there is no substituter that can build it"3818worker2 # [ 27.116612] nix-grpc-daemon[769]: ts=2026-09-21T09:52:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:562483819worker2 # [ 27.131740] nix-grpc-daemon[769]: ts=2026-09-21T09:52:27Z level=debug event=path_info_miss path=rna2s44q968xsxpa8a3qx3wdj7ybhqp0-nixos-system-client-test error="error: path '/nix/store/rna2s44q968xsxpa8a3qx3wdj7ybhqp0-nixos-system-client-test' is required, but there is no substituter that can build it"3820client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3821client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.09 seconds)3822client: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3823client: (finished: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds)3824worker2 # [ 27.546230] nix-grpc-daemon[769]: ts=2026-09-21T09:52:28Z level=info event=oidc_jwks provider=mock keys=13825worker2 # [ 27.556195] nix-grpc-daemon[769]: ts=2026-09-21T09:52:28Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:562483826worker2 # [ 27.572732] nix-grpc-daemon[769]: ts=2026-09-21T09:52:28Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:562483827worker2 # [ 27.588584] nix-grpc-daemon[769]: ts=2026-09-21T09:52:28Z level=debug event=path_info_miss path=rna2s44q968xsxpa8a3qx3wdj7ybhqp0-nixos-system-client-test error="error: path '/nix/store/rna2s44q968xsxpa8a3qx3wdj7ybhqp0-nixos-system-client-test' is required, but there is no substituter that can build it"3828worker2 # [ 27.603800] nix-grpc-daemon[769]: ts=2026-09-21T09:52:28Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:562483829worker2 # [ 27.612527] nix-grpc-daemon[769]: ts=2026-09-21T09:52:28Z level=debug event=path_info_miss path=rna2s44q968xsxpa8a3qx3wdj7ybhqp0-nixos-system-client-test error="error: path '/nix/store/rna2s44q968xsxpa8a3qx3wdj7ybhqp0-nixos-system-client-test' is required, but there is no substituter that can build it"3830client: must succeed: rm -r /var/lib/nix-grpc-store3831client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)3832worker2 # [ 27.893245] nix-grpc-daemon[769]: ts=2026-09-21T09:52:28Z level=info event=denied method=QueryMissing cn=- role=none3833worker2 # [ 28.102751] nix-grpc-daemon[769]: ts=2026-09-21T09:52:28Z level=info event=denied method=QueryMissing cn=stranger role=none3834(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.37 seconds)3835subtest: DAG build is scheduled across workers and lands in the cache3836client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&23837worker2 # [ 28.296295] nix-grpc-daemon[769]: ts=2026-09-21T09:52:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:562483838client # don't know how to build these paths:3839client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3840worker2 # [ 28.308634] nix-grpc-daemon[769]: ts=2026-09-21T09:52:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562483841worker1 # [ 28.361763] nix-grpc-daemon[765]: ts=2026-09-21T09:52:28Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=03842worker1 # [ 28.362596] nix-grpc-daemon[765]: ts=2026-09-21T09:52:28Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=13843worker2 # [ 28.430257] nix-grpc-daemon[769]: ts=2026-09-21T09:52:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562483844worker2 # [ 28.453347] nix-grpc-daemon[769]: ts=2026-09-21T09:52:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562483845client # copying 1 paths to 'lb:50051'...3846worker2 # [ 28.473938] nix-grpc-daemon[769]: ts=2026-09-21T09:52:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:562483847client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...3848worker2 # [ 28.510375] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)3849worker2 # [ 28.555463] nix-grpc-daemon[769]: ts=2026-09-21T09:52:29Z level=info event=niks3_push_started pid=8473850lb # [ 28.630064] envoy[900]: conn peer=[2001:db8:1::1]:38686 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=21753851lb # [ 28.630669] envoy[900]: conn peer=[2001:db8:1::1]:38690 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=973852lb # [ 28.630833] envoy[900]: conn peer=[2001:db8:1::1]:38694 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=203853lb # [ 28.630937] envoy[900]: conn peer=[2001:db8:1::1]:38702 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=153854worker2 # [ 28.778098] nix-grpc-daemon[847]: time=2026-09-21T09:52:29.368Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""3855worker2 # [ 28.912097] nix-daemon[834]: accepted connection from pid 860, user nix-grpc-daemon (trusted)3856worker2 # [ 28.934462] nix-daemon[834]: reaped child process 863, status = succeeded3857worker2 # [ 29.075021] nix-daemon[834]: accepted connection from pid 865, user nix-grpc-daemon (trusted)3858worker2 # [ 29.098363] nix-daemon[834]: reaped child process 868, status = succeeded3859lb # [ 29.042124] niks3-server[964]: 2026/09/21 09:52:29 INFO Received uploads request method=POST path=/api/pending_closures3860worker2 # [ 29.134018] nix-grpc-daemon[847]: time=2026-09-21T09:52:29.723Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3861worker2 # [ 29.135706] nix-grpc-daemon[847]: time=2026-09-21T09:52:29.723Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"3862lb # [ 29.144001] niks3-server[964]: 2026/09/21 09:52:29 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst3863lb # [ 29.157742] niks3-server[964]: 2026/09/21 09:52:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign3864lb # [ 29.170025] niks3-server[964]: 2026/09/21 09:52:29 INFO Signed narinfos id=1 count=13865worker2 # [ 29.228403] nix-grpc-daemon[847]: time=2026-09-21T09:52:29.818Z level=INFO msg="Uploading 1 narinfos"3866lb # [ 29.197856] niks3-server[964]: 2026/09/21 09:52:29 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete3867lb # [ 29.224116] niks3-server[964]: 2026/09/21 09:52:29 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls3868lb # [ 29.235396] niks3-server[964]: 2026/09/21 09:52:29 INFO Completed upload id=13869worker2 # [ 29.293620] nix-grpc-daemon[847]: time=2026-09-21T09:52:29.884Z level=INFO msg="Upload complete. (511ms)"3870lb # [ 29.256996] niks3-server[964]: 2026/09/21 09:52:29 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo3871worker2 # [ 29.304597] nix-grpc-daemon[769]: ts=2026-09-21T09:52:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=0 paths=1 nar_bytes_in=5043872worker2 # [ 29.325369] nix-grpc-daemon[769]: ts=2026-09-21T09:52:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562483873worker2 # [ 29.351070] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)3874worker2 # [ 29.360855] nix-daemon[834]: reaped child process 845, status = succeeded3875client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...3876worker2 # [ 29.382022] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)3877client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3878client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds3879client # farm-a> LOG-farm-a-t13880worker2 # [ 30.669283] nix-daemon[834]: reaped child process 872, status = succeeded3881worker2 # [ 30.821967] nix-daemon[834]: accepted connection from pid 888, user nix-grpc-daemon (trusted)3882worker2 # [ 30.846792] nix-daemon[834]: reaped child process 891, status = succeeded3883lb # [ 30.790908] niks3-server[964]: 2026/09/21 09:52:31 INFO Received uploads request method=POST path=/api/pending_closures3884worker2 # [ 30.872621] nix-grpc-daemon[847]: time=2026-09-21T09:52:31.462Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3885worker2 # [ 30.874398] nix-grpc-daemon[847]: time=2026-09-21T09:52:31.462Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"3886lb # [ 30.875012] niks3-server[964]: 2026/09/21 09:52:31 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst3887lb # [ 30.886754] niks3-server[964]: 2026/09/21 09:52:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign3888worker2 # [ 30.944452] nix-grpc-daemon[847]: time=2026-09-21T09:52:31.534Z level=INFO msg="Uploading 1 narinfos"3889lb # [ 30.897520] niks3-server[964]: 2026/09/21 09:52:31 INFO Signed narinfos id=2 count=13890lb # [ 30.902865] niks3-server[964]: 2026/09/21 09:52:31 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3891lb # [ 30.908696] niks3-server[964]: 2026/09/21 09:52:31 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls3892lb # [ 30.915680] niks3-server[964]: 2026/09/21 09:52:31 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete3893worker2 # [ 30.978859] nix-grpc-daemon[847]: time=2026-09-21T09:52:31.570Z level=INFO msg="Upload complete. (375ms)"3894lb # [ 30.923376] niks3-server[964]: 2026/09/21 09:52:31 INFO Completed upload id=23895lb # [ 30.925933] niks3-server[964]: 2026/09/21 09:52:31 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo3896worker2 # [ 30.991109] nix-grpc-daemon[769]: ts=2026-09-21T09:52:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=13897worker1 # [ 30.990829] nix-grpc-daemon[765]: ts=2026-09-21T09:52:31Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=03898worker1 # [ 30.997529] nix-grpc-daemon[765]: ts=2026-09-21T09:52:31Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=13899worker2 # [ 31.024432] nix-daemon[834]: reaped child process 870, status = succeeded3900worker2 # [ 31.051798] nix-grpc-daemon[769]: ts=2026-09-21T09:52:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562483901worker2 # [ 31.065027] nix-grpc-daemon[769]: ts=2026-09-21T09:52:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562483902worker2 # [ 31.080208] nix-grpc-daemon[847]: time=2026-09-21T09:52:31.671Z level=INFO msg="All 1 paths already cached"3903worker2 # [ 31.087952] nix-grpc-daemon[769]: ts=2026-09-21T09:52:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:562483904client # copying 1 paths to 'lb:50051'...3905client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...3906worker2 # [ 31.116869] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)3907worker2 # [ 31.287604] nix-daemon[834]: accepted connection from pid 896, user nix-grpc-daemon (trusted)3908worker2 # [ 31.317559] nix-daemon[834]: reaped child process 899, status = succeeded3909worker2 # [ 31.446379] nix-daemon[834]: accepted connection from pid 901, user nix-grpc-daemon (trusted)3910worker2 # [ 31.473033] nix-daemon[834]: reaped child process 904, status = succeeded3911lb # [ 31.412608] niks3-server[964]: 2026/09/21 09:52:32 INFO Received uploads request method=POST path=/api/pending_closures3912worker2 # [ 31.489258] nix-grpc-daemon[847]: time=2026-09-21T09:52:32.080Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"3913worker2 # [ 31.490762] nix-grpc-daemon[847]: time=2026-09-21T09:52:32.080Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"3914lb # [ 31.468246] niks3-server[964]: 2026/09/21 09:52:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign3915worker2 # [ 31.535258] nix-grpc-daemon[847]: time=2026-09-21T09:52:32.126Z level=INFO msg="Uploading 1 narinfos"3916lb # [ 31.481257] niks3-server[964]: 2026/09/21 09:52:32 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst3917lb # [ 31.488597] niks3-server[964]: 2026/09/21 09:52:32 INFO Signed narinfos id=3 count=13918lb # [ 31.490245] niks3-server[964]: 2026/09/21 09:52:32 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls3919lb # [ 31.504357] niks3-server[964]: 2026/09/21 09:52:32 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete3920lb # [ 31.511219] niks3-server[964]: 2026/09/21 09:52:32 INFO Completed upload id=33921worker2 # [ 31.569156] nix-grpc-daemon[847]: time=2026-09-21T09:52:32.160Z level=INFO msg="Upload complete. (440ms)"3922lb # [ 31.513542] niks3-server[964]: 2026/09/21 09:52:32 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo3923worker2 # [ 31.580098] nix-grpc-daemon[769]: ts=2026-09-21T09:52:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=0 paths=1 nar_bytes_in=6243924worker2 # [ 31.593018] nix-grpc-daemon[769]: ts=2026-09-21T09:52:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562483925worker2 # [ 31.612378] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)3926worker2 # [ 31.620422] nix-daemon[834]: reaped child process 894, status = succeeded3927client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...3928worker2 # [ 31.635387] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)3929client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv3930client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds3931client # farm-b> LOG-farm-b-t13932worker2 # [ 32.874459] nix-daemon[834]: reaped child process 908, status = succeeded3933worker2 # [ 32.983055] nix-daemon[834]: accepted connection from pid 924, user nix-grpc-daemon (trusted)3934worker2 # [ 33.008603] nix-daemon[834]: reaped child process 927, status = succeeded3935lb # [ 32.950245] niks3-server[964]: 2026/09/21 09:52:33 INFO Received uploads request method=POST path=/api/pending_closures3936worker2 # [ 33.019947] nix-grpc-daemon[847]: time=2026-09-21T09:52:33.610Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"3937worker2 # [ 33.021708] nix-grpc-daemon[847]: time=2026-09-21T09:52:33.610Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"3938lb # [ 32.991616] niks3-server[964]: 2026/09/21 09:52:33 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst3939lb # [ 33.005571] niks3-server[964]: 2026/09/21 09:52:33 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv3940worker2 # [ 33.070387] nix-grpc-daemon[847]: time=2026-09-21T09:52:33.661Z level=INFO msg="Uploading 1 narinfos"3941lb # [ 33.012419] niks3-server[964]: 2026/09/21 09:52:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign3942lb # [ 33.025163] niks3-server[964]: 2026/09/21 09:52:33 INFO Signed narinfos id=4 count=13943lb # [ 33.030629] niks3-server[964]: 2026/09/21 09:52:33 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls3944lb # [ 33.044282] niks3-server[964]: 2026/09/21 09:52:33 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete3945lb # [ 33.052951] niks3-server[964]: 2026/09/21 09:52:33 INFO Completed upload id=43946worker2 # [ 33.110714] nix-grpc-daemon[847]: time=2026-09-21T09:52:33.701Z level=INFO msg="Upload complete. (259ms)"3947lb # [ 33.056080] niks3-server[964]: 2026/09/21 09:52:33 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo3948worker2 # [ 33.125377] nix-grpc-daemon[769]: ts=2026-09-21T09:52:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=13949worker1 # [ 33.113801] nix-grpc-daemon[765]: ts=2026-09-21T09:52:33Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=03950worker1 # [ 33.120477] nix-grpc-daemon[765]: ts=2026-09-21T09:52:33Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=13951worker2 # [ 33.147269] nix-daemon[834]: reaped child process 906, status = succeeded3952worker2 # [ 33.182299] nix-grpc-daemon[769]: ts=2026-09-21T09:52:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562483953worker2 # [ 33.204948] nix-grpc-daemon[769]: ts=2026-09-21T09:52:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562483954worker2 # [ 33.220516] nix-grpc-daemon[847]: time=2026-09-21T09:52:33.811Z level=INFO msg="All 2 paths already cached"3955client # copying 1 paths to 'lb:50051'...3956worker2 # [ 33.231411] nix-grpc-daemon[769]: ts=2026-09-21T09:52:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:562483957client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...3958worker2 # [ 33.254432] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)3959worker2 # [ 33.424986] nix-daemon[834]: accepted connection from pid 931, user nix-grpc-daemon (trusted)3960worker2 # [ 33.445686] nix-daemon[834]: reaped child process 934, status = succeeded3961worker2 # [ 33.552606] nix-daemon[834]: accepted connection from pid 936, user nix-grpc-daemon (trusted)3962worker2 # [ 33.571522] nix-daemon[834]: reaped child process 939, status = succeeded3963lb # [ 33.517414] niks3-server[964]: 2026/09/21 09:52:34 INFO Received uploads request method=POST path=/api/pending_closures3964worker2 # [ 33.585642] nix-grpc-daemon[847]: time=2026-09-21T09:52:34.176Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"3965worker2 # [ 33.587203] nix-grpc-daemon[847]: time=2026-09-21T09:52:34.176Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"3966lb # [ 33.549610] niks3-server[964]: 2026/09/21 09:52:34 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst3967worker2 # [ 33.628739] nix-grpc-daemon[847]: time=2026-09-21T09:52:34.219Z level=INFO msg="Uploading 1 narinfos"3968lb # [ 33.563769] niks3-server[964]: 2026/09/21 09:52:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign3969lb # [ 33.577162] niks3-server[964]: 2026/09/21 09:52:34 INFO Signed narinfos id=5 count=13970lb # [ 33.584251] niks3-server[964]: 2026/09/21 09:52:34 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls3971lb # [ 33.596885] niks3-server[964]: 2026/09/21 09:52:34 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete3972worker2 # [ 33.658598] nix-grpc-daemon[847]: time=2026-09-21T09:52:34.249Z level=INFO msg="Upload complete. (384ms)"3973lb # [ 33.603252] niks3-server[964]: 2026/09/21 09:52:34 INFO Completed upload id=53974lb # [ 33.606984] niks3-server[964]: 2026/09/21 09:52:34 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo3975worker2 # [ 33.668361] nix-grpc-daemon[769]: ts=2026-09-21T09:52:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=0 paths=1 nar_bytes_in=7523976worker2 # [ 33.683758] nix-grpc-daemon[769]: ts=2026-09-21T09:52:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562483977worker2 # [ 33.693102] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)3978worker2 # [ 33.696193] nix-daemon[834]: reaped child process 929, status = succeeded3979client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...3980client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3981worker2 # [ 33.710682] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)3982client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds3983client # farm-top> LOG-farm-top-t13984worker2 # [ 34.980890] nix-daemon[834]: reaped child process 943, status = succeeded3985worker2 # [ 35.090554] nix-daemon[834]: accepted connection from pid 960, user nix-grpc-daemon (trusted)3986worker2 # [ 35.110470] nix-daemon[834]: reaped child process 963, status = succeeded3987lb # [ 35.058544] niks3-server[964]: 2026/09/21 09:52:35 INFO Received uploads request method=POST path=/api/pending_closures3988worker2 # [ 35.127524] nix-grpc-daemon[847]: time=2026-09-21T09:52:35.717Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"3989worker2 # [ 35.129318] nix-grpc-daemon[847]: time=2026-09-21T09:52:35.717Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"3990lb # [ 35.104236] niks3-server[964]: 2026/09/21 09:52:35 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3991lb # [ 35.114423] niks3-server[964]: 2026/09/21 09:52:35 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst3992worker2 # [ 35.181235] nix-grpc-daemon[847]: time=2026-09-21T09:52:35.772Z level=INFO msg="Uploading 1 narinfos"3993lb # [ 35.125834] niks3-server[964]: 2026/09/21 09:52:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign3994lb # [ 35.133473] niks3-server[964]: 2026/09/21 09:52:35 INFO Signed narinfos id=6 count=13995lb # [ 35.138125] niks3-server[964]: 2026/09/21 09:52:35 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls3996lb # [ 35.145926] niks3-server[964]: 2026/09/21 09:52:35 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete3997lb # [ 35.152964] niks3-server[964]: 2026/09/21 09:52:35 INFO Completed upload id=63998worker2 # [ 35.210639] nix-grpc-daemon[847]: time=2026-09-21T09:52:35.801Z level=INFO msg="Upload complete. (289ms)"3999lb # [ 35.155617] niks3-server[964]: 2026/09/21 09:52:35 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4000worker2 # [ 35.221112] nix-grpc-daemon[769]: ts=2026-09-21T09:52:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14001worker2 # [ 35.239847] nix-daemon[834]: reaped child process 941, status = succeeded4002client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 7.16 seconds)4003client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4004client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.20 seconds)4005client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4006client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds)4007client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4008client # copying 3 paths...4009client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4010client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4011client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4012client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.21 seconds)4013(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.70 seconds)4014subtest: repeat is answered Cached by the scheduler without building4015worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4016worker1 # finding garbage collector roots...4017worker1 # deleting specified paths...4018worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4019worker1 # deleting unused links...4020worker1 # note: hard linking is currently saving 0.0 KiB4021worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.25 seconds)4022worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4023worker2 # finding garbage collector roots...4024worker2 # deleting specified paths...4025worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4026worker2 # deleting unused links...4027worker2 # note: hard linking is currently saving 0.0 KiB4028worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds)4029client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&24030worker2 # [ 36.505204] nix-grpc-daemon[769]: ts=2026-09-21T09:52:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:562484031client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4032client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4033worker1 # [ 36.507892] nix-grpc-daemon[765]: ts=2026-09-21T09:52:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397924034worker1 # [ 36.540500] nix-grpc-daemon[765]: ts=2026-09-21T09:52:37Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14035client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.31 seconds)4036client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4037client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.15 seconds)4038lb: must succeed: curl -sf localhost:9901/clusters4039lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4040lb: must succeed: curl -sf localhost:9901/clusters4041lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4042worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4043worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4044(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.21 seconds)4045subtest: inputs already in the cache are not sent to the scheduler4046lb: must succeed: curl -sf localhost:9901/clusters4047lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4048lb: must succeed: curl -sf localhost:9901/clusters4049lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4050worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4051worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4052client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&24053worker1 # [ 37.398935] nix-grpc-daemon[765]: ts=2026-09-21T09:52:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:397924054client # don't know how to build these paths:4055client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4056worker2 # [ 37.426759] nix-grpc-daemon[769]: ts=2026-09-21T09:52:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562484057worker2 # [ 37.438391] nix-grpc-daemon[847]: time=2026-09-21T09:52:38.028Z level=INFO msg="All 2 paths already cached"4058worker1 # [ 37.437598] nix-grpc-daemon[765]: ts=2026-09-21T09:52:38Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04059worker1 # [ 37.443424] nix-grpc-daemon[765]: ts=2026-09-21T09:52:38Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14060worker2 # [ 37.501887] nix-grpc-daemon[769]: ts=2026-09-21T09:52:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562484061worker2 # [ 37.518443] nix-grpc-daemon[769]: ts=2026-09-21T09:52:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562484062worker2 # [ 37.533854] nix-grpc-daemon[847]: time=2026-09-21T09:52:38.125Z level=INFO msg="All 2 paths already cached"4063client # copying 1 paths to 'lb:50051'...4064client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4065worker2 # [ 37.543662] nix-grpc-daemon[769]: ts=2026-09-21T09:52:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:562484066worker2 # [ 37.557527] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4067worker2 # [ 37.744801] nix-daemon[834]: accepted connection from pid 980, user nix-grpc-daemon (trusted)4068worker2 # [ 37.789486] nix-daemon[834]: reaped child process 983, status = succeeded4069worker2 # [ 37.884175] nix-daemon[834]: accepted connection from pid 985, user nix-grpc-daemon (trusted)4070worker2 # [ 37.905798] nix-daemon[834]: reaped child process 989, status = succeeded4071lb # [ 37.847453] niks3-server[964]: 2026/09/21 09:52:38 INFO Received uploads request method=POST path=/api/pending_closures4072worker2 # [ 37.915072] nix-grpc-daemon[847]: time=2026-09-21T09:52:38.506Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4073worker2 # [ 37.916658] nix-grpc-daemon[847]: time=2026-09-21T09:52:38.506Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4074lb # [ 37.900256] niks3-server[964]: 2026/09/21 09:52:38 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4075lb # [ 37.919115] niks3-server[964]: 2026/09/21 09:52:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4076worker2 # [ 37.980979] nix-grpc-daemon[847]: time=2026-09-21T09:52:38.571Z level=INFO msg="Uploading 1 narinfos"4077lb # [ 37.929033] niks3-server[964]: 2026/09/21 09:52:38 INFO Signed narinfos id=7 count=14078lb # [ 37.930733] niks3-server[964]: 2026/09/21 09:52:38 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4079worker2 # [ 38.005137] nix-grpc-daemon[847]: time=2026-09-21T09:52:38.596Z level=INFO msg="Upload complete. (429ms)"4080lb # [ 37.942296] niks3-server[964]: 2026/09/21 09:52:38 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4081lb # [ 37.952363] niks3-server[964]: 2026/09/21 09:52:38 INFO Completed upload id=74082lb # [ 37.958087] niks3-server[964]: 2026/09/21 09:52:38 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4083worker2 # [ 38.016378] nix-grpc-daemon[769]: ts=2026-09-21T09:52:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=0 paths=1 nar_bytes_in=7524084worker2 # [ 38.032489] nix-grpc-daemon[769]: ts=2026-09-21T09:52:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562484085worker2 # [ 38.040275] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4086worker2 # [ 38.048358] nix-daemon[834]: reaped child process 978, status = succeeded4087client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4088worker2 # [ 38.057919] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4089client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4090lb # [ 38.628952] envoy[900]: conn peer=[2001:db8:1::1]:38718 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9575 tx=4059 ms=69514091lb # [ 38.633854] envoy[900]: conn peer=[2001:db8:1::1]:55924 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=764092client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4093client # farm-top> LOG-farm-top-t1b4094worker2 # [ 39.159454] nix-daemon[834]: reaped child process 993, status = succeeded4095worker2 # [ 39.231248] nix-daemon[834]: accepted connection from pid 1009, user nix-grpc-daemon (trusted)4096worker2 # [ 39.250895] nix-daemon[834]: reaped child process 1012, status = succeeded4097lb # [ 39.198049] niks3-server[964]: 2026/09/21 09:52:39 INFO Received uploads request method=POST path=/api/pending_closures4098worker2 # [ 39.268421] nix-grpc-daemon[847]: time=2026-09-21T09:52:39.858Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4099worker2 # [ 39.270120] nix-grpc-daemon[847]: time=2026-09-21T09:52:39.858Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4100lb # [ 39.245383] niks3-server[964]: 2026/09/21 09:52:39 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4101lb # [ 39.259422] niks3-server[964]: 2026/09/21 09:52:39 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4102worker2 # [ 39.325969] nix-grpc-daemon[847]: time=2026-09-21T09:52:39.916Z level=INFO msg="Uploading 1 narinfos"4103lb # [ 39.277172] niks3-server[964]: 2026/09/21 09:52:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4104lb # [ 39.288916] niks3-server[964]: 2026/09/21 09:52:39 INFO Signed narinfos id=8 count=14105lb # [ 39.291170] niks3-server[964]: 2026/09/21 09:52:39 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4106lb # [ 39.300367] niks3-server[964]: 2026/09/21 09:52:39 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4107lb # [ 39.307333] niks3-server[964]: 2026/09/21 09:52:39 INFO Completed upload id=84108worker2 # [ 39.364760] nix-grpc-daemon[847]: time=2026-09-21T09:52:39.955Z level=INFO msg="Upload complete. (297ms)"4109lb # [ 39.310174] niks3-server[964]: 2026/09/21 09:52:39 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4110worker2 # [ 39.377126] nix-grpc-daemon[769]: ts=2026-09-21T09:52:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14111worker2 # [ 39.398713] nix-daemon[834]: reaped child process 991, status = succeeded4112client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.22 seconds)4113client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4114client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.17 seconds)4115lb: must succeed: curl -sf localhost:9901/clusters4116lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4117lb: must succeed: curl -sf localhost:9901/clusters4118lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4119worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4120worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4121(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.73 seconds)4122subtest: two clients wanting the same drv build it once4123client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4124client # [ 39.841414] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4125client # Running as unit: dedup1.service; invocation ID: 19bd0c95982442f9a6d01860003b65524126client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.09 seconds)4127client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4128client # [ 40.049516] nix[927]: don't know how to build these paths:4129client # [ 40.050550] nix[927]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4130worker1 # [ 40.019285] nix-grpc-daemon[765]: ts=2026-09-21T09:52:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:397924131worker1 # [ 40.032284] nix-grpc-daemon[765]: ts=2026-09-21T09:52:40Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04132worker1 # [ 40.038446] nix-grpc-daemon[765]: ts=2026-09-21T09:52:40Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14133worker1 # [ 40.047271] nix-grpc-daemon[765]: ts=2026-09-21T09:52:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397924134worker1 # [ 40.065606] nix-grpc-daemon[765]: ts=2026-09-21T09:52:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397924135client # [ 40.108485] nix[927]: copying 1 paths to 'lb:50051'...4136client # [ 40.111382] nix[927]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4137worker1 # [ 40.081146] nix-grpc-daemon[765]: ts=2026-09-21T09:52:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397924138worker1 # [ 40.097952] nix-daemon[844]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4139worker1 # [ 40.130683] nix-grpc-daemon[765]: ts=2026-09-21T09:52:40Z level=info event=niks3_push_started pid=8984140worker1 # [ 40.299242] nix-grpc-daemon[898]: time=2026-09-21T09:52:40.894Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""4141worker1 # [ 40.411659] nix-daemon[844]: accepted connection from pid 911, user nix-grpc-daemon (trusted)4142worker1 # [ 40.432902] nix-daemon[844]: reaped child process 914, status = succeeded4143worker1 # [ 40.533564] nix-daemon[844]: accepted connection from pid 916, user nix-grpc-daemon (trusted)4144worker1 # [ 40.549872] nix-daemon[844]: reaped child process 919, status = succeeded4145lb # [ 40.519059] niks3-server[964]: 2026/09/21 09:52:41 INFO Received uploads request method=POST path=/api/pending_closures4146worker1 # [ 40.570806] nix-grpc-daemon[898]: time=2026-09-21T09:52:41.165Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4147worker1 # [ 40.572709] nix-grpc-daemon[898]: time=2026-09-21T09:52:41.165Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4148lb # [ 40.559433] niks3-server[964]: 2026/09/21 09:52:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4149worker1 # [ 40.611207] nix-grpc-daemon[898]: time=2026-09-21T09:52:41.206Z level=INFO msg="Uploading 1 narinfos"4150lb # [ 40.575425] niks3-server[964]: 2026/09/21 09:52:41 INFO Signed narinfos id=9 count=14151lb # [ 40.583274] niks3-server[964]: 2026/09/21 09:52:41 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4152worker1 # [ 40.639704] nix-grpc-daemon[898]: time=2026-09-21T09:52:41.235Z level=INFO msg="Upload complete. (337ms)"4153lb # [ 40.598467] niks3-server[964]: 2026/09/21 09:52:41 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4154worker1 # [ 40.647191] nix-grpc-daemon[765]: ts=2026-09-21T09:52:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39792 duration_s=0 paths=1 nar_bytes_in=4324155lb # [ 40.608326] niks3-server[964]: 2026/09/21 09:52:41 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4156lb # [ 40.613828] niks3-server[964]: 2026/09/21 09:52:41 INFO Completed upload id=94157lb # [ 40.616765] niks3-server[964]: 2026/09/21 09:52:41 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4158worker1 # [ 40.661000] nix-grpc-daemon[765]: ts=2026-09-21T09:52:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397924159worker1 # [ 40.667095] nix-daemon[844]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4160worker1 # [ 40.671959] nix-daemon[844]: reaped child process 896, status = succeeded4161client # [ 40.711635] nix[927]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4162worker1 # [ 40.685193] nix-daemon[844]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4163client # [ 40.951228] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4164client # Running as unit: dedup2.service; invocation ID: 0dab8ccceb11407d8a2da07610044ef34165client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.12 seconds)4166client # [ 41.158582] nix[943]: don't know how to build these paths:4167client # [ 41.159599] nix[943]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4168worker2 # [ 41.148653] nix-grpc-daemon[769]: ts=2026-09-21T09:52:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:562484169worker1 # [ 41.142401] nix-grpc-daemon[765]: ts=2026-09-21T09:52:41Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04170client # [ 41.179905] nix[943]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4171worker1 # [ 41.150173] nix-grpc-daemon[765]: ts=2026-09-21T09:52:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397924172client: must succeed: sleep 24173client: (finished: must succeed: sleep 2, in 2.05 seconds)4174client: waiting for success: ! systemctl is-active dedup1 dedup24175worker1 # [ 44.579043] nix-daemon[844]: reaped child process 923, status = succeeded4176worker1 # [ 44.651041] nix-daemon[844]: accepted connection from pid 957, user nix-grpc-daemon (trusted)4177worker1 # [ 44.669507] nix-daemon[844]: reaped child process 960, status = succeeded4178lb # [ 44.634222] niks3-server[964]: 2026/09/21 09:52:45 INFO Received uploads request method=POST path=/api/pending_closures4179worker1 # [ 44.688084] nix-grpc-daemon[898]: time=2026-09-21T09:52:45.281Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4180worker1 # [ 44.689563] nix-grpc-daemon[898]: time=2026-09-21T09:52:45.281Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4181lb # [ 44.674348] niks3-server[964]: 2026/09/21 09:52:45 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4182lb # [ 44.685636] niks3-server[964]: 2026/09/21 09:52:45 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4183worker1 # [ 44.736875] nix-grpc-daemon[898]: time=2026-09-21T09:52:45.331Z level=INFO msg="Uploading 1 narinfos"4184lb # [ 44.694686] niks3-server[964]: 2026/09/21 09:52:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4185lb # [ 44.703833] niks3-server[964]: 2026/09/21 09:52:45 INFO Signed narinfos id=10 count=14186lb # [ 44.709309] niks3-server[964]: 2026/09/21 09:52:45 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4187worker1 # [ 44.760200] nix-grpc-daemon[898]: time=2026-09-21T09:52:45.356Z level=INFO msg="Upload complete. (260ms)"4188lb # [ 44.720056] niks3-server[964]: 2026/09/21 09:52:45 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4189lb # [ 44.726645] niks3-server[964]: 2026/09/21 09:52:45 INFO Completed upload id=104190lb # [ 44.730137] niks3-server[964]: 2026/09/21 09:52:45 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4191worker1 # [ 44.776943] nix-grpc-daemon[765]: ts=2026-09-21T09:52:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39792 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14192worker1 # [ 44.788348] nix-grpc-daemon[765]: ts=2026-09-21T09:52:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39792 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14193worker1 # [ 44.796417] nix-daemon[844]: reaped child process 921, status = succeeded4194client # [ 44.857238] systemd[1]: dedup2.service: Deactivated successfully.4195client # [ 44.860673] systemd[1]: dedup2.service: Consumed 185ms CPU time over 3.909s wall clock time, 33.2M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.4196client # [ 44.868193] systemd[1]: dedup1.service: Deactivated successfully.4197client # [ 44.871412] systemd[1]: dedup1.service: Consumed 181ms CPU time over 5.027s wall clock time, 41.5M memory peak, 7.1K incoming IP traffic, 8.5K outgoing IP traffic.4198client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.16 seconds)4199client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4200client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4201worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4202worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4203worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4204worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4205worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4206worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4207worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4208worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4209(finished: subtest: two clients wanting the same drv build it once, in 6.22 seconds)4210subtest: build hook: nix-daemon with builders = grpc://lb4211client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4212client # [ 46.174843] systemd[1]: Started Nix Daemon.4213client # [ 46.299706] nix-daemon[987]: accepted connection from pid 984, user root (trusted)4214worker1 # [ 46.419547] nix-grpc-daemon[765]: ts=2026-09-21T09:52:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:397924215worker2 # [ 46.440180] nix-grpc-daemon[769]: ts=2026-09-21T09:52:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562484216worker1 # [ 46.438229] nix-grpc-daemon[765]: ts=2026-09-21T09:52:47Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04217worker1 # [ 46.444272] nix-grpc-daemon[765]: ts=2026-09-21T09:52:47Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14218worker2 # [ 46.501367] nix-grpc-daemon[769]: ts=2026-09-21T09:52:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562484219worker2 # [ 46.517520] nix-grpc-daemon[769]: ts=2026-09-21T09:52:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562484220worker2 # [ 46.533243] nix-grpc-daemon[769]: ts=2026-09-21T09:52:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:562484221worker2 # [ 46.550459] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4222worker2 # [ 46.722628] nix-daemon[834]: accepted connection from pid 1039, user nix-grpc-daemon (trusted)4223worker2 # [ 46.740714] nix-daemon[834]: reaped child process 1042, status = succeeded4224worker2 # [ 46.844614] nix-daemon[834]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)4225worker2 # [ 46.862697] nix-daemon[834]: reaped child process 1047, status = succeeded4226lb # [ 46.810690] niks3-server[964]: 2026/09/21 09:52:47 INFO Received uploads request method=POST path=/api/pending_closures4227worker2 # [ 46.878592] nix-grpc-daemon[847]: time=2026-09-21T09:52:47.469Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4228worker2 # [ 46.880240] nix-grpc-daemon[847]: time=2026-09-21T09:52:47.469Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4229lb # [ 46.842859] niks3-server[964]: 2026/09/21 09:52:47 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4230worker2 # [ 46.913788] nix-grpc-daemon[847]: time=2026-09-21T09:52:47.504Z level=INFO msg="Uploading 1 narinfos"4231lb # [ 46.856937] niks3-server[964]: 2026/09/21 09:52:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4232lb # [ 46.867694] niks3-server[964]: 2026/09/21 09:52:47 INFO Signed narinfos id=11 count=14233lb # [ 46.874227] niks3-server[964]: 2026/09/21 09:52:47 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4234worker2 # [ 46.941690] nix-grpc-daemon[847]: time=2026-09-21T09:52:47.532Z level=INFO msg="Upload complete. (373ms)"4235lb # [ 46.885082] niks3-server[964]: 2026/09/21 09:52:47 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4236lb # [ 46.893068] niks3-server[964]: 2026/09/21 09:52:47 INFO Completed upload id=114237worker2 # [ 46.948296] nix-grpc-daemon[769]: ts=2026-09-21T09:52:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=0 paths=1 nar_bytes_in=5124238lb # [ 46.896766] niks3-server[964]: 2026/09/21 09:52:47 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4239worker2 # [ 46.962680] nix-grpc-daemon[769]: ts=2026-09-21T09:52:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562484240worker2 # [ 46.968731] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4241worker2 # [ 46.973396] nix-daemon[834]: reaped child process 1037, status = succeeded4242worker2 # [ 46.988145] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4243lb # [ 48.629155] envoy[900]: conn peer=[2001:db8:1::1]:55936 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1809 ms=19764244lb # [ 48.634170] envoy[900]: conn peer=[2001:db8:1::1]:47260 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1085 ms=36624245lb # [ 48.639016] envoy[900]: conn peer=[2001:db8:1::1]:55950 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1427 ms=47734246worker2 # [ 48.865270] nix-daemon[834]: reaped child process 1051, status = succeeded4247worker2 # [ 48.980181] nix-daemon[834]: accepted connection from pid 1067, user nix-grpc-daemon (trusted)4248worker2 # [ 49.003280] nix-daemon[834]: reaped child process 1070, status = succeeded4249lb # [ 48.947952] niks3-server[964]: 2026/09/21 09:52:49 INFO Received uploads request method=POST path=/api/pending_closures4250worker2 # [ 49.016433] nix-grpc-daemon[847]: time=2026-09-21T09:52:49.605Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4251worker2 # [ 49.018103] nix-grpc-daemon[847]: time=2026-09-21T09:52:49.605Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4252lb # [ 48.986299] niks3-server[964]: 2026/09/21 09:52:49 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4253lb # [ 49.008162] niks3-server[964]: 2026/09/21 09:52:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4254worker2 # [ 49.071994] nix-grpc-daemon[847]: time=2026-09-21T09:52:49.662Z level=INFO msg="Uploading 1 narinfos"4255lb # [ 49.018623] niks3-server[964]: 2026/09/21 09:52:49 INFO Signed narinfos id=12 count=14256lb # [ 49.023907] niks3-server[964]: 2026/09/21 09:52:49 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4257lb # [ 49.035308] niks3-server[964]: 2026/09/21 09:52:49 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4258worker2 # [ 49.099558] nix-grpc-daemon[847]: time=2026-09-21T09:52:49.690Z level=INFO msg="Upload complete. (299ms)"4259lb # [ 49.042873] niks3-server[964]: 2026/09/21 09:52:49 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4260lb # [ 49.046797] niks3-server[964]: 2026/09/21 09:52:49 INFO Completed upload id=124261lb # [ 49.052728] niks3-server[964]: 2026/09/21 09:52:49 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4262worker2 # [ 49.113193] nix-grpc-daemon[769]: ts=2026-09-21T09:52:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=2 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14263worker1 # [ 49.102723] nix-grpc-daemon[765]: ts=2026-09-21T09:52:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:397924264worker1 # [ 49.111687] nix-grpc-daemon[765]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4265worker2 # [ 49.130866] nix-daemon[834]: reaped child process 1049, status = succeeded4266worker2 # [ 49.175003] nix-grpc-daemon[769]: ts=2026-09-21T09:52:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:562484267worker2 # [ 49.181217] nix-grpc-daemon[769]: ts=2026-09-21T09:52:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=0 paths=1 nar_bytes_out=1284268worker1 # [ 49.408995] nix-grpc-daemon[765]: ts=2026-09-21T09:52:50Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:397924269worker2 # [ 49.430053] nix-grpc-daemon[769]: ts=2026-09-21T09:52:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562484270worker2 # [ 49.439797] nix-grpc-daemon[847]: time=2026-09-21T09:52:50.031Z level=INFO msg="All 1 paths already cached"4271worker1 # [ 49.441897] nix-grpc-daemon[765]: ts=2026-09-21T09:52:50Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04272worker1 # [ 49.448893] nix-grpc-daemon[765]: ts=2026-09-21T09:52:50Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14273worker2 # [ 49.468146] nix-grpc-daemon[769]: ts=2026-09-21T09:52:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562484274worker2 # [ 49.483693] nix-grpc-daemon[769]: ts=2026-09-21T09:52:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562484275worker2 # [ 49.498059] nix-grpc-daemon[847]: time=2026-09-21T09:52:50.089Z level=INFO msg="All 1 paths already cached"4276worker2 # [ 49.507462] nix-grpc-daemon[769]: ts=2026-09-21T09:52:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:562484277worker2 # [ 49.523110] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4278worker2 # [ 49.698026] nix-daemon[834]: accepted connection from pid 1074, user nix-grpc-daemon (trusted)4279worker2 # [ 49.717781] nix-daemon[834]: reaped child process 1077, status = succeeded4280worker2 # [ 49.820993] nix-daemon[834]: accepted connection from pid 1079, user nix-grpc-daemon (trusted)4281worker2 # [ 49.839966] nix-daemon[834]: reaped child process 1082, status = succeeded4282lb # [ 49.785111] niks3-server[964]: 2026/09/21 09:52:50 INFO Received uploads request method=POST path=/api/pending_closures4283worker2 # [ 49.851624] nix-grpc-daemon[847]: time=2026-09-21T09:52:50.442Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4284worker2 # [ 49.853288] nix-grpc-daemon[847]: time=2026-09-21T09:52:50.442Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4285lb # [ 49.816191] niks3-server[964]: 2026/09/21 09:52:50 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4286worker2 # [ 49.884370] nix-grpc-daemon[847]: time=2026-09-21T09:52:50.475Z level=INFO msg="Uploading 1 narinfos"4287lb # [ 49.828483] niks3-server[964]: 2026/09/21 09:52:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4288lb # [ 49.840688] niks3-server[964]: 2026/09/21 09:52:50 INFO Signed narinfos id=13 count=14289lb # [ 49.843210] niks3-server[964]: 2026/09/21 09:52:50 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4290lb # [ 49.853705] niks3-server[964]: 2026/09/21 09:52:50 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4291lb # [ 49.862273] niks3-server[964]: 2026/09/21 09:52:50 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4292lb # [ 49.866560] niks3-server[964]: 2026/09/21 09:52:50 INFO Completed upload id=134293worker2 # [ 49.923861] nix-grpc-daemon[847]: time=2026-09-21T09:52:50.515Z level=INFO msg="Upload complete. (383ms)"4294worker2 # [ 49.933084] nix-grpc-daemon[769]: ts=2026-09-21T09:52:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=0 paths=1 nar_bytes_in=6244295worker2 # [ 49.947010] nix-grpc-daemon[769]: ts=2026-09-21T09:52:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562484296worker2 # [ 49.953789] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4297worker2 # [ 49.958374] nix-daemon[834]: reaped child process 1072, status = succeeded4298worker2 # [ 49.968554] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4299worker2 # [ 51.636890] nix-daemon[834]: reaped child process 1086, status = succeeded4300worker2 # [ 51.769796] nix-daemon[834]: accepted connection from pid 1102, user nix-grpc-daemon (trusted)4301worker2 # [ 51.789095] nix-daemon[834]: reaped child process 1105, status = succeeded4302lb # [ 51.736897] niks3-server[964]: 2026/09/21 09:52:52 INFO Received uploads request method=POST path=/api/pending_closures4303worker2 # [ 51.804573] nix-grpc-daemon[847]: time=2026-09-21T09:52:52.394Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4304worker2 # [ 51.807373] nix-grpc-daemon[847]: time=2026-09-21T09:52:52.394Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4305lb # [ 51.784979] niks3-server[964]: 2026/09/21 09:52:52 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4306lb # [ 51.796374] niks3-server[964]: 2026/09/21 09:52:52 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4307worker2 # [ 51.863132] nix-grpc-daemon[847]: time=2026-09-21T09:52:52.454Z level=INFO msg="Uploading 1 narinfos"4308lb # [ 51.805080] niks3-server[964]: 2026/09/21 09:52:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4309lb # [ 51.811260] niks3-server[964]: 2026/09/21 09:52:52 INFO Signed narinfos id=14 count=14310lb # [ 51.817248] niks3-server[964]: 2026/09/21 09:52:52 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4311lb # [ 51.824773] niks3-server[964]: 2026/09/21 09:52:52 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4312lb # [ 51.833377] niks3-server[964]: 2026/09/21 09:52:52 INFO Completed upload id=144313lb # [ 51.837616] niks3-server[964]: 2026/09/21 09:52:52 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4314worker2 # [ 51.890525] nix-grpc-daemon[847]: time=2026-09-21T09:52:52.481Z level=INFO msg="Upload complete. (308ms)"4315worker2 # [ 51.901456] nix-grpc-daemon[769]: ts=2026-09-21T09:52:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14316worker2 # [ 51.922450] nix-grpc-daemon[769]: ts=2026-09-21T09:52:52Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:562484317worker2 # [ 51.928104] nix-daemon[834]: reaped child process 1084, status = succeeded4318worker1 # [ 51.921500] nix-grpc-daemon[765]: ts=2026-09-21T09:52:52Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:397924319worker1 # [ 51.936310] nix-grpc-daemon[765]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4320worker1 # [ 51.954110] nix-grpc-daemon[765]: ts=2026-09-21T09:52:52Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39792 duration_s=0 paths=1 nar_bytes_out=1764321worker1 # [ 52.234843] nix-grpc-daemon[765]: ts=2026-09-21T09:52:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:397924322worker1 # [ 52.242683] nix-grpc-daemon[765]: ts=2026-09-21T09:52:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397924323worker1 # [ 52.250531] nix-grpc-daemon[898]: time=2026-09-21T09:52:52.844Z level=INFO msg="All 2 paths already cached"4324worker1 # [ 52.262260] nix-grpc-daemon[765]: ts=2026-09-21T09:52:52Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04325worker1 # [ 52.268752] nix-grpc-daemon[765]: ts=2026-09-21T09:52:52Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14326worker2 # [ 52.287835] nix-grpc-daemon[769]: ts=2026-09-21T09:52:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562484327worker2 # [ 52.303216] nix-grpc-daemon[769]: ts=2026-09-21T09:52:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562484328worker2 # [ 52.317053] nix-grpc-daemon[847]: time=2026-09-21T09:52:52.908Z level=INFO msg="All 2 paths already cached"4329worker2 # [ 52.326280] nix-grpc-daemon[769]: ts=2026-09-21T09:52:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:562484330worker2 # [ 52.340296] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4331worker2 # [ 52.504102] nix-daemon[834]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)4332worker2 # [ 52.521580] nix-daemon[834]: reaped child process 1112, status = succeeded4333worker2 # [ 52.623850] nix-daemon[834]: accepted connection from pid 1114, user nix-grpc-daemon (trusted)4334worker2 # [ 52.639920] nix-daemon[834]: reaped child process 1117, status = succeeded4335lb # [ 52.589380] niks3-server[964]: 2026/09/21 09:52:53 INFO Received uploads request method=POST path=/api/pending_closures4336worker2 # [ 52.656045] nix-grpc-daemon[847]: time=2026-09-21T09:52:53.247Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4337worker2 # [ 52.657490] nix-grpc-daemon[847]: time=2026-09-21T09:52:53.247Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4338lb # [ 52.616408] niks3-server[964]: 2026/09/21 09:52:53 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4339lb # [ 52.630258] niks3-server[964]: 2026/09/21 09:52:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4340worker2 # [ 52.695850] nix-grpc-daemon[847]: time=2026-09-21T09:52:53.286Z level=INFO msg="Uploading 1 narinfos"4341lb # [ 52.643899] niks3-server[964]: 2026/09/21 09:52:53 INFO Signed narinfos id=15 count=14342lb # [ 52.650117] niks3-server[964]: 2026/09/21 09:52:53 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4343lb # [ 52.662245] niks3-server[964]: 2026/09/21 09:52:53 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4344worker2 # [ 52.724668] nix-grpc-daemon[847]: time=2026-09-21T09:52:53.315Z level=INFO msg="Upload complete. (361ms)"4345lb # [ 52.669072] niks3-server[964]: 2026/09/21 09:52:53 INFO Completed upload id=154346lb # [ 52.672941] niks3-server[964]: 2026/09/21 09:52:53 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4347worker2 # [ 52.732107] nix-grpc-daemon[769]: ts=2026-09-21T09:52:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=0 paths=1 nar_bytes_in=7524348worker2 # [ 52.747487] nix-grpc-daemon[769]: ts=2026-09-21T09:52:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562484349worker2 # [ 52.754110] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4350worker2 # [ 52.758763] nix-daemon[834]: reaped child process 1107, status = succeeded4351worker2 # [ 52.769826] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4352worker2 # [ 53.634400] nix-daemon[834]: reaped child process 1121, status = succeeded4353worker2 # [ 53.761876] nix-daemon[834]: accepted connection from pid 1137, user nix-grpc-daemon (trusted)4354worker2 # [ 53.782485] nix-daemon[834]: reaped child process 1140, status = succeeded4355lb # [ 53.731300] niks3-server[964]: 2026/09/21 09:52:54 INFO Received uploads request method=POST path=/api/pending_closures4356worker2 # [ 53.800426] nix-grpc-daemon[847]: time=2026-09-21T09:52:54.389Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4357worker2 # [ 53.802126] nix-grpc-daemon[847]: time=2026-09-21T09:52:54.389Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4358lb # [ 53.771975] niks3-server[964]: 2026/09/21 09:52:54 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4359lb # [ 53.789720] niks3-server[964]: 2026/09/21 09:52:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4360worker2 # [ 53.852999] nix-grpc-daemon[847]: time=2026-09-21T09:52:54.443Z level=INFO msg="Uploading 1 narinfos"4361lb # [ 53.800380] niks3-server[964]: 2026/09/21 09:52:54 INFO Signed narinfos id=16 count=14362lb # [ 53.802518] niks3-server[964]: 2026/09/21 09:52:54 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4363lb # [ 53.813075] niks3-server[964]: 2026/09/21 09:52:54 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4364lb # [ 53.819797] niks3-server[964]: 2026/09/21 09:52:54 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4365lb # [ 53.825961] niks3-server[964]: 2026/09/21 09:52:54 INFO Completed upload id=164366worker2 # [ 53.881572] nix-grpc-daemon[847]: time=2026-09-21T09:52:54.472Z level=INFO msg="Upload complete. (300ms)"4367lb # [ 53.827606] niks3-server[964]: 2026/09/21 09:52:54 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4368worker2 # [ 53.901166] nix-grpc-daemon[769]: ts=2026-09-21T09:52:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14369worker2 # [ 53.916074] nix-daemon[834]: reaped child process 1119, status = succeeded4370worker1 # [ 53.923572] nix-grpc-daemon[765]: ts=2026-09-21T09:52:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:397924371worker1 # [ 53.937033] nix-grpc-daemon[765]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4372worker1 # [ 53.978853] nix-grpc-daemon[765]: ts=2026-09-21T09:52:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:397924373worker1 # [ 53.984433] nix-grpc-daemon[765]: ts=2026-09-21T09:52:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39792 duration_s=0 paths=1 nar_bytes_out=2324374client # [ 54.136509] nix-daemon[987]: reaped child process 989, status = succeeded4375client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.17 seconds)4376client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4377client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4378client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4379client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4380client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4381client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4382(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.29 seconds)4383subtest: an input only one worker has still reaches the builder4384client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4385client # this derivation will be built:4386client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4387client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4388client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.30 seconds)4389client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4390client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)4391worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4392worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)4393worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4394worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)4395client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24396client # [ 55.108368] nix-daemon[987]: accepted connection from pid 1069, user root (trusted)4397client # this derivation will be built:4398client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4399worker2 # [ 55.245259] nix-grpc-daemon[769]: ts=2026-09-21T09:52:55Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:562484400client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4401worker2 # [ 55.255138] nix-grpc-daemon[769]: ts=2026-09-21T09:52:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562484402client # copying 1 paths...4403client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4404worker1 # [ 55.256297] nix-grpc-daemon[765]: ts=2026-09-21T09:52:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397924405worker1 # [ 55.271032] nix-daemon[844]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4406worker1 # [ 55.429511] nix-daemon[844]: accepted connection from pid 987, user nix-grpc-daemon (trusted)4407worker1 # [ 55.450019] nix-daemon[844]: reaped child process 990, status = succeeded4408lb # [ 55.407427] niks3-server[964]: 2026/09/21 09:52:56 INFO Received uploads request method=POST path=/api/pending_closures4409worker1 # [ 55.459808] nix-grpc-daemon[898]: time=2026-09-21T09:52:56.054Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4410worker1 # [ 55.462712] nix-grpc-daemon[898]: time=2026-09-21T09:52:56.054Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4411worker1 # [ 55.489924] nix-grpc-daemon[898]: time=2026-09-21T09:52:56.085Z level=INFO msg="Uploading 1 narinfos"4412lb # [ 55.439131] niks3-server[964]: 2026/09/21 09:52:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4413lb # [ 55.452419] niks3-server[964]: 2026/09/21 09:52:56 INFO Signed narinfos id=17 count=14414lb # [ 55.461239] niks3-server[964]: 2026/09/21 09:52:56 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4415lb # [ 55.478221] niks3-server[964]: 2026/09/21 09:52:56 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4416worker1 # [ 55.525299] nix-grpc-daemon[898]: time=2026-09-21T09:52:56.121Z level=INFO msg="Upload complete. (239ms)"4417lb # [ 55.487376] niks3-server[964]: 2026/09/21 09:52:56 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4418lb # [ 55.492253] niks3-server[964]: 2026/09/21 09:52:56 INFO Completed upload id=174419lb # [ 55.495494] niks3-server[964]: 2026/09/21 09:52:56 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4420worker1 # [ 55.536323] nix-grpc-daemon[765]: ts=2026-09-21T09:52:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39792 duration_s=0 paths=1 nar_bytes_in=1204421worker1 # [ 55.554560] nix-grpc-daemon[765]: ts=2026-09-21T09:52:56Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04422worker1 # [ 55.565547] nix-grpc-daemon[765]: ts=2026-09-21T09:52:56Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14423worker1 # [ 55.571546] nix-daemon[844]: reaped child process 985, status = succeeded4424worker1 # [ 55.598088] nix-grpc-daemon[765]: ts=2026-09-21T09:52:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397924425worker1 # [ 55.614179] nix-grpc-daemon[765]: ts=2026-09-21T09:52:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397924426client # copying 2 paths to 'lb:50051'...4427client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4428worker1 # [ 55.632405] nix-grpc-daemon[765]: ts=2026-09-21T09:52:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397924429client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4430worker1 # [ 55.645680] nix-daemon[844]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4431worker1 # [ 55.796161] nix-daemon[844]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4432worker1 # [ 55.815044] nix-daemon[844]: reaped child process 999, status = succeeded4433worker1 # [ 55.917211] nix-daemon[844]: accepted connection from pid 1001, user nix-grpc-daemon (trusted)4434worker1 # [ 55.936342] nix-daemon[844]: reaped child process 1004, status = succeeded4435lb # [ 55.893311] niks3-server[964]: 2026/09/21 09:52:56 INFO Received uploads request method=POST path=/api/pending_closures4436lb # [ 55.902889] niks3-server[964]: 2026/09/21 09:52:56 INFO Received uploads request method=POST path=/api/pending_closures4437worker1 # [ 55.953388] nix-grpc-daemon[898]: time=2026-09-21T09:52:56.549Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4438worker1 # [ 55.954862] nix-grpc-daemon[898]: time=2026-09-21T09:52:56.549Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4439worker1 # [ 55.956698] nix-grpc-daemon[898]: time=2026-09-21T09:52:56.549Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4440lb # [ 55.939261] niks3-server[964]: 2026/09/21 09:52:56 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4441lb # [ 55.954051] niks3-server[964]: 2026/09/21 09:52:56 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4442lb # [ 55.973091] niks3-server[964]: 2026/09/21 09:52:56 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4443lb # [ 55.982862] niks3-server[964]: 2026/09/21 09:52:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4444lb # [ 55.993009] niks3-server[964]: 2026/09/21 09:52:56 INFO Signed narinfos id=18 count=14445worker1 # [ 56.033346] nix-grpc-daemon[898]: time=2026-09-21T09:52:56.628Z level=INFO msg="Uploading 3 narinfos"4446lb # [ 55.995150] niks3-server[964]: 2026/09/21 09:52:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4447lb # [ 56.005202] niks3-server[964]: 2026/09/21 09:52:56 INFO Signed narinfos id=19 count=24448lb # [ 56.007323] niks3-server[964]: 2026/09/21 09:52:56 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4449lb # [ 56.027415] niks3-server[964]: 2026/09/21 09:52:56 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4450lb # [ 56.039060] niks3-server[964]: 2026/09/21 09:52:56 INFO Completed upload id=184451lb # [ 56.045190] niks3-server[964]: 2026/09/21 09:52:56 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4452worker1 # [ 56.086669] nix-grpc-daemon[898]: time=2026-09-21T09:52:56.682Z level=INFO msg="Upload complete. (423ms)"4453lb # [ 56.048151] niks3-server[964]: 2026/09/21 09:52:56 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4454lb # [ 56.051966] niks3-server[964]: 2026/09/21 09:52:56 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4455lb # [ 56.055732] niks3-server[964]: 2026/09/21 09:52:56 INFO Completed upload id=194456lb # [ 56.057822] niks3-server[964]: 2026/09/21 09:52:56 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4457worker1 # [ 56.096401] nix-grpc-daemon[765]: ts=2026-09-21T09:52:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39792 duration_s=0 paths=2 nar_bytes_in=9844458worker1 # [ 56.116836] nix-grpc-daemon[765]: ts=2026-09-21T09:52:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397924459worker1 # [ 56.125325] nix-daemon[844]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4460worker1 # [ 56.131188] nix-daemon[844]: reaped child process 994, status = succeeded4461worker1 # [ 56.143236] nix-daemon[844]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4462client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4463client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4464client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4465worker1 # [ 57.424645] nix-daemon[844]: reaped child process 1008, status = succeeded4466worker1 # [ 57.517659] nix-daemon[844]: accepted connection from pid 1024, user nix-grpc-daemon (trusted)4467worker1 # [ 57.538842] nix-daemon[844]: reaped child process 1027, status = succeeded4468lb # [ 57.501722] niks3-server[964]: 2026/09/21 09:52:58 INFO Received uploads request method=POST path=/api/pending_closures4469worker1 # [ 57.554710] nix-grpc-daemon[898]: time=2026-09-21T09:52:58.149Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4470worker1 # [ 57.585664] nix-grpc-daemon[898]: time=2026-09-21T09:52:58.181Z level=INFO msg="Uploading 1 narinfos"4471lb # [ 57.534992] niks3-server[964]: 2026/09/21 09:52:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4472lb # [ 57.549248] niks3-server[964]: 2026/09/21 09:52:58 INFO Signed narinfos id=20 count=14473lb # [ 57.558002] niks3-server[964]: 2026/09/21 09:52:58 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4474lb # [ 57.566641] niks3-server[964]: 2026/09/21 09:52:58 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4475lb # [ 57.581785] niks3-server[964]: 2026/09/21 09:52:58 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4476worker1 # [ 57.629369] nix-grpc-daemon[898]: time=2026-09-21T09:52:58.225Z level=INFO msg="Upload complete. (272ms)"4477lb # [ 57.590265] niks3-server[964]: 2026/09/21 09:52:58 INFO Completed upload id=204478lb # [ 57.594931] niks3-server[964]: 2026/09/21 09:52:58 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4479worker1 # [ 57.641150] nix-grpc-daemon[765]: ts=2026-09-21T09:52:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39792 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14480worker1 # [ 57.660889] nix-daemon[844]: reaped child process 1006, status = succeeded4481client # copying 1 paths...4482worker2 # [ 57.705875] nix-grpc-daemon[769]: ts=2026-09-21T09:52:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:562484483worker2 # [ 57.721651] nix-grpc-daemon[769]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4484client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4485worker2 # [ 57.772121] nix-grpc-daemon[769]: ts=2026-09-21T09:52:58Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:562484486worker2 # [ 57.778757] nix-grpc-daemon[769]: ts=2026-09-21T09:52:58Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=0 paths=1 nar_bytes_out=1204487client # [ 57.918121] nix-daemon[987]: reaped child process 1072, status = succeeded4488client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.01 seconds)4489client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&24490client # [ 58.096590] nix-daemon[987]: accepted connection from pid 1091, user root (trusted)4491client # this derivation will be built:4492client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4493worker2 # [ 58.233977] nix-grpc-daemon[769]: ts=2026-09-21T09:52:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:562484494client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4495worker2 # [ 58.242465] nix-grpc-daemon[769]: ts=2026-09-21T09:52:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562484496worker1 # [ 58.246191] nix-grpc-daemon[765]: ts=2026-09-21T09:52:58Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04497worker2 # [ 58.265064] nix-grpc-daemon[769]: ts=2026-09-21T09:52:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562484498worker1 # [ 58.252803] nix-grpc-daemon[765]: ts=2026-09-21T09:52:58Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14499worker2 # [ 58.279577] nix-grpc-daemon[769]: ts=2026-09-21T09:52:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562484500client # copying 1 paths to 'lb:50051'...4501client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4502worker2 # [ 58.299550] nix-grpc-daemon[769]: ts=2026-09-21T09:52:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:562484503worker2 # [ 58.315125] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4504worker2 # [ 58.337725] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 1/5)4505worker2 # [ 58.390800] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 91 ms (attempt 2/5)4506worker2 # [ 58.483601] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 149 ms (attempt 3/5)4507worker2 # [ 58.634827] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 686 ms (attempt 4/5)4508lb # [ 58.629051] envoy[900]: conn peer=[2001:db8:1::1]:47274 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=404509lb # [ 58.633845] envoy[900]: conn peer=[2001:db8:1::1]:47262 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3407 tx=1856 ms=27714510lb # [ 58.638957] envoy[900]: conn peer=[2001:db8:1::1]:47862 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=984511lb # [ 58.644578] envoy[900]: conn peer=[2001:db8:1::1]:47286 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3900 tx=1957 ms=26074512lb # [ 58.649286] envoy[900]: conn peer=[2001:db8:1::1]:47866 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4339 tx=2303 ms=18044513lb # [ 58.653968] envoy[900]: conn peer=[2001:db8:1::1]:47878 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=664514lb # [ 58.658564] envoy[900]: conn peer=[2001:db8:1::1]:47894 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4263 tx=1960 ms=25954515lb # [ 58.664076] envoy[900]: conn peer=[2001:db8:1::1]:47898 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=724516worker2 # [ 59.325405] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4517worker2 # [ 59.329839] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4518worker2 # [ 59.339637] nix-grpc-daemon[769]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4519worker2 # [ 59.524066] nix-daemon[834]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)4520worker2 # [ 59.546249] nix-daemon[834]: reaped child process 1156, status = succeeded4521worker2 # [ 59.667368] nix-daemon[834]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)4522worker2 # [ 59.693762] nix-daemon[834]: reaped child process 1161, status = succeeded4523lb # [ 59.634726] niks3-server[964]: 2026/09/21 09:53:00 INFO Received uploads request method=POST path=/api/pending_closures4524worker2 # [ 59.704317] nix-grpc-daemon[847]: time=2026-09-21T09:53:00.294Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4525worker2 # [ 59.705836] nix-grpc-daemon[847]: time=2026-09-21T09:53:00.294Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4526lb # [ 59.667011] niks3-server[964]: 2026/09/21 09:53:00 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4527lb # [ 59.683391] niks3-server[964]: 2026/09/21 09:53:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4528worker2 # [ 59.748036] nix-grpc-daemon[847]: time=2026-09-21T09:53:00.338Z level=INFO msg="Uploading 1 narinfos"4529lb # [ 59.695997] niks3-server[964]: 2026/09/21 09:53:00 INFO Signed narinfos id=21 count=14530lb # [ 59.698498] niks3-server[964]: 2026/09/21 09:53:00 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4531worker2 # [ 59.768192] nix-grpc-daemon[847]: time=2026-09-21T09:53:00.359Z level=INFO msg="Upload complete. (409ms)"4532lb # [ 59.710041] niks3-server[964]: 2026/09/21 09:53:00 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4533lb # [ 59.719688] niks3-server[964]: 2026/09/21 09:53:00 INFO Completed upload id=214534worker2 # [ 59.774905] nix-grpc-daemon[769]: ts=2026-09-21T09:53:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=1 paths=1 nar_bytes_in=5684535lb # [ 59.721742] niks3-server[964]: 2026/09/21 09:53:00 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4536worker2 # [ 59.785223] nix-grpc-daemon[769]: ts=2026-09-21T09:53:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:562484537worker2 # [ 59.798372] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4538worker2 # [ 59.803378] nix-daemon[834]: reaped child process 1146, status = succeeded4539worker2 # [ 59.813018] nix-grpc-daemon[769]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4540worker2 # [ 59.827898] nix-daemon[834]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4541client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4542client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4543client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4544worker2 # [ 61.320365] nix-daemon[834]: reaped child process 1166, status = succeeded4545worker2 # [ 61.444043] nix-daemon[834]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)4546worker2 # [ 61.461948] nix-daemon[834]: reaped child process 1185, status = succeeded4547lb # [ 61.413541] niks3-server[964]: 2026/09/21 09:53:02 INFO Received uploads request method=POST path=/api/pending_closures4548worker2 # [ 61.482332] nix-grpc-daemon[847]: time=2026-09-21T09:53:02.072Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4549lb # [ 61.453078] niks3-server[964]: 2026/09/21 09:53:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4550worker2 # [ 61.520743] nix-grpc-daemon[847]: time=2026-09-21T09:53:02.111Z level=INFO msg="Uploading 1 narinfos"4551lb # [ 61.467134] niks3-server[964]: 2026/09/21 09:53:02 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4552lb # [ 61.477425] niks3-server[964]: 2026/09/21 09:53:02 INFO Signed narinfos id=22 count=14553worker2 # [ 61.543590] nix-grpc-daemon[847]: time=2026-09-21T09:53:02.134Z level=INFO msg="Upload complete. (300ms)"4554lb # [ 61.486275] niks3-server[964]: 2026/09/21 09:53:02 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4555lb # [ 61.492791] niks3-server[964]: 2026/09/21 09:53:02 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4556lb # [ 61.496741] niks3-server[964]: 2026/09/21 09:53:02 INFO Completed upload id=224557worker2 # [ 61.553302] nix-grpc-daemon[769]: ts=2026-09-21T09:53:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14558lb # [ 61.499676] niks3-server[964]: 2026/09/21 09:53:02 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4559worker2 # [ 61.572634] nix-daemon[834]: reaped child process 1163, status = succeeded4560client # copying 1 paths...4561worker1 # [ 61.587170] nix-grpc-daemon[765]: ts=2026-09-21T09:53:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:397924562worker1 # [ 61.603013] nix-grpc-daemon[765]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4563client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4564worker2 # [ 61.669695] nix-grpc-daemon[769]: ts=2026-09-21T09:53:02Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:562484565worker2 # [ 61.677206] nix-grpc-daemon[769]: ts=2026-09-21T09:53:02Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=0 paths=1 nar_bytes_out=1204566client # [ 61.789579] nix-daemon[987]: reaped child process 1094, status = succeeded4567client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.87 seconds)4568client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&24569client # [ 61.983389] nix-daemon[987]: accepted connection from pid 1113, user root (trusted)4570client # this derivation will be built:4571client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4572worker2 # [ 62.133229] nix-grpc-daemon[769]: ts=2026-09-21T09:53:02Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:562484573client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4574worker1 # [ 62.126768] nix-grpc-daemon[765]: ts=2026-09-21T09:53:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397924575worker1 # [ 62.139506] nix-grpc-daemon[898]: time=2026-09-21T09:53:02.734Z level=INFO msg="All 1 paths already cached"4576worker1 # [ 62.161035] nix-grpc-daemon[765]: ts=2026-09-21T09:53:02Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04577worker1 # [ 62.169318] nix-grpc-daemon[765]: ts=2026-09-21T09:53:02Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14578worker1 # [ 62.178827] nix-grpc-daemon[765]: ts=2026-09-21T09:53:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397924579worker1 # [ 62.191215] nix-grpc-daemon[765]: ts=2026-09-21T09:53:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:397924580worker1 # [ 62.207980] nix-grpc-daemon[898]: time=2026-09-21T09:53:02.803Z level=INFO msg="All 1 paths already cached"4581client # copying 1 paths to 'lb:50051'...4582client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4583worker1 # [ 62.220116] nix-grpc-daemon[765]: ts=2026-09-21T09:53:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:397924584worker1 # [ 62.237383] nix-daemon[844]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4585worker1 # [ 62.407208] nix-daemon[844]: accepted connection from pid 1036, user nix-grpc-daemon (trusted)4586worker1 # [ 62.428235] nix-daemon[844]: reaped child process 1039, status = succeeded4587worker1 # [ 62.544738] nix-daemon[844]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)4588worker1 # [ 62.563122] nix-daemon[844]: reaped child process 1044, status = succeeded4589lb # [ 62.529678] niks3-server[964]: 2026/09/21 09:53:03 INFO Received uploads request method=POST path=/api/pending_closures4590worker1 # [ 62.581254] nix-grpc-daemon[898]: time=2026-09-21T09:53:03.176Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4591worker1 # [ 62.582980] nix-grpc-daemon[898]: time=2026-09-21T09:53:03.176Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4592lb # [ 62.562376] niks3-server[964]: 2026/09/21 09:53:03 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4593lb # [ 62.576690] niks3-server[964]: 2026/09/21 09:53:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4594worker1 # [ 62.625811] nix-grpc-daemon[898]: time=2026-09-21T09:53:03.221Z level=INFO msg="Uploading 1 narinfos"4595lb # [ 62.588367] niks3-server[964]: 2026/09/21 09:53:03 INFO Signed narinfos id=23 count=14596lb # [ 62.594402] niks3-server[964]: 2026/09/21 09:53:03 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4597lb # [ 62.606213] niks3-server[964]: 2026/09/21 09:53:03 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4598lb # [ 62.613773] niks3-server[964]: 2026/09/21 09:53:03 INFO Completed upload id=234599worker1 # [ 62.654519] nix-grpc-daemon[898]: time=2026-09-21T09:53:03.250Z level=INFO msg="Upload complete. (399ms)"4600lb # [ 62.620739] niks3-server[964]: 2026/09/21 09:53:03 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4601worker1 # [ 62.665359] nix-grpc-daemon[765]: ts=2026-09-21T09:53:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39792 duration_s=0 paths=1 nar_bytes_in=5684602worker1 # [ 62.684499] nix-grpc-daemon[765]: ts=2026-09-21T09:53:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:397924603worker1 # [ 62.691986] nix-daemon[844]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4604worker1 # [ 62.697619] nix-daemon[844]: reaped child process 1034, status = succeeded4605client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4606worker1 # [ 62.713284] nix-daemon[844]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4607client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4608client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4609worker1 # [ 64.019487] nix-daemon[844]: reaped child process 1048, status = succeeded4610worker1 # [ 64.099352] nix-daemon[844]: accepted connection from pid 1064, user nix-grpc-daemon (trusted)4611worker1 # [ 64.117983] nix-daemon[844]: reaped child process 1067, status = succeeded4612lb # [ 64.085309] niks3-server[964]: 2026/09/21 09:53:04 INFO Received uploads request method=POST path=/api/pending_closures4613worker1 # [ 64.139897] nix-grpc-daemon[898]: time=2026-09-21T09:53:04.733Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4614lb # [ 64.119610] niks3-server[964]: 2026/09/21 09:53:04 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4615worker1 # [ 64.171222] nix-grpc-daemon[898]: time=2026-09-21T09:53:04.766Z level=INFO msg="Uploading 1 narinfos"4616lb # [ 64.131897] niks3-server[964]: 2026/09/21 09:53:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4617lb # [ 64.143841] niks3-server[964]: 2026/09/21 09:53:04 INFO Signed narinfos id=24 count=14618lb # [ 64.158217] niks3-server[964]: 2026/09/21 09:53:04 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4619lb # [ 64.166990] niks3-server[964]: 2026/09/21 09:53:04 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4620lb # [ 64.171803] niks3-server[964]: 2026/09/21 09:53:04 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4621worker1 # [ 64.213212] nix-grpc-daemon[898]: time=2026-09-21T09:53:04.809Z level=INFO msg="Upload complete. (287ms)"4622lb # [ 64.175690] niks3-server[964]: 2026/09/21 09:53:04 INFO Completed upload id=244623client # copying 1 paths...4624worker1 # [ 64.229691] nix-grpc-daemon[765]: ts=2026-09-21T09:53:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39792 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14625client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4626worker1 # [ 64.245937] nix-grpc-daemon[765]: ts=2026-09-21T09:53:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:397924627worker1 # [ 64.250794] nix-daemon[844]: reaped child process 1046, status = succeeded4628worker2 # [ 64.285807] nix-grpc-daemon[769]: ts=2026-09-21T09:53:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:562484629worker2 # [ 64.298734] nix-grpc-daemon[769]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4630worker2 # [ 64.315903] nix-grpc-daemon[769]: ts=2026-09-21T09:53:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56248 duration_s=0 paths=1 nar_bytes_out=1204631client # [ 64.455105] nix-daemon[987]: reaped child process 1116, status = succeeded4632client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.66 seconds)4633(finished: subtest: an input only one worker has still reaches the builder, in 10.19 seconds)4634subtest: upload whose reference the worker lacks is completed from the cache4635client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache4636client # this derivation will be built:4637client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4638client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4639client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.35 seconds)4640client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache4641client # this derivation will be built:4642client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4643client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4644client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.30 seconds)4645client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4646client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.13 seconds)4647worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4648worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)4649worker2 # [ 65.494508] systemd[1]: Stopping Nix worker-protocol over gRPC...4650worker2 # [ 66.032960] nix-grpc-daemon[769]: ts=2026-09-21T09:53:06Z level=info event=signal_exit4651worker1 # [ 68.003305] nix-grpc-daemon[765]: ts=2026-09-21T09:53:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:397784652worker2 # [ 68.026014] nix-grpc-daemon[769]: ts=2026-09-21T09:53:08Z level=info event=scheduler_restarting4653worker2 # [ 68.055152] nix-grpc-daemon[769]: ts=2026-09-21T09:53:08Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4654worker2 # [ 68.075601] nix-daemon[834]: reaped child process 836, status = succeeded4655worker2 # [ 68.112961] nix-grpc-daemon[769]: E0000 00:00:1789984388.703981 814 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4656worker2 # [ 68.126400] nix-grpc-daemon[769]: E0000 00:00:1789984388.717503 814 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4657worker1 # [ 68.008209] nix-grpc-daemon[765]: ts=2026-09-21T09:53:08Z level=info event=worker_gone addr=192.168.1.4:500514658lb # [ 68.627143] envoy[900]: conn peer=[2001:db8:1::1]:36906 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=314659lb # [ 68.632225] envoy[900]: conn peer=[2001:db8:1::1]:47902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3756 tx=2012 ms=34614660lb # [ 68.636995] envoy[900]: conn peer=[2001:db8:1::1]:36908 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3758 tx=1876 ms=22404661lb # [ 68.642176] envoy[900]: conn peer=[2001:db8:1::1]:36916 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=974662lb # [ 68.647010] envoy[900]: conn peer=[2001:db8:1::4]:54990 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=20899 tx=1663 ms=489514663worker2 # [ 70.035714] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4664worker2 # [ 70.040912] systemd[1]: Stopped Nix worker-protocol over gRPC.4665worker2 # [ 70.046935] systemd[1]: nix-grpc-daemon.service: Consumed 3.944s CPU time over 51.223s wall clock time, 65.2M memory peak, 79.4K incoming IP traffic, 159K outgoing IP traffic.4666worker2 # [ 70.062381] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4667worker2 # [ 70.072625] systemd[1]: Closed Nix worker-protocol over gRPC.4668worker2 # [ 70.074437] systemd[1]: nix-grpc-daemon.socket: Consumed 510us CPU time over 1min 5ms wall clock time, 256K memory peak, 73.8K incoming IP traffic, 57.9K outgoing IP traffic.4669lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4670lb: must succeed: curl -sf localhost:9901/clusters4671lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4672lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)4673client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24674worker1 # [ 70.373310] nix-grpc-daemon[765]: ts=2026-09-21T09:53:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:397924675worker1 # [ 70.383124] nix-grpc-daemon[765]: ts=2026-09-21T09:53:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:397924676client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4677client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.27 seconds)4678worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4679worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)4680worker2 # [ 70.526935] systemd[1]: Listening on Nix worker-protocol over gRPC.4681worker1 # [ 70.582748] systemd[1]: Stopping Nix worker-protocol over gRPC...4682worker1 # [ 70.920822] nix-grpc-daemon[765]: ts=2026-09-21T09:53:11Z level=info event=signal_exit4683worker1 # [ 70.989159] nix-grpc-daemon[765]: ts=2026-09-21T09:53:11Z level=info event=scheduler_yield4684worker1 # [ 70.995252] nix-grpc-daemon[765]: ts=2026-09-21T09:53:11Z level=info event=scheduler_restarting addr=lb:500514685lb # [ 70.954220] niks3-server[964]: 2026/09/21 09:53:11 INFO lead: released remote=192.168.1.3:482304686worker1 # [ 71.000743] nix-grpc-daemon[765]: ts=2026-09-21T09:53:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:397784687worker1 # [ 71.005219] nix-grpc-daemon[765]: ts=2026-09-21T09:53:11Z level=info event=worker_gone addr=192.168.1.3:500514688worker2 # [ 71.032636] systemd[1]: Starting scheduler-token.service...4689worker2 # [ 71.129396] systemd[1]: scheduler-token.service: Deactivated successfully.4690worker2 # [ 71.137086] systemd[1]: Finished scheduler-token.service.4691worker2 # [ 71.139251] systemd[1]: scheduler-token.service: Consumed 60ms CPU time over 100ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4692worker2 # [ 71.162242] systemd[1]: Starting Nix worker-protocol over gRPC...4693worker1 # [ 71.196944] nix-grpc-daemon[765]: ts=2026-09-21T09:53:11Z level=info event=scheduler_connected addr=lb:500514694worker1 # [ 71.202132] nix-grpc-daemon[765]: ts=2026-09-21T09:53:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4695worker2 # [ 71.285792] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:11Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4696worker2 # [ 71.288176] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:11Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4697worker2 # [ 71.292077] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:11Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14698worker2 # [ 71.299129] nix-grpc-daemon[1210]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4699worker2 # [ 71.313885] nix-grpc-daemon[1210]: E0000 00:00:1789984391.887999 1210 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4700lb # [ 71.258394] niks3-server[964]: 2026/09/21 09:53:11 INFO lead: acquired remote=192.168.1.4:576024701worker2 # [ 71.318929] nix-grpc-daemon[1210]: E0000 00:00:1789984391.890259 1210 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4702worker2 # [ 71.348181] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:11Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4703worker2 # [ 71.349879] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:11Z level=info event=scheduler_take_over4704worker2 # [ 71.351003] nix-grpc-daemon[1210]: E0000 00:00:1789984391.908311 1252 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4705worker2 # [ 71.358105] nix-grpc-daemon[1210]: E0000 00:00:1789984391.928144 1252 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4706worker2 # [ 71.360946] nix-grpc-daemon[1210]: E0000 00:00:1789984391.928305 1252 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4707worker2 # [ 71.363540] systemd[1]: Started Nix worker-protocol over gRPC.4708worker2 # [ 71.364474] nix-grpc-daemon[1210]: E0000 00:00:1789984391.948447 1252 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4709worker2 # [ 71.372261] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:11Z level=info event=scheduler_connected addr=lb:500514710worker2 # [ 71.374351] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4711worker1 # [ 71.404418] nix-grpc-daemon[765]: ts=2026-09-21T09:53:11Z level=info event=scheduler_connected addr=lb:500514712worker1 # [ 71.408391] nix-grpc-daemon[765]: ts=2026-09-21T09:53:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4713worker2 # [ 71.575547] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:12Z level=info event=scheduler_connected addr=lb:500514714worker2 # [ 71.580497] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4715worker1 # [ 71.604789] nix-grpc-daemon[765]: ts=2026-09-21T09:53:12Z level=info event=scheduler_connected addr=lb:500514716worker1 # [ 71.609833] nix-grpc-daemon[765]: ts=2026-09-21T09:53:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4717worker2 # [ 71.781359] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:12Z level=info event=scheduler_connected addr=lb:500514718worker2 # [ 71.786359] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4719worker1 # [ 71.810429] nix-grpc-daemon[765]: ts=2026-09-21T09:53:12Z level=info event=scheduler_connected addr=lb:500514720worker1 # [ 71.815647] nix-grpc-daemon[765]: ts=2026-09-21T09:53:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4721worker2 # [ 71.987365] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:12Z level=info event=scheduler_connected addr=lb:500514722worker2 # [ 71.992558] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4723worker1 # [ 71.995091] nix-grpc-daemon[765]: ts=2026-09-21T09:53:12Z level=info event=scheduler_restarting4724worker1 # [ 72.035965] nix-grpc-daemon[765]: E0000 00:00:1789984392.618013 810 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true4725worker1 # [ 72.052372] nix-grpc-daemon[765]: ts=2026-09-21T09:53:12Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4726worker1 # [ 72.068003] nix-daemon[844]: reaped child process 846, status = succeeded4727worker1 # [ 72.071119] nix-grpc-daemon[765]: E0000 00:00:1789984392.667054 810 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true4728worker2 # [ 72.194086] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:12Z level=info event=scheduler_connected addr=lb:500514729worker2 # [ 72.199172] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4730worker2 # [ 72.401394] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:12Z level=info event=scheduler_connected addr=lb:500514731worker2 # [ 72.419182] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:13Z level=info event=oidc_jwks provider=mock keys=14732worker2 # [ 72.423806] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:13Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:562744733worker2 # [ 72.429218] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:13Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04734worker1 # [ 73.963800] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4735worker1 # [ 73.968936] systemd[1]: Stopped Nix worker-protocol over gRPC.4736worker1 # [ 73.974976] systemd[1]: nix-grpc-daemon.service: Consumed 2.077s CPU time over 55.293s wall clock time, 57.9M memory peak, 55.9K incoming IP traffic, 116.4K outgoing IP traffic.4737worker1 # [ 73.991209] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4738worker1 # [ 74.002855] systemd[1]: Closed Nix worker-protocol over gRPC.4739lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4740lb: must succeed: curl -sf localhost:9901/clusters4741worker1 # [ 74.006577] systemd[1]: nix-grpc-daemon.socket: Consumed 520us CPU time over 1min 4.076s wall clock time, 256K memory peak, 73.6K incoming IP traffic, 66.1K outgoing IP traffic.4742lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4743lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)4744client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24745worker2 # [ 74.329197] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:562884746worker2 # [ 74.345554] nix-daemon[834]: accepted connection from pid 1210, user nix-grpc-daemon (trusted)4747worker2 # [ 74.373622] nix-grpc-daemon[1210]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)4748worker2 # [ 74.388553] nix-grpc-daemon[1210]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 2/5)4749worker2 # [ 74.405179] nix-grpc-daemon[1210]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 3/5)4750worker2 # [ 74.547153] nix-grpc-daemon[1210]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 605 ms (attempt 4/5)4751worker2 # [ 75.155970] nix-grpc-daemon[1210]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4752worker2 # [ 75.159718] nix-grpc-daemon[1210]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4753client # copying 2 paths...4754client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4755worker2 # [ 75.185773] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:562884756client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4757worker2 # [ 75.203371] nix-daemon[834]: accepted connection from pid 1210, user nix-grpc-daemon (trusted)4758worker2 # [ 75.241110] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:15Z level=info event=niks3_push_started pid=12704759worker2 # [ 75.393195] nix-grpc-daemon[1270]: time=2026-09-21T09:53:15.983Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""4760worker2 # [ 75.505404] nix-daemon[834]: accepted connection from pid 1283, user nix-grpc-daemon (trusted)4761worker2 # [ 75.522140] nix-daemon[834]: reaped child process 1286, status = succeeded4762lb # [ 75.473820] niks3-server[964]: 2026/09/21 09:53:16 INFO Received uploads request method=POST path=/api/pending_closures4763lb # [ 75.483404] niks3-server[964]: 2026/09/21 09:53:16 INFO Received uploads request method=POST path=/api/pending_closures4764worker2 # [ 75.551499] nix-grpc-daemon[1270]: time=2026-09-21T09:53:16.141Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4765worker2 # [ 75.554656] nix-grpc-daemon[1270]: time=2026-09-21T09:53:16.141Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4766worker2 # [ 75.557003] nix-grpc-daemon[1270]: time=2026-09-21T09:53:16.141Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4767lb # [ 75.538787] niks3-server[964]: 2026/09/21 09:53:16 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4768lb # [ 75.550157] niks3-server[964]: 2026/09/21 09:53:16 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4769lb # [ 75.566947] niks3-server[964]: 2026/09/21 09:53:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4770worker2 # [ 75.635718] nix-grpc-daemon[1270]: time=2026-09-21T09:53:16.225Z level=INFO msg="Uploading 3 narinfos"4771lb # [ 75.580454] niks3-server[964]: 2026/09/21 09:53:16 INFO Signed narinfos id=25 count=14772lb # [ 75.582831] niks3-server[964]: 2026/09/21 09:53:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4773lb # [ 75.594838] niks3-server[964]: 2026/09/21 09:53:16 INFO Signed narinfos id=26 count=24774lb # [ 75.599151] niks3-server[964]: 2026/09/21 09:53:16 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4775lb # [ 75.606395] niks3-server[964]: 2026/09/21 09:53:16 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4776lb # [ 75.618916] niks3-server[964]: 2026/09/21 09:53:16 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4777lb # [ 75.630701] niks3-server[964]: 2026/09/21 09:53:16 INFO Completed upload id=254778lb # [ 75.637997] niks3-server[964]: 2026/09/21 09:53:16 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4779worker2 # [ 75.692681] nix-grpc-daemon[1270]: time=2026-09-21T09:53:16.283Z level=INFO msg="Upload complete. (295ms)"4780lb # [ 75.645078] niks3-server[964]: 2026/09/21 09:53:16 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4781lb # [ 75.652210] niks3-server[964]: 2026/09/21 09:53:16 INFO Completed upload id=264782worker2 # [ 75.705458] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56288 duration_s=0 paths=2 nar_bytes_in=3124783lb # [ 75.659470] niks3-server[964]: 2026/09/21 09:53:16 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4784lb # [ 75.662828] niks3-server[964]: 2026/09/21 09:53:16 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4785worker2 # [ 75.721509] nix-daemon[834]: reaped child process 1268, status = succeeded4786client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.65 seconds)4787worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4788worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)4789worker1 # [ 75.831081] systemd[1]: Listening on Nix worker-protocol over gRPC.4790worker1 # [ 75.841683] systemd[1]: Starting Nix worker-protocol over gRPC...4791worker1 # [ 75.981222] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4792worker1 # [ 75.983371] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4793worker1 # [ 75.987389] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14794worker1 # [ 75.993168] nix-grpc-daemon[1090]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4795worker1 # [ 75.994654] nix-grpc-daemon[1090]: E0000 00:00:1789984396.588848 1090 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4796lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4797lb: must succeed: curl -sf localhost:9901/clusters4798worker1 # [ 76.044180] nix-grpc-daemon[1090]: E0000 00:00:1789984396.595046 1090 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4799worker1 # [ 76.046450] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4800worker1 # [ 76.052488] nix-grpc-daemon[1090]: E0000 00:00:1789984396.608351 1132 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4801worker1 # [ 76.055466] nix-grpc-daemon[1090]: E0000 00:00:1789984396.627634 1132 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4802worker1 # [ 76.058126] nix-grpc-daemon[1090]: E0000 00:00:1789984396.627804 1132 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4803worker1 # [ 76.060624] systemd[1]: Started Nix worker-protocol over gRPC.4804worker1 # [ 76.061425] nix-grpc-daemon[1090]: E0000 00:00:1789984396.647607 1132 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4805worker1 # [ 76.072297] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:16Z level=info event=scheduler_connected addr=lb:500514806worker2 # [ 76.089089] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:562744807worker2 # [ 76.095451] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04808lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4809lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.08 seconds)4810(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.67 seconds)4811subtest: scheduler down: the other node takes the lock and keeps it4812lb: must succeed: curl -sf localhost:9901/clusters4813lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4814lb: must succeed: curl -sf localhost:9901/clusters4815lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4816lb: must succeed: curl -sf localhost:9901/clusters4817lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4818lb: must succeed: curl -sf localhost:9901/clusters4819lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4820worker2 # [ 76.349500] systemd[1]: Stopping Nix worker-protocol over gRPC...4821worker2 # [ 77.310559] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:17Z level=info event=signal_exit4822worker2 # [ 77.317624] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:17Z level=info event=scheduler_yield4823lb # [ 77.268620] niks3-server[964]: 2026/09/21 09:53:17 INFO lead: released remote=192.168.1.4:576024824worker1 # [ 77.313428] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:17Z level=info event=scheduler_restarting addr=lb:500514825worker2 # [ 77.332112] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:562744826worker2 # [ 77.338069] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:17Z level=info event=worker_gone addr=192.168.1.3:500514827worker2 # [ 77.346382] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:17Z level=info event=scheduler_restarting addr=lb:500514828worker2 # [ 77.350510] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:562744829worker2 # [ 77.354917] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:17Z level=info event=worker_gone addr=192.168.1.4:500514830worker1 # [ 77.513366] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:18Z level=info event=scheduler_connected addr=lb:500514831worker1 # [ 77.518630] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4832worker2 # [ 77.539265] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:18Z level=info event=scheduler_connected addr=lb:500514833worker2 # [ 77.544504] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4834worker1 # [ 77.719340] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:18Z level=info event=scheduler_connected addr=lb:500514835worker1 # [ 77.724576] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4836worker2 # [ 77.745560] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:18Z level=info event=scheduler_connected addr=lb:500514837worker2 # [ 77.750605] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4838worker1 # [ 77.925363] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:18Z level=info event=scheduler_connected addr=lb:500514839worker1 # [ 77.930470] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4840worker2 # [ 77.951874] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:18Z level=info event=scheduler_connected addr=lb:500514841worker2 # [ 77.958393] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4842worker1 # [ 78.131161] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:18Z level=info event=scheduler_connected addr=lb:500514843worker1 # [ 78.136344] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4844worker2 # [ 78.159218] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:18Z level=info event=scheduler_connected addr=lb:500514845worker2 # [ 78.164355] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4846worker2 # [ 78.327232] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:18Z level=info event=scheduler_restarting4847worker1 # [ 78.338773] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:18Z level=info event=scheduler_connected addr=lb:500514848worker2 # [ 78.345265] nix-grpc-daemon[1210]: E0000 00:00:1789984398.936169 1255 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true4849worker1 # [ 78.342933] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4850worker2 # [ 78.360728] nix-grpc-daemon[1210]: E0000 00:00:1789984398.936481 1255 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true4851worker2 # [ 78.385055] nix-grpc-daemon[1210]: ts=2026-09-21T09:53:18Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4852worker2 # [ 78.389526] nix-daemon[834]: reaped child process 1262, status = succeeded4853worker1 # [ 78.539358] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:19Z level=info event=scheduler_connected addr=lb:500514854worker1 # [ 78.544601] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4855lb # [ 78.626259] envoy[900]: conn peer=[2001:db8:1::1]:36924 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=523 ms=354856lb # [ 78.631127] envoy[900]: conn peer=[2001:db8:1::3]:42246 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5031 tx=1484 ms=530594857lb # [ 78.636241] envoy[900]: conn peer=[2001:db8:1::1]:59438 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1010 tx=282 ms=13934858lb # [ 78.640875] envoy[900]: conn peer=[2001:db8:1::4]:44556 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7963 tx=535 ms=70124859worker1 # [ 78.745329] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:19Z level=info event=scheduler_connected addr=lb:500514860worker1 # [ 78.750408] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4861worker1 # [ 78.951194] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:19Z level=info event=scheduler_connected addr=lb:500514862worker1 # [ 78.956379] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4863worker1 # [ 79.157072] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:19Z level=info event=scheduler_connected addr=lb:500514864worker1 # [ 79.162258] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4865worker2 # [ 79.335623] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4866worker2 # [ 79.341163] systemd[1]: Stopped Nix worker-protocol over gRPC.4867worker2 # [ 79.343760] systemd[1]: nix-grpc-daemon.service: Consumed 517ms CPU time over 8.184s wall clock time, 33.8M memory peak, 20.5K incoming IP traffic, 30.9K outgoing IP traffic.4868worker2 # [ 79.361841] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4869worker2 # [ 79.368890] systemd[1]: Closed Nix worker-protocol over gRPC.4870worker2 # [ 79.370740] systemd[1]: nix-grpc-daemon.socket: Consumed 864us CPU time over 8.845s wall clock time, 256K memory peak, 19.5K incoming IP traffic, 14.3K outgoing IP traffic.4871worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over4872worker1 # [ 79.364676] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:19Z level=info event=scheduler_connected addr=lb:500514873worker1 # [ 79.368634] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4874worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)4875worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4876worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4877worker1 # [ 79.565275] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:20Z level=info event=scheduler_connected addr=lb:500514878worker1 # [ 79.570582] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4879worker1 # [ 79.771365] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:20Z level=info event=scheduler_connected addr=lb:500514880worker1 # [ 79.776672] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4881worker1 # [ 79.977453] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:20Z level=info event=scheduler_connected addr=lb:500514882worker1 # [ 79.982530] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4883worker1 # [ 80.183240] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:20Z level=info event=scheduler_connected addr=lb:500514884worker1 # [ 80.188543] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4885worker1 # [ 80.391039] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:20Z level=info event=scheduler_connected addr=lb:500514886worker1 # [ 80.395223] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4887worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4888worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4889worker1 # [ 80.591676] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:21Z level=info event=scheduler_connected addr=lb:500514890worker1 # [ 80.597036] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4891worker1 # [ 80.797739] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:21Z level=info event=scheduler_connected addr=lb:500514892worker1 # [ 80.802828] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4893worker1 # [ 81.003763] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:21Z level=info event=scheduler_connected addr=lb:500514894worker1 # [ 81.009663] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4895lb # [ 80.973052] niks3-server[964]: 2026/09/21 09:53:21 INFO lead: acquired remote=192.168.1.3:405164896worker1 # [ 81.015554] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:21Z level=info event=scheduler_take_over4897worker1 # [ 81.210189] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:21Z level=info event=scheduler_connected addr=lb:500514898worker1 # [ 81.219318] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:21Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:540824899worker1 # [ 81.224774] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:21Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04900worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4901worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4902client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&24903worker1 # [ 81.845907] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:540964904worker1 # [ 81.860379] nix-daemon[844]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4905client # don't know how to build these paths:4906client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv4907worker1 # [ 81.884251] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540964908worker1 # [ 81.900328] nix-grpc-daemon[1090]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 1/5)4909worker1 # [ 81.995891] nix-grpc-daemon[1090]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 2/5)4910worker1 # [ 82.091716] nix-grpc-daemon[1090]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 210 ms (attempt 3/5)4911worker1 # [ 82.303856] nix-grpc-daemon[1090]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 351 ms (attempt 4/5)4912worker1 # [ 82.659967] nix-grpc-daemon[1090]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4913worker1 # [ 82.663689] nix-grpc-daemon[1090]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4914worker1 # [ 82.696309] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:23Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=04915worker1 # [ 82.702857] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:23Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=14916worker1 # [ 82.744138] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540964917worker1 # [ 82.759878] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540964918client # copying 1 paths to 'lb:50051'...4919client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...4920worker1 # [ 82.774465] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:540964921worker1 # [ 82.789029] nix-daemon[844]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4922worker1 # [ 82.818118] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:23Z level=info event=niks3_push_started pid=11674923worker1 # [ 82.964924] nix-grpc-daemon[1167]: time=2026-09-21T09:53:23.559Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""4924worker1 # [ 83.076854] nix-daemon[844]: accepted connection from pid 1179, user nix-grpc-daemon (trusted)4925worker1 # [ 83.099294] nix-daemon[844]: reaped child process 1182, status = succeeded4926worker1 # [ 83.197941] nix-daemon[844]: accepted connection from pid 1184, user nix-grpc-daemon (trusted)4927worker1 # [ 83.214216] nix-daemon[844]: reaped child process 1187, status = succeeded4928lb # [ 83.180275] niks3-server[964]: 2026/09/21 09:53:23 INFO Received uploads request method=POST path=/api/pending_closures4929worker1 # [ 83.233549] nix-grpc-daemon[1167]: time=2026-09-21T09:53:23.828Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4930worker1 # [ 83.235013] nix-grpc-daemon[1167]: time=2026-09-21T09:53:23.828Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"4931lb # [ 83.216749] niks3-server[964]: 2026/09/21 09:53:23 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst4932worker1 # [ 83.278339] nix-grpc-daemon[1167]: time=2026-09-21T09:53:23.873Z level=INFO msg="Uploading 1 narinfos"4933lb # [ 83.236231] niks3-server[964]: 2026/09/21 09:53:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign4934lb # [ 83.245726] niks3-server[964]: 2026/09/21 09:53:23 INFO Signed narinfos id=27 count=14935lb # [ 83.255103] niks3-server[964]: 2026/09/21 09:53:23 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls4936lb # [ 83.269018] niks3-server[964]: 2026/09/21 09:53:23 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete4937lb # [ 83.275614] niks3-server[964]: 2026/09/21 09:53:23 INFO Completed upload id=274938worker1 # [ 83.316603] nix-grpc-daemon[1167]: time=2026-09-21T09:53:23.912Z level=INFO msg="Upload complete. (348ms)"4939lb # [ 83.281779] niks3-server[964]: 2026/09/21 09:53:23 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo4940worker1 # [ 83.328932] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54096 duration_s=0 paths=1 nar_bytes_in=5204941worker1 # [ 83.342449] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540964942worker1 # [ 83.351753] nix-daemon[844]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4943worker1 # [ 83.354288] nix-daemon[844]: reaped child process 1165, status = succeeded4944client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...4945client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv4946worker1 # [ 83.367812] nix-daemon[844]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4947client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4948client # farm-a> LOG-farm-a-on-standby4949worker1 # [ 84.399335] nix-daemon[844]: reaped child process 1191, status = succeeded4950worker1 # [ 84.508700] nix-daemon[844]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)4951worker1 # [ 84.528568] nix-daemon[844]: reaped child process 1210, status = succeeded4952lb # [ 84.486520] niks3-server[964]: 2026/09/21 09:53:25 INFO Received uploads request method=POST path=/api/pending_closures4953worker1 # [ 84.539797] nix-grpc-daemon[1167]: time=2026-09-21T09:53:25.132Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4954worker1 # [ 84.542388] nix-grpc-daemon[1167]: time=2026-09-21T09:53:25.133Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"4955lb # [ 84.526967] niks3-server[964]: 2026/09/21 09:53:25 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv4956lb # [ 84.537207] niks3-server[964]: 2026/09/21 09:53:25 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst4957worker1 # [ 84.592292] nix-grpc-daemon[1167]: time=2026-09-21T09:53:25.186Z level=INFO msg="Uploading 1 narinfos"4958lb # [ 84.550275] niks3-server[964]: 2026/09/21 09:53:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign4959lb # [ 84.557216] niks3-server[964]: 2026/09/21 09:53:25 INFO Signed narinfos id=28 count=14960lb # [ 84.559588] niks3-server[964]: 2026/09/21 09:53:25 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls4961lb # [ 84.571327] niks3-server[964]: 2026/09/21 09:53:25 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete4962worker1 # [ 84.619883] nix-grpc-daemon[1167]: time=2026-09-21T09:53:25.215Z level=INFO msg="Upload complete. (242ms)"4963lb # [ 84.581123] niks3-server[964]: 2026/09/21 09:53:25 INFO Completed upload id=284964lb # [ 84.583370] niks3-server[964]: 2026/09/21 09:53:25 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo4965worker1 # [ 84.638070] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54096 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=14966worker1 # [ 84.656408] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:25Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=04967worker1 # [ 84.660903] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:25Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=14968worker1 # [ 84.665407] nix-daemon[844]: reaped child process 1189, status = succeeded4969worker1 # [ 84.693834] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540964970worker1 # [ 84.709051] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540964971worker1 # [ 84.723244] nix-grpc-daemon[1167]: time=2026-09-21T09:53:25.319Z level=INFO msg="All 1 paths already cached"4972client # copying 1 paths to 'lb:50051'...4973client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...4974worker1 # [ 84.733191] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:540964975worker1 # [ 84.746576] nix-daemon[844]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4976worker1 # [ 84.889045] nix-daemon[844]: accepted connection from pid 1216, user nix-grpc-daemon (trusted)4977worker1 # [ 84.906374] nix-daemon[844]: reaped child process 1219, status = succeeded4978worker1 # [ 85.012673] nix-daemon[844]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)4979worker1 # [ 85.029520] nix-daemon[844]: reaped child process 1224, status = succeeded4980lb # [ 84.992703] niks3-server[964]: 2026/09/21 09:53:25 INFO Received uploads request method=POST path=/api/pending_closures4981worker1 # [ 85.043841] nix-grpc-daemon[1167]: time=2026-09-21T09:53:25.639Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4982worker1 # [ 85.045633] nix-grpc-daemon[1167]: time=2026-09-21T09:53:25.639Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"4983lb # [ 85.023194] niks3-server[964]: 2026/09/21 09:53:25 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst4984lb # [ 85.038894] niks3-server[964]: 2026/09/21 09:53:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign4985worker1 # [ 85.088970] nix-grpc-daemon[1167]: time=2026-09-21T09:53:25.684Z level=INFO msg="Uploading 1 narinfos"4986lb # [ 85.054148] niks3-server[964]: 2026/09/21 09:53:25 INFO Signed narinfos id=29 count=14987lb # [ 85.061746] niks3-server[964]: 2026/09/21 09:53:25 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls4988lb # [ 85.072318] niks3-server[964]: 2026/09/21 09:53:25 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete4989lb # [ 85.079039] niks3-server[964]: 2026/09/21 09:53:25 INFO Completed upload id=294990worker1 # [ 85.120427] nix-grpc-daemon[1167]: time=2026-09-21T09:53:25.716Z level=INFO msg="Upload complete. (358ms)"4991lb # [ 85.082372] niks3-server[964]: 2026/09/21 09:53:25 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo4992worker1 # [ 85.128140] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54096 duration_s=0 paths=1 nar_bytes_in=6404993worker1 # [ 85.142543] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540964994worker1 # [ 85.149423] nix-daemon[844]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4995worker1 # [ 85.154542] nix-daemon[844]: reaped child process 1214, status = succeeded4996client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...4997worker1 # [ 85.165234] nix-daemon[844]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4998client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv4999client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5000client # farm-b> LOG-farm-b-on-standby5001worker1 # [ 86.597272] nix-daemon[844]: reaped child process 1228, status = succeeded5002worker1 # [ 86.719423] nix-daemon[844]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5003worker1 # [ 86.736867] nix-daemon[844]: reaped child process 1247, status = succeeded5004lb # [ 86.701569] niks3-server[964]: 2026/09/21 09:53:27 INFO Received uploads request method=POST path=/api/pending_closures5005worker1 # [ 86.753669] nix-grpc-daemon[1167]: time=2026-09-21T09:53:27.348Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5006worker1 # [ 86.755143] nix-grpc-daemon[1167]: time=2026-09-21T09:53:27.348Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5007lb # [ 86.734393] niks3-server[964]: 2026/09/21 09:53:27 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5008lb # [ 86.747797] niks3-server[964]: 2026/09/21 09:53:27 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5009lb # [ 86.764530] niks3-server[964]: 2026/09/21 09:53:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5010worker1 # [ 86.812175] nix-grpc-daemon[1167]: time=2026-09-21T09:53:27.407Z level=INFO msg="Uploading 1 narinfos"5011lb # [ 86.774189] niks3-server[964]: 2026/09/21 09:53:27 INFO Signed narinfos id=30 count=15012lb # [ 86.779357] niks3-server[964]: 2026/09/21 09:53:27 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5013lb # [ 86.790132] niks3-server[964]: 2026/09/21 09:53:27 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5014lb # [ 86.797671] niks3-server[964]: 2026/09/21 09:53:27 INFO Completed upload id=305015lb # [ 86.800153] niks3-server[964]: 2026/09/21 09:53:27 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5016worker1 # [ 86.836653] nix-grpc-daemon[1167]: time=2026-09-21T09:53:27.432Z level=INFO msg="Upload complete. (259ms)"5017worker1 # [ 86.854085] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54096 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15018worker1 # [ 86.874629] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:27Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05019worker1 # [ 86.879668] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:27Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15020worker1 # [ 86.884859] nix-daemon[844]: reaped child process 1226, status = succeeded5021worker1 # [ 86.910066] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540965022worker1 # [ 86.926025] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540965023client # copying 1 paths to 'lb:50051'...5024worker1 # [ 86.940760] nix-grpc-daemon[1167]: time=2026-09-21T09:53:27.536Z level=INFO msg="All 2 paths already cached"5025client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5026worker1 # [ 86.947967] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:540965027worker1 # [ 86.961533] nix-daemon[844]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)5028worker1 # [ 87.111347] nix-daemon[844]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5029worker1 # [ 87.128582] nix-daemon[844]: reaped child process 1256, status = succeeded5030worker1 # [ 87.228404] nix-daemon[844]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)5031worker1 # [ 87.244621] nix-daemon[844]: reaped child process 1261, status = succeeded5032lb # [ 87.209825] niks3-server[964]: 2026/09/21 09:53:27 INFO Received uploads request method=POST path=/api/pending_closures5033worker1 # [ 87.260077] nix-grpc-daemon[1167]: time=2026-09-21T09:53:27.855Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5034worker1 # [ 87.261552] nix-grpc-daemon[1167]: time=2026-09-21T09:53:27.855Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5035lb # [ 87.237219] niks3-server[964]: 2026/09/21 09:53:27 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5036worker1 # [ 87.300758] nix-grpc-daemon[1167]: time=2026-09-21T09:53:27.896Z level=INFO msg="Uploading 1 narinfos"5037lb # [ 87.252595] niks3-server[964]: 2026/09/21 09:53:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5038lb # [ 87.263771] niks3-server[964]: 2026/09/21 09:53:27 INFO Signed narinfos id=31 count=15039lb # [ 87.270071] niks3-server[964]: 2026/09/21 09:53:27 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5040lb # [ 87.280071] niks3-server[964]: 2026/09/21 09:53:27 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5041worker1 # [ 87.328120] nix-grpc-daemon[1167]: time=2026-09-21T09:53:27.924Z level=INFO msg="Upload complete. (351ms)"5042lb # [ 87.287424] niks3-server[964]: 2026/09/21 09:53:27 INFO Completed upload id=315043lb # [ 87.291589] niks3-server[964]: 2026/09/21 09:53:27 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5044worker1 # [ 87.336302] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54096 duration_s=0 paths=1 nar_bytes_in=7685045worker1 # [ 87.352119] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540965046worker1 # [ 87.358237] nix-daemon[844]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)5047worker1 # [ 87.363114] nix-daemon[844]: reaped child process 1251, status = succeeded5048client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5049worker1 # [ 87.374898] nix-daemon[844]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)5050client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5051client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5052client # farm-top> LOG-farm-top-on-standby5053worker1 # [ 88.687634] nix-daemon[844]: reaped child process 1265, status = succeeded5054worker1 # [ 88.768681] nix-daemon[844]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5055worker1 # [ 88.789442] nix-daemon[844]: reaped child process 1284, status = succeeded5056lb # [ 88.753018] niks3-server[964]: 2026/09/21 09:53:29 INFO Received uploads request method=POST path=/api/pending_closures5057worker1 # [ 88.806342] nix-grpc-daemon[1167]: time=2026-09-21T09:53:29.400Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5058worker1 # [ 88.808319] nix-grpc-daemon[1167]: time=2026-09-21T09:53:29.400Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5059lb # [ 88.795038] niks3-server[964]: 2026/09/21 09:53:29 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5060worker1 # [ 88.851538] nix-grpc-daemon[1167]: time=2026-09-21T09:53:29.447Z level=INFO msg="Uploading 1 narinfos"5061lb # [ 88.810287] niks3-server[964]: 2026/09/21 09:53:29 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5062lb # [ 88.818126] niks3-server[964]: 2026/09/21 09:53:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5063lb # [ 88.826852] niks3-server[964]: 2026/09/21 09:53:29 INFO Signed narinfos id=32 count=15064lb # [ 88.831974] niks3-server[964]: 2026/09/21 09:53:29 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5065worker1 # [ 88.881533] nix-grpc-daemon[1167]: time=2026-09-21T09:53:29.477Z level=INFO msg="Upload complete. (290ms)"5066lb # [ 88.840903] niks3-server[964]: 2026/09/21 09:53:29 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5067lb # [ 88.847785] niks3-server[964]: 2026/09/21 09:53:29 INFO Completed upload id=325068lb # [ 88.853332] niks3-server[964]: 2026/09/21 09:53:29 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5069worker1 # [ 88.894073] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54096 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15070worker1 # [ 88.913925] nix-daemon[844]: reaped child process 1263, status = succeeded5071client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 7.33 seconds)5072client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5073client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5074worker2 # [ 89.217053] systemd[1]: Listening on Nix worker-protocol over gRPC.5075worker2 # [ 89.224790] systemd[1]: Starting scheduler-token.service...5076worker2 # [ 89.306283] systemd[1]: scheduler-token.service: Deactivated successfully.5077worker2 # [ 89.312744] systemd[1]: Finished scheduler-token.service.5078worker2 # [ 89.314418] systemd[1]: scheduler-token.service: Consumed 47ms CPU time over 82ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5079worker2 # [ 89.323011] systemd[1]: Starting Nix worker-protocol over gRPC...5080worker2 # [ 89.448402] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5081worker2 # [ 89.450600] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5082worker2 # [ 89.454433] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15083worker2 # [ 89.460067] nix-grpc-daemon[1310]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5084worker2 # [ 89.461532] nix-grpc-daemon[1310]: E0000 00:00:1789984410.050183 1310 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5085lb: must succeed: curl -sf localhost:9901/clusters5086worker2 # [ 89.504151] nix-grpc-daemon[1310]: E0000 00:00:1789984410.056344 1310 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5087worker2 # [ 89.506511] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5088worker2 # [ 89.517409] nix-grpc-daemon[1310]: E0000 00:00:1789984410.068987 1350 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5089worker2 # [ 89.520241] nix-grpc-daemon[1310]: E0000 00:00:1789984410.089058 1350 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5090worker2 # [ 89.522657] nix-grpc-daemon[1310]: E0000 00:00:1789984410.089197 1350 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5091worker2 # [ 89.525098] systemd[1]: Started Nix worker-protocol over gRPC.5092worker2 # [ 89.525937] nix-grpc-daemon[1310]: E0000 00:00:1789984410.107797 1350 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5093worker2 # [ 89.533636] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:30Z level=info event=scheduler_connected addr=lb:500515094worker1 # [ 89.528164] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:30Z level=info event=oidc_jwks provider=mock keys=15095worker1 # [ 89.533662] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:30Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:540825096worker1 # [ 89.538812] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:30Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05097lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5098worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5099worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5100lb: must succeed: curl -sf localhost:9901/clusters5101lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5102lb: must succeed: curl -sf localhost:9901/clusters5103lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5104worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5105worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5106(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.65 seconds)5107subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5108client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5109client # [ 89.858951] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5110client # Running as unit: rs.service; invocation ID: 1abb4f0968aa4855ba504f194b742eed5111client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.08 seconds)5112worker2 # [ 90.040151] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:438525113worker2 # [ 90.055055] nix-daemon[834]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5114client # [ 90.085087] nix[1216]: don't know how to build these paths:5115client # [ 90.086601] nix[1216]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5116worker1 # [ 90.069331] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:30Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05117worker1 # [ 90.075345] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:30Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15118worker1 # [ 90.122625] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540965119worker1 # [ 90.138740] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540965120client # [ 90.180248] nix[1216]: copying 1 paths to 'lb:50051'...5121client # [ 90.185823] nix[1216]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5122worker1 # [ 90.156328] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:540965123worker1 # [ 90.171308] nix-daemon[844]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)5124worker1 # [ 90.337092] nix-daemon[844]: accepted connection from pid 1297, user nix-grpc-daemon (trusted)5125worker1 # [ 90.360289] nix-daemon[844]: reaped child process 1300, status = succeeded5126worker1 # [ 90.456159] nix-daemon[844]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5127worker1 # [ 90.471881] nix-daemon[844]: reaped child process 1305, status = succeeded5128lb # [ 90.439072] niks3-server[964]: 2026/09/21 09:53:31 INFO Received uploads request method=POST path=/api/pending_closures5129worker1 # [ 90.490006] nix-grpc-daemon[1167]: time=2026-09-21T09:53:31.085Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5130worker1 # [ 90.491460] nix-grpc-daemon[1167]: time=2026-09-21T09:53:31.085Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5131lb # [ 90.469983] niks3-server[964]: 2026/09/21 09:53:31 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5132lb # [ 90.487410] niks3-server[964]: 2026/09/21 09:53:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5133worker1 # [ 90.534382] nix-grpc-daemon[1167]: time=2026-09-21T09:53:31.130Z level=INFO msg="Uploading 1 narinfos"5134lb # [ 90.495365] niks3-server[964]: 2026/09/21 09:53:31 INFO Signed narinfos id=33 count=15135lb # [ 90.505557] niks3-server[964]: 2026/09/21 09:53:31 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5136lb # [ 90.515536] niks3-server[964]: 2026/09/21 09:53:31 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5137worker1 # [ 90.561413] nix-grpc-daemon[1167]: time=2026-09-21T09:53:31.157Z level=INFO msg="Upload complete. (373ms)"5138lb # [ 90.523153] niks3-server[964]: 2026/09/21 09:53:31 INFO Completed upload id=335139lb # [ 90.527931] niks3-server[964]: 2026/09/21 09:53:31 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5140worker1 # [ 90.569857] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54096 duration_s=0 paths=1 nar_bytes_in=4165141worker1 # [ 90.585238] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540965142worker1 # [ 90.594981] nix-daemon[844]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)5143worker1 # [ 90.597664] nix-daemon[844]: reaped child process 1295, status = succeeded5144client # [ 90.637609] nix[1216]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5145worker1 # [ 90.611886] nix-daemon[844]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)5146lb: must succeed: curl -sf localhost:9901/clusters5147lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5148lb: must succeed: curl -sf localhost:9901/clusters5149lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5150lb: must succeed: curl -sf localhost:9901/clusters5151lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5152lb: must succeed: curl -sf localhost:9901/clusters5153lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5154worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5155worker2 # [ 92.654309] systemd-journald[268]: Received client request to rotate journal, rotating.5156worker2 # [ 92.678248] systemd-journald[268]: Vacuuming done, freed 0B of archived journals from /var/log/journal/8463986d2fd14b9faed621ce78c0b4f3.5157worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5158worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5159worker1 # [ 92.709846] systemd[1]: Stopping Nix worker-protocol over gRPC...5160worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5161client: must succeed: sleep 5.05162worker1 # [ 93.017393] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:33Z level=info event=draining5163client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5164worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5165worker1 # [ 98.094435] nix-daemon[844]: reaped child process 1309, status = succeeded5166worker1 # [ 98.164105] nix-daemon[844]: accepted connection from pid 1350, user nix-grpc-daemon (trusted)5167worker1 # [ 98.183846] nix-daemon[844]: reaped child process 1353, status = succeeded5168lb # [ 98.146661] niks3-server[964]: 2026/09/21 09:53:38 INFO Received uploads request method=POST path=/api/pending_closures5169worker1 # [ 98.204489] nix-grpc-daemon[1167]: time=2026-09-21T09:53:38.797Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5170worker1 # [ 98.206329] nix-grpc-daemon[1167]: time=2026-09-21T09:53:38.798Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5171lb # [ 98.212277] niks3-server[964]: 2026/09/21 09:53:38 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5172lb # [ 98.221144] niks3-server[964]: 2026/09/21 09:53:38 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5173worker1 # [ 98.277349] nix-grpc-daemon[1167]: time=2026-09-21T09:53:38.873Z level=INFO msg="Uploading 1 narinfos"5174lb # [ 98.235451] niks3-server[964]: 2026/09/21 09:53:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5175lb # [ 98.242454] niks3-server[964]: 2026/09/21 09:53:38 INFO Signed narinfos id=34 count=15176lb # [ 98.247053] niks3-server[964]: 2026/09/21 09:53:38 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5177lb # [ 98.255720] niks3-server[964]: 2026/09/21 09:53:38 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5178lb # [ 98.262817] niks3-server[964]: 2026/09/21 09:53:38 INFO Completed upload id=345179worker1 # [ 98.303901] nix-grpc-daemon[1167]: time=2026-09-21T09:53:38.899Z level=INFO msg="Upload complete. (276ms)"5180lb # [ 98.265938] niks3-server[964]: 2026/09/21 09:53:38 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5181worker1 # [ 98.321386] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54096 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15182worker1 # [ 98.337129] nix-daemon[844]: reaped child process 1307, status = succeeded5183client # [ 98.379592] systemd[1]: rs.service: Deactivated successfully.5184client # [ 98.383969] systemd[1]: rs.service: Consumed 187ms CPU time over 8.524s wall clock time, 43.4M memory peak, 7K incoming IP traffic, 8.4K outgoing IP traffic.5185lb # [ 98.628396] envoy[900]: conn peer=[2001:db8:1::1]:46276 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9679 tx=4114 ms=70725186lb # [ 98.633780] envoy[900]: conn peer=[2001:db8:1::1]:46290 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1404 ms=83085187worker1 # [ 99.019831] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:39Z level=info event=signal_exit5188worker1 # [ 100.020524] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:40Z level=info event=scheduler_yield5189lb # [ 99.981965] niks3-server[964]: 2026/09/21 09:53:40 INFO lead: released remote=192.168.1.3:405165190worker1 # [ 100.026106] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:40Z level=info event=scheduler_restarting addr=lb:500515191worker2 # [ 100.044899] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:40Z level=info event=scheduler_restarting addr=lb:500515192worker1 # [ 100.033703] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:540825193worker1 # [ 100.038270] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:40Z level=info event=worker_gone addr=192.168.1.3:500515194worker1 # [ 100.041973] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:540825195worker1 # [ 100.046062] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:40Z level=info event=worker_gone addr=192.168.1.4:500515196worker2 # [ 100.244533] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:40Z level=info event=scheduler_connected addr=lb:500515197worker1 # [ 100.227315] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:40Z level=info event=scheduler_connected addr=lb:500515198worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.21 seconds)5199client: waiting for success: ! systemctl is-active rs5200worker1 # [ 100.232863] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5201worker2 # [ 100.250764] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5202client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5203client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5204client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5205client: must fail: journalctl -u rs | grep -q 'reconnecting'5206client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5207lb: must succeed: curl -sf localhost:9901/clusters5208worker1 # [ 100.433819] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:41Z level=info event=scheduler_connected addr=lb:500515209worker2 # [ 100.450412] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:41Z level=info event=scheduler_connected addr=lb:500515210worker1 # [ 100.439476] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5211worker2 # [ 100.455686] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5212lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5213worker1 # [ 100.639997] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:41Z level=info event=scheduler_connected addr=lb:500515214worker2 # [ 100.656753] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:41Z level=info event=scheduler_connected addr=lb:500515215worker1 # [ 100.645440] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5216worker2 # [ 100.661792] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5217worker1 # [ 100.846193] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:41Z level=info event=scheduler_connected addr=lb:500515218worker2 # [ 100.862934] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:41Z level=info event=scheduler_connected addr=lb:500515219worker1 # [ 100.851614] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5220worker2 # [ 100.868243] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5221worker1 # [ 101.027431] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:41Z level=info event=scheduler_restarting5222worker2 # [ 101.070941] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:41Z level=info event=scheduler_connected addr=lb:500515223worker2 # [ 101.075101] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5224worker1 # [ 101.083410] nix-grpc-daemon[1090]: E0000 00:00:1789984421.669431 1135 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5225worker1 # [ 101.089606] nix-grpc-daemon[1090]: E0000 00:00:1789984421.671190 1135 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5226worker1 # [ 101.096612] nix-grpc-daemon[1090]: ts=2026-09-21T09:53:41Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5227worker1 # [ 101.100945] nix-daemon[844]: reaped child process 1157, status = succeeded5228worker2 # [ 101.272181] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:41Z level=info event=scheduler_connected addr=lb:500515229worker2 # [ 101.277151] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5230lb: must succeed: curl -sf localhost:9901/clusters5231worker2 # [ 101.478496] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:42Z level=info event=scheduler_connected addr=lb:500515232worker2 # [ 101.483640] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5233lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5234worker1 # [ 101.651893] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5235worker1 # [ 101.658962] systemd[1]: Stopped Nix worker-protocol over gRPC.5236worker1 # [ 101.665204] systemd[1]: nix-grpc-daemon.service: Consumed 1.888s CPU time over 25.815s wall clock time, 38.9M memory peak, 49.8K incoming IP traffic, 93.8K outgoing IP traffic.5237worker2 # [ 101.684681] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:42Z level=info event=scheduler_connected addr=lb:500515238worker2 # [ 101.689731] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5239worker1 # [ 101.692525] systemd[1]: Starting Nix worker-protocol over gRPC...5240worker1 # [ 101.823986] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:42Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5241worker1 # [ 101.829141] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:42Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5242worker1 # [ 101.831993] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:42Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15243lb # [ 101.804832] niks3-server[964]: 2026/09/21 09:53:42 INFO lead: acquired remote=192.168.1.3:450765244worker1 # [ 101.856060] nix-grpc-daemon[1360]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5245worker1 # [ 101.857514] nix-grpc-daemon[1360]: E0000 00:00:1789984422.430825 1360 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5246worker1 # [ 101.859814] nix-grpc-daemon[1360]: E0000 00:00:1789984422.433202 1360 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5247worker2 # [ 101.890917] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:42Z level=info event=scheduler_connected addr=lb:500515248worker1 # [ 101.880816] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:42Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5249worker2 # [ 101.896085] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5250worker1 # [ 101.883007] nix-grpc-daemon[1360]: E0000 00:00:1789984422.439685 1399 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5251worker1 # [ 101.885702] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:42Z level=info event=scheduler_take_over5252worker1 # [ 101.886860] systemd[1]: Started Nix worker-protocol over gRPC.5253worker1 # [ 101.891070] nix-grpc-daemon[1360]: E0000 00:00:1789984422.461276 1399 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5254worker1 # [ 101.895660] nix-grpc-daemon[1360]: E0000 00:00:1789984422.461425 1399 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5255worker1 # [ 101.898269] nix-grpc-daemon[1360]: E0000 00:00:1789984422.475906 1399 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5256worker1 # [ 101.900652] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:42Z level=info event=scheduler_connected addr=lb:500515257worker1 # [ 101.901963] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5258worker2 # [ 102.098892] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:42Z level=info event=scheduler_connected addr=lb:500515259worker2 # [ 102.103049] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5260worker1 # [ 102.096727] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:42Z level=info event=scheduler_connected addr=lb:500515261worker1 # [ 102.101702] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5262worker2 # [ 102.299617] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:42Z level=info event=scheduler_connected addr=lb:500515263worker2 # [ 102.304759] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5264worker1 # [ 102.302458] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:42Z level=info event=scheduler_connected addr=lb:500515265worker1 # [ 102.307657] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5266worker2 # [ 102.505835] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:43Z level=info event=scheduler_connected addr=lb:500515267worker2 # [ 102.510897] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5268worker1 # [ 102.508569] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:43Z level=info event=scheduler_connected addr=lb:500515269worker1 # [ 102.513738] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5270lb: must succeed: curl -sf localhost:9901/clusters5271lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5272worker2 # [ 102.711908] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:43Z level=info event=scheduler_connected addr=lb:500515273worker1 # [ 102.708128] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:43Z level=info event=oidc_jwks provider=mock keys=15274worker1 # [ 102.714419] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:504345275worker1 # [ 102.721160] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05276worker1 # [ 102.726617] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:43Z level=info event=scheduler_connected addr=lb:500515277worker1 # [ 102.730630] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:504345278worker1 # [ 102.735451] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05279lb: must succeed: curl -sf localhost:9901/clusters5280lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5281worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5282worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5283(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.96 seconds)5284subtest: scheduler restart mid-build: build finishes, no double build5285client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5286client # [ 103.821641] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5287client # Running as unit: mid.service; invocation ID: c816b99ae04f4dc897683be33af62e315288client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)5289worker1 # [ 104.010727] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:504405290worker1 # [ 104.023185] nix-daemon[844]: accepted connection from pid 1360, user nix-grpc-daemon (trusted)5291client # [ 104.071332] nix[1252]: don't know how to build these paths:5292client # [ 104.072830] nix[1252]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5293worker1 # [ 104.056123] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:44Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05294worker1 # [ 104.062092] nix-grpc-daemon[1360]: ts=2026-09-21T09:53:44Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15295worker2 # [ 104.129088] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:438525296worker2 # [ 104.149026] nix-grpc-daemon[1310]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 65 ms (attempt 1/5)5297worker2 # [ 104.223416] nix-grpc-daemon[1310]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 134 ms (attempt 2/5)5298worker2 # [ 104.359359] nix-grpc-daemon[1310]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 163 ms (attempt 3/5)5299worker2 # [ 104.524287] nix-grpc-daemon[1310]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 302 ms (attempt 4/5)5300worker2 # [ 104.828824] nix-grpc-daemon[1310]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5301worker2 # [ 104.835405] nix-grpc-daemon[1310]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5302worker2 # [ 104.854561] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:438525303worker2 # [ 104.869853] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:438525304client # [ 104.886307] nix[1252]: copying 1 paths to 'lb:50051'...5305client # [ 104.889456] nix[1252]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5306worker2 # [ 104.892568] nix-daemon[834]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5307worker2 # [ 104.922582] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:45Z level=info event=niks3_push_started pid=14245308worker2 # [ 105.080110] nix-grpc-daemon[1424]: time=2026-09-21T09:53:45.669Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5309worker2 # [ 105.201824] nix-daemon[834]: accepted connection from pid 1441, user nix-grpc-daemon (trusted)5310worker2 # [ 105.220776] nix-daemon[834]: reaped child process 1444, status = succeeded5311worker2 # [ 105.325026] nix-daemon[834]: accepted connection from pid 1446, user nix-grpc-daemon (trusted)5312worker2 # [ 105.340954] nix-daemon[834]: reaped child process 1449, status = succeeded5313lb # [ 105.290068] niks3-server[964]: 2026/09/21 09:53:45 INFO Received uploads request method=POST path=/api/pending_closures5314worker2 # [ 105.358334] nix-grpc-daemon[1424]: time=2026-09-21T09:53:45.948Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5315worker2 # [ 105.359779] nix-grpc-daemon[1424]: time=2026-09-21T09:53:45.948Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5316lb # [ 105.334240] niks3-server[964]: 2026/09/21 09:53:45 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5317worker2 # [ 105.404113] nix-grpc-daemon[1424]: time=2026-09-21T09:53:45.995Z level=INFO msg="Uploading 1 narinfos"5318lb # [ 105.351239] niks3-server[964]: 2026/09/21 09:53:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5319lb # [ 105.365472] niks3-server[964]: 2026/09/21 09:53:45 INFO Signed narinfos id=35 count=15320lb # [ 105.376916] niks3-server[964]: 2026/09/21 09:53:45 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5321lb # [ 105.390208] niks3-server[964]: 2026/09/21 09:53:45 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5322lb # [ 105.404099] niks3-server[964]: 2026/09/21 09:53:46 INFO Completed upload id=355323worker2 # [ 105.462018] nix-grpc-daemon[1424]: time=2026-09-21T09:53:46.053Z level=INFO msg="Upload complete. (383ms)"5324lb # [ 105.420391] niks3-server[964]: 2026/09/21 09:53:46 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5325worker2 # [ 105.476847] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43852 duration_s=0 paths=1 nar_bytes_in=4245326worker2 # [ 105.494892] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:438525327worker2 # [ 105.504934] nix-daemon[834]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5328worker2 # [ 105.508521] nix-daemon[834]: reaped child process 1422, status = succeeded5329client # [ 105.533182] nix[1252]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5330worker2 # [ 105.521929] nix-daemon[834]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5331lb: must succeed: curl -sf localhost:9901/clusters5332lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5333lb: must succeed: curl -sf localhost:9901/clusters5334lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5335worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5336worker1 # [ 107.739266] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1360 (nix-grpc-daemon) on client request.5337worker1 # [ 107.758590] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5338worker2 # [ 107.793206] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:48Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"5339lb # [ 107.732928] niks3-server[964]: 2026/09/21 09:53:48 INFO lead: released remote=192.168.1.3:450765340client # [ 107.807685] nix[1252]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5341worker1 # [ 107.787368] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5342worker1 # [ 107.811001] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5343worker1 # [ 107.820396] systemd[1]: nix-grpc-daemon.service: Consumed 235ms CPU time over 6.094s wall clock time, 14.5M memory peak, 10K incoming IP traffic, 11.7K outgoing IP traffic.5344worker1 # [ 107.822629] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5345worker1 # [ 107.824558] systemd[1]: Starting Nix worker-protocol over gRPC...5346worker1 # [ 107.825853] nix-daemon[844]: reaped child process 1419, status = succeeded5347worker1 # [ 107.916440] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5348worker1 # [ 107.918750] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5349worker1 # [ 107.922377] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15350worker1 # [ 107.928066] nix-grpc-daemon[1447]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5351worker1 # [ 107.929512] nix-grpc-daemon[1447]: E0000 00:00:1789984428.522814 1447 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5352worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.25 seconds)5353client: must succeed: sleep 5.05354worker1 # [ 107.939691] nix-grpc-daemon[1447]: E0000 00:00:1789984428.528912 1447 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5355worker1 # [ 107.943872] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:48Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5356lb # [ 107.903768] niks3-server[964]: 2026/09/21 09:53:48 INFO lead: acquired remote=192.168.1.3:450945357worker1 # [ 107.972133] systemd[1]: Started Nix worker-protocol over gRPC.5358worker1 # [ 107.973028] nix-grpc-daemon[1447]: E0000 00:00:1789984428.542375 1487 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5359worker1 # [ 107.975298] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:48Z level=info event=scheduler_take_over5360worker2 # [ 107.993476] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:48Z level=info event=scheduler_connected addr=lb:500515361worker1 # [ 107.983857] nix-grpc-daemon[1447]: E0000 00:00:1789984428.557464 1487 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5362worker1 # [ 107.986520] nix-grpc-daemon[1447]: E0000 00:00:1789984428.557602 1487 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5363worker1 # [ 107.990027] nix-grpc-daemon[1447]: E0000 00:00:1789984428.577112 1487 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5364worker1 # [ 107.997312] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:48Z level=info event=scheduler_connected addr=lb:500515365worker1 # [ 107.999855] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:48Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:504485366worker1 # [ 108.003317] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:48Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05367worker1 # [ 108.005201] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:48Z level=info event=oidc_jwks provider=mock keys=15368worker1 # [ 108.006434] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:504485369worker1 # [ 108.008128] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=15370lb # [ 108.626712] envoy[900]: conn peer=[2001:db8:1::3]:34952 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4417 tx=1371 ms=249975371lb # [ 108.632035] envoy[900]: conn peer=[2001:db8:1::3]:37342 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=972 tx=362 ms=58925372client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5373client: waiting for success: ! systemctl is-active mid5374worker2 # [ 113.364526] nix-daemon[834]: reaped child process 1453, status = succeeded5375worker2 # [ 113.479198] nix-daemon[834]: accepted connection from pid 1490, user nix-grpc-daemon (trusted)5376worker2 # [ 113.502594] nix-daemon[834]: reaped child process 1493, status = succeeded5377lb # [ 113.443543] niks3-server[964]: 2026/09/21 09:53:54 INFO Received uploads request method=POST path=/api/pending_closures5378worker2 # [ 113.516054] nix-grpc-daemon[1424]: time=2026-09-21T09:53:54.104Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5379worker2 # [ 113.517552] nix-grpc-daemon[1424]: time=2026-09-21T09:53:54.104Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5380lb # [ 113.490631] niks3-server[964]: 2026/09/21 09:53:54 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5381worker2 # [ 113.566058] nix-grpc-daemon[1424]: time=2026-09-21T09:53:54.156Z level=INFO msg="Uploading 1 narinfos"5382lb # [ 113.508104] niks3-server[964]: 2026/09/21 09:53:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5383lb # [ 113.516175] niks3-server[964]: 2026/09/21 09:53:54 INFO Signed narinfos id=36 count=15384lb # [ 113.524413] niks3-server[964]: 2026/09/21 09:53:54 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5385lb # [ 113.534201] niks3-server[964]: 2026/09/21 09:53:54 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5386lb # [ 113.546846] niks3-server[964]: 2026/09/21 09:53:54 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5387lb # [ 113.555152] niks3-server[964]: 2026/09/21 09:53:54 INFO Completed upload id=365388worker2 # [ 113.610762] nix-grpc-daemon[1424]: time=2026-09-21T09:53:54.201Z level=INFO msg="Upload complete. (272ms)"5389lb # [ 113.560524] niks3-server[964]: 2026/09/21 09:53:54 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5390worker2 # [ 113.625304] nix-grpc-daemon[1310]: ts=2026-09-21T09:53:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43852 duration_s=8 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15391worker2 # [ 113.641483] nix-daemon[834]: reaped child process 1451, status = succeeded5392client # [ 113.671071] systemd[1]: mid.service: Deactivated successfully.5393client # [ 113.675413] systemd[1]: mid.service: Consumed 212ms CPU time over 9.853s wall clock time, 39.3M memory peak, 7.8K incoming IP traffic, 8.9K outgoing IP traffic.5394client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)5395client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5396client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5397(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.69 seconds)5398subtest: interrupting the client stops the build on the worker5399client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr5400client # [ 114.482436] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.5401client # Running as unit: intr.service; invocation ID: 0fec334e6c8f4a47a3a840b22c8f6a915402client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)5403worker1 # [ 114.639581] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536505404worker1 # [ 114.651004] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5405client # [ 114.698695] nix[1288]: don't know how to build these paths:5406client # [ 114.699754] nix[1288]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5407worker1 # [ 114.677432] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:55Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05408worker1 # [ 114.684211] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:55Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15409worker1 # [ 114.729764] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505410worker1 # [ 114.746352] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 1/5)5411worker1 # [ 114.840818] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 2/5)5412worker1 # [ 114.942882] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 3/5)5413worker1 # [ 115.011510] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 433 ms (attempt 4/5)5414worker1 # [ 115.448541] nix-grpc-daemon[1447]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5415worker1 # [ 115.454932] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5416worker1 # [ 115.473644] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536505417client # [ 115.516288] nix[1288]: copying 1 paths to 'lb:50051'...5418client # [ 115.520777] nix[1288]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5419worker1 # [ 115.492659] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536505420worker1 # [ 115.506900] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5421worker1 # [ 115.541059] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:56Z level=info event=niks3_push_started pid=15215422worker1 # [ 115.717965] nix-grpc-daemon[1521]: time=2026-09-21T09:53:56.312Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5423worker1 # [ 115.830349] nix-daemon[844]: accepted connection from pid 1538, user nix-grpc-daemon (trusted)5424worker1 # [ 115.846221] nix-daemon[844]: reaped child process 1541, status = succeeded5425worker1 # [ 115.948494] nix-daemon[844]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5426worker1 # [ 115.963910] nix-daemon[844]: reaped child process 1546, status = succeeded5427lb # [ 115.929687] niks3-server[964]: 2026/09/21 09:53:56 INFO Received uploads request method=POST path=/api/pending_closures5428worker1 # [ 115.982304] nix-grpc-daemon[1521]: time=2026-09-21T09:53:56.577Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5429worker1 # [ 115.983760] nix-grpc-daemon[1521]: time=2026-09-21T09:53:56.577Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5430lb # [ 115.967934] niks3-server[964]: 2026/09/21 09:53:56 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5431worker1 # [ 116.026410] nix-grpc-daemon[1521]: time=2026-09-21T09:53:56.622Z level=INFO msg="Uploading 1 narinfos"5432lb # [ 115.976729] niks3-server[964]: 2026/09/21 09:53:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5433lb # [ 115.988597] niks3-server[964]: 2026/09/21 09:53:56 INFO Signed narinfos id=37 count=15434lb # [ 115.998169] niks3-server[964]: 2026/09/21 09:53:56 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5435lb # [ 116.013876] niks3-server[964]: 2026/09/21 09:53:56 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5436lb # [ 116.020062] niks3-server[964]: 2026/09/21 09:53:56 INFO Completed upload id=375437worker1 # [ 116.061178] nix-grpc-daemon[1521]: time=2026-09-21T09:53:56.657Z level=INFO msg="Upload complete. (340ms)"5438lb # [ 116.022756] niks3-server[964]: 2026/09/21 09:53:56 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5439worker1 # [ 116.068760] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=0 paths=1 nar_bytes_in=4245440worker1 # [ 116.083670] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505441worker1 # [ 116.090579] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5442worker1 # [ 116.095541] nix-daemon[844]: reaped child process 1519, status = succeeded5443client # [ 116.136753] nix[1288]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5444worker1 # [ 116.109757] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5445client: must succeed: systemctl kill -s INT intr5446client # [ 118.166146] systemd[1]: intr.service: Sent signal SIGINT to main process 1288 (nix) on client request.5447worker1 # [ 118.147073] nix-grpc-daemon[1447]: ts=2026-09-21T09:53:58Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5448client # [ 118.176686] nix[1288]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5449client # [ 118.188488] nix[1288]: error: interrupted by the user5450client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)5451client # [ 118.219166] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5452client # [ 118.225254] systemd[1]: intr.service: Failed with result 'exit-code'.5453client # [ 118.227882] systemd[1]: intr.service: Consumed 174ms CPU time over 3.740s wall clock time, 41.4M memory peak, 6.5K incoming IP traffic, 8.3K outgoing IP traffic.5454worker1 # [ 118.336530] nix-daemon[844]: reaped child process 1548, status = succeeded5455worker1 # [ 118.337830] nix-daemon[1550]: unexpected Nix daemon error: error: interrupted by the user5456worker1 # [ 118.345480] nix-daemon[844]: reaped child process 1550, status = failed with exit code 15457lb # [ 118.628906] envoy[900]: conn peer=[2001:db8:1::1]:59686 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3078 tx=1496 ms=96195458lb # [ 118.633951] envoy[900]: conn peer=[2001:db8:1::1]:52942 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3013 tx=851 ms=35395459(finished: subtest: interrupting the client stops the build on the worker, in 5.01 seconds)5460subtest: a deploy mid-build drains: the build finishes, the new generation takes over5461client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw5462client # [ 119.526101] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.5463client # Running as unit: sw.service; invocation ID: c3f6d73b8ac6418784a63a920fd014815464client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds)5465worker1 # [ 119.702097] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536505466client # [ 119.739229] nix[1309]: don't know how to build these paths:5467client # [ 119.740477] nix[1309]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5468worker1 # [ 119.724096] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:00Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05469worker1 # [ 119.730125] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:00Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15470worker2 # [ 119.786869] nix-grpc-daemon[1310]: ts=2026-09-21T09:54:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:438525471worker2 # [ 119.804770] nix-grpc-daemon[1310]: ts=2026-09-21T09:54:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:438525472client # [ 119.830523] nix[1309]: copying 1 paths to 'lb:50051'...5473client # [ 119.836179] nix[1309]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5474worker2 # [ 119.822552] nix-grpc-daemon[1310]: ts=2026-09-21T09:54:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:438525475worker2 # [ 119.837523] nix-daemon[834]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5476worker2 # [ 119.995373] nix-daemon[834]: accepted connection from pid 1525, user nix-grpc-daemon (trusted)5477worker2 # [ 120.017288] nix-daemon[834]: reaped child process 1528, status = succeeded5478worker2 # [ 120.114919] nix-daemon[834]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)5479worker2 # [ 120.133943] nix-daemon[834]: reaped child process 1533, status = succeeded5480lb # [ 120.083458] niks3-server[964]: 2026/09/21 09:54:00 INFO Received uploads request method=POST path=/api/pending_closures5481worker2 # [ 120.152893] nix-grpc-daemon[1424]: time=2026-09-21T09:54:00.743Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5482worker2 # [ 120.154393] nix-grpc-daemon[1424]: time=2026-09-21T09:54:00.743Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5483lb # [ 120.117599] niks3-server[964]: 2026/09/21 09:54:00 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5484lb # [ 120.131172] niks3-server[964]: 2026/09/21 09:54:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5485worker2 # [ 120.194580] nix-grpc-daemon[1424]: time=2026-09-21T09:54:00.785Z level=INFO msg="Uploading 1 narinfos"5486lb # [ 120.139299] niks3-server[964]: 2026/09/21 09:54:00 INFO Signed narinfos id=38 count=15487lb # [ 120.147339] niks3-server[964]: 2026/09/21 09:54:00 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5488lb # [ 120.157976] niks3-server[964]: 2026/09/21 09:54:00 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5489lb # [ 120.166140] niks3-server[964]: 2026/09/21 09:54:00 INFO Completed upload id=385490worker2 # [ 120.221451] nix-grpc-daemon[1424]: time=2026-09-21T09:54:00.812Z level=INFO msg="Upload complete. (366ms)"5491lb # [ 120.170602] niks3-server[964]: 2026/09/21 09:54:00 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5492worker2 # [ 120.233291] nix-grpc-daemon[1310]: ts=2026-09-21T09:54:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43852 duration_s=0 paths=1 nar_bytes_in=4165493worker2 # [ 120.247890] nix-grpc-daemon[1310]: ts=2026-09-21T09:54:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:438525494worker2 # [ 120.257221] nix-daemon[834]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5495worker2 # [ 120.260296] nix-daemon[834]: reaped child process 1523, status = succeeded5496client # [ 120.284393] nix[1309]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5497worker2 # [ 120.274737] nix-daemon[834]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5498worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25499worker2 # Checking switch inhibitors... done5500worker2 # activating the configuration...5501worker2 # [ 122.516514] nixos[1569]: switching to system configuration /nix/store/7cxvz5my1dw85f5wp78sc1kckff176ap-nixos-system-worker2-test5502worker2 # [ 122.518327] systemd[1]: Stopped target Remote File Systems.5503worker2 # [ 122.519100] systemd[1]: Stopped target Local File Systems.5504worker2 # setting up /etc...5505worker2 # [ 123.084798] systemd[1]: Reload requested from client PID 1569 ('.switch-to-conf') (unit backdoor.service)...5506worker2 # [ 123.086388] systemd[1]: Reloading...5507worker2 # [ 128.000409] systemd[1]: Reloading finished in 4907 ms.5508worker2 # restarting sysinit-reactivation.target5509worker2 # [ 128.108165] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5510worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5511worker2 # [ 128.240406] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5512worker2 # [ 128.245544] systemd[1]: Reached target Reactivate sysinit units.5513worker2 # [ 128.246390] systemd[1]: Reloading D-Bus System Message Bus...5514worker2 # [ 128.247474] dbus-broker-launch[491]: Caught SIGHUP, trigger reload.5515worker2 # [ 128.258453] systemd[1]: Reloading Nix worker-protocol over gRPC...5516worker2 # [ 128.277288] dbus-broker-launch[491]: Looking up NSS user entry for 'systemd-timesync'...5517worker2 # [ 128.280559] dbus-broker-launch[491]: NSS returned no entry for 'systemd-timesync'5518worker2 # [ 128.282832] dbus-broker-launch[491]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5519worker2 # [ 128.295388] systemd[1]: Reloaded Nix worker-protocol over gRPC.5520worker2 # [ 128.305436] systemd[1]: Reloaded D-Bus System Message Bus.5521worker2 # [ 128.307613] systemd[1]: Reached target Local File Systems.5522worker2 # [ 128.310476] systemd[1]: Reached target Remote File Systems.5523worker2 # [ 128.316472] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5524worker2 # [ 128.317818] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5525worker2 # [ 128.328403] systemd[1]: Starting linger-users.service...5526worker2 # [ 128.336113] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5527worker2 # [ 128.348056] systemd[1]: Starting Create SUID/SGID Wrappers...5528worker2 # [ 128.348938] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5529worker2 # [ 128.350146] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675530worker2 # [ 128.357966] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5531worker2 # [ 128.362602] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5532worker2 # [ 128.370919] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5533worker2 # [ 128.395724] systemd[1]: linger-users.service: Deactivated successfully.5534worker2 # [ 128.397391] systemd[1]: Finished linger-users.service.5535worker2 # [ 128.486529] nix-grpc-daemon[1310]: ts=2026-09-21T09:54:09Z level=info event=draining5536worker2 # [ 128.698899] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5537worker2 # [ 128.704061] systemd[1]: Finished Create SUID/SGID Wrappers.5538worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5539worker2 # [ 128.953221] nixos[1569]: finished switching to system configuration /nix/store/7cxvz5my1dw85f5wp78sc1kckff176ap-nixos-system-worker2-test5540worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.89 seconds)5541worker2: must succeed: pgrep -f 'read -t [9]0 x'5542worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)5543client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&25544client # don't know how to build these paths:5545client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5546worker1 # [ 129.232849] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536505547worker1 # [ 129.239424] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536505548worker1 # [ 129.266729] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:09Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05549worker1 # [ 129.273862] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:09Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=15550worker1 # [ 129.319999] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505551worker1 # [ 129.335626] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536505552client # copying 1 paths to 'lb:50051'...5553worker1 # [ 129.352214] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536505554client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5555worker1 # [ 129.375496] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5556worker1 # [ 129.543535] nix-daemon[844]: accepted connection from pid 1605, user nix-grpc-daemon (trusted)5557worker1 # [ 129.567988] nix-daemon[844]: reaped child process 1608, status = succeeded5558worker1 # [ 129.670817] nix-daemon[844]: accepted connection from pid 1610, user nix-grpc-daemon (trusted)5559worker1 # [ 129.691820] nix-daemon[844]: reaped child process 1613, status = succeeded5560lb # [ 129.656090] niks3-server[964]: 2026/09/21 09:54:10 INFO Received uploads request method=POST path=/api/pending_closures5561worker1 # [ 129.707197] nix-grpc-daemon[1521]: time=2026-09-21T09:54:10.302Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5562worker1 # [ 129.708929] nix-grpc-daemon[1521]: time=2026-09-21T09:54:10.302Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5563lb # [ 129.685675] niks3-server[964]: 2026/09/21 09:54:10 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5564lb # [ 129.702326] niks3-server[964]: 2026/09/21 09:54:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5565worker1 # [ 129.751002] nix-grpc-daemon[1521]: time=2026-09-21T09:54:10.346Z level=INFO msg="Uploading 1 narinfos"5566lb # [ 129.714936] niks3-server[964]: 2026/09/21 09:54:10 INFO Signed narinfos id=39 count=15567lb # [ 129.721034] niks3-server[964]: 2026/09/21 09:54:10 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5568worker1 # [ 129.774591] nix-grpc-daemon[1521]: time=2026-09-21T09:54:10.370Z level=INFO msg="Upload complete. (380ms)"5569lb # [ 129.733980] niks3-server[964]: 2026/09/21 09:54:10 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5570worker1 # [ 129.782313] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=0 paths=1 nar_bytes_in=5285571lb # [ 129.743939] niks3-server[964]: 2026/09/21 09:54:10 INFO Completed upload id=395572lb # [ 129.746629] niks3-server[964]: 2026/09/21 09:54:10 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5573worker1 # [ 129.797419] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505574worker1 # [ 129.803665] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5575worker1 # [ 129.808857] nix-daemon[844]: reaped child process 1603, status = succeeded5576client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5577worker1 # [ 129.820857] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5578client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5579client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5580client # farm-a> LOG-farm-a-during-drain5581worker1 # [ 130.696459] nix-daemon[844]: reaped child process 1617, status = succeeded5582worker1 # [ 130.804420] nix-daemon[844]: accepted connection from pid 1633, user nix-grpc-daemon (trusted)5583worker1 # [ 130.825180] nix-daemon[844]: reaped child process 1636, status = succeeded5584lb # [ 130.783360] niks3-server[964]: 2026/09/21 09:54:11 INFO Received uploads request method=POST path=/api/pending_closures5585worker1 # [ 130.835241] nix-grpc-daemon[1521]: time=2026-09-21T09:54:11.431Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5586worker1 # [ 130.841620] nix-grpc-daemon[1521]: time=2026-09-21T09:54:11.433Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5587lb # [ 130.822438] niks3-server[964]: 2026/09/21 09:54:11 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5588lb # [ 130.835898] niks3-server[964]: 2026/09/21 09:54:11 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5589lb # [ 130.851732] niks3-server[964]: 2026/09/21 09:54:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5590worker1 # [ 130.899102] nix-grpc-daemon[1521]: time=2026-09-21T09:54:11.494Z level=INFO msg="Uploading 1 narinfos"5591lb # [ 130.861133] niks3-server[964]: 2026/09/21 09:54:11 INFO Signed narinfos id=40 count=15592lb # [ 130.866914] niks3-server[964]: 2026/09/21 09:54:11 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5593lb # [ 130.877959] niks3-server[964]: 2026/09/21 09:54:11 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5594lb # [ 130.886548] niks3-server[964]: 2026/09/21 09:54:11 INFO Completed upload id=405595worker1 # [ 130.926050] nix-grpc-daemon[1521]: time=2026-09-21T09:54:11.522Z level=INFO msg="Upload complete. (296ms)"5596lb # [ 130.888701] niks3-server[964]: 2026/09/21 09:54:11 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5597worker1 # [ 130.944323] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=15598worker1 # [ 130.961025] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:11Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05599worker1 # [ 130.964914] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:11Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=15600worker1 # [ 130.968846] nix-daemon[844]: reaped child process 1615, status = succeeded5601worker1 # [ 130.990249] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505602worker1 # [ 131.003221] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536505603worker1 # [ 131.017984] nix-grpc-daemon[1521]: time=2026-09-21T09:54:11.614Z level=INFO msg="All 1 paths already cached"5604client # copying 1 paths to 'lb:50051'...5605worker1 # [ 131.028619] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536505606client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5607worker1 # [ 131.051474] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5608worker1 # [ 131.208551] nix-daemon[844]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)5609worker1 # [ 131.233096] nix-daemon[844]: reaped child process 1645, status = succeeded5610worker1 # [ 131.329900] nix-daemon[844]: accepted connection from pid 1647, user nix-grpc-daemon (trusted)5611worker1 # [ 131.347144] nix-daemon[844]: reaped child process 1650, status = succeeded5612lb # [ 131.309931] niks3-server[964]: 2026/09/21 09:54:11 INFO Received uploads request method=POST path=/api/pending_closures5613worker1 # [ 131.360061] nix-grpc-daemon[1521]: time=2026-09-21T09:54:11.955Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5614worker1 # [ 131.361525] nix-grpc-daemon[1521]: time=2026-09-21T09:54:11.956Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5615lb # [ 131.338891] niks3-server[964]: 2026/09/21 09:54:11 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5616worker1 # [ 131.394901] nix-grpc-daemon[1521]: time=2026-09-21T09:54:11.990Z level=INFO msg="Uploading 1 narinfos"5617lb # [ 131.354352] niks3-server[964]: 2026/09/21 09:54:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5618lb # [ 131.364623] niks3-server[964]: 2026/09/21 09:54:11 INFO Signed narinfos id=41 count=15619lb # [ 131.367176] niks3-server[964]: 2026/09/21 09:54:11 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5620lb # [ 131.379606] niks3-server[964]: 2026/09/21 09:54:11 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5621worker1 # [ 131.425562] nix-grpc-daemon[1521]: time=2026-09-21T09:54:12.021Z level=INFO msg="Upload complete. (356ms)"5622lb # [ 131.386669] niks3-server[964]: 2026/09/21 09:54:11 INFO Completed upload id=415623lb # [ 131.389612] niks3-server[964]: 2026/09/21 09:54:11 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5624worker1 # [ 131.435441] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=0 paths=1 nar_bytes_in=6405625worker1 # [ 131.447442] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505626worker1 # [ 131.453079] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5627worker1 # [ 131.457619] nix-daemon[844]: reaped child process 1640, status = succeeded5628client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5629worker1 # [ 131.467898] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5630client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5631client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5632client # farm-b> LOG-farm-b-during-drain5633worker1 # [ 132.501502] nix-daemon[844]: reaped child process 1654, status = succeeded5634worker1 # [ 132.614332] nix-daemon[844]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)5635worker1 # [ 132.633717] nix-daemon[844]: reaped child process 1673, status = succeeded5636lb # [ 132.598327] niks3-server[964]: 2026/09/21 09:54:13 INFO Received uploads request method=POST path=/api/pending_closures5637worker1 # [ 132.651128] nix-grpc-daemon[1521]: time=2026-09-21T09:54:13.246Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5638worker1 # [ 132.652782] nix-grpc-daemon[1521]: time=2026-09-21T09:54:13.246Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5639lb # [ 132.626619] niks3-server[964]: 2026/09/21 09:54:13 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5640worker1 # [ 132.683333] nix-grpc-daemon[1521]: time=2026-09-21T09:54:13.279Z level=INFO msg="Uploading 1 narinfos"5641lb # [ 132.635202] niks3-server[964]: 2026/09/21 09:54:13 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5642lb # [ 132.644500] niks3-server[964]: 2026/09/21 09:54:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5643lb # [ 132.647254] niks3-server[964]: 2026/09/21 09:54:13 INFO Signed narinfos id=42 count=15644lb # [ 132.654789] niks3-server[964]: 2026/09/21 09:54:13 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5645worker1 # [ 132.704214] nix-grpc-daemon[1521]: time=2026-09-21T09:54:13.300Z level=INFO msg="Upload complete. (219ms)"5646lb # [ 132.663705] niks3-server[964]: 2026/09/21 09:54:13 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5647lb # [ 132.668313] niks3-server[964]: 2026/09/21 09:54:13 INFO Completed upload id=425648lb # [ 132.671176] niks3-server[964]: 2026/09/21 09:54:13 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5649worker1 # [ 132.714629] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=15650worker1 # [ 132.729681] nix-daemon[844]: reaped child process 1652, status = succeeded5651worker1 # [ 132.732551] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:13Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05652worker1 # [ 132.736693] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:13Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=15653worker1 # [ 132.784525] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505654worker1 # [ 132.799184] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536505655worker1 # [ 132.813740] nix-grpc-daemon[1521]: time=2026-09-21T09:54:13.409Z level=INFO msg="All 2 paths already cached"5656client # copying 1 paths to 'lb:50051'...5657worker1 # [ 132.824093] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536505658client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5659worker1 # [ 132.847195] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5660worker1 # [ 133.010510] nix-daemon[844]: accepted connection from pid 1679, user nix-grpc-daemon (trusted)5661worker1 # [ 133.032568] nix-daemon[844]: reaped child process 1682, status = succeeded5662worker1 # [ 133.133619] nix-daemon[844]: accepted connection from pid 1684, user nix-grpc-daemon (trusted)5663worker1 # [ 133.152686] nix-daemon[844]: reaped child process 1687, status = succeeded5664lb # [ 133.116703] niks3-server[964]: 2026/09/21 09:54:13 INFO Received uploads request method=POST path=/api/pending_closures5665worker1 # [ 133.167090] nix-grpc-daemon[1521]: time=2026-09-21T09:54:13.762Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5666worker1 # [ 133.168694] nix-grpc-daemon[1521]: time=2026-09-21T09:54:13.762Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5667lb # [ 133.149966] niks3-server[964]: 2026/09/21 09:54:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5668worker1 # [ 133.199379] nix-grpc-daemon[1521]: time=2026-09-21T09:54:13.795Z level=INFO msg="Uploading 1 narinfos"5669lb # [ 133.165961] niks3-server[964]: 2026/09/21 09:54:13 INFO Signed narinfos id=43 count=15670lb # [ 133.173958] niks3-server[964]: 2026/09/21 09:54:13 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5671worker1 # [ 133.230152] nix-grpc-daemon[1521]: time=2026-09-21T09:54:13.826Z level=INFO msg="Upload complete. (365ms)"5672lb # [ 133.188898] niks3-server[964]: 2026/09/21 09:54:13 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5673worker1 # [ 133.237156] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=0 paths=1 nar_bytes_in=7685674lb # [ 133.196890] niks3-server[964]: 2026/09/21 09:54:13 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5675lb # [ 133.202479] niks3-server[964]: 2026/09/21 09:54:13 INFO Completed upload id=435676lb # [ 133.205763] niks3-server[964]: 2026/09/21 09:54:13 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5677worker1 # [ 133.250045] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505678worker1 # [ 133.259037] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5679worker1 # [ 133.261851] nix-daemon[844]: reaped child process 1677, status = succeeded5680client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5681worker1 # [ 133.274905] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5682client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5683client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5684client # farm-top> LOG-farm-top-during-drain5685worker1 # [ 134.724876] nix-daemon[844]: reaped child process 1691, status = succeeded5686worker1 # [ 134.843138] nix-daemon[844]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)5687worker1 # [ 134.860913] nix-daemon[844]: reaped child process 1710, status = succeeded5688lb # [ 134.825272] niks3-server[964]: 2026/09/21 09:54:15 INFO Received uploads request method=POST path=/api/pending_closures5689worker1 # [ 134.876420] nix-grpc-daemon[1521]: time=2026-09-21T09:54:15.471Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5690worker1 # [ 134.878122] nix-grpc-daemon[1521]: time=2026-09-21T09:54:15.471Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5691lb # [ 134.886157] niks3-server[964]: 2026/09/21 09:54:15 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5692lb # [ 134.898056] niks3-server[964]: 2026/09/21 09:54:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5693worker1 # [ 134.944078] nix-grpc-daemon[1521]: time=2026-09-21T09:54:15.539Z level=INFO msg="Uploading 1 narinfos"5694lb # [ 134.906268] niks3-server[964]: 2026/09/21 09:54:15 INFO Signed narinfos id=44 count=15695lb # [ 134.910523] niks3-server[964]: 2026/09/21 09:54:15 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5696lb # [ 134.920592] niks3-server[964]: 2026/09/21 09:54:15 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5697lb # [ 134.922943] niks3-server[964]: 2026/09/21 09:54:15 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5698lb # [ 134.932934] niks3-server[964]: 2026/09/21 09:54:15 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5699lb # [ 134.944391] niks3-server[964]: 2026/09/21 09:54:15 INFO Completed upload id=445700worker1 # [ 134.986338] nix-grpc-daemon[1521]: time=2026-09-21T09:54:15.582Z level=INFO msg="Upload complete. (301ms)"5701worker1 # [ 135.001268] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=15702worker1 # [ 135.018530] nix-daemon[844]: reaped child process 1689, status = succeeded5703client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.05 seconds)5704client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath5705client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.19 seconds)5706worker2: must succeed: pkill -f 'read -t [9]0 x'5707worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)5708client: waiting for success: ! systemctl is-active sw5709worker2 # [ 135.580440] nix-daemon[834]: reaped child process 1537, status = succeeded5710worker2 # [ 135.680749] nix-daemon[834]: accepted connection from pid 1709, user nix-grpc-daemon (trusted)5711worker2 # [ 135.701554] nix-daemon[834]: reaped child process 1712, status = succeeded5712lb # [ 135.647318] niks3-server[964]: 2026/09/21 09:54:16 INFO Received uploads request method=POST path=/api/pending_closures5713worker2 # [ 135.717888] nix-grpc-daemon[1424]: time=2026-09-21T09:54:16.308Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5714worker2 # [ 135.719647] nix-grpc-daemon[1424]: time=2026-09-21T09:54:16.308Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5715lb # [ 135.686310] niks3-server[964]: 2026/09/21 09:54:16 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5716lb # [ 135.703496] niks3-server[964]: 2026/09/21 09:54:16 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5717lb # [ 135.711826] niks3-server[964]: 2026/09/21 09:54:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5718worker2 # [ 135.776467] nix-grpc-daemon[1424]: time=2026-09-21T09:54:16.367Z level=INFO msg="Uploading 1 narinfos"5719lb # [ 135.721404] niks3-server[964]: 2026/09/21 09:54:16 INFO Signed narinfos id=45 count=15720lb # [ 135.726698] niks3-server[964]: 2026/09/21 09:54:16 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5721lb # [ 135.737231] niks3-server[964]: 2026/09/21 09:54:16 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5722lb # [ 135.743535] niks3-server[964]: 2026/09/21 09:54:16 INFO Completed upload id=455723lb # [ 135.746383] niks3-server[964]: 2026/09/21 09:54:16 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5724worker2 # [ 135.800785] nix-grpc-daemon[1424]: time=2026-09-21T09:54:16.392Z level=INFO msg="Upload complete. (294ms)"5725worker2 # [ 135.813590] nix-grpc-daemon[1310]: ts=2026-09-21T09:54:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43852 duration_s=15 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=15726worker2 # [ 135.832777] nix-daemon[834]: reaped child process 1535, status = succeeded5727client # [ 135.859431] systemd[1]: sw.service: Deactivated successfully.5728client # [ 135.863862] systemd[1]: sw.service: Consumed 198ms CPU time over 16.337s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.5729client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)5730client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5731client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)5732worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5733worker2 # [ 136.489797] nix-grpc-daemon[1310]: ts=2026-09-21T09:54:17Z level=info event=signal_exit5734worker2 # [ 136.536829] systemd[1]: Starting scheduler-token.service...5735worker2 # [ 136.607765] systemd[1]: scheduler-token.service: Deactivated successfully.5736worker2 # [ 136.612796] systemd[1]: Finished scheduler-token.service.5737worker2 # [ 136.614347] systemd[1]: scheduler-token.service: Consumed 38ms CPU time over 74ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5738worker1 # [ 138.473802] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:504485739worker2 # [ 138.490990] nix-grpc-daemon[1310]: ts=2026-09-21T09:54:19Z level=info event=scheduler_restarting5740worker1 # [ 138.478605] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:19Z level=info event=worker_gone addr=192.168.1.4:500515741worker2 # [ 138.515057] nix-grpc-daemon[1310]: ts=2026-09-21T09:54:19Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5742worker2 # [ 138.542465] nix-daemon[834]: reaped child process 1369, status = succeeded5743worker2 # [ 138.589955] nix-grpc-daemon[1310]: E0000 00:00:1789984459.181047 1355 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5744worker2 # [ 138.606961] nix-grpc-daemon[1310]: E0000 00:00:1789984459.198081 1355 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5745lb # [ 138.626792] envoy[900]: conn peer=[2001:db8:1::1]:50574 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9608 tx=4061 ms=57865746lb # [ 138.631853] envoy[900]: conn peer=[2001:db8:1::1]:52952 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1431 ms=161095747lb # [ 138.636787] envoy[900]: conn peer=[2001:db8:1::4]:59700 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=12227 tx=850 ms=489755748worker2 # [ 139.860725] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5749worker2 # [ 139.866770] systemd[1]: nix-grpc-daemon.service: Consumed 1.219s CPU time over 50.542s wall clock time, 36.2M memory peak, 27.5K incoming IP traffic, 49.3K outgoing IP traffic.5750worker2 # [ 139.889582] systemd[1]: Starting Nix worker-protocol over gRPC...5751worker2 # [ 140.038076] nix-grpc-daemon[1729]: ts=2026-09-21T09:54:20Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5752worker2 # [ 140.040421] nix-grpc-daemon[1729]: ts=2026-09-21T09:54:20Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5753worker2 # [ 140.044425] nix-grpc-daemon[1729]: ts=2026-09-21T09:54:20Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15754worker2 # [ 140.051051] nix-grpc-daemon[1729]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5755worker2 # [ 140.059955] nix-grpc-daemon[1729]: E0000 00:00:1789984460.640069 1729 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5756worker2 # [ 140.067865] nix-grpc-daemon[1729]: E0000 00:00:1789984460.642200 1729 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5757worker2 # [ 140.096777] nix-grpc-daemon[1729]: ts=2026-09-21T09:54:20Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5758worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.65 seconds)5759worker2 # [ 140.098445] nix-grpc-daemon[1729]: E0000 00:00:1789984460.654925 1769 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5760lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5761lb: must succeed: curl -sf localhost:9901/clusters5762worker2 # [ 140.105337] nix-grpc-daemon[1729]: E0000 00:00:1789984460.674111 1769 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5763worker2 # [ 140.107622] nix-grpc-daemon[1729]: E0000 00:00:1789984460.674275 1769 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5764worker2 # [ 140.110188] systemd[1]: Started Nix worker-protocol over gRPC.5765worker2 # [ 140.111014] nix-grpc-daemon[1729]: E0000 00:00:1789984460.695144 1769 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5766worker2 # [ 140.119456] nix-grpc-daemon[1729]: ts=2026-09-21T09:54:20Z level=info event=scheduler_connected addr=lb:500515767worker1 # [ 140.107438] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:20Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:504485768worker1 # [ 140.113311] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:20Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05769lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5770lb: must succeed: curl -sf localhost:9901/clusters5771lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5772lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.14 seconds)5773lb: must succeed: curl -sf localhost:9901/clusters5774lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5775worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5776worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5777(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 21.91 seconds)5778subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5779client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15780worker2 # [ 141.581380] nix-grpc-daemon[1729]: ts=2026-09-21T09:54:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:507105781worker2 # [ 141.598040] nix-daemon[834]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)5782worker1 # [ 141.604863] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536505783worker1 # [ 141.650669] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:22Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05784worker1 # [ 141.657873] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:22Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15785worker1 # [ 141.663835] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:22Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05786worker1 # [ 141.669132] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:22Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05787worker1 # [ 141.697721] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505788worker1 # [ 141.711860] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536505789worker1 # [ 141.726517] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536505790worker1 # [ 141.741127] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5791worker1 # [ 141.893444] nix-daemon[844]: accepted connection from pid 1719, user nix-grpc-daemon (trusted)5792worker1 # [ 141.913330] nix-daemon[844]: reaped child process 1722, status = succeeded5793worker1 # [ 142.015879] nix-daemon[844]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)5794worker1 # [ 142.031332] nix-daemon[844]: reaped child process 1727, status = succeeded5795lb # [ 141.997757] niks3-server[964]: 2026/09/21 09:54:22 INFO Received uploads request method=POST path=/api/pending_closures5796worker1 # [ 142.049336] nix-grpc-daemon[1521]: time=2026-09-21T09:54:22.644Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5797worker1 # [ 142.050760] nix-grpc-daemon[1521]: time=2026-09-21T09:54:22.644Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"5798lb # [ 142.027479] niks3-server[964]: 2026/09/21 09:54:22 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst5799lb # [ 142.042075] niks3-server[964]: 2026/09/21 09:54:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5800worker1 # [ 142.088905] nix-grpc-daemon[1521]: time=2026-09-21T09:54:22.684Z level=INFO msg="Uploading 1 narinfos"5801lb # [ 142.050510] niks3-server[964]: 2026/09/21 09:54:22 INFO Signed narinfos id=46 count=15802lb # [ 142.057498] niks3-server[964]: 2026/09/21 09:54:22 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls5803lb # [ 142.069102] niks3-server[964]: 2026/09/21 09:54:22 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5804worker1 # [ 142.114704] nix-grpc-daemon[1521]: time=2026-09-21T09:54:22.710Z level=INFO msg="Upload complete. (356ms)"5805lb # [ 142.074727] niks3-server[964]: 2026/09/21 09:54:22 INFO Completed upload id=465806lb # [ 142.078652] niks3-server[964]: 2026/09/21 09:54:22 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo5807worker1 # [ 142.124653] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=0 paths=1 nar_bytes_in=5125808worker1 # [ 142.136072] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505809worker1 # [ 142.145789] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5810worker1 # [ 142.148598] nix-daemon[844]: reaped child process 1717, status = succeeded5811worker1 # [ 142.160190] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5812worker1 # [ 143.452708] nix-daemon[844]: reaped child process 1731, status = succeeded5813worker1 # [ 143.529951] nix-daemon[844]: accepted connection from pid 1747, user nix-grpc-daemon (trusted)5814worker1 # [ 143.549759] nix-daemon[844]: reaped child process 1750, status = succeeded5815lb # [ 143.507128] niks3-server[964]: 2026/09/21 09:54:24 INFO Received uploads request method=POST path=/api/pending_closures5816worker1 # [ 143.560759] nix-grpc-daemon[1521]: time=2026-09-21T09:54:24.153Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5817worker1 # [ 143.563586] nix-grpc-daemon[1521]: time=2026-09-21T09:54:24.153Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"5818lb # [ 143.543999] niks3-server[964]: 2026/09/21 09:54:24 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv5819lb # [ 143.556717] niks3-server[964]: 2026/09/21 09:54:24 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst5820worker1 # [ 143.610688] nix-grpc-daemon[1521]: time=2026-09-21T09:54:24.206Z level=INFO msg="Uploading 1 narinfos"5821lb # [ 143.569091] niks3-server[964]: 2026/09/21 09:54:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5822lb # [ 143.578493] niks3-server[964]: 2026/09/21 09:54:24 INFO Signed narinfos id=47 count=15823lb # [ 143.583838] niks3-server[964]: 2026/09/21 09:54:24 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls5824lb # [ 143.595365] niks3-server[964]: 2026/09/21 09:54:24 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5825worker1 # [ 143.642739] nix-grpc-daemon[1521]: time=2026-09-21T09:54:24.238Z level=INFO msg="Upload complete. (269ms)"5826lb # [ 143.602113] niks3-server[964]: 2026/09/21 09:54:24 INFO Completed upload id=475827lb # [ 143.605856] niks3-server[964]: 2026/09/21 09:54:24 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo5828worker1 # [ 143.662160] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=6 outputs=15829worker1 # [ 143.677888] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:24Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=15830worker1 # [ 143.689119] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505831worker1 # [ 143.695179] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:24Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=05832worker1 # [ 143.702964] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536505833worker1 # [ 143.711546] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536505834worker1 # [ 143.717536] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5835worker1 # [ 143.721815] nix-daemon[844]: reaped child process 1729, status = succeeded5836worker1 # [ 143.841761] nix-daemon[844]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)5837worker1 # [ 143.858948] nix-daemon[844]: reaped child process 1759, status = succeeded5838worker1 # [ 143.960112] nix-daemon[844]: accepted connection from pid 1761, user nix-grpc-daemon (trusted)5839worker1 # [ 143.978981] nix-daemon[844]: reaped child process 1764, status = succeeded5840lb # [ 143.938804] niks3-server[964]: 2026/09/21 09:54:24 INFO Received uploads request method=POST path=/api/pending_closures5841worker1 # [ 143.989368] nix-grpc-daemon[1521]: time=2026-09-21T09:54:24.585Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5842worker1 # [ 143.990833] nix-grpc-daemon[1521]: time=2026-09-21T09:54:24.585Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"5843lb # [ 143.966404] niks3-server[964]: 2026/09/21 09:54:24 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst5844worker1 # [ 144.020993] nix-grpc-daemon[1521]: time=2026-09-21T09:54:24.616Z level=INFO msg="Uploading 1 narinfos"5845lb # [ 143.983113] niks3-server[964]: 2026/09/21 09:54:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5846lb # [ 143.994035] niks3-server[964]: 2026/09/21 09:54:24 INFO Signed narinfos id=48 count=15847worker1 # [ 144.047870] nix-grpc-daemon[1521]: time=2026-09-21T09:54:24.643Z level=INFO msg="Upload complete. (331ms)"5848lb # [ 143.999946] niks3-server[964]: 2026/09/21 09:54:24 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls5849worker1 # [ 144.054466] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=0 paths=1 nar_bytes_in=5125850lb # [ 144.013060] niks3-server[964]: 2026/09/21 09:54:24 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5851lb # [ 144.017485] niks3-server[964]: 2026/09/21 09:54:24 INFO Completed upload id=485852lb # [ 144.019802] niks3-server[964]: 2026/09/21 09:54:24 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo5853worker1 # [ 144.066990] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505854worker1 # [ 144.072132] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5855worker1 # [ 144.076417] nix-daemon[844]: reaped child process 1754, status = succeeded5856worker1 # [ 144.086080] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5857worker1 # [ 145.193071] nix-daemon[844]: reaped child process 1768, status = succeeded5858worker1 # [ 145.253416] nix-daemon[844]: accepted connection from pid 1784, user nix-grpc-daemon (trusted)5859worker1 # [ 145.271687] nix-daemon[844]: reaped child process 1787, status = succeeded5860lb # [ 145.238756] niks3-server[964]: 2026/09/21 09:54:25 INFO Received uploads request method=POST path=/api/pending_closures5861worker1 # [ 145.292513] nix-grpc-daemon[1521]: time=2026-09-21T09:54:25.885Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5862worker1 # [ 145.294589] nix-grpc-daemon[1521]: time=2026-09-21T09:54:25.885Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"5863lb # [ 145.281229] niks3-server[964]: 2026/09/21 09:54:25 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv5864lb # [ 145.292350] niks3-server[964]: 2026/09/21 09:54:25 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst5865worker1 # [ 145.346339] nix-grpc-daemon[1521]: time=2026-09-21T09:54:25.942Z level=INFO msg="Uploading 1 narinfos"5866lb # [ 145.306368] niks3-server[964]: 2026/09/21 09:54:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5867lb # [ 145.314772] niks3-server[964]: 2026/09/21 09:54:25 INFO Signed narinfos id=49 count=15868lb # [ 145.319155] niks3-server[964]: 2026/09/21 09:54:25 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls5869lb # [ 145.328402] niks3-server[964]: 2026/09/21 09:54:25 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5870lb # [ 145.333758] niks3-server[964]: 2026/09/21 09:54:25 INFO Completed upload id=495871worker1 # [ 145.374601] nix-grpc-daemon[1521]: time=2026-09-21T09:54:25.970Z level=INFO msg="Upload complete. (277ms)"5872lb # [ 145.337048] niks3-server[964]: 2026/09/21 09:54:25 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo5873worker1 # [ 145.389202] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=7 outputs=15874worker1 # [ 145.407289] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:25Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=15875worker1 # [ 145.413115] nix-daemon[844]: reaped child process 1766, status = succeeded5876worker1 # [ 145.429005] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505877worker1 # [ 145.437213] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:26Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=05878worker1 # [ 145.449256] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536505879worker1 # [ 145.464132] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536505880worker1 # [ 145.477895] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5881worker1 # [ 145.631567] nix-daemon[844]: accepted connection from pid 1793, user nix-grpc-daemon (trusted)5882worker1 # [ 145.648383] nix-daemon[844]: reaped child process 1796, status = succeeded5883worker1 # [ 145.754225] nix-daemon[844]: accepted connection from pid 1798, user nix-grpc-daemon (trusted)5884worker1 # [ 145.774947] nix-daemon[844]: reaped child process 1801, status = succeeded5885lb # [ 145.731444] niks3-server[964]: 2026/09/21 09:54:26 INFO Received uploads request method=POST path=/api/pending_closures5886worker1 # [ 145.783239] nix-grpc-daemon[1521]: time=2026-09-21T09:54:26.379Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5887worker1 # [ 145.785016] nix-grpc-daemon[1521]: time=2026-09-21T09:54:26.379Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"5888lb # [ 145.760270] niks3-server[964]: 2026/09/21 09:54:26 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst5889worker1 # [ 145.815893] nix-grpc-daemon[1521]: time=2026-09-21T09:54:26.411Z level=INFO msg="Uploading 1 narinfos"5890lb # [ 145.775053] niks3-server[964]: 2026/09/21 09:54:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5891lb # [ 145.787958] niks3-server[964]: 2026/09/21 09:54:26 INFO Signed narinfos id=50 count=15892lb # [ 145.791233] niks3-server[964]: 2026/09/21 09:54:26 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls5893worker1 # [ 145.848343] nix-grpc-daemon[1521]: time=2026-09-21T09:54:26.444Z level=INFO msg="Upload complete. (356ms)"5894lb # [ 145.807985] niks3-server[964]: 2026/09/21 09:54:26 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5895lb # [ 145.813058] niks3-server[964]: 2026/09/21 09:54:26 INFO Completed upload id=505896lb # [ 145.818486] niks3-server[964]: 2026/09/21 09:54:26 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo5897worker1 # [ 145.856272] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=0 paths=1 nar_bytes_in=5125898worker1 # [ 145.869625] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505899worker1 # [ 145.875222] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5900worker1 # [ 145.879771] nix-daemon[844]: reaped child process 1791, status = succeeded5901worker1 # [ 145.889861] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5902worker1 # [ 147.399279] nix-daemon[844]: reaped child process 1805, status = succeeded5903worker1 # [ 147.473903] nix-daemon[844]: accepted connection from pid 1821, user nix-grpc-daemon (trusted)5904worker1 # [ 147.494013] nix-daemon[844]: reaped child process 1824, status = succeeded5905lb # [ 147.451424] niks3-server[964]: 2026/09/21 09:54:28 INFO Received uploads request method=POST path=/api/pending_closures5906worker1 # [ 147.503299] nix-grpc-daemon[1521]: time=2026-09-21T09:54:28.098Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5907worker1 # [ 147.505159] nix-grpc-daemon[1521]: time=2026-09-21T09:54:28.098Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"5908lb # [ 147.482509] niks3-server[964]: 2026/09/21 09:54:28 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv5909lb # [ 147.490522] niks3-server[964]: 2026/09/21 09:54:28 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst5910lb # [ 147.500338] niks3-server[964]: 2026/09/21 09:54:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5911worker1 # [ 147.544728] nix-grpc-daemon[1521]: time=2026-09-21T09:54:28.140Z level=INFO msg="Uploading 1 narinfos"5912lb # [ 147.505925] niks3-server[964]: 2026/09/21 09:54:28 INFO Signed narinfos id=51 count=15913lb # [ 147.509548] niks3-server[964]: 2026/09/21 09:54:28 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls5914lb # [ 147.515860] niks3-server[964]: 2026/09/21 09:54:28 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5915lb # [ 147.521044] niks3-server[964]: 2026/09/21 09:54:28 INFO Completed upload id=515916lb # [ 147.522817] niks3-server[964]: 2026/09/21 09:54:28 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo5917worker1 # [ 147.561807] nix-grpc-daemon[1521]: time=2026-09-21T09:54:28.157Z level=INFO msg="Upload complete. (259ms)"5918worker1 # [ 147.577155] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=8 outputs=15919worker1 # [ 147.588322] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:28Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=15920worker1 # [ 147.591648] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:28Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=05921worker1 # [ 147.594995] nix-daemon[844]: reaped child process 1803, status = succeeded5922worker1 # [ 147.613260] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505923worker1 # [ 147.625749] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536505924worker1 # [ 147.637707] nix-grpc-daemon[1521]: time=2026-09-21T09:54:28.233Z level=INFO msg="All 1 paths already cached"5925worker1 # [ 147.646622] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536505926worker1 # [ 147.658150] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5927worker1 # [ 147.799691] nix-daemon[844]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)5928worker1 # [ 147.816463] nix-daemon[844]: reaped child process 1833, status = succeeded5929worker1 # [ 147.917201] nix-daemon[844]: accepted connection from pid 1835, user nix-grpc-daemon (trusted)5930worker1 # [ 147.934123] nix-daemon[844]: reaped child process 1838, status = succeeded5931lb # [ 147.898253] niks3-server[964]: 2026/09/21 09:54:28 INFO Received uploads request method=POST path=/api/pending_closures5932worker1 # [ 147.948575] nix-grpc-daemon[1521]: time=2026-09-21T09:54:28.544Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5933worker1 # [ 147.950383] nix-grpc-daemon[1521]: time=2026-09-21T09:54:28.544Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"5934lb # [ 147.924535] niks3-server[964]: 2026/09/21 09:54:28 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst5935lb # [ 147.938560] niks3-server[964]: 2026/09/21 09:54:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign5936worker1 # [ 147.987763] nix-grpc-daemon[1521]: time=2026-09-21T09:54:28.583Z level=INFO msg="Uploading 1 narinfos"5937lb # [ 147.951260] niks3-server[964]: 2026/09/21 09:54:28 INFO Signed narinfos id=52 count=15938lb # [ 147.957320] niks3-server[964]: 2026/09/21 09:54:28 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls5939lb # [ 147.968129] niks3-server[964]: 2026/09/21 09:54:28 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete5940worker1 # [ 148.009213] nix-grpc-daemon[1521]: time=2026-09-21T09:54:28.605Z level=INFO msg="Upload complete. (338ms)"5941lb # [ 147.975889] niks3-server[964]: 2026/09/21 09:54:28 INFO Completed upload id=525942worker1 # [ 148.018172] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=0 paths=1 nar_bytes_in=6245943lb # [ 147.979566] niks3-server[964]: 2026/09/21 09:54:28 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo5944worker1 # [ 148.029722] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505945worker1 # [ 148.037046] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5946worker1 # [ 148.039545] nix-daemon[844]: reaped child process 1828, status = succeeded5947worker1 # [ 148.051107] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5948worker1 # [ 149.732168] nix-daemon[844]: reaped child process 1842, status = succeeded5949worker1 # [ 149.832201] nix-daemon[844]: accepted connection from pid 1858, user nix-grpc-daemon (trusted)5950worker1 # [ 149.850081] nix-daemon[844]: reaped child process 1861, status = succeeded5951lb # [ 149.814235] niks3-server[964]: 2026/09/21 09:54:30 INFO Received uploads request method=POST path=/api/pending_closures5952worker1 # [ 149.868465] nix-grpc-daemon[1521]: time=2026-09-21T09:54:30.462Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5953worker1 # [ 149.872803] nix-grpc-daemon[1521]: time=2026-09-21T09:54:30.462Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"5954lb # [ 149.857002] niks3-server[964]: 2026/09/21 09:54:30 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst5955lb # [ 149.872553] niks3-server[964]: 2026/09/21 09:54:30 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv5956worker1 # [ 149.924900] nix-grpc-daemon[1521]: time=2026-09-21T09:54:30.520Z level=INFO msg="Uploading 1 narinfos"5957lb # [ 149.883311] niks3-server[964]: 2026/09/21 09:54:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign5958lb # [ 149.893950] niks3-server[964]: 2026/09/21 09:54:30 INFO Signed narinfos id=53 count=15959lb # [ 149.900894] niks3-server[964]: 2026/09/21 09:54:30 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls5960lb # [ 149.913244] niks3-server[964]: 2026/09/21 09:54:30 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete5961lb # [ 149.921501] niks3-server[964]: 2026/09/21 09:54:30 INFO Completed upload id=535962worker1 # [ 149.961374] nix-grpc-daemon[1521]: time=2026-09-21T09:54:30.557Z level=INFO msg="Upload complete. (299ms)"5963lb # [ 149.924148] niks3-server[964]: 2026/09/21 09:54:30 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo5964worker1 # [ 149.981023] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=9 outputs=15965worker1 # [ 150.001150] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:30Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=15966worker1 # [ 150.007834] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505967worker1 # [ 150.013329] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:30Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=05968worker1 # [ 150.018077] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536505969worker1 # [ 150.022566] nix-daemon[844]: reaped child process 1840, status = succeeded5970worker1 # [ 150.036431] nix-grpc-daemon[1521]: time=2026-09-21T09:54:30.613Z level=INFO msg="All 1 paths already cached"5971worker1 # [ 150.039050] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5972worker1 # [ 150.044846] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536505973worker1 # [ 150.170749] nix-daemon[844]: accepted connection from pid 1867, user nix-grpc-daemon (trusted)5974worker1 # [ 150.186271] nix-daemon[844]: reaped child process 1870, status = succeeded5975worker1 # [ 150.286844] nix-daemon[844]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)5976worker1 # [ 150.305522] nix-daemon[844]: reaped child process 1875, status = succeeded5977lb # [ 150.265437] niks3-server[964]: 2026/09/21 09:54:30 INFO Received uploads request method=POST path=/api/pending_closures5978worker1 # [ 150.315256] nix-grpc-daemon[1521]: time=2026-09-21T09:54:30.911Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5979worker1 # [ 150.316973] nix-grpc-daemon[1521]: time=2026-09-21T09:54:30.911Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"5980lb # [ 150.291265] niks3-server[964]: 2026/09/21 09:54:30 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst5981worker1 # [ 150.346860] nix-grpc-daemon[1521]: time=2026-09-21T09:54:30.942Z level=INFO msg="Uploading 1 narinfos"5982lb # [ 150.305725] niks3-server[964]: 2026/09/21 09:54:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign5983lb # [ 150.317491] niks3-server[964]: 2026/09/21 09:54:30 INFO Signed narinfos id=54 count=15984lb # [ 150.323610] niks3-server[964]: 2026/09/21 09:54:30 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls5985lb # [ 150.335400] niks3-server[964]: 2026/09/21 09:54:30 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete5986worker1 # [ 150.383685] nix-grpc-daemon[1521]: time=2026-09-21T09:54:30.979Z level=INFO msg="Upload complete. (339ms)"5987lb # [ 150.344161] niks3-server[964]: 2026/09/21 09:54:30 INFO Completed upload id=545988lb # [ 150.348332] niks3-server[964]: 2026/09/21 09:54:30 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo5989worker1 # [ 150.392239] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=0 paths=1 nar_bytes_in=6245990worker1 # [ 150.403798] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536505991worker1 # [ 150.409665] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5992worker1 # [ 150.414483] nix-daemon[844]: reaped child process 1865, status = succeeded5993worker1 # [ 150.424784] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5994worker1 # [ 151.880580] nix-daemon[844]: reaped child process 1879, status = succeeded5995worker1 # [ 151.977775] nix-daemon[844]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)5996worker1 # [ 151.998259] nix-daemon[844]: reaped child process 1898, status = succeeded5997lb # [ 151.962496] niks3-server[964]: 2026/09/21 09:54:32 INFO Received uploads request method=POST path=/api/pending_closures5998worker1 # [ 152.015709] nix-grpc-daemon[1521]: time=2026-09-21T09:54:32.610Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5999worker1 # [ 152.017349] nix-grpc-daemon[1521]: time=2026-09-21T09:54:32.610Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6000lb # [ 151.999367] niks3-server[964]: 2026/09/21 09:54:32 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6001lb # [ 152.015117] niks3-server[964]: 2026/09/21 09:54:32 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6002worker1 # [ 152.066725] nix-grpc-daemon[1521]: time=2026-09-21T09:54:32.662Z level=INFO msg="Uploading 1 narinfos"6003lb # [ 152.029704] niks3-server[964]: 2026/09/21 09:54:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6004lb # [ 152.040800] niks3-server[964]: 2026/09/21 09:54:32 INFO Signed narinfos id=55 count=16005lb # [ 152.045354] niks3-server[964]: 2026/09/21 09:54:32 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6006worker1 # [ 152.095657] nix-grpc-daemon[1521]: time=2026-09-21T09:54:32.691Z level=INFO msg="Upload complete. (258ms)"6007lb # [ 152.055243] niks3-server[964]: 2026/09/21 09:54:32 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6008lb # [ 152.058470] niks3-server[964]: 2026/09/21 09:54:32 INFO Completed upload id=556009lb # [ 152.063285] niks3-server[964]: 2026/09/21 09:54:32 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6010worker1 # [ 152.113375] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=10 outputs=16011worker1 # [ 152.129746] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:32Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16012worker1 # [ 152.140217] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536506013worker1 # [ 152.146884] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:32Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06014worker1 # [ 152.152321] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536506015worker1 # [ 152.157273] nix-daemon[844]: reaped child process 1877, status = succeeded6016worker1 # [ 152.159296] nix-grpc-daemon[1521]: time=2026-09-21T09:54:32.747Z level=INFO msg="All 1 paths already cached"6017worker1 # [ 152.162186] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536506018worker1 # [ 152.173135] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6019worker1 # [ 152.306811] nix-daemon[844]: accepted connection from pid 1904, user nix-grpc-daemon (trusted)6020worker1 # [ 152.324103] nix-daemon[844]: reaped child process 1907, status = succeeded6021worker1 # [ 152.423931] nix-daemon[844]: accepted connection from pid 1909, user nix-grpc-daemon (trusted)6022worker1 # [ 152.438659] nix-daemon[844]: reaped child process 1912, status = succeeded6023lb # [ 152.404315] niks3-server[964]: 2026/09/21 09:54:33 INFO Received uploads request method=POST path=/api/pending_closures6024worker1 # [ 152.454826] nix-grpc-daemon[1521]: time=2026-09-21T09:54:33.050Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6025worker1 # [ 152.456390] nix-grpc-daemon[1521]: time=2026-09-21T09:54:33.050Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6026lb # [ 152.435689] niks3-server[964]: 2026/09/21 09:54:33 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6027lb # [ 152.449755] niks3-server[964]: 2026/09/21 09:54:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6028worker1 # [ 152.497270] nix-grpc-daemon[1521]: time=2026-09-21T09:54:33.093Z level=INFO msg="Uploading 1 narinfos"6029lb # [ 152.459717] niks3-server[964]: 2026/09/21 09:54:33 INFO Signed narinfos id=56 count=16030lb # [ 152.465304] niks3-server[964]: 2026/09/21 09:54:33 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6031worker1 # [ 152.517029] nix-grpc-daemon[1521]: time=2026-09-21T09:54:33.113Z level=INFO msg="Upload complete. (334ms)"6032lb # [ 152.476423] niks3-server[964]: 2026/09/21 09:54:33 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6033lb # [ 152.481299] niks3-server[964]: 2026/09/21 09:54:33 INFO Completed upload id=566034worker1 # [ 152.524101] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=0 paths=1 nar_bytes_in=6246035lb # [ 152.484616] niks3-server[964]: 2026/09/21 09:54:33 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6036worker1 # [ 152.537365] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536506037worker1 # [ 152.543040] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6038worker1 # [ 152.547487] nix-daemon[844]: reaped child process 1902, status = succeeded6039worker1 # [ 152.557534] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6040worker1 # [ 153.419724] nix-daemon[844]: reaped child process 1916, status = succeeded6041worker1 # [ 153.554904] nix-daemon[844]: accepted connection from pid 1932, user nix-grpc-daemon (trusted)6042worker1 # [ 153.576514] nix-daemon[844]: reaped child process 1935, status = succeeded6043lb # [ 153.538731] niks3-server[964]: 2026/09/21 09:54:34 INFO Received uploads request method=POST path=/api/pending_closures6044worker1 # [ 153.591660] nix-grpc-daemon[1521]: time=2026-09-21T09:54:34.185Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6045worker1 # [ 153.593583] nix-grpc-daemon[1521]: time=2026-09-21T09:54:34.185Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6046lb # [ 153.577251] niks3-server[964]: 2026/09/21 09:54:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6047worker1 # [ 153.629947] nix-grpc-daemon[1521]: time=2026-09-21T09:54:34.225Z level=INFO msg="Uploading 1 narinfos"6048lb # [ 153.594213] niks3-server[964]: 2026/09/21 09:54:34 INFO Signed narinfos id=57 count=16049lb # [ 153.600353] niks3-server[964]: 2026/09/21 09:54:34 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6050lb # [ 153.614928] niks3-server[964]: 2026/09/21 09:54:34 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6051worker1 # [ 153.663398] nix-grpc-daemon[1521]: time=2026-09-21T09:54:34.259Z level=INFO msg="Upload complete. (294ms)"6052lb # [ 153.625525] niks3-server[964]: 2026/09/21 09:54:34 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6053lb # [ 153.630557] niks3-server[964]: 2026/09/21 09:54:34 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6054lb # [ 153.635526] niks3-server[964]: 2026/09/21 09:54:34 INFO Completed upload id=576055lb # [ 153.640840] niks3-server[964]: 2026/09/21 09:54:34 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6056worker1 # [ 153.682980] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=11 outputs=16057worker1 # [ 153.702963] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:34Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16058worker1 # [ 153.709110] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536506059worker1 # [ 153.713435] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:34Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06060worker1 # [ 153.718961] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536506061worker1 # [ 153.724183] nix-grpc-daemon[1521]: time=2026-09-21T09:54:34.319Z level=INFO msg="All 2 paths already cached"6062worker1 # [ 153.728524] nix-daemon[844]: reaped child process 1914, status = succeeded6063worker1 # [ 153.730802] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536506064worker1 # [ 153.747509] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6065worker1 # [ 153.874309] nix-daemon[844]: accepted connection from pid 1941, user nix-grpc-daemon (trusted)6066worker1 # [ 153.894273] nix-daemon[844]: reaped child process 1944, status = succeeded6067worker1 # [ 153.996276] nix-daemon[844]: accepted connection from pid 1946, user nix-grpc-daemon (trusted)6068worker1 # [ 154.016073] nix-daemon[844]: reaped child process 1949, status = succeeded6069lb # [ 153.973572] niks3-server[964]: 2026/09/21 09:54:34 INFO Received uploads request method=POST path=/api/pending_closures6070worker1 # [ 154.024186] nix-grpc-daemon[1521]: time=2026-09-21T09:54:34.619Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6071worker1 # [ 154.025660] nix-grpc-daemon[1521]: time=2026-09-21T09:54:34.619Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6072lb # [ 154.001019] niks3-server[964]: 2026/09/21 09:54:34 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6073lb # [ 154.018377] niks3-server[964]: 2026/09/21 09:54:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6074worker1 # [ 154.067006] nix-grpc-daemon[1521]: time=2026-09-21T09:54:34.662Z level=INFO msg="Uploading 1 narinfos"6075lb # [ 154.030522] niks3-server[964]: 2026/09/21 09:54:34 INFO Signed narinfos id=58 count=16076lb # [ 154.036222] niks3-server[964]: 2026/09/21 09:54:34 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6077worker1 # [ 154.087553] nix-grpc-daemon[1521]: time=2026-09-21T09:54:34.683Z level=INFO msg="Upload complete. (334ms)"6078lb # [ 154.045881] niks3-server[964]: 2026/09/21 09:54:34 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6079lb # [ 154.052259] niks3-server[964]: 2026/09/21 09:54:34 INFO Completed upload id=586080worker1 # [ 154.094551] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=0 paths=1 nar_bytes_in=7526081lb # [ 154.054442] niks3-server[964]: 2026/09/21 09:54:34 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6082worker1 # [ 154.105611] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536506083worker1 # [ 154.112411] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6084worker1 # [ 154.114827] nix-daemon[844]: reaped child process 1939, status = succeeded6085worker1 # [ 154.124096] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6086worker1 # [ 155.626350] nix-daemon[844]: reaped child process 1953, status = succeeded6087worker1 # [ 155.694351] nix-daemon[844]: accepted connection from pid 1969, user nix-grpc-daemon (trusted)6088worker1 # [ 155.710746] nix-daemon[844]: reaped child process 1972, status = succeeded6089lb # [ 155.677202] niks3-server[964]: 2026/09/21 09:54:36 INFO Received uploads request method=POST path=/api/pending_closures6090worker1 # [ 155.731068] nix-grpc-daemon[1521]: time=2026-09-21T09:54:36.325Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6091worker1 # [ 155.733351] nix-grpc-daemon[1521]: time=2026-09-21T09:54:36.325Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6092lb # [ 155.719264] niks3-server[964]: 2026/09/21 09:54:36 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6093worker1 # [ 155.783883] nix-grpc-daemon[1521]: time=2026-09-21T09:54:36.379Z level=INFO msg="Uploading 1 narinfos"6094lb # [ 155.732296] niks3-server[964]: 2026/09/21 09:54:36 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6095lb # [ 155.747482] niks3-server[964]: 2026/09/21 09:54:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6096lb # [ 155.755033] niks3-server[964]: 2026/09/21 09:54:36 INFO Signed narinfos id=59 count=16097lb # [ 155.760968] niks3-server[964]: 2026/09/21 09:54:36 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6098lb # [ 155.768125] niks3-server[964]: 2026/09/21 09:54:36 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6099lb # [ 155.777294] niks3-server[964]: 2026/09/21 09:54:36 INFO Completed upload id=596100worker1 # [ 155.819099] nix-grpc-daemon[1521]: time=2026-09-21T09:54:36.415Z level=INFO msg="Upload complete. (284ms)"6101lb # [ 155.780123] niks3-server[964]: 2026/09/21 09:54:36 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6102worker1 # [ 155.833216] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=12 outputs=16103worker1 # [ 155.851321] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:36Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16104worker1 # [ 155.856564] nix-daemon[844]: reaped child process 1951, status = succeeded6105worker1 # [ 155.874085] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536506106worker1 # [ 155.888923] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536506107worker1 # [ 155.903687] nix-grpc-daemon[1521]: time=2026-09-21T09:54:36.499Z level=INFO msg="All 2 paths already cached"6108worker1 # [ 155.912540] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536506109worker1 # [ 155.926568] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6110worker1 # [ 156.083041] nix-daemon[844]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6111worker1 # [ 156.101616] nix-daemon[844]: reaped child process 1979, status = succeeded6112worker1 # [ 156.202091] nix-daemon[844]: accepted connection from pid 1981, user nix-grpc-daemon (trusted)6113worker1 # [ 156.223282] nix-daemon[844]: reaped child process 1984, status = succeeded6114lb # [ 156.182765] niks3-server[964]: 2026/09/21 09:54:36 INFO Received uploads request method=POST path=/api/pending_closures6115worker1 # [ 156.234313] nix-grpc-daemon[1521]: time=2026-09-21T09:54:36.830Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6116worker1 # [ 156.235783] nix-grpc-daemon[1521]: time=2026-09-21T09:54:36.830Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6117lb # [ 156.213545] niks3-server[964]: 2026/09/21 09:54:36 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6118worker1 # [ 156.264989] nix-grpc-daemon[1521]: time=2026-09-21T09:54:36.860Z level=INFO msg="Uploading 1 narinfos"6119lb # [ 156.228111] niks3-server[964]: 2026/09/21 09:54:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6120lb # [ 156.239269] niks3-server[964]: 2026/09/21 09:54:36 INFO Signed narinfos id=60 count=16121lb # [ 156.248417] niks3-server[964]: 2026/09/21 09:54:36 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6122worker1 # [ 156.298100] nix-grpc-daemon[1521]: time=2026-09-21T09:54:36.894Z level=INFO msg="Upload complete. (355ms)"6123lb # [ 156.256850] niks3-server[964]: 2026/09/21 09:54:36 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6124lb # [ 156.262582] niks3-server[964]: 2026/09/21 09:54:36 INFO Completed upload id=606125lb # [ 156.266553] niks3-server[964]: 2026/09/21 09:54:36 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6126worker1 # [ 156.308150] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=0 paths=1 nar_bytes_in=7526127worker1 # [ 156.317511] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536506128worker1 # [ 156.326737] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6129worker1 # [ 156.329035] nix-daemon[844]: reaped child process 1974, status = succeeded6130worker1 # [ 156.340151] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6131worker1 # [ 157.571001] nix-daemon[844]: reaped child process 1988, status = succeeded6132worker1 # [ 157.688968] nix-daemon[844]: accepted connection from pid 2004, user nix-grpc-daemon (trusted)6133worker1 # [ 157.708346] nix-daemon[844]: reaped child process 2007, status = succeeded6134lb # [ 157.671886] niks3-server[964]: 2026/09/21 09:54:38 INFO Received uploads request method=POST path=/api/pending_closures6135worker1 # [ 157.724440] nix-grpc-daemon[1521]: time=2026-09-21T09:54:38.319Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6136worker1 # [ 157.726092] nix-grpc-daemon[1521]: time=2026-09-21T09:54:38.319Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6137lb # [ 157.711464] niks3-server[964]: 2026/09/21 09:54:38 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6138lb # [ 157.723926] niks3-server[964]: 2026/09/21 09:54:38 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6139worker1 # [ 157.776085] nix-grpc-daemon[1521]: time=2026-09-21T09:54:38.371Z level=INFO msg="Uploading 1 narinfos"6140lb # [ 157.737049] niks3-server[964]: 2026/09/21 09:54:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6141lb # [ 157.746198] niks3-server[964]: 2026/09/21 09:54:38 INFO Signed narinfos id=61 count=16142lb # [ 157.752407] niks3-server[964]: 2026/09/21 09:54:38 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6143lb # [ 157.761930] niks3-server[964]: 2026/09/21 09:54:38 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6144lb # [ 157.767791] niks3-server[964]: 2026/09/21 09:54:38 INFO Completed upload id=616145worker1 # [ 157.808068] nix-grpc-daemon[1521]: time=2026-09-21T09:54:38.404Z level=INFO msg="Upload complete. (256ms)"6146lb # [ 157.770992] niks3-server[964]: 2026/09/21 09:54:38 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6147worker1 # [ 157.821391] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=13 outputs=16148worker1 # [ 157.838442] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:38Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16149worker1 # [ 157.843411] nix-daemon[844]: reaped child process 1986, status = succeeded6150worker1 # [ 157.865991] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536506151worker1 # [ 157.881085] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536506152worker1 # [ 157.895403] nix-grpc-daemon[1521]: time=2026-09-21T09:54:38.491Z level=INFO msg="All 2 paths already cached"6153worker1 # [ 157.904410] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536506154worker1 # [ 157.917740] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6155worker1 # [ 158.072155] nix-daemon[844]: accepted connection from pid 2011, user nix-grpc-daemon (trusted)6156worker1 # [ 158.092484] nix-daemon[844]: reaped child process 2014, status = succeeded6157worker1 # [ 158.193301] nix-daemon[844]: accepted connection from pid 2016, user nix-grpc-daemon (trusted)6158worker1 # [ 158.211217] nix-daemon[844]: reaped child process 2019, status = succeeded6159lb # [ 158.168068] niks3-server[964]: 2026/09/21 09:54:38 INFO Received uploads request method=POST path=/api/pending_closures6160worker1 # [ 158.217992] nix-grpc-daemon[1521]: time=2026-09-21T09:54:38.813Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6161worker1 # [ 158.219442] nix-grpc-daemon[1521]: time=2026-09-21T09:54:38.813Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6162lb # [ 158.196368] niks3-server[964]: 2026/09/21 09:54:38 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6163worker1 # [ 158.250372] nix-grpc-daemon[1521]: time=2026-09-21T09:54:38.846Z level=INFO msg="Uploading 1 narinfos"6164lb # [ 158.210256] niks3-server[964]: 2026/09/21 09:54:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6165lb # [ 158.222900] niks3-server[964]: 2026/09/21 09:54:38 INFO Signed narinfos id=62 count=16166lb # [ 158.229124] niks3-server[964]: 2026/09/21 09:54:38 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6167worker1 # [ 158.280644] nix-grpc-daemon[1521]: time=2026-09-21T09:54:38.876Z level=INFO msg="Upload complete. (347ms)"6168lb # [ 158.239446] niks3-server[964]: 2026/09/21 09:54:38 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6169lb # [ 158.246184] niks3-server[964]: 2026/09/21 09:54:38 INFO Completed upload id=626170worker1 # [ 158.287904] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=0 paths=1 nar_bytes_in=7526171lb # [ 158.249605] niks3-server[964]: 2026/09/21 09:54:38 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6172worker1 # [ 158.299610] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536506173worker1 # [ 158.304475] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6174worker1 # [ 158.307997] nix-daemon[844]: reaped child process 2009, status = succeeded6175worker1 # [ 158.317184] nix-daemon[844]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)6176worker1 # [ 159.553223] nix-daemon[844]: reaped child process 2023, status = succeeded6177worker1 # [ 159.673591] nix-daemon[844]: accepted connection from pid 2039, user nix-grpc-daemon (trusted)6178worker1 # [ 159.691461] nix-daemon[844]: reaped child process 2042, status = succeeded6179lb # [ 159.657523] niks3-server[964]: 2026/09/21 09:54:40 INFO Received uploads request method=POST path=/api/pending_closures6180worker1 # [ 159.710174] nix-grpc-daemon[1521]: time=2026-09-21T09:54:40.305Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6181worker1 # [ 159.711711] nix-grpc-daemon[1521]: time=2026-09-21T09:54:40.305Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6182lb # [ 159.693886] niks3-server[964]: 2026/09/21 09:54:40 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6183lb # [ 159.706820] niks3-server[964]: 2026/09/21 09:54:40 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6184worker1 # [ 159.758712] nix-grpc-daemon[1521]: time=2026-09-21T09:54:40.354Z level=INFO msg="Uploading 1 narinfos"6185lb # [ 159.716944] niks3-server[964]: 2026/09/21 09:54:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6186lb # [ 159.726367] niks3-server[964]: 2026/09/21 09:54:40 INFO Signed narinfos id=63 count=16187lb # [ 159.731570] niks3-server[964]: 2026/09/21 09:54:40 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6188lb # [ 159.740784] niks3-server[964]: 2026/09/21 09:54:40 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6189worker1 # [ 159.789301] nix-grpc-daemon[1521]: time=2026-09-21T09:54:40.385Z level=INFO msg="Upload complete. (264ms)"6190lb # [ 159.747549] niks3-server[964]: 2026/09/21 09:54:40 INFO Completed upload id=636191lb # [ 159.752445] niks3-server[964]: 2026/09/21 09:54:40 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6192worker1 # [ 159.802276] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53650 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=14 outputs=16193worker1 # [ 159.821363] nix-daemon[844]: reaped child process 2021, status = succeeded6194client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.53 seconds)6195worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6196worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6197client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6198client # [ 160.019858] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6199client # Running as unit: gpu.service; invocation ID: ab3c2c4d69e74e5c9110a94da7f711eb6200client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.08 seconds)6201lb: must succeed: curl -sf localhost:9901/clusters6202lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6203lb: must succeed: curl -sf localhost:9901/clusters6204lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6205worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6206client # [ 160.208213] nix[1388]: don't know how to build these paths:6207client # [ 160.209653] nix[1388]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6208worker1 # [ 160.177425] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536506209worker2 # [ 160.200972] nix-grpc-daemon[1729]: ts=2026-09-21T09:54:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507106210worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6211worker2 # [ 160.216620] nix-grpc-daemon[1729]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 14 ms (attempt 1/5)6212worker2 # [ 160.239709] nix-grpc-daemon[1729]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 77 ms (attempt 2/5)6213worker2 # [ 160.318360] nix-grpc-daemon[1729]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 3/5)6214worker2 # [ 160.476373] nix-grpc-daemon[1729]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 419 ms (attempt 4/5)6215worker2 # [ 160.897866] nix-grpc-daemon[1729]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6216worker2 # [ 160.905480] nix-grpc-daemon[1729]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6217worker1 # [ 160.919888] nix-grpc-daemon[1447]: ts=2026-09-21T09:54:41Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06218client # [ 160.949265] nix[1388]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6219lb: must succeed: curl -sf localhost:9901/clusters6220lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6221lb: must succeed: curl -sf localhost:9901/clusters6222lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6223worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6224worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6225client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6226lb # [ 168.632785] envoy[900]: conn peer=[2001:db8:1::1]:33038 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27140 tx=11379 ms=182606227client # [ 190.951114] nix[1388]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6228client # [ 190.964175] nix[1388]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6229client # [ 190.996493] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6230client # [ 191.004618] systemd[1]: gpu.service: Failed with result 'exit-code'.6231client # [ 191.008256] systemd[1]: gpu.service: Consumed 184ms CPU time over 30.981s wall clock time, 35.4M memory peak, 4K written to disk, 5.4K incoming IP traffic, 5.1K outgoing IP traffic.6232client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 30.12 seconds)6233client: must succeed: journalctl -u gpu -o cat6234client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6235lb: must succeed: curl -sf localhost:9901/clusters6236lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6237lb: must succeed: curl -sf localhost:9901/clusters6238lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6239worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6240worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6241(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 50.37 seconds)6242subtest: low disk drains a worker and builds go to the other6243worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6244worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.17 seconds)6245worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6246worker1 # [ 193.003148] nix-grpc-daemon[1447]: ts=2026-09-21T09:55:13Z level=info event=unhealthy reason=min_free6247worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)6248client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26249client # don't know how to build these paths:6250client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6251worker2 # [ 194.230440] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:507106252worker1 # [ 194.219505] nix-grpc-daemon[1447]: ts=2026-09-21T09:55:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536506253worker1 # [ 194.246466] nix-grpc-daemon[1447]: ts=2026-09-21T09:55:14Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06254worker1 # [ 194.252767] nix-grpc-daemon[1447]: ts=2026-09-21T09:55:14Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16255worker2 # [ 194.310738] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507106256worker2 # [ 194.327735] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507106257client # copying 1 paths to 'lb:50051'...6258client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6259worker2 # [ 194.344691] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507106260worker2 # [ 194.360891] nix-daemon[834]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6261worker2 # [ 194.400131] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:14Z level=info event=niks3_push_started pid=17896262worker2 # [ 194.536687] nix-grpc-daemon[1789]: time=2026-09-21T09:55:15.126Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6263worker2 # [ 194.644118] nix-daemon[834]: accepted connection from pid 1802, user nix-grpc-daemon (trusted)6264worker2 # [ 194.662338] nix-daemon[834]: reaped child process 1805, status = succeeded6265worker2 # [ 194.770888] nix-daemon[834]: accepted connection from pid 1807, user nix-grpc-daemon (trusted)6266worker2 # [ 194.791274] nix-daemon[834]: reaped child process 1810, status = succeeded6267lb # [ 194.733679] niks3-server[964]: 2026/09/21 09:55:15 INFO Received uploads request method=POST path=/api/pending_closures6268worker2 # [ 194.803489] nix-grpc-daemon[1789]: time=2026-09-21T09:55:15.393Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6269worker2 # [ 194.805458] nix-grpc-daemon[1789]: time=2026-09-21T09:55:15.393Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6270lb # [ 194.794810] niks3-server[964]: 2026/09/21 09:55:15 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6271worker2 # [ 194.860860] nix-grpc-daemon[1789]: time=2026-09-21T09:55:15.451Z level=INFO msg="Uploading 1 narinfos"6272lb # [ 194.808590] niks3-server[964]: 2026/09/21 09:55:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6273lb # [ 194.818062] niks3-server[964]: 2026/09/21 09:55:15 INFO Signed narinfos id=64 count=16274lb # [ 194.825358] niks3-server[964]: 2026/09/21 09:55:15 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6275lb # [ 194.838777] niks3-server[964]: 2026/09/21 09:55:15 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6276worker2 # [ 194.894334] nix-grpc-daemon[1789]: time=2026-09-21T09:55:15.485Z level=INFO msg="Upload complete. (355ms)"6277lb # [ 194.848896] niks3-server[964]: 2026/09/21 09:55:15 INFO Completed upload id=646278worker2 # [ 194.906201] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50710 duration_s=0 paths=1 nar_bytes_in=5126279lb # [ 194.852716] niks3-server[964]: 2026/09/21 09:55:15 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6280worker2 # [ 194.921653] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507106281worker2 # [ 194.927449] nix-daemon[834]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6282worker2 # [ 194.932629] nix-daemon[834]: reaped child process 1787, status = succeeded6283client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6284worker2 # [ 194.945105] nix-daemon[834]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6285client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6286client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6287client # farm-a> LOG-farm-a-drain6288worker2 # [ 196.203451] nix-daemon[834]: reaped child process 1814, status = succeeded6289worker2 # [ 196.336488] nix-daemon[834]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6290worker2 # [ 196.357752] nix-daemon[834]: reaped child process 1833, status = succeeded6291lb # [ 196.299252] niks3-server[964]: 2026/09/21 09:55:16 INFO Received uploads request method=POST path=/api/pending_closures6292worker2 # [ 196.368401] nix-grpc-daemon[1789]: time=2026-09-21T09:55:16.956Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6293worker2 # [ 196.369954] nix-grpc-daemon[1789]: time=2026-09-21T09:55:16.956Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6294lb # [ 196.337626] niks3-server[964]: 2026/09/21 09:55:16 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6295lb # [ 196.348501] niks3-server[964]: 2026/09/21 09:55:16 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6296lb # [ 196.360301] niks3-server[964]: 2026/09/21 09:55:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6297worker2 # [ 196.418506] nix-grpc-daemon[1789]: time=2026-09-21T09:55:17.008Z level=INFO msg="Uploading 1 narinfos"6298lb # [ 196.363643] niks3-server[964]: 2026/09/21 09:55:16 INFO Signed narinfos id=65 count=16299lb # [ 196.372776] niks3-server[964]: 2026/09/21 09:55:16 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6300worker2 # [ 196.440874] nix-grpc-daemon[1789]: time=2026-09-21T09:55:17.032Z level=INFO msg="Upload complete. (284ms)"6301lb # [ 196.384697] niks3-server[964]: 2026/09/21 09:55:16 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6302lb # [ 196.392923] niks3-server[964]: 2026/09/21 09:55:16 INFO Completed upload id=656303lb # [ 196.395312] niks3-server[964]: 2026/09/21 09:55:16 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6304worker1 # [ 196.444622] nix-grpc-daemon[1447]: ts=2026-09-21T09:55:17Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06305worker2 # [ 196.453066] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50710 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=15 outputs=16306worker1 # [ 196.450211] nix-grpc-daemon[1447]: ts=2026-09-21T09:55:17Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16307worker2 # [ 196.469761] nix-daemon[834]: reaped child process 1812, status = succeeded6308worker2 # [ 196.506081] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507106309worker2 # [ 196.520972] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507106310worker2 # [ 196.535142] nix-grpc-daemon[1789]: time=2026-09-21T09:55:17.126Z level=INFO msg="All 1 paths already cached"6311client # copying 1 paths to 'lb:50051'...6312client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6313worker2 # [ 196.544811] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507106314worker2 # [ 196.557507] nix-daemon[834]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6315worker2 # [ 196.703101] nix-daemon[834]: accepted connection from pid 1837, user nix-grpc-daemon (trusted)6316worker2 # [ 196.724260] nix-daemon[834]: reaped child process 1840, status = succeeded6317worker2 # [ 196.823746] nix-daemon[834]: accepted connection from pid 1842, user nix-grpc-daemon (trusted)6318lb # [ 196.781439] niks3-server[964]: 2026/09/21 09:55:17 INFO Received uploads request method=POST path=/api/pending_closures6319worker2 # [ 196.842024] nix-daemon[834]: reaped child process 1845, status = succeeded6320worker2 # [ 196.847301] nix-grpc-daemon[1789]: time=2026-09-21T09:55:17.438Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6321worker2 # [ 196.848935] nix-grpc-daemon[1789]: time=2026-09-21T09:55:17.438Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6322lb # [ 196.807016] niks3-server[964]: 2026/09/21 09:55:17 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6323worker2 # [ 196.878808] nix-grpc-daemon[1789]: time=2026-09-21T09:55:17.469Z level=INFO msg="Uploading 1 narinfos"6324lb # [ 196.821185] niks3-server[964]: 2026/09/21 09:55:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6325lb # [ 196.834392] niks3-server[964]: 2026/09/21 09:55:17 INFO Signed narinfos id=66 count=16326lb # [ 196.842216] niks3-server[964]: 2026/09/21 09:55:17 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6327lb # [ 196.854933] niks3-server[964]: 2026/09/21 09:55:17 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6328worker2 # [ 196.918607] nix-grpc-daemon[1789]: time=2026-09-21T09:55:17.509Z level=INFO msg="Upload complete. (345ms)"6329lb # [ 196.865878] niks3-server[964]: 2026/09/21 09:55:17 INFO Completed upload id=666330lb # [ 196.869787] niks3-server[964]: 2026/09/21 09:55:17 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6331worker2 # [ 196.927318] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50710 duration_s=0 paths=1 nar_bytes_in=6326332worker2 # [ 196.938011] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507106333worker2 # [ 196.943620] nix-daemon[834]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6334worker2 # [ 196.947982] nix-daemon[834]: reaped child process 1835, status = succeeded6335client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6336worker2 # [ 196.961435] nix-daemon[834]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6337client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6338client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6339client # farm-b> LOG-farm-b-drain6340worker2 # [ 198.470754] nix-daemon[834]: reaped child process 1849, status = succeeded6341worker2 # [ 198.541169] nix-daemon[834]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6342worker2 # [ 198.560598] nix-daemon[834]: reaped child process 1868, status = succeeded6343lb # [ 198.503208] niks3-server[964]: 2026/09/21 09:55:19 INFO Received uploads request method=POST path=/api/pending_closures6344worker2 # [ 198.570354] nix-grpc-daemon[1789]: time=2026-09-21T09:55:19.160Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6345worker2 # [ 198.571869] nix-grpc-daemon[1789]: time=2026-09-21T09:55:19.160Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6346lb # [ 198.538811] niks3-server[964]: 2026/09/21 09:55:19 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6347lb # [ 198.554280] niks3-server[964]: 2026/09/21 09:55:19 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6348worker2 # [ 198.621624] nix-grpc-daemon[1789]: time=2026-09-21T09:55:19.212Z level=INFO msg="Uploading 1 narinfos"6349lb # [ 198.564882] niks3-server[964]: 2026/09/21 09:55:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6350lb # [ 198.573670] niks3-server[964]: 2026/09/21 09:55:19 INFO Signed narinfos id=67 count=16351lb # [ 198.579004] niks3-server[964]: 2026/09/21 09:55:19 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6352lb # [ 198.587392] niks3-server[964]: 2026/09/21 09:55:19 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6353lb # [ 198.592577] niks3-server[964]: 2026/09/21 09:55:19 INFO Completed upload id=676354worker2 # [ 198.649146] nix-grpc-daemon[1789]: time=2026-09-21T09:55:19.240Z level=INFO msg="Upload complete. (277ms)"6355lb # [ 198.595181] niks3-server[964]: 2026/09/21 09:55:19 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6356worker2 # [ 198.661192] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50710 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=16 outputs=16357worker1 # [ 198.649326] nix-grpc-daemon[1447]: ts=2026-09-21T09:55:19Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06358worker1 # [ 198.655143] nix-grpc-daemon[1447]: ts=2026-09-21T09:55:19Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16359worker2 # [ 198.680790] nix-daemon[834]: reaped child process 1847, status = succeeded6360lb # [ 198.629371] envoy[900]: conn peer=[2001:db8:1::1]:37586 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=914 tx=540 ms=307676361worker2 # [ 198.718323] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507106362worker2 # [ 198.733329] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:507106363worker2 # [ 198.747839] nix-grpc-daemon[1789]: time=2026-09-21T09:55:19.339Z level=INFO msg="All 2 paths already cached"6364client # copying 1 paths to 'lb:50051'...6365client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6366worker2 # [ 198.759007] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:507106367worker2 # [ 198.772798] nix-daemon[834]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6368worker2 # [ 198.932087] nix-daemon[834]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)6369worker2 # [ 198.956813] nix-daemon[834]: reaped child process 1875, status = succeeded6370worker2 # [ 199.052130] nix-daemon[834]: accepted connection from pid 1877, user nix-grpc-daemon (trusted)6371worker2 # [ 199.068760] nix-daemon[834]: reaped child process 1880, status = succeeded6372lb # [ 199.016867] niks3-server[964]: 2026/09/21 09:55:19 INFO Received uploads request method=POST path=/api/pending_closures6373worker2 # [ 199.082869] nix-grpc-daemon[1789]: time=2026-09-21T09:55:19.673Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6374worker2 # [ 199.084499] nix-grpc-daemon[1789]: time=2026-09-21T09:55:19.673Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6375lb # [ 199.043142] niks3-server[964]: 2026/09/21 09:55:19 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6376lb # [ 199.057195] niks3-server[964]: 2026/09/21 09:55:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6377worker2 # [ 199.120516] nix-grpc-daemon[1789]: time=2026-09-21T09:55:19.711Z level=INFO msg="Uploading 1 narinfos"6378lb # [ 199.066600] niks3-server[964]: 2026/09/21 09:55:19 INFO Signed narinfos id=68 count=16379lb # [ 199.074058] niks3-server[964]: 2026/09/21 09:55:19 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6380lb # [ 199.083648] niks3-server[964]: 2026/09/21 09:55:19 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6381worker2 # [ 199.146530] nix-grpc-daemon[1789]: time=2026-09-21T09:55:19.737Z level=INFO msg="Upload complete. (357ms)"6382lb # [ 199.093994] niks3-server[964]: 2026/09/21 09:55:19 INFO Completed upload id=686383lb # [ 199.097775] niks3-server[964]: 2026/09/21 09:55:19 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6384worker2 # [ 199.155357] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50710 duration_s=0 paths=1 nar_bytes_in=7606385worker2 # [ 199.166891] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:507106386worker2 # [ 199.172717] nix-daemon[834]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6387worker2 # [ 199.177777] nix-daemon[834]: reaped child process 1870, status = succeeded6388client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6389worker2 # [ 199.190325] nix-daemon[834]: accepted connection from pid 1729, user nix-grpc-daemon (trusted)6390client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6391client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6392client # farm-top> LOG-farm-top-drain6393worker2 # [ 200.263738] nix-daemon[834]: reaped child process 1884, status = succeeded6394worker2 # [ 200.336664] nix-daemon[834]: accepted connection from pid 1900, user nix-grpc-daemon (trusted)6395worker2 # [ 200.354208] nix-daemon[834]: reaped child process 1903, status = succeeded6396lb # [ 200.305932] niks3-server[964]: 2026/09/21 09:55:20 INFO Received uploads request method=POST path=/api/pending_closures6397worker2 # [ 200.375013] nix-grpc-daemon[1789]: time=2026-09-21T09:55:20.964Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6398worker2 # [ 200.376698] nix-grpc-daemon[1789]: time=2026-09-21T09:55:20.964Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6399lb # [ 200.341076] niks3-server[964]: 2026/09/21 09:55:20 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6400worker2 # [ 200.410283] nix-grpc-daemon[1789]: time=2026-09-21T09:55:21.001Z level=INFO msg="Uploading 1 narinfos"6401lb # [ 200.352787] niks3-server[964]: 2026/09/21 09:55:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6402lb # [ 200.365576] niks3-server[964]: 2026/09/21 09:55:20 INFO Signed narinfos id=69 count=16403lb # [ 200.374292] niks3-server[964]: 2026/09/21 09:55:20 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6404worker2 # [ 200.446902] nix-grpc-daemon[1789]: time=2026-09-21T09:55:21.038Z level=INFO msg="Upload complete. (246ms)"6405lb # [ 200.390004] niks3-server[964]: 2026/09/21 09:55:20 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6406lb # [ 200.402193] niks3-server[964]: 2026/09/21 09:55:20 INFO Completed upload id=696407worker2 # [ 200.457162] nix-grpc-daemon[1729]: ts=2026-09-21T09:55:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50710 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=17 outputs=16408lb # [ 200.405653] niks3-server[964]: 2026/09/21 09:55:20 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6409lb # [ 200.409756] niks3-server[964]: 2026/09/21 09:55:20 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6410worker2 # [ 200.475060] nix-daemon[834]: reaped child process 1882, status = succeeded6411client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 6.50 seconds)6412client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath6413client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.17 seconds)6414worker1: must succeed: rm /nix/.rw-store/fill6415worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)6416worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6417worker1 # [ 201.007830] nix-grpc-daemon[1447]: ts=2026-09-21T09:55:21Z level=info event=healthy reason=min_free6418worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)6419(finished: subtest: low disk drains a worker and builds go to the other, in 10.28 seconds)6420(finished: run the VM test script, in 202.88 seconds)6421test script finished in 203.13s6422cleanup6423kill QemuMachine (pid 49)6424client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)6425client # [2026-09-21T09:55:23Z INFO virtiofsd] Client disconnected, shutting down6426client # [2026-09-21T09:55:23Z INFO virtiofsd] Client disconnected, shutting down6427client # [2026-09-21T09:55:23Z INFO virtiofsd] Client disconnected, shutting down6428kill QemuMachine (pid 50)6429lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)6430lb # [2026-09-21T09:55:23Z INFO virtiofsd] Client disconnected, shutting down6431lb # [2026-09-21T09:55:23Z INFO virtiofsd] Client disconnected, shutting down6432lb # [2026-09-21T09:55:23Z INFO virtiofsd] Client disconnected, shutting down6433kill QemuMachine (pid 51)6434worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)6435worker1 # [2026-09-21T09:55:23Z INFO virtiofsd] Client disconnected, shutting down6436worker1 # [2026-09-21T09:55:23Z INFO virtiofsd] Client disconnected, shutting down6437worker1 # [2026-09-21T09:55:23Z INFO virtiofsd] Client disconnected, shutting down6438kill QemuMachine (pid 52)6439worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)6440worker2 # [2026-09-21T09:55:24Z INFO virtiofsd] Client disconnected, shutting down6441worker2 # [2026-09-21T09:55:24Z INFO virtiofsd] Client disconnected, shutting down6442worker2 # [2026-09-21T09:55:24Z INFO virtiofsd] Client disconnected, shutting down6443(finished: cleanup, in 1.13 seconds)6444additionally exposed symbols:6445 client, lb, worker1, worker2,6446 vlan1,6447 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh