tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm client # Disk image does not exist, creating the virtualisation disk image... client: QEMU running (pid 49) client # Formatting '/build/vm-state-client/tmp.t8W6lP5d83', fmt=raw size=1073741824 worker2: QEMU running (pid 52) worker1 # Disk image does not exist, creating the virtualisation disk image... lb: QEMU running (pid 50) client # mke2fs 1.47.4 (6-Mar-2025) lb # Disk image does not exist, creating the virtualisation disk image... worker2 # Disk image does not exist, creating the virtualisation disk image... client # Discarding device blocks: 0/262144 done lb # Formatting '/build/vm-state-lb/tmp.5o23et6ELD', fmt=raw size=1073741824 worker2 # Formatting '/build/vm-state-worker2/tmp.6jShamaDoo', fmt=raw size=1073741824 client # Creating filesystem with 262144 4k blocks and 65536 inodes lb # mke2fs 1.47.4 (6-Mar-2025) worker2 # mke2fs 1.47.4 (6-Mar-2025) client # Filesystem UUID: bd498c12-dff1-4cc1-bc1d-be1cb8394cc7 lb # Discarding device blocks: 0/262144 done worker2 # Discarding device blocks: 0/262144 done client # Superblock backups stored on blocks: lb # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes client # 32768, 98304, 163840, 229376 lb # Filesystem UUID: 209df36f-c74f-438a-b883-feb5929c5341 worker2 # Filesystem UUID: 80cb3d53-321a-4129-8263-8fcad6d40ced client # lb # Superblock backups stored on blocks: worker2 # Superblock backups stored on blocks: client # Allocating group tables: 0/8 done lb # 32768, 98304, 163840, 229376 worker2 # 32768, 98304, 163840, 229376 client # Writing inode tables: 0/8 done lb # worker2 # client # Creating journal (8192 blocks): done lb # Allocating group tables: 0/8 done worker2 # Allocating group tables: 0/8 done client # Writing superblocks and filesystem accounting information: 0/8 done lb # Writing inode tables: 0/8 done worker2 # Writing inode tables: 0/8 done client # lb # Creating journal (8192 blocks): done worker2 # Creating journal (8192 blocks): done client # Virtualisation disk image created. lb # Writing superblocks and filesystem accounting information: 0/8 done worker2 # Writing superblocks and filesystem accounting information: 0/8 done client # Starting virtiofs daemons... lb # worker2 # client # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # Virtualisation disk image created. worker2 # Virtualisation disk image created. client # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # Starting virtiofs daemons... worker2 # Starting virtiofs daemons... client # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests worker1: QEMU running (pid 51) worker1 # Formatting '/build/vm-state-worker1/tmp.PFbKYLG5Vq', fmt=raw size=1073741824 lb # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests worker1 # mke2fs 1.47.4 (6-Mar-2025) lb # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests worker1 # Discarding device blocks: 0/262144 done (finished: start all VMs, in 0.52 seconds) worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # Filesystem UUID: 9c69db1f-c800-4716-bd16-1a58065274da client: waiting for unit network-addresses-eth1.service worker2 # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests client: waiting for the VM to finish booting worker2 # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests worker1 # Superblock backups stored on blocks: worker1 # 32768, 98304, 163840, 229376 worker1 # worker1 # Allocating group tables: 0/8 done worker1 # Writing inode tables: 0/8 done worker1 # Creating journal (8192 blocks): done worker1 # Writing superblocks and filesystem accounting information: 0/8 done worker1 # worker1 # Virtualisation disk image created. worker1 # Starting virtiofs daemons... worker1 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T11:31:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T11:31:33Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-22T11:31:33Z INFO virtiofsd] Client connected, servicing requests client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] KASLR enabled client # [ 0.000000] random: crng init done client # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] efi: UEFI not found. client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] Zone ranges: client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] DMA32 empty client # [ 0.000000] Normal empty client # [ 0.000000] Device empty client # [ 0.000000] Movable zone start for each node client # [ 0.000000] Early memory node ranges client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] psci: Trusted OS migration not required client # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 client # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] CPU features: detected: Spectre-BHB worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] alternatives: applying boot alternatives worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker2 # [ 0.000000] KASLR enabled worker2 # [ 0.000000] random: crng init done worker2 # [ 0.000000] Machine model: linux,dummy-virt worker2 # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/52ij8s6jkav0f53amdkjkhqqbw3wp27m-nixos-system-client-test/init regInfo=/nix/store/7pzypp1k8pcxgz5kigmsrkgswbnh2dyi-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] Zone ranges: worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] KASLR enabled worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/7pzypp1k8pcxgz5kigmsrkgswbnh2dyi-closure-info/registration", will be passed to user space. worker2 # [ 0.000000] DMA32 empty worker1 # [ 0.000000] random: crng init done worker2 # [ 0.000000] Normal empty worker1 # [ 0.000000] Machine model: linux,dummy-virt worker2 # [ 0.000000] Device empty client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker1 # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] Movable zone start for each node worker2 # [ 0.000000] Early memory node ranges client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker1 # [ 0.000000] Zone ranges: worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] software IO TLB: area num 1. worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker1 # [ 0.000000] DMA32 empty worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] Normal empty worker1 # [ 0.000000] Device empty client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.000000] Movable zone start for each node client # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] psci: Trusted OS migration not required worker1 # [ 0.000000] Early memory node ranges worker2 # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Policy zone: DMA worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.000000] Detected PIPT I-cache on CPU0 worker1 # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] allocated 3145728 bytes of page_ext worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] psci: Trusted OS migration not required client # [ 0.000000] rcu: Hierarchical RCU implementation. worker2 # [ 0.000000] CPU features: detected: Spectre-v4 worker1 # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] Rude variant of Tasks RCU enabled. worker1 # [ 0.000000] Detected PIPT I-cache on CPU0 worker2 # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fjl79kv3mslif1ryinjddfhvml6nj1zy-nixos-system-worker2-test/init regInfo=/nix/store/5w3f1zdp329rh9c37p2qh6m4v14r7217-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/5w3f1zdp329rh9c37p2qh6m4v14r7217-closure-info/registration", will be passed to user space. client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes client # [ 0.000000] GICv3: 256 SPIs implemented client # [ 0.000000] GICv3: 0 Extended SPIs implemented worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7w3hl2ifzqxyp2m0j26ag73ir8lww0pf-nixos-system-worker1-test/init regInfo=/nix/store/83grcsshvji6snxndrs5k1lrxmsrsjr5-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/83grcsshvji6snxndrs5k1lrxmsrsjr5-closure-info/registration", will be passed to user space. client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.000000] Fallback order for Node 0: 0 client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] Policy zone: DMA client # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000 worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000 worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.000000] allocated 3145728 bytes of page_ext worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker1 # [ 0.000000] Fallback order for Node 0: 0 client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] Policy zone: DMA worker2 # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.000037] arm-pv: using stolen time PV worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups client # [ 0.000665] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.000000] rcu: Hierarchical RCU implementation. client # [ 0.000871] Console: colour dummy device 80x25 worker1 # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.000879] printk: legacy console [tty0] enabled worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.001096] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.001102] pid_max: default: 32768 minimum: 301 worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled. client # [ 0.001177] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.001348] landlock: Up and running. worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.001351] Yama: becoming mindful. worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 client # [ 0.002008] LSM support for eBPF active worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.000000] GICv3: 256 SPIs implemented worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented client # [ 0.002185] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq client # [ 0.002225] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.004221] rcu: Hierarchical SRCU implementation. worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.004226] rcu: Max phase no-delay instances is 1000. lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.005502] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 client # [ 0.005593] EFI services will not be available. worker1 # [ 0.000000] GICv3: 256 SPIs implemented client # [ 0.005682] smp: Bringing up secondary CPUs ... lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] KASLR enabled client # [ 0.005690] smp: Brought up 1 node, 1 CPU worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.000000] random: crng init done client # [ 0.005694] SMP: Total of 1 processors activated. lb # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) client # [ 0.005699] CPU: All CPU(s) started at EL1 lb # [ 0.000000] efi: UEFI not found. worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 client # [ 0.005709] CPU features: detected: Branch Target Identification lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.005713] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.005716] CPU features: detected: Instruction cache invalidation not required for I/D coherence lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) lb # [ 0.000000] Zone ranges: worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns client # [ 0.005720] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). lb # [ 0.000000] DMA32 empty client # [ 0.005724] CPU features: detected: Common not Private translations lb # [ 0.000000] Normal empty worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 lb # [ 0.000000] Device empty client # [ 0.005727] CPU features: detected: CRC32 instructions lb # [ 0.000000] Movable zone start for each node worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 lb # [ 0.000000] Early memory node ranges client # [ 0.005730] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.005734] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.000032] arm-pv: using stolen time PV lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.005737] CPU features: detected: Data independent timing control (DIT) worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.005740] CPU features: detected: E0PD worker2 # [ 0.000450] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). lb # [ 0.000000] psci: probing for conduit method from DT. worker2 # [ 0.000617] Console: colour dummy device 80x25 client # [ 0.005743] CPU features: detected: Enhanced Counter Virtualization lb # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker2 # [ 0.000624] printk: legacy console [tty0] enabled client # [ 0.005746] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.005750] CPU features: detected: Enhanced Virtualization Traps lb # [ 0.000000] psci: Trusted OS migration not required worker2 # [ 0.000838] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns lb # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.005753] CPU features: detected: Fine Grained Traps worker2 # [ 0.000845] pid_max: default: 32768 minimum: 301 worker1 # [ 0.000031] arm-pv: using stolen time PV worker2 # [ 0.000917] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.005757] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.001055] landlock: Up and running. worker1 # [ 0.000455] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.005763] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.001058] Yama: becoming mindful. lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker1 # [ 0.000653] Console: colour dummy device 80x25 worker2 # [ 0.001550] LSM support for eBPF active client # [ 0.005766] CPU features: detected: LSE atomic instructions lb # [ 0.000000] Detected PIPT I-cache on CPU0 worker1 # [ 0.000660] printk: legacy console [tty0] enabled client # [ 0.005768] CPU features: detected: Privileged Access Never worker2 # [ 0.001701] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.005771] CPU features: detected: PMUv3 worker1 # [ 0.000851] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) lb # [ 0.000000] CPU features: detected: GICv3 CPU interface worker2 # [ 0.001734] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.005774] CPU features: detected: RAS Extension Support worker1 # [ 0.000858] pid_max: default: 32768 minimum: 301 lb # [ 0.000000] CPU features: detected: Spectre-v4 worker2 # [ 0.003586] rcu: Hierarchical SRCU implementation. client # [ 0.005777] CPU features: detected: RASv1p1 Extension Support lb # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000939] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.003591] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.001077] landlock: Up and running. client # [ 0.005780] CPU features: detected: Random Number Generator lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker2 # [ 0.004858] fsl-mc MSI: its@8080000 domain created worker1 # [ 0.001080] Yama: becoming mindful. worker1 # [ 0.001564] LSM support for eBPF active worker2 # [ 0.004950] EFI services will not be available. lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.005782] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.005060] smp: Bringing up secondary CPUs ... lb # [ 0.000000] alternatives: applying boot alternatives worker1 # [ 0.001726] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.005068] smp: Brought up 1 node, 1 CPU client # [ 0.005785] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.005070] SMP: Total of 1 processors activated. worker1 # [ 0.001759] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.005787] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.005076] CPU: All CPU(s) started at EL1 worker1 # [ 0.003653] rcu: Hierarchical SRCU implementation. worker2 # [ 0.005087] CPU features: detected: Branch Target Identification client # [ 0.005792] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.003658] rcu: Max phase no-delay instances is 1000. worker2 # [ 0.005092] CPU features: detected: ARMv8.4 Translation Table Level client # [ 0.005833] alternatives: applying system-wide alternatives worker1 # [ 0.004959] fsl-mc MSI: its@8080000 domain created worker1 # [ 0.005049] EFI services will not be available. lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rwj7rgrjnik4yfrdy5nxpi16ig7dj0yx-nixos-system-lb-test/init regInfo=/nix/store/y0b8gyps1r0yyzfhbdbfwqvj69pgkv2z-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.008843] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.005095] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker1 # [ 0.005130] smp: Bringing up secondary CPUs ... worker1 # [ 0.005139] smp: Brought up 1 node, 1 CPU worker2 # [ 0.005099] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker1 # [ 0.005141] SMP: Total of 1 processors activated. lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/y0b8gyps1r0yyzfhbdbfwqvj69pgkv2z-closure-info/registration", will be passed to user space. client # [ 0.009003] Memory: 1405648K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved) worker1 # [ 0.005147] CPU: All CPU(s) started at EL1 client # [ 0.009429] devtmpfs: initialized worker2 # [ 0.005103] CPU features: detected: Common not Private translations lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker1 # [ 0.005157] CPU features: detected: Branch Target Identification worker2 # [ 0.005106] CPU features: detected: CRC32 instructions client # [ 0.011293] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.005162] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.005109] CPU features: detected: Data cache clean to Point of Deep Persistence client # [ 0.011316] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.005113] CPU features: detected: Data cache clean to Point of Persistence worker1 # [ 0.005165] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.011512] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.005116] CPU features: detected: Data independent timing control (DIT) client # [ 0.011516] 0 pages in range for non-PLT usage lb # [ 0.000000] software IO TLB: area num 1. worker1 # [ 0.005168] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.005119] CPU features: detected: E0PD client # [ 0.011517] 508272 pages in range for PLT usage worker1 # [ 0.005172] CPU features: detected: Common not Private translations lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.011638] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.005121] CPU features: detected: Enhanced Counter Virtualization worker1 # [ 0.005176] CPU features: detected: CRC32 instructions client # [ 0.012372] DMI not present or invalid. lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.005125] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.015951] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.000000] Fallback order for Node 0: 0 worker1 # [ 0.005178] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.005128] CPU features: detected: Enhanced Virtualization Traps lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 client # [ 0.018342] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.005182] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.005131] CPU features: detected: Fine Grained Traps lb # [ 0.000000] Policy zone: DMA client # [ 0.018535] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker1 # [ 0.005185] CPU features: detected: Data independent timing control (DIT) lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker2 # [ 0.005135] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker1 # [ 0.005188] CPU features: detected: E0PD client # [ 0.018738] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.005140] CPU features: detected: RCpc load-acquire (LDAPR) lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.005190] CPU features: detected: Enhanced Counter Virtualization lb # [ 0.000000] allocated 3145728 bytes of page_ext client # [ 0.018762] audit: initializing netlink subsys (disabled) worker2 # [ 0.005143] CPU features: detected: LSE atomic instructions worker1 # [ 0.005193] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.005146] CPU features: detected: Privileged Access Never client # [ 0.019323] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.005148] CPU features: detected: PMUv3 lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.005197] CPU features: detected: Enhanced Virtualization Traps client # [ 0.019325] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.005200] CPU features: detected: Fine Grained Traps worker2 # [ 0.005151] CPU features: detected: RAS Extension Support client # [ 0.019329] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.005154] CPU features: detected: RASv1p1 Extension Support worker1 # [ 0.005204] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.019332] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.005157] CPU features: detected: Random Number Generator worker1 # [ 0.005208] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.019337] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.005159] CPU features: detected: Speculation barrier (SB) worker1 # [ 0.005211] CPU features: detected: LSE atomic instructions lb # [ 0.000000] Rude variant of Tasks RCU enabled. worker2 # [ 0.005162] CPU features: detected: Stage-2 Force Write-Back client # [ 0.019364] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.005214] CPU features: detected: Privileged Access Never lb # [ 0.000000] Tracing variant of Tasks RCU enabled. client # [ 0.019373] cpuidle: using governor ladder worker2 # [ 0.005165] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.005217] CPU features: detected: PMUv3 client # [ 0.019379] cpuidle: using governor menu lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.005220] CPU features: detected: RAS Extension Support worker2 # [ 0.005170] CPU features: detected: Speculative Store Bypassing Safe (SSBS) client # [ 0.019562] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.005222] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.005209] alternatives: applying system-wide alternatives client # [ 0.019579] ASID allocator initialised with 65536 entries worker1 # [ 0.005225] CPU features: detected: Random Number Generator client # [ 0.020789] Serial: AMBA PL011 UART driver lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.008235] CPU features: detected: BBM Level 2 without TLB conflict abort client # [ 0.026123] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.005228] CPU features: detected: Speculation barrier (SB) client # [ 0.026262] printk: console [ttyAMA0] enabled worker2 # [ 0.008446] Memory: 1405916K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved) worker1 # [ 0.005231] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.008825] devtmpfs: initialized lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.005233] CPU features: detected: TLB range maintenance instructions lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker2 # [ 0.010938] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.005238] CPU features: detected: Speculative Store Bypassing Safe (SSBS) lb # [ 0.000000] GICv3: 0 Extended SPIs implemented worker2 # [ 0.010962] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.005276] alternatives: applying system-wide alternatives lb # [ 0.000000] Root IRQ handler: gic_handle_irq worker2 # [ 0.011155] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.008198] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker2 # [ 0.011160] 0 pages in range for non-PLT usage lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.008366] Memory: 1405624K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved) worker2 # [ 0.011161] 508272 pages in range for PLT usage lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker1 # [ 0.008757] devtmpfs: initialized worker2 # [ 0.011270] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.012145] DMI not present or invalid. worker1 # [ 0.010603] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.015450] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.145307] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.010626] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) client # [ 0.145328] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.018201] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.010793] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker1 # [ 0.010797] 0 pages in range for non-PLT usage client # [ 0.145333] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker2 # [ 0.018390] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker1 # [ 0.010799] 508272 pages in range for PLT usage client # [ 0.145338] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker2 # [ 0.018596] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker1 # [ 0.010913] pinctrl core: initialized pinctrl subsystem lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.145342] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.018620] audit: initializing netlink subsys (disabled) worker1 # [ 0.011685] DMI not present or invalid. client # [ 0.145347] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.019211] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.015358] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.019213] thermal_sys: Registered thermal governor 'bang_bang' client # [ 0.145351] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker1 # [ 0.017724] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.019217] thermal_sys: Registered thermal governor 'step_wise' client # [ 0.145355] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker1 # [ 0.017917] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.019220] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.018124] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.019225] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.018148] audit: initializing netlink subsys (disabled) lb # [ 0.000031] arm-pv: using stolen time PV client # [ 0.152637] fbcon: Taking over console worker2 # [ 0.019251] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.018747] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.152652] ACPI: Interpreter disabled. worker2 # [ 0.019259] cpuidle: using governor ladder lb # [ 0.000390] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.018749] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.019265] cpuidle: using governor menu lb # [ 0.000603] Console: colour dummy device 80x25 worker1 # [ 0.018753] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.019461] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.000611] printk: legacy console [tty0] enabled worker1 # [ 0.018755] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.019478] ASID allocator initialised with 65536 entries worker2 # [ 0.020732] Serial: AMBA PL011 UART driver lb # [ 0.000799] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.018761] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.000806] pid_max: default: 32768 minimum: 301 worker2 # [ 0.025984] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker1 # [ 0.018785] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.000888] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.026122] printk: console [ttyAMA0] enabled worker1 # [ 0.018793] cpuidle: using governor ladder lb # [ 0.001009] landlock: Up and running. worker1 # [ 0.018799] cpuidle: using governor menu lb # [ 0.001012] Yama: becoming mindful. lb # [ 0.001446] LSM support for eBPF active worker1 # [ 0.019001] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.019017] ASID allocator initialised with 65536 entries lb # [ 0.001588] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.020227] Serial: AMBA PL011 UART driver lb # [ 0.001622] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.025531] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.003528] rcu: Hierarchical SRCU implementation. worker1 # [ 0.025701] printk: console [ttyAMA0] enabled lb # [ 0.003534] rcu: Max phase no-delay instances is 1000. client # [ 0.161677] iommu: Default domain type: Translated lb # [ 0.004799] fsl-mc MSI: its@8080000 domain created client # [ 0.161688] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.004896] EFI services will not be available. client # [ 0.162031] SCSI subsystem initialized lb # [ 0.004979] smp: Bringing up secondary CPUs ... lb # [ 0.004987] smp: Brought up 1 node, 1 CPU worker2 # [ 0.144246] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages client # [ 0.164023] usbcore: registered new interface driver usbfs lb # [ 0.004991] SMP: Total of 1 processors activated. lb # [ 0.004996] CPU: All CPU(s) started at EL1 worker2 # [ 0.144266] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.164069] usbcore: registered new interface driver hub client # [ 0.164086] usbcore: registered new device driver usb lb # [ 0.005006] CPU features: detected: Branch Target Identification worker2 # [ 0.144271] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.164361] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.005010] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.144275] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.164367] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.144280] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.005013] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.164385] PTP clock support registered worker2 # [ 0.144284] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.164435] EDAC MC: Ver: 3.0.0 worker1 # [ 0.143003] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.005017] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.144289] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages client # [ 0.169000] scmi_core: SCMI protocol bus registered worker1 # [ 0.143020] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.005020] CPU features: detected: Common not Private translations worker2 # [ 0.144293] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.005024] CPU features: detected: CRC32 instructions worker1 # [ 0.143025] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.169955] FPGA manager framework client # [ 0.170912] vgaarb: loaded worker1 # [ 0.143030] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.005027] CPU features: detected: Data cache clean to Point of Deep Persistence worker1 # [ 0.143034] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.005030] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.171527] clocksource: Switched to clocksource arch_sys_counter worker2 # [ 0.151661] fbcon: Taking over console worker1 # [ 0.143039] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.005033] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.151677] ACPI: Interpreter disabled. lb # [ 0.005036] CPU features: detected: E0PD worker1 # [ 0.143043] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.005039] CPU features: detected: Enhanced Counter Virtualization worker1 # [ 0.143047] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.005042] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) lb # [ 0.005045] CPU features: detected: Enhanced Virtualization Traps lb # [ 0.005049] CPU features: detected: Fine Grained Traps worker1 # [ 0.150395] fbcon: Taking over console worker1 # [ 0.150410] ACPI: Interpreter disabled. lb # [ 0.005052] CPU features: detected: Generic authentication (architected QARMA5 algorithm) lb # [ 0.005057] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.177872] VFS: Disk quotas dquot_6.6.0 lb # [ 0.005061] CPU features: detected: LSE atomic instructions client # [ 0.177917] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.005063] CPU features: detected: Privileged Access Never lb # [ 0.005066] CPU features: detected: PMUv3 client # [ 0.179646] netfs: FS-Cache loaded lb # [ 0.005069] CPU features: detected: RAS Extension Support client # [ 0.179771] pnp: PnP ACPI: disabled lb # [ 0.005071] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.160637] iommu: Default domain type: Translated lb # [ 0.005074] CPU features: detected: Random Number Generator worker2 # [ 0.160647] iommu: DMA domain TLB invalidation policy: strict mode worker2 # [ 0.160975] SCSI subsystem initialized lb # [ 0.005077] CPU features: detected: Speculation barrier (SB) lb # [ 0.005079] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.163003] usbcore: registered new interface driver usbfs lb # [ 0.005083] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.163035] usbcore: registered new interface driver hub lb # [ 0.005087] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker2 # [ 0.163059] usbcore: registered new device driver usb worker1 # [ 0.159401] iommu: Default domain type: Translated lb # [ 0.005124] alternatives: applying system-wide alternatives worker2 # [ 0.163309] pps_core: LinuxPPS API ver. 1 registered worker1 # [ 0.159411] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.008119] CPU features: detected: BBM Level 2 without TLB conflict abort worker1 # [ 0.159745] SCSI subsystem initialized worker2 # [ 0.163315] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.163324] PTP clock support registered worker2 # [ 0.163372] EDAC MC: Ver: 3.0.0 worker1 # [ 0.161724] usbcore: registered new interface driver usbfs lb # [ 0.008330] Memory: 1405640K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved) client # [ 0.183606] NET: Registered PF_INET protocol family lb # [ 0.008756] devtmpfs: initialized worker1 # [ 0.161757] usbcore: registered new interface driver hub worker2 # [ 0.168026] scmi_core: SCMI protocol bus registered client # [ 0.183885] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.161772] usbcore: registered new device driver usb lb # [ 0.010657] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.168980] FPGA manager framework worker1 # [ 0.162042] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.169932] vgaarb: loaded lb # [ 0.010680] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.162048] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.010857] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker2 # [ 0.170557] clocksource: Switched to clocksource arch_sys_counter worker1 # [ 0.162057] PTP clock support registered lb # [ 0.010861] 0 pages in range for non-PLT usage worker1 # [ 0.162104] EDAC MC: Ver: 3.0.0 lb # [ 0.010863] 508272 pages in range for PLT usage worker1 # [ 0.166643] scmi_core: SCMI protocol bus registered lb # [ 0.010974] pinctrl core: initialized pinctrl subsystem lb # [ 0.011776] DMI not present or invalid. worker1 # [ 0.167633] FPGA manager framework lb # [ 0.014908] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.168625] vgaarb: loaded lb # [ 0.017491] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.169256] clocksource: Switched to clocksource arch_sys_counter lb # [ 0.017683] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.017884] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.017905] audit: initializing netlink subsys (disabled) worker2 # [ 0.176928] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.176975] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.018469] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.178685] netfs: FS-Cache loaded lb # [ 0.018471] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.178806] pnp: PnP ACPI: disabled worker1 # [ 0.173872] VFS: Disk quotas dquot_6.6.0 lb # [ 0.018475] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.173912] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.018477] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.018483] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.018508] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.018516] cpuidle: using governor ladder worker1 # [ 0.177635] netfs: FS-Cache loaded lb # [ 0.018523] cpuidle: using governor menu worker1 # [ 0.177757] pnp: PnP ACPI: disabled lb # [ 0.018714] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.018731] ASID allocator initialised with 65536 entries lb # [ 0.019947] Serial: AMBA PL011 UART driver lb # [ 0.025251] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.182722] NET: Registered PF_INET protocol family lb # [ 0.025414] printk: console [ttyAMA0] enabled worker2 # [ 0.183004] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.181640] NET: Registered PF_INET protocol family worker1 # [ 0.181911] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.144792] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.144811] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.144816] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages lb # [ 0.144821] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.144825] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.144830] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.213027] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.144834] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages client # [ 0.213073] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.144838] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page client # [ 0.213099] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.213172] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.152418] fbcon: Taking over console client # [ 0.213328] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.152432] ACPI: Interpreter disabled. client # [ 0.213411] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.154383] iommu: Default domain type: Translated client # [ 0.213446] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.154393] iommu: DMA domain TLB invalidation policy: strict mode client # [ 0.213482] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.156092] SCSI subsystem initialized client # [ 0.213573] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.213594] NET: Registered PF_XDP protocol family client # [ 0.213611] PCI: CLS 0 bytes, default 64 client # [ 0.213857] Trying to unpack rootfs image as initramfs... client # [ 0.229335] kvm [1]: HYP mode not available lb # [ 0.161257] usbcore: registered new interface driver usbfs lb # [ 0.161290] usbcore: registered new interface driver hub lb # [ 0.161306] usbcore: registered new device driver usb lb # [ 0.161579] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.161586] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.161596] PTP clock support registered lb # [ 0.161663] EDAC MC: Ver: 3.0.0 lb # [ 0.166325] scmi_core: SCMI protocol bus registered lb # [ 0.167303] FPGA manager framework lb # [ 0.168270] vgaarb: loaded lb # [ 0.168911] clocksource: Switched to clocksource arch_sys_counter worker1 # [ 0.211263] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker2 # [ 0.213095] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.211310] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.173780] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.213140] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.211336] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.173817] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.213166] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.211407] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.213237] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.211550] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.213380] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.211640] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.213462] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.177568] netfs: FS-Cache loaded worker1 # [ 0.211675] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.177697] pnp: PnP ACPI: disabled worker2 # [ 0.213515] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.211710] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.213551] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.211800] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.213668] NET: Registered PF_UNIX/PF_LOCAL protocol family worker1 # [ 0.211820] NET: Registered PF_XDP protocol family worker2 # [ 0.213688] NET: Registered PF_XDP protocol family worker1 # [ 0.211837] PCI: CLS 0 bytes, default 64 worker2 # [ 0.213706] PCI: CLS 0 bytes, default 64 worker1 # [ 0.212083] Trying to unpack rootfs image as initramfs... worker2 # [ 0.213945] Trying to unpack rootfs image as initramfs... worker1 # [ 0.227007] kvm [1]: HYP mode not available lb # [ 0.181592] NET: Registered PF_INET protocol family lb # [ 0.181879] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.232498] kvm [1]: HYP mode not available lb # [ 0.212022] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.212077] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.212104] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.212179] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.212323] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.212411] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.212448] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.212484] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.212574] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.212613] NET: Registered PF_XDP protocol family lb # [ 0.212632] PCI: CLS 0 bytes, default 64 lb # [ 0.212885] Trying to unpack rootfs image as initramfs... lb # [ 0.230662] kvm [1]: HYP mode not available client # [ 0.325245] Initialise system trusted keyrings client # [ 0.331576] workingset: timestamp_bits=42 max_order=19 bucket_order=0 client # [ 0.332946] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.333735] 9p: Installing v9fs 9p2000 file system support worker1 # [ 0.322404] Initialise system trusted keyrings worker1 # [ 0.323131] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker2 # [ 0.329017] Initialise system trusted keyrings worker1 # [ 0.324438] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker2 # [ 0.329763] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.325208] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.331012] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker2 # [ 0.331761] 9p: Installing v9fs 9p2000 file system support client # [ 0.354419] Key type asymmetric registered client # [ 0.354442] Asymmetric key parser 'x509' registered client # [ 0.354518] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.363607] io scheduler mq-deadline registered client # [ 0.363630] io scheduler kyber registered client # [ 0.368727] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.370004] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.370385] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.370403] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.370416] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.355779] Key type asymmetric registered client # [ 0.370424] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.355806] Asymmetric key parser 'x509' registered client # [ 0.370448] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.355885] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker2 # [ 0.358051] io scheduler mq-deadline registered client # [ 0.370477] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker2 # [ 0.358063] io scheduler kyber registered client # [ 0.370549] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker1 # [ 0.354002] Key type asymmetric registered client # [ 0.370559] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.354033] Asymmetric key parser 'x509' registered client # [ 0.370565] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.354140] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.370570] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker1 # [ 0.356312] io scheduler mq-deadline registered client # [ 0.370575] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker1 # [ 0.356323] io scheduler kyber registered client # [ 0.370628] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.371058] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.371240] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.371256] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.371286] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.371302] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.371271] pl061_gpio 9030000.pl061: PL061 GPIO chip registered lb # [ 0.325443] Initialise system trusted keyrings client # [ 0.391853] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.392057] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.326201] workingset: timestamp_bits=42 max_order=19 bucket_order=0 client # [ 0.392074] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.327508] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.392104] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.372617] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.392578] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.328301] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.373002] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.392757] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker2 # [ 0.373019] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.392774] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.392803] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.373031] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.393254] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.369528] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker2 # [ 0.373040] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.393448] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.370198] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.373066] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.393464] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.370565] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.393494] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.373102] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker1 # [ 0.370583] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.393513] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.373187] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker1 # [ 0.370596] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.373196] pci_bus 0000:00: root bus resource [bus 00-ff] client # [ 0.393965] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.394152] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.373202] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.370605] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.373207] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.394182] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.370626] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.373212] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.394637] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker1 # [ 0.370650] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.394824] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.373266] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.370723] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.394854] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.373720] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.370732] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.373905] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.395273] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker1 # [ 0.370738] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.395453] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker2 # [ 0.373922] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.370743] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.373952] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.370748] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.373968] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.370829] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.374428] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.371281] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.374636] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.371471] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.374652] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.412634] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.371488] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.412825] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.374682] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.371518] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.412855] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.375136] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.371535] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.375318] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.413328] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.371998] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.375334] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.413524] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.372205] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.375364] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.413554] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.372222] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.375817] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.414009] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.372253] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.376000] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.414197] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.376015] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.372728] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.414227] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.372917] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker2 # [ 0.376045] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.414686] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.372934] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.376063] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.414980] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.372964] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.376521] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.414998] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.376707] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.373455] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.415028] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.373648] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.376737] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.415494] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.373664] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.377219] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker1 # [ 0.373694] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.357007] Key type asymmetric registered worker2 # [ 0.377406] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.357035] Asymmetric key parser 'x509' registered worker1 # [ 0.373713] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.377436] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.357102] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker1 # [ 0.374175] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint lb # [ 0.359176] io scheduler mq-deadline registered worker2 # [ 0.377842] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker1 # [ 0.374376] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.359187] io scheduler kyber registered worker2 # [ 0.378024] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker1 # [ 0.374407] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.378285] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.374868] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.378482] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.375060] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.378512] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.375091] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.375525] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker1 # [ 0.375712] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] client # [ 0.431739] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.375975] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.431767] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.376167] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.431798] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.376213] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.432416] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker1 # [ 0.376681] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.432428] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.376874] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.432434] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.376904] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.369075] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.432481] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned client # [ 0.432528] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker2 # [ 0.419091] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.432577] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned lb # [ 0.370494] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.419304] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.432625] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned lb # [ 0.370915] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.419336] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.432673] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.370932] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker2 # [ 0.419792] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.432721] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.419981] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.370945] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.420011] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.432768] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned lb # [ 0.370954] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.420473] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint client # [ 0.432815] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.370973] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.420753] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] client # [ 0.432862] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.420770] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.370999] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.432980] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.420800] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.371093] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.433028] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker1 # [ 0.421448] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.421261] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.371103] pci_bus 0000:00: root bus resource [bus 00-ff] client # [ 0.433050] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.421444] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.421666] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.371109] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker2 # [ 0.421460] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.433071] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.421698] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.371114] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.421507] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.433093] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned lb # [ 0.371120] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker1 # [ 0.422196] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.422084] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned client # [ 0.433115] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.422488] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.371175] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.422505] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.433136] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.422096] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned lb # [ 0.371611] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.433158] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.422101] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.422536] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.371797] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.433185] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.423006] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.371815] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.422146] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.423197] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.433207] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.371845] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.422193] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker1 # [ 0.423213] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.433230] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.371863] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.422241] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.423243] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.433252] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned lb # [ 0.372313] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.423826] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.422289] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned client # [ 0.433287] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.372497] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.423837] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.433309] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.422337] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.372513] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.433330] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.372544] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.423843] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker2 # [ 0.422385] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned client # [ 0.433352] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker1 # [ 0.423889] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.422432] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.433373] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned client # [ 0.433395] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.422478] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker1 # [ 0.423937] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.391418] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.433416] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.423986] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker2 # [ 0.422524] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.391606] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.433442] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.424036] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned lb # [ 0.391623] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.433452] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.391654] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.424084] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned client # [ 0.433457] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] client # [ 0.434275] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.392112] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.424133] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.392298] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.424194] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned lb # [ 0.392315] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.424243] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.392347] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.392367] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.424290] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.392826] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker1 # [ 0.424360] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.424407] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker1 # [ 0.424429] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker1 # [ 0.424450] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.424472] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.450702] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.424494] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.450766] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker1 # [ 0.424516] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.450789] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker1 # [ 0.424538] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.450811] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.424564] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.450835] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.424587] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker2 # [ 0.450858] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.424609] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.450880] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.424632] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.450902] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.424655] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker2 # [ 0.450928] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.424677] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.450952] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.424698] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.450975] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker1 # [ 0.424720] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.405083] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.450999] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.424741] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.405129] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.451020] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.424763] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.405632] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.451041] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker1 # [ 0.424785] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.405825] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.451062] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker1 # [ 0.424811] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker2 # [ 0.451083] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.405856] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.424821] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker2 # [ 0.451104] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker1 # [ 0.424826] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.406280] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.451125] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.406462] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker2 # [ 0.451146] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.406709] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.451175] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.406896] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.451184] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.406927] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.451189] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker2 # [ 0.452004] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.407399] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.490834] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) lb # [ 0.407586] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.407616] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.408064] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.408252] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.408284] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.408747] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.469760] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.409041] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.409059] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.409090] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.409555] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.409738] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.409755] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.409786] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.410370] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned client # [ 0.500890] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.410382] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned lb # [ 0.410387] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.410433] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.410482] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.410530] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned client # [ 0.505111] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.410579] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned client # [ 0.507292] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.410627] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.410675] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.410722] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned lb # [ 0.410769] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.410816] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.410885] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned lb # [ 0.410933] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned lb # [ 0.410956] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned lb # [ 0.410978] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned lb # [ 0.411000] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.496769] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) lb # [ 0.411023] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.490779] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) client # [ 0.518585] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) lb # [ 0.411054] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.493014] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.411076] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.411102] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned lb # [ 0.411125] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.411148] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.411172] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned lb # [ 0.411195] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.411217] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.411240] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.411262] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker2 # [ 0.499925] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.524730] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) lb # [ 0.411284] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker2 # [ 0.501982] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.411306] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.504049] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.411328] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned client # [ 0.526491] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) lb # [ 0.411357] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker2 # [ 0.506339] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker1 # [ 0.504247] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.411367] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.411371] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.412190] pci 0000:00:07.0: enabling device (0000 -> 0002) worker1 # [ 0.510298] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) client # [ 0.536649] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.512414] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) client # [ 0.538654] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker2 # [ 0.524334] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker2 # [ 0.526150] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker2 # [ 0.529640] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.522915] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker1 # [ 0.524782] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.549725] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) client # [ 0.552963] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.537297] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.488535] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker1 # [ 0.536094] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.538942] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) client # [ 0.566363] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.543127] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.540801] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker2 # [ 0.545562] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.568968] msm_serial: driver initialized lb # [ 0.498369] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.569124] SuperH (H)SCI(F) driver initialized client # [ 0.569178] STM32 USART driver initialized worker1 # [ 0.550645] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.558704] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.509559] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.511581] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.562468] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.513626] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) lb # [ 0.515927] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker2 # [ 0.567910] msm_serial: driver initialized worker1 # [ 0.565193] msm_serial: driver initialized worker2 # [ 0.568080] SuperH (H)SCI(F) driver initialized worker2 # [ 0.568132] STM32 USART driver initialized worker1 # [ 0.565942] SuperH (H)SCI(F) driver initialized worker1 # [ 0.565999] STM32 USART driver initialized lb # [ 0.526141] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) lb # [ 0.528416] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) client # [ 0.603788] loop: module loaded client # [ 0.603983] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 0.604842] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 0.539702] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.542599] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker2 # [ 0.597424] loop: module loaded worker2 # [ 0.597635] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 0.612176] megasas: 07.734.00.00-rc1 worker2 # [ 0.598487] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 0.612939] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] client # [ 0.615273] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.615302] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.617038] Using buffer write method client # [ 0.617134] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker1 # [ 0.598327] loop: module loaded worker1 # [ 0.598513] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 0.618988] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.619015] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.599286] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 0.553026] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) lb # [ 0.559091] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.606662] megasas: 07.734.00.00-rc1 worker2 # [ 0.607449] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.609354] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.609379] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.637555] Using buffer write method client # [ 0.637596] Concatenating MTD devices: client # [ 0.637600] (0): "0.flash" worker1 # [ 0.609941] megasas: 07.734.00.00-rc1 worker2 # [ 0.611033] Using buffer write method client # [ 0.637604] (1): "0.flash" lb # [ 0.566174] msm_serial: driver initialized client # [ 0.637613] into device "0.flash" worker1 # [ 0.610679] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.566323] SuperH (H)SCI(F) driver initialized worker2 # [ 0.611100] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.566379] STM32 USART driver initialized worker1 # [ 0.612800] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.612878] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.612824] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.612900] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.614445] Using buffer write method worker2 # [ 0.614469] Concatenating MTD devices: worker2 # [ 0.614473] (0): "0.flash" worker2 # [ 0.614477] (1): "0.flash" worker2 # [ 0.614489] into device "0.flash" worker1 # [ 0.617592] Using buffer write method worker1 # [ 0.617657] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker1 # [ 0.619307] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.619328] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.621098] Using buffer write method worker1 # [ 0.621122] Concatenating MTD devices: worker1 # [ 0.621126] (0): "0.flash" worker1 # [ 0.621130] (1): "0.flash" worker1 # [ 0.621141] into device "0.flash" lb # [ 0.595716] loop: module loaded lb # [ 0.595893] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 0.596665] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 0.609628] megasas: 07.734.00.00-rc1 lb # [ 0.610315] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.612583] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.612611] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.614163] Using buffer write method lb # [ 0.614240] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.615929] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.615953] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.633524] Using buffer write method lb # [ 0.633565] Concatenating MTD devices: lb # [ 0.633570] (0): "0.flash" lb # [ 0.633574] (1): "0.flash" lb # [ 0.633583] into device "0.flash" client # [ 0.901218] Freeing initrd memory: 26972K client # [ 0.907319] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.911312] thunder_xcv, ver 1.0 client # [ 0.911362] thunder_bgx, ver 1.0 client # [ 0.911384] nicpf, ver 1.0 client # [ 0.913143] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.913155] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 0.890059] Freeing initrd memory: 26976K client # [ 0.913181] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 0.897364] Freeing initrd memory: 26944K client # [ 0.913189] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 0.913220] igb: Intel(R) Gigabit Ethernet Network Driver client # [ 0.913225] igb: Copyright (c) 2007-2014 Intel Corporation. client # [ 0.913249] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.896021] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.913254] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. client # [ 0.913406] sky2: driver version 1.30 client # [ 0.915016] usbcore: registered new interface driver usb-storage worker2 # [ 0.903402] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.915152] usbcore: registered new interface driver usbserial_generic client # [ 0.915166] usbserial: USB Serial support registered for generic client # [ 0.921013] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.921033] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.899738] thunder_xcv, ver 1.0 client # [ 0.921306] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.899781] thunder_bgx, ver 1.0 worker1 # [ 0.899802] nicpf, ver 1.0 worker1 # [ 0.900374] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.927595] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 0.900382] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 0.900403] e1000e: Intel(R) PRO/1000 Network Driver worker1 # [ 0.900411] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 0.929223] rtc-pl031 9010000.pl031: registered as rtc0 worker1 # [ 0.900439] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 0.900445] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 0.907457] thunder_xcv, ver 1.0 client # [ 0.929252] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T11:31:35 UTC (1790076695) worker2 # [ 0.907502] thunder_bgx, ver 1.0 worker1 # [ 0.900466] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 0.929562] i2c_dev: i2c /dev entries driver worker2 # [ 0.907525] nicpf, ver 1.0 worker2 # [ 0.908081] e1000: Intel(R) PRO/1000 Network Driver worker1 # [ 0.900472] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 0.900605] sky2: driver version 1.30 worker2 # [ 0.908090] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.932614] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.932926] hub 1-0:1.0: USB hub found worker2 # [ 0.908113] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.932947] hub 1-0:1.0: 6 ports detected worker2 # [ 0.908122] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 0.908152] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 0.909978] ehci-pci 0000:00:07.0: EHCI Host Controller worker2 # [ 0.908158] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 0.910010] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker2 # [ 0.908196] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.910298] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker2 # [ 0.908203] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 0.908335] sky2: driver version 1.30 worker1 # [ 0.913031] usbcore: registered new interface driver usb-storage worker2 # [ 0.909962] usbcore: registered new interface driver usb-storage worker1 # [ 0.913120] usbcore: registered new interface driver usbserial_generic worker2 # [ 0.910054] usbcore: registered new interface driver usbserial_generic client # [ 0.937448] sdhci: Secure Digital Host Controller Interface driver worker1 # [ 0.913133] usbserial: USB Serial support registered for generic worker2 # [ 0.910068] usbserial: USB Serial support registered for generic client # [ 0.937459] sdhci: Copyright(c) Pierre Ossman worker2 # [ 0.911286] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.937721] Synopsys Designware Multimedia Card Interface Driver client # [ 0.938087] sdhci-pltfm: SDHCI platform and OF driver helper worker2 # [ 0.911316] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.915950] hv_vmbus: registering driver hyperv_keyboard worker2 # [ 0.911602] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 client # [ 0.942250] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.917487] rtc-pl031 9010000.pl031: registered as rtc0 worker2 # [ 0.923267] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.942497] usbcore: registered new interface driver usbhid worker2 # [ 0.923584] hub 1-0:1.0: USB hub found client # [ 0.942504] usbhid: USB HID core driver worker2 # [ 0.923598] hub 1-0:1.0: 6 ports detected worker1 # [ 0.917512] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T11:31:35 UTC (1790076695) worker1 # [ 0.917806] i2c_dev: i2c /dev entries driver worker2 # [ 0.925726] hv_vmbus: registering driver hyperv_keyboard client # [ 0.945360] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.927356] rtc-pl031 9010000.pl031: registered as rtc0 client # [ 0.946892] drop_monitor: Initializing network drop monitor service client # [ 0.947031] NET: Registered PF_INET6 protocol family worker2 # [ 0.927384] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T11:31:35 UTC (1790076695) worker2 # [ 0.927699] i2c_dev: i2c /dev entries driver client # [ 0.950138] Segment Routing with IPv6 worker1 # [ 0.923095] sdhci: Secure Digital Host Controller Interface driver client # [ 0.950158] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.923106] sdhci: Copyright(c) Pierre Ossman client # [ 0.950186] NET: Registered PF_PACKET protocol family worker1 # [ 0.923364] Synopsys Designware Multimedia Card Interface Driver worker1 # [ 0.923722] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.925227] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.928573] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker2 # [ 0.932911] sdhci: Secure Digital Host Controller Interface driver worker1 # [ 0.929609] hub 1-0:1.0: USB hub found worker2 # [ 0.932924] sdhci: Copyright(c) Pierre Ossman worker1 # [ 0.930102] hub 1-0:1.0: 6 ports detected client # [ 0.951849] 9pnet: Installing 9P2000 support worker2 # [ 0.933186] Synopsys Designware Multimedia Card Interface Driver client # [ 0.951895] Key type dns_resolver registered worker2 # [ 0.933576] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.931196] usbcore: registered new interface driver usbhid worker1 # [ 0.931206] usbhid: USB HID core driver worker2 # [ 0.937841] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.938083] usbcore: registered new interface driver usbhid client # [ 0.958859] registered taskstats version 1 worker2 # [ 0.938090] usbhid: USB HID core driver client # [ 0.959001] Loading compiled-in X.509 certificates worker1 # [ 0.933499] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 0.935009] drop_monitor: Initializing network drop monitor service worker1 # [ 0.935171] NET: Registered PF_INET6 protocol family worker2 # [ 0.940959] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 0.938226] Segment Routing with IPv6 worker2 # [ 0.942501] drop_monitor: Initializing network drop monitor service worker1 # [ 0.938243] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.938270] NET: Registered PF_PACKET protocol family worker2 # [ 0.944555] NET: Registered PF_INET6 protocol family worker2 # [ 0.945744] Segment Routing with IPv6 lb # [ 0.894021] Freeing initrd memory: 26948K worker2 # [ 0.945775] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.939902] 9pnet: Installing 9P2000 support worker2 # [ 0.945804] NET: Registered PF_PACKET protocol family worker1 # [ 0.939950] Key type dns_resolver registered client # [ 0.967652] Demotion targets for Node 0: null client # [ 0.967770] Key type .fscrypt registered client # [ 0.967776] Key type fscrypt-provisioning registered client # [ 0.967876] ima: No TPM chip found, activating TPM-bypass! worker2 # [ 0.947653] 9pnet: Installing 9P2000 support client # [ 0.967895] ima: Allocated hash algorithm: sha1 lb # [ 0.900201] tun: Universal TUN/TAP device driver, 1.6 worker2 # [ 0.947698] Key type dns_resolver registered client # [ 0.967916] ima: No architecture policies found client # [ 0.971964] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.954262] registered taskstats version 1 worker1 # [ 0.946704] registered taskstats version 1 worker2 # [ 0.954408] Loading compiled-in X.509 certificates worker1 # [ 0.946841] Loading compiled-in X.509 certificates lb # [ 0.903922] thunder_xcv, ver 1.0 lb # [ 0.903967] thunder_bgx, ver 1.0 lb # [ 0.903989] nicpf, ver 1.0 lb # [ 0.904530] e1000: Intel(R) PRO/1000 Network Driver lb # [ 0.904538] e1000: Copyright (c) 1999-2006 Intel Corporation. lb # [ 0.904561] e1000e: Intel(R) PRO/1000 Network Driver lb # [ 0.904568] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. lb # [ 0.904597] igb: Intel(R) Gigabit Ethernet Network Driver lb # [ 0.904602] igb: Copyright (c) 2007-2014 Intel Corporation. lb # [ 0.904647] igbvf: Intel(R) Gigabit Virtual Function Network Driver lb # [ 0.904653] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 0.955764] Demotion targets for Node 0: null lb # [ 0.904785] sky2: driver version 1.30 worker1 # [ 0.955874] Key type .fscrypt registered worker1 # [ 0.955880] Key type fscrypt-provisioning registered worker1 # [ 0.955985] ima: No TPM chip found, activating TPM-bypass! worker2 # [ 0.963069] Demotion targets for Node 0: null worker1 # [ 0.956004] ima: Allocated hash algorithm: sha1 worker2 # [ 0.963174] Key type .fscrypt registered worker1 # [ 0.956026] ima: No architecture policies found worker2 # [ 0.963181] Key type fscrypt-provisioning registered worker2 # [ 0.963276] ima: No TPM chip found, activating TPM-bypass! worker1 # [ 0.960050] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.963295] ima: Allocated hash algorithm: sha1 worker2 # [ 0.963315] ima: No architecture policies found lb # [ 0.913516] usbcore: registered new interface driver usb-storage lb # [ 0.913605] usbcore: registered new interface driver usbserial_generic worker2 # [ 0.967675] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 0.913619] usbserial: USB Serial support registered for generic lb # [ 0.914211] hv_vmbus: registering driver hyperv_keyboard lb # [ 0.915558] ehci-pci 0000:00:07.0: EHCI Host Controller lb # [ 0.915587] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 lb # [ 0.915874] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 client # [ 0.990075] clk: Disabling unused clocks client # [ 0.990102] PM: genpd: Disabling unused power domains lb # [ 0.921411] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 0.921440] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T11:31:35 UTC (1790076695) lb # [ 0.921798] i2c_dev: i2c /dev entries driver client # [ 0.994297] Freeing unused kernel memory: 4736K client # [ 0.994522] Run /init as init process lb # [ 0.927133] sdhci: Secure Digital Host Controller Interface driver lb # [ 0.927144] sdhci: Copyright(c) Pierre Ossman lb # [ 0.927407] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.927773] sdhci-pltfm: SDHCI platform and OF driver helper lb # [ 0.931669] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 lb # [ 0.932683] hub 1-0:1.0: USB hub found lb # [ 0.933204] hub 1-0:1.0: 6 ports detected worker1 # [ 0.977817] clk: Disabling unused clocks worker1 # [ 0.977845] PM: genpd: Disabling unused power domains lb # [ 0.934433] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.986118] clk: Disabling unused clocks lb # [ 0.934674] usbcore: registered new interface driver usbhid worker2 # [ 0.986145] PM: genpd: Disabling unused power domains lb # [ 0.934680] usbhid: USB HID core driver worker1 # [ 0.981938] Freeing unused kernel memory: 4736K worker1 # [ 0.982141] Run /init as init process client # [ 1.009763] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.937629] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.990325] Freeing unused kernel memory: 4736K lb # [ 0.939186] drop_monitor: Initializing network drop monitor service worker2 # [ 0.991137] Run /init as init process lb # [ 0.939358] NET: Registered PF_INET6 protocol family lb # [ 0.942405] Segment Routing with IPv6 lb # [ 0.942426] In-situ OAM (IOAM) with IPv6 lb # [ 0.942454] NET: Registered PF_PACKET protocol family lb # [ 0.944062] 9pnet: Installing 9P2000 support lb # [ 0.944108] Key type dns_resolver registered lb # [ 0.950706] registered taskstats version 1 lb # [ 0.950846] Loading compiled-in X.509 certificates worker2 # [ 1.004828] systemd[1]: Successfully made /usr/ read-only. worker1 # [ 0.999601] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.959973] Demotion targets for Node 0: null lb # [ 0.960096] Key type .fscrypt registered lb # [ 0.960103] Key type fscrypt-provisioning registered lb # [ 0.960201] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.960221] ima: Allocated hash algorithm: sha1 lb # [ 0.960242] ima: No architecture policies found lb # [ 0.964372] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 0.982244] clk: Disabling unused clocks lb # [ 0.982270] PM: genpd: Disabling unused power domains lb # [ 0.986463] Freeing unused kernel memory: 4736K lb # [ 0.986670] Run /init as init process lb # [ 1.003152] systemd[1]: Successfully made /usr/ read-only. client # [ 1.179571] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 1.170600] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.177318] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 1.181012] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.329504] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.323327] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 client # [ 1.344453] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.350287] systemd[1]: Detected virtualization qemu. client # [ 1.351273] systemd[1]: Detected architecture arm64. client # [ 1.352270] systemd[1]: Running in initrd. client # [ 1.353620] systemd[1]: Initializing machine ID from random generator. client # [ 1.355031] systemd[1]: Hostname set to . worker1 # [ 1.329883] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.339733] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.352041] systemd[1]: Detected virtualization qemu. worker1 # [ 1.336233] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.354090] systemd[1]: Detected architecture arm64. worker1 # [ 1.336294] systemd[1]: Detected virtualization qemu. worker2 # [ 1.356102] systemd[1]: Running in initrd. worker1 # [ 1.336369] systemd[1]: Detected architecture arm64. worker1 # [ 1.336394] systemd[1]: Running in initrd. worker2 # [ 1.358884] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.355374] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.361945] systemd[1]: Hostname set to . worker1 # [ 1.358346] systemd[1]: Hostname set to . lb # [ 1.333464] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 client # [ 1.407799] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 lb # [ 1.339430] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 1.351813] systemd[1]: Detected virtualization qemu. lb # [ 1.353947] systemd[1]: Detected architecture arm64. lb # [ 1.355836] systemd[1]: Running in initrd. worker2 # [ 1.406866] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 lb # [ 1.358470] systemd[1]: Initializing machine ID from random generator. lb # [ 1.361361] systemd[1]: Hostname set to . worker1 # [ 1.409482] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 lb # [ 1.417185] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 client # [ 1.531591] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.529311] usb 1-2: new high-speed USB device number 3 using ehci-pci worker2 # [ 1.534612] usb 1-2: new high-speed USB device number 3 using ehci-pci lb # [ 1.540964] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.619025] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.667045] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 1.661890] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.692268] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.697003] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.679142] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.683201] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker1 # [ 1.684981] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.688638] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.692880] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker1 # [ 1.689642] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.695863] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.689675] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.689701] systemd[1]: Reached target Path Units. worker1 # [ 1.689717] systemd[1]: Reached target Slice Units. worker1 # [ 1.689732] systemd[1]: Reached target Swaps. worker1 # [ 1.689746] systemd[1]: Reached target Timer Units. worker1 # [ 1.689889] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.690008] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.690122] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.690223] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.690308] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.690324] systemd[1]: Reached target Socket Units. worker1 # [ 1.700064] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.700134] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 1.708839] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.671750] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.717617] systemd[1]: Starting Journal Service... worker1 # [ 1.739558] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.739677] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.748621] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.743380] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.756546] systemd[1]: Created slice Slice /system/modprobe. lb # [ 1.701775] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 1.757614] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.778049] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.758684] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 1.759509] systemd[1]: Reached target Path Units. worker2 # [ 1.760161] systemd[1]: Reached target Slice Units. worker2 # [ 1.760797] systemd[1]: Reached target Swaps. worker2 # [ 1.761374] systemd[1]: Reached target Timer Units. worker2 # [ 1.762177] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.708168] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker2 # [ 1.763177] systemd[1]: Listening on Journal Socket (/dev/log). worker2 # [ 1.764044] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.764801] systemd[1]: Listening on udev Control Socket. worker2 # [ 1.765653] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.766433] systemd[1]: Reached target Socket Units. worker2 # [ 1.768446] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.789968] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.761420] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 1.768512] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 1.791162] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.792553] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.793627] systemd[1]: Reached target Path Units. client # [ 1.794438] systemd[1]: Reached target Slice Units. client # [ 1.795261] systemd[1]: Reached target Swaps. client # [ 1.796075] systemd[1]: Reached target Timer Units. client # [ 1.797106] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 1.778204] systemd[1]: Mounting Kernel Configuration File System... client # [ 1.798427] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.799632] systemd[1]: Listening on Journal Sockets. client # [ 1.800621] systemd[1]: Listening on udev Control Socket. client # [ 1.801704] systemd[1]: Listening on udev Kernel Socket. client # [ 1.802609] systemd[1]: Reached target Socket Units. client # [ 1.805100] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.806303] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 1.782813] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.783378] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 1.789677] systemd-journald[72]: Collecting audit messages is disabled. client # [ 1.815702] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 1.799661] systemd[1]: Starting Journal Service... worker1 # [ 1.793741] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.795442] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 1.801824] systemd[1]: Starting Load Kernel Modules... worker2 # [ 1.802711] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.823910] systemd[1]: Starting Journal Service... worker2 # [ 1.810030] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.806533] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker1 # [ 1.806776] [drm] features: -virgl +edid -resource_blob -host_visible worker1 # [ 1.806786] [drm] features: -context_init worker1 # [ 1.807453] [drm] number of scanouts: 1 worker1 # [ 1.807471] [drm] number of cap sets: 0 worker2 # [ 1.825000] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 1.825684] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.847971] systemd[1]: Starting Load Kernel Modules... client # [ 1.848935] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.828881] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker2 # [ 1.835117] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.828906] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 lb # [ 1.790892] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.839816] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 1.800275] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.871330] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.801575] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 1.845692] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.801620] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 1.801655] systemd[1]: Reached target Path Units. lb # [ 1.801676] systemd[1]: Reached target Slice Units. lb # [ 1.801698] systemd[1]: Reached target Swaps. lb # [ 1.801718] systemd[1]: Reached target Timer Units. lb # [ 1.801928] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.802087] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.802232] systemd[1]: Listening on Journal Sockets. lb # [ 1.802368] systemd[1]: Listening on udev Control Socket. lb # [ 1.802484] systemd[1]: Listening on udev Kernel Socket. client # [ 1.880115] systemd[1]: Starting Coldplug All udev Devices... lb # [ 1.802508] systemd[1]: Reached target Socket Units. lb # [ 1.815151] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.816336] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 1.891837] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.892837] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 1.861784] Console: switching to colour frame buffer device 160x50 lb # [ 1.825097] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 1.868527] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker2 # [ 1.878507] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.833199] systemd[1]: Starting Journal Service... client # [ 1.907974] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.894706] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.903206] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.897862] systemd[1]: Finished Load Kernel Modules. worker2 # [ 1.905480] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.901188] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 1.907930] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.861155] systemd[1]: Starting Load Kernel Modules... lb # [ 1.861284] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.931835] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.914440] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.919891] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker2 # [ 1.920140] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 1.920150] [drm] features: -context_init worker2 # [ 1.920871] [drm] number of scanouts: 1 worker1 # [ 1.916640] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 1.920888] [drm] number of cap sets: 0 worker1 # [ 1.916684] systemd[1]: Reached target Local File Systems. lb # [ 1.873552] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.947616] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.879645] systemd-journald[72]: Collecting audit messages is disabled. worker1 # [ 1.926883] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 1.887324] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.956934] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker2 # [ 1.934986] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic client # [ 1.957184] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 1.935006] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 client # [ 1.957193] [drm] features: -context_init lb # [ 1.889218] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.957923] [drm] number of scanouts: 1 client # [ 1.957941] [drm] number of cap sets: 0 client # [ 1.972257] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 1.974787] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.913117] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.961583] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 1.966836] Console: switching to colour frame buffer device 160x50 client # [ 1.988117] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic client # [ 1.988137] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker1 # [ 1.972820] systemd[1]: Started Journal Service. lb # [ 1.935691] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.987262] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.987471] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 1.987495] systemd[1]: Reached target Local File Systems. worker2 # [ 1.991461] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 1.976298] systemd-modules-load[73]: Inserted module 'dm_mod' worker1 # [ 1.977549] systemd-modules-load[73]: Module 'virtio_balloon' is built in worker1 # [ 1.978753] systemd-modules-load[73]: Module 'virtio_console' is built in worker1 # [ 1.979851] systemd-modules-load[73]: Inserted module 'virtio_gpu' client # [ 2.011937] Console: switching to colour frame buffer device 160x50 worker2 # [ 1.999365] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device lb # [ 1.953626] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.988805] systemd-modules-load[73]: Module 'virtio_rng' is built in lb # [ 1.956181] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.989843] systemd[1]: Starting Create System Files and Directories... worker2 # [ 2.010171] systemd[1]: Finished Load Kernel Modules. client # [ 2.032208] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 2.032407] systemd[1]: Reached target Preparation for Local File Systems. client # [ 2.032430] systemd[1]: Reached target Local File Systems. worker2 # [ 2.015064] systemd[1]: Starting Apply Kernel Variables... lb # [ 1.963796] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 2.039986] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 2.045297] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 2.018372] systemd[1]: Finished Create System Files and Directories. lb # [ 1.986171] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 lb # [ 1.986416] [drm] features: -virgl +edid -resource_blob -host_visible lb # [ 1.986428] [drm] features: -context_init lb # [ 1.987152] [drm] number of scanouts: 1 lb # [ 1.987171] [drm] number of cap sets: 0 client # [ 2.061470] systemd[1]: Finished Load Kernel Modules. worker1 # [ 2.024710] systemd-udevd[79]: Using default interface naming scheme 'v261'. client # [ 2.056498] systemd-modules-load[74]: Inserted module 'dm_mod' client # [ 2.057792] systemd-modules-load[74]: Module 'virtio_balloon' is built in lb # [ 1.997280] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic client # [ 2.058854] systemd-modules-load[74]: Module 'virtio_console' is built in lb # [ 1.997299] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 client # [ 2.059890] systemd-modules-load[74]: Inserted module 'virtio_gpu' worker2 # [ 2.054728] systemd[1]: Started Journal Service. client # [ 2.077646] systemd[1]: Starting Apply Kernel Variables... client # [ 2.077826] systemd[1]: Started Journal Service. worker2 # [ 2.048396] systemd-modules-load[73]: Inserted module 'dm_mod' client # [ 2.072430] systemd-modules-load[74]: Module 'virtio_rng' is built in worker1 # [ 2.049597] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.053939] systemd-modules-load[73]: Module 'virtio_balloon' is built in lb # [ 2.017662] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 2.055032] systemd-modules-load[73]: Module 'virtio_console' is built in lb # [ 2.017877] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 2.017904] systemd[1]: Reached target Local File Systems. lb # [ 2.021979] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 2.068238] systemd-modules-load[73]: Inserted module 'virtio_gpu' worker2 # [ 2.069333] systemd-modules-load[73]: Module 'virtio_rng' is built in worker2 # [ 2.071286] systemd[1]: Starting Create System Files and Directories... worker2 # [ 2.073402] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.029412] Console: switching to colour frame buffer device 160x50 lb # [ 2.036311] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker2 # [ 2.086422] systemd-udevd[78]: Using default interface naming scheme 'v261'. client # [ 2.117037] systemd[1]: Starting Create System Files and Directories... lb # [ 2.052525] systemd-modules-load[74]: Inserted module 'dm_mod' lb # [ 2.066604] systemd[1]: Finished Load Kernel Modules. lb # [ 2.066885] systemd[1]: Started Journal Service. lb # [ 2.057719] systemd-modules-load[74]: Module 'virtio_balloon' is built in lb # [ 2.058865] systemd-modules-load[74]: Module 'virtio_console' is built in lb # [ 2.059924] systemd-modules-load[74]: Inserted module 'virtio_gpu' worker1 # [ 2.108114] systemd[1]: Starting Virtual Console Setup... client # [ 2.132487] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker2 # [ 2.117287] systemd[1]: Finished Create System Files and Directories. client # [ 2.137053] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.072310] systemd-modules-load[74]: Module 'virtio_rng' is built in client # [ 2.139907] systemd[1]: Finished Create System Files and Directories. lb # [ 2.073372] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.127494] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.087774] systemd[1]: Starting Create System Files and Directories... client # [ 2.162251] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 2.160560] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.164177] systemd[1]: Finished Virtual Console Setup. lb # [ 2.128348] systemd-udevd[78]: Using default interface naming scheme 'v261'. lb # [ 2.131492] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 2.183790] systemd[1]: Starting Virtual Console Setup... lb # [ 2.140363] systemd[1]: Finished Create System Files and Directories. client # [ 2.224123] systemd[1]: Starting Virtual Console Setup... lb # [ 2.161156] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.232449] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.236054] systemd[1]: Finished Virtual Console Setup. client # [ 2.272543] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.276030] systemd[1]: Finished Virtual Console Setup. lb # [ 2.220252] systemd[1]: Starting Virtual Console Setup... lb # [ 2.268491] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.271900] systemd[1]: Finished Virtual Console Setup. worker1 # [ 2.766488] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 2.768142] systemd[1]: Reached target System Initialization. worker1 # [ 2.772113] systemd[1]: Reached target Basic System. worker2 # [ 2.848732] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 2.849705] systemd[1]: Reached target System Initialization. worker2 # [ 2.850529] systemd[1]: Reached target Basic System. client # [ 2.874127] systemd[1]: Finished Coldplug All udev Devices. client # [ 2.875065] systemd[1]: Reached target System Initialization. client # [ 2.880124] systemd[1]: Reached target Basic System. worker1 # [ 2.925870] (udev-worker)[89]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 2.882802] systemd[1]: Finished Coldplug All udev Devices. lb # [ 2.888141] systemd[1]: Reached target System Initialization. lb # [ 2.889394] systemd[1]: Reached target Basic System. worker1 # [ 2.948903] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.951834] (udev-worker)[89]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.996147] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.031919] (udev-worker)[105]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 3.022296] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 3.020805] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.026854] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.028263] systemd[1]: Reached target Initrd Root Device. worker1 # [ 3.033776] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 3.057559] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 3.061816] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.083661] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 3.090690] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.096356] systemd[1]: Found device /dev/disk/by-label/nixos. lb # [ 3.046936] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 3.095911] systemd[1]: Mounting /sysroot... client # [ 3.120100] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.103312] systemd[1]: Reached target Initrd Root Device. client # [ 3.122790] systemd[1]: Reached target Initrd Root Device. lb # [ 3.058255] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.128211] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 3.109445] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... lb # [ 3.065426] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.142156] EXT4-fs (vda): mounted filesystem 9c69db1f-c800-4716-bd16-1a58065274da r/w with ordered data mode. Quota mode: none. worker1 # [ 3.134845] systemd[1]: Mounted /sysroot. worker1 # [ 3.136438] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.140435] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 3.158207] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.163735] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. client # [ 3.183970] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks client # [ 3.189161] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.172304] systemd[1]: Mounting /sysroot... worker1 # [ 3.168260] systemd-sysroot-fstab-check[117]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 3.195045] systemd[1]: Mounting /sysroot... worker1 # [ 3.173245] systemd[1]: Reload requested from client PID 117 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 3.177181] systemd[1]: Reloading... lb # [ 3.147682] systemd[1]: Found device /dev/disk/by-label/nixos. lb # [ 3.152232] systemd[1]: Reached target Initrd Root Device. lb # [ 3.154537] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 3.223212] EXT4-fs (vda): mounted filesystem 80cb3d53-321a-4129-8263-8fcad6d40ced r/w with ordered data mode. Quota mode: none. client # [ 3.246370] EXT4-fs (vda): mounted filesystem bd498c12-dff1-4cc1-bc1d-be1cb8394cc7 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.214399] systemd[1]: Mounted /sysroot. client # [ 3.237212] systemd[1]: Mounted /sysroot. worker2 # [ 3.219471] systemd[1]: Reached target Initrd Root File System. client # [ 3.239080] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.220962] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 3.244198] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 3.247894] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 3.267423] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.203053] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.256090] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.276668] systemd[1]: Reload requested from client PID 119 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.278332] systemd[1]: Reloading... lb # [ 3.210380] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.260142] systemd[1]: Reloading... lb # [ 3.216189] systemd[1]: Mounting /sysroot... lb # [ 3.265264] EXT4-fs (vda): mounted filesystem 209df36f-c74f-438a-b883-feb5929c5341 r/w with ordered data mode. Quota mode: none. lb # [ 3.258246] systemd[1]: Mounted /sysroot. lb # [ 3.260430] systemd[1]: Reached target Initrd Root File System. lb # [ 3.264216] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.293219] systemd-sysroot-fstab-check[117]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.297589] systemd[1]: Reload requested from client PID 117 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.301228] systemd[1]: Reloading... worker1 # [ 3.391115] systemd[1]: Reloading finished in 215 ms. worker1 # [ 3.427067] systemd-sysroot-fstab-check[117]: Requesting initrd-fs.target/start/replace... worker1 # [ 3.431486] systemd-sysroot-fstab-check[117]: Requesting swap.target/start/replace... worker1 # [ 3.437020] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.438415] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.439910] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.472867] systemd[1]: Reloading finished in 216 ms. client # [ 3.494342] systemd[1]: Reloading finished in 218 ms. worker2 # [ 3.506181] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... client # [ 3.526989] systemd-sysroot-fstab-check[119]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.509862] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... client # [ 3.530699] systemd-sysroot-fstab-check[119]: Requesting swap.target/start/replace... worker2 # [ 3.514867] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.518102] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 3.537880] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.520762] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.540144] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 3.544165] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. lb # [ 3.509537] systemd[1]: Reloading finished in 207 ms. lb # [ 3.543256] systemd-sysroot-fstab-check[117]: Requesting initrd-fs.target/start/replace... lb # [ 3.547407] systemd-sysroot-fstab-check[117]: Requesting swap.target/start/replace... lb # [ 3.554793] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.557384] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.560135] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.734926] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.746467] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.761378] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.785040] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.814619] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.825418] systemd[1]: Mounting /sysroot/tmp/xchg... worker2 # [ 3.832194] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.836624] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.844633] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.868940] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.867617] fuse: init (API version 7.45) worker2 # [ 3.861708] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.858661] systemd[1]: Mounted /sysroot/run. worker1 # [ 3.875683] virtiofs virtio6: discovered new tag: nix-store worker1 # [ 3.876532] virtiofs virtio6: virtio_fs_setup_dax: No cache capability client # [ 3.888561] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.873834] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.891032] virtiofs virtio7: discovered new tag: shared worker1 # [ 3.891809] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 3.908231] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.898780] virtiofs virtio8: discovered new tag: xchg worker1 # [ 3.899538] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 3.928186] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 3.865558] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.914595] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 3.918051] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 3.920362] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 3.938490] fuse: init (API version 7.45) client # [ 3.949168] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.926594] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 3.883044] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.933884] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 3.935060] systemd[1]: Mounted /sysroot/run. client # [ 3.975411] fuse: init (API version 7.45) lb # [ 3.896886] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.945824] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.947219] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 3.971912] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.997794] virtiofs virtio6: discovered new tag: nix-store client # [ 3.998603] virtiofs virtio6: virtio_fs_setup_dax: No cache capability lb # [ 3.923340] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 4.014688] virtiofs virtio7: discovered new tag: shared client # [ 4.015471] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 4.023364] virtiofs virtio8: discovered new tag: xchg lb # [ 3.942305] systemd[1]: Mounting /sysroot/tmp/xchg... worker2 # [ 3.992319] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 4.029370] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 4.022715] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 4.026737] systemd[1]: Mounted /sysroot/run. lb # [ 3.961544] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 4.045770] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 3.994335] fuse: init (API version 7.45) lb # [ 4.006518] virtiofs virtio6: discovered new tag: nix-store client # [ 4.063739] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 4.044872] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.007349] virtiofs virtio6: virtio_fs_setup_dax: No cache capability lb # [ 4.000129] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 4.071690] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 4.075900] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 4.079128] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.060479] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.022700] virtiofs virtio7: discovered new tag: shared lb # [ 4.023506] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 4.034032] virtiofs virtio8: discovered new tag: xchg lb # [ 4.034816] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker2 # [ 4.100161] virtiofs virtio6: discovered new tag: nix-store lb # [ 4.042236] systemd[1]: Mounted /sysroot/run. lb # [ 4.053743] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 4.058316] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.061574] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.065242] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.067833] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 4.134366] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 4.148169] virtiofs virtio7: discovered new tag: shared worker2 # [ 4.148956] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 4.153918] virtiofs virtio8: discovered new tag: xchg worker2 # [ 4.159210] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker2 # [ 4.154081] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 4.158196] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 4.162772] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 4.356586] (udev-worker)[89]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.364570] (udev-worker)[89]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.394671] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.400899] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.402007] systemd[1]: Stopping Virtual Console Setup... client # [ 4.423756] (udev-worker)[92]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.402808] systemd[1]: Starting Virtual Console Setup... client # [ 4.430174] (udev-worker)[92]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.419978] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.421601] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.425076] systemd[1]: Starting Virtual Console Setup... client # [ 4.460752] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.465466] systemd[1]: Stopped Virtual Console Setup. client # [ 4.466321] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.446702] (udev-worker)[99]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory client # [ 4.468369] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.452184] (udev-worker)[99]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.453248] systemd-vconsole-setup[153]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 4.456669] systemd[1]: Finished Virtual Console Setup. client # [ 4.479449] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.481082] systemd[1]: Stopped Virtual Console Setup. client # [ 4.487077] systemd[1]: Starting Virtual Console Setup... lb # [ 4.418728] (udev-worker)[90]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory lb # [ 4.424368] (udev-worker)[90]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory client # [ 4.514009] systemd-vconsole-setup[153]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 4.496541] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.516759] systemd[1]: Finished Virtual Console Setup. worker2 # [ 4.498846] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.500382] systemd[1]: Stopping Virtual Console Setup... lb # [ 4.452121] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.453298] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.454776] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.504357] systemd[1]: Starting Virtual Console Setup... lb # [ 4.458776] systemd[1]: Starting Virtual Console Setup... lb # [ 4.469619] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.472534] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.475340] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.533655] systemd-vconsole-setup[156]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 4.537109] systemd[1]: Finished Virtual Console Setup. lb # [ 4.505334] systemd-vconsole-setup[152]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 4.508913] systemd[1]: Finished Virtual Console Setup. worker1 # [ 4.738050] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.804429] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.808301] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.811422] systemd[1]: Starting Find NixOS closure... worker2 # [ 4.816068] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.824437] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.872154] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.851183] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 4.852987] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 4.882062] systemd[1]: Finished Find NixOS closure. worker1 # [ 4.884700] systemd[1]: Reached target Initrd Default Target. worker2 # [ 4.888898] systemd[1]: Mounted /sysroot/nix/store. worker2 # [ 4.891393] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.888365] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker2 # [ 4.896474] systemd[1]: Starting Find NixOS closure... worker2 # [ 4.908514] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.868487] systemd[1]: Mounting /sysroot/nix/store... client # [ 4.941561] systemd[1]: Mounted /sysroot/nix/store. client # [ 4.944742] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.924461] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 4.925937] systemd[1]: Stopped target Basic System. client # [ 4.948879] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.928438] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 4.931705] systemd[1]: Stopped target Path Units. worker1 # [ 4.933598] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 4.936405] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 4.960334] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 4.940558] systemd[1]: Stopped target Slice Units. worker1 # [ 4.945131] systemd[1]: Stopped target Socket Units. worker1 # [ 4.947656] systemd[1]: Stopped target System Initialization. worker1 # [ 4.950476] systemd[1]: Stopped target Swaps. worker2 # [ 4.956152] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 4.956539] systemd[1]: Stopped target Timer Units. worker2 # [ 4.959751] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 4.958258] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 4.960299] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 4.964756] systemd[1]: Finished Find NixOS closure. worker2 # [ 4.968538] systemd[1]: Reached target Initrd Default Target. worker1 # [ 4.965186] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 4.966442] systemd[1]: Stopped Find NixOS closure. worker2 # [ 4.971014] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 4.967904] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 4.970396] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 4.971484] systemd[1]: Stopped Apply Kernel Variables. worker1 # [ 4.978380] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 4.979554] systemd[1]: Stopped Load Kernel Modules. client # [ 5.001949] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.936545] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.983152] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 4.940320] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.986715] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 4.987897] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 4.943293] systemd[1]: Starting Find NixOS closure... client # [ 5.010025] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 4.992265] systemd[1]: Stopped Create System Files and Directories. worker1 # [ 4.993455] systemd[1]: Stopped target Local File Systems. worker1 # [ 4.995064] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 5.020749] systemd[1]: Finished Find NixOS closure. worker1 # [ 4.998785] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 5.003079] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 4.999974] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 5.001307] systemd[1]: Stopping Rule-based Manager for Device Events and Files... client # [ 5.024242] systemd[1]: Reached target Initrd Default Target. lb # [ 4.956357] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 5.002348] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 5.003368] systemd[1]: Stopped Virtual Console Setup. client # [ 5.026625] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker2 # [ 5.008170] systemd[1]: Stopped target Basic System. worker2 # [ 5.009647] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 5.010758] systemd[1]: Stopped target Path Units. worker2 # [ 5.011817] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 5.012411] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 5.013422] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 5.014594] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 5.016190] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 5.020158] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.021611] systemd[1]: Stopped target Slice Units. worker2 # [ 5.022502] systemd[1]: Stopped target Socket Units. worker2 # [ 5.023411] systemd[1]: Stopped target System Initialization. worker1 # [ 5.020186] systemd[1]: systemd-udevd.service: Consumed 1.446s CPU time over 3.063s wall clock time, 25.5M memory peak. worker2 # [ 5.025364] systemd[1]: Stopped target Swaps. worker1 # [ 5.021615] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 5.027186] systemd[1]: Stopped target Timer Units. worker1 # [ 5.024146] systemd[1]: Closed udev Control Socket. worker1 # [ 5.024865] systemd[1]: Starting Cleanup udev Database... worker2 # [ 5.030274] systemd[1]: dbus.socket: Deactivated successfully. client # [ 5.051285] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 5.028401] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker1 # [ 5.029521] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 5.053199] systemd[1]: Stopped target Basic System. worker1 # [ 5.030382] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 5.054267] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 5.037412] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 5.056457] systemd[1]: Stopped target Path Units. worker2 # [ 5.039374] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 5.036326] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 5.037415] systemd[1]: kmod-static-nodes.service: Deactivated successfully. client # [ 5.060358] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 5.038372] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 4.993594] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 5.039267] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.044848] systemd[1]: Stopped Find NixOS closure. client # [ 5.064158] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.045791] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 5.065319] systemd[1]: Stopped target Slice Units. worker2 # [ 5.046819] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 5.066170] systemd[1]: Stopped target Socket Units. lb # [ 4.997683] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker2 # [ 5.047912] systemd[1]: Stopped Apply Kernel Variables. worker1 # [ 5.044142] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 5.067843] systemd[1]: Stopped target System Initialization. client # [ 5.070043] systemd[1]: Stopped target Swaps. client # [ 5.072714] systemd[1]: Stopped target Timer Units. lb # [ 5.008643] systemd[1]: Finished Find NixOS closure. client # [ 5.076584] systemd[1]: dbus.socket: Deactivated successfully. client # [ 5.078242] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 5.011570] systemd[1]: Reached target Initrd Default Target. worker2 # [ 5.061498] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 5.080883] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 5.013497] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 5.060469] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 5.084579] systemd[1]: Stopped Find NixOS closure. worker2 # [ 5.065890] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 5.064478] systemd[1]: Finished Cleanup udev Database. client # [ 5.087644] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 5.065315] systemd[1]: Reached target Switch Root. worker2 # [ 5.069209] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 5.071565] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 5.068130] systemd[1]: Starting NixOS Activation... worker2 # [ 5.075067] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 5.076212] systemd[1]: Stopped Create System Files and Directories. client # [ 5.096445] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 5.080686] systemd[1]: Stopped target Local File Systems. client # [ 5.099717] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 5.081569] systemd[1]: Stopped target Preparation for Local File Systems. worker2 # [ 5.082505] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 5.102631] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 5.084083] systemd[1]: Stopped Coldplug All udev Devices. client # [ 5.104373] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 5.086762] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 5.087973] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 5.089062] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 5.089794] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.108431] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 5.090730] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 5.091650] systemd[1]: systemd-udevd.service: Deactivated successfully. client # [ 5.112227] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 5.045096] systemd[1]: Stopped target Initrd Default Target. client # [ 5.113365] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 5.096259] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 5.047409] systemd[1]: Stopped target Basic System. client # [ 5.115422] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 5.097310] systemd[1]: systemd-udevd.service: Consumed 1.453s CPU time over 3.091s wall clock time, 22.2M memory peak. lb # [ 5.048946] systemd[1]: Stopped target Initrd Root Device. lb # [ 5.052430] systemd[1]: Stopped target Path Units. client # [ 5.120498] systemd[1]: Stopped target Local File Systems. client # [ 5.121310] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 5.053559] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 5.122288] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 5.123281] systemd[1]: Stopped Coldplug All udev Devices. worker2 # [ 5.104376] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.105448] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 5.106414] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 5.125206] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 5.057702] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.107432] systemd[1]: Closed udev Control Socket. client # [ 5.126395] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 5.108219] systemd[1]: Starting Cleanup udev Database... client # [ 5.127425] systemd[1]: Stopped Virtual Console Setup. client # [ 5.128367] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.129445] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 5.130416] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 5.112216] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 5.131332] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 5.064188] systemd[1]: Stopped target Slice Units. worker2 # [ 5.113322] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 5.065260] systemd[1]: Stopped target Socket Units. worker2 # [ 5.114195] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 5.066276] systemd[1]: Stopped target System Initialization. lb # [ 5.067334] systemd[1]: Stopped target Swaps. worker2 # [ 5.116083] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 5.068450] systemd[1]: Stopped target Timer Units. client # [ 5.136276] systemd[1]: systemd-udevd.service: Consumed 1.410s CPU time over 3.081s wall clock time, 22.7M memory peak. client # [ 5.137779] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 5.070682] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 5.120426] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 5.121415] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 5.140406] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 5.073221] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 5.144307] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 5.145410] systemd[1]: Closed udev Control Socket. lb # [ 5.077612] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 5.146163] systemd[1]: Starting Cleanup udev Database... client # [ 5.148183] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 5.083346] systemd[1]: Stopped Find NixOS closure. client # [ 5.152107] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 5.153017] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 5.154136] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 5.088222] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 5.156241] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 5.090050] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 5.160389] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 5.096980] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 5.145594] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 5.148126] systemd[1]: Finished Cleanup udev Database. worker2 # [ 5.149593] systemd[1]: Reached target Switch Root. lb # [ 5.102478] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 5.152147] systemd[1]: Starting NixOS Activation... lb # [ 5.103698] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 5.149000] initrd-nixos-activation-start[177]: booting system configuration /nix/store/7w3hl2ifzqxyp2m0j26ag73ir8lww0pf-nixos-system-worker1-test lb # [ 5.105469] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 5.108348] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 5.110759] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 5.114621] systemd[1]: Stopped Create System Files and Directories. client # [ 5.182936] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 5.117161] systemd[1]: Stopped target Local File Systems. client # [ 5.185389] systemd[1]: Finished Cleanup udev Database. lb # [ 5.119569] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 5.187650] systemd[1]: Reached target Switch Root. lb # [ 5.120980] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 5.189087] systemd[1]: Starting NixOS Activation... lb # [ 5.124368] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 5.125213] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 5.128483] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 5.129572] systemd[1]: Stopped Virtual Console Setup. lb # [ 5.130961] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 5.132009] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. lb # [ 5.133070] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 5.134067] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 5.134894] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 5.135808] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 5.182311] initrd-nixos-activation-start[177]: running activation script... lb # [ 5.140260] systemd[1]: systemd-udevd.service: Consumed 1.393s CPU time over 3.107s wall clock time, 25.4M memory peak. lb # [ 5.142703] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 5.148185] systemd[1]: Closed udev Control Socket. lb # [ 5.148982] systemd[1]: Starting Cleanup udev Database... lb # [ 5.149750] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 5.150806] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 5.152192] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 5.156128] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 5.157134] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 5.160154] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 5.230551] initrd-nixos-activation-start[179]: booting system configuration /nix/store/fjl79kv3mslif1ryinjddfhvml6nj1zy-nixos-system-worker2-test lb # [ 5.185457] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 5.188530] systemd[1]: Finished Cleanup udev Database. lb # [ 5.189345] systemd[1]: Reached target Switch Root. lb # [ 5.190830] systemd[1]: Starting NixOS Activation... client # [ 5.272894] initrd-nixos-activation-start[176]: booting system configuration /nix/store/52ij8s6jkav0f53amdkjkhqqbw3wp27m-nixos-system-client-test worker2 # [ 5.263951] initrd-nixos-activation-start[179]: running activation script... client # [ 5.306420] initrd-nixos-activation-start[176]: running activation script... lb # [ 5.277655] initrd-nixos-activation-start[175]: booting system configuration /nix/store/rwj7rgrjnik4yfrdy5nxpi16ig7dj0yx-nixos-system-lb-test lb # [ 5.309901] initrd-nixos-activation-start[175]: running activation script... worker1 # [ 5.430372] initrd-nixos-activation-start[200]: setting up /etc... worker2 # [ 5.538744] initrd-nixos-activation-start[202]: setting up /etc... client # [ 5.579870] initrd-nixos-activation-start[199]: setting up /etc... worker1 # [ 5.557660] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 5.560296] systemd[1]: Finished NixOS Activation. worker1 # [ 5.561948] systemd[1]: Starting Switch Root... worker1 # [ 5.584915] systemd[1]: Switching root. lb # [ 5.552649] initrd-nixos-activation-start[198]: setting up /etc... worker2 # [ 5.669828] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 5.673005] systemd[1]: Finished NixOS Activation. worker2 # [ 5.674274] systemd[1]: Starting Switch Root... worker2 # [ 5.692852] systemd[1]: Switching root. client # [ 5.713285] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 5.716163] systemd[1]: Finished NixOS Activation. client # [ 5.717778] systemd[1]: Starting Switch Root... client # [ 5.740561] systemd[1]: Switching root. lb # [ 5.686142] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 5.689157] systemd[1]: Finished NixOS Activation. lb # [ 5.690335] systemd[1]: Starting Switch Root... worker1 # [ 5.762408] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). lb # [ 5.712595] systemd[1]: Switching root. worker2 # [ 5.873826] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). client # [ 5.922792] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). lb # [ 5.901069] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker1 # [ 6.250253] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 6.263245] systemd[1]: Detected virtualization qemu. worker1 # [ 6.266531] systemd[1]: Detected architecture arm64. worker1 # [ 6.269372] systemd[1]: Detected first boot. worker1 # [ 6.273055] systemd[1]: Initializing machine ID from random generator. worker2 # [ 6.383266] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 6.389943] systemd[1]: Detected virtualization qemu. worker2 # [ 6.391926] systemd[1]: Detected architecture arm64. worker2 # [ 6.393716] systemd[1]: Detected first boot. worker2 # [ 6.397864] systemd[1]: Initializing machine ID from random generator. client # [ 6.453605] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 6.466526] systemd[1]: Detected virtualization qemu. client # [ 6.469836] systemd[1]: Detected architecture arm64. client # [ 6.473882] systemd[1]: Detected first boot. client # [ 6.479100] systemd[1]: Initializing machine ID from random generator. lb # [ 6.437318] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 6.449674] systemd[1]: Detected virtualization qemu. lb # [ 6.452965] systemd[1]: Detected architecture arm64. lb # [ 6.456944] systemd[1]: Detected first boot. lb # [ 6.463270] systemd[1]: Initializing machine ID from random generator. worker1 # [ 6.568696] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 6.708014] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.747674] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.767071] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 6.821835] systemd[1]: Applying preset policy. client # [ 6.953068] systemd[1]: Applying preset policy. worker2 # [ 6.947699] systemd[1]: Applying preset policy. lb # [ 6.973155] systemd[1]: Applying preset policy. worker1 # [ 7.067530] systemd[1]: Populated /etc with preset unit settings. client # [ 7.200037] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 7.226920] systemd[1]: Populated /etc with preset unit settings. lb # [ 7.251367] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 7.315211] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 7.316976] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 7.322106] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 7.325163] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.327497] systemd[1]: Created slice User and Session Slice. worker1 # [ 7.328777] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 7.330560] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 7.332648] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.334406] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 7.336114] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 7.338013] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 7.339704] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 7.341875] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 7.343039] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 7.344745] systemd[1]: Reached target Path Units. worker1 # [ 7.346272] systemd[1]: Reached target Remote File Systems. worker1 # [ 7.348602] systemd[1]: Reached target Slice Units. worker1 # [ 7.349681] systemd[1]: Reached target Swaps. worker1 # [ 7.353520] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 7.356704] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.359108] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 7.361546] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.362771] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 7.368060] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 7.370513] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.373642] systemd[1]: Listening on Console Output Muting Service Socket. worker1 # [ 7.375718] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 7.377319] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.380736] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 7.387025] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 7.388419] systemd[1]: Listening on udev Control Socket. worker1 # [ 7.390186] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 7.394914] systemd[1]: Mounting Huge Pages File System... worker1 # [ 7.403209] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 7.413609] systemd[1]: Mounting Kernel Debug File System... client # [ 7.446777] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 7.448688] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 7.425456] systemd[1]: Mounting Kernel Trace File System... client # [ 7.453484] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 7.457199] systemd[1]: Created slice Slice /system/getty. client # [ 7.459168] systemd[1]: Created slice User and Session Slice. client # [ 7.460718] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 7.461133] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 7.439333] systemd[1]: Starting Create List of Static Device Nodes... client # [ 7.461498] systemd[1]: Expecting device /dev/hvc0... client # [ 7.461778] systemd[1]: Expecting device /dev/ttyAMA0... client # [ 7.462078] systemd[1]: Reached target Local Encrypted Volumes. client # [ 7.462423] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 7.440835] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 7.462749] systemd[1]: Stopped target initrd-root-fs.target. client # [ 7.463063] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.463384] systemd[1]: Reached target Virtual Machines and Containers. client # [ 7.470590] systemd[1]: Reached target Path Units. client # [ 7.472968] systemd[1]: Reached target Remote File Systems. client # [ 7.474914] systemd[1]: Reached target Slice Units. client # [ 7.476843] systemd[1]: Reached target Swaps. worker1 # [ 7.452549] systemd[1]: Mounting Kernel Configuration File System... client # [ 7.480269] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 7.483479] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.457424] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.457791] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 7.485782] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 7.458064] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 7.488281] systemd[1]: Listening on Factory Reset Management. client # [ 7.489556] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 7.470438] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 7.472255] systemd[1]: Stopped initrd-switch-root.service. client # [ 7.493915] systemd[1]: Starting Journal Log Access Socket... client # [ 7.496149] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 7.476682] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 7.500502] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 7.481197] systemd[1]: Created slice Slice /system/getty. client # [ 7.502229] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 7.483459] systemd[1]: Created slice User and Session Slice. worker2 # [ 7.484728] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 7.505366] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.505719] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 7.487025] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 7.488960] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 7.490532] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 7.484188] systemd[1]: Mounting FUSE Control File System... client # [ 7.512262] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 7.492143] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 7.484641] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 7.513677] systemd[1]: Listening on udev Control Socket. worker2 # [ 7.493843] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.495607] systemd[1]: Stopped target initrd-root-fs.target. client # [ 7.515297] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 7.496702] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 7.498100] systemd[1]: Reached target Virtual Machines and Containers. client # [ 7.519278] systemd[1]: Mounting Huge Pages File System... worker2 # [ 7.500265] systemd[1]: Reached target Path Units. worker2 # [ 7.502893] systemd[1]: Reached target Remote File Systems. worker2 # [ 7.503938] systemd[1]: Reached target Slice Units. worker2 # [ 7.505464] systemd[1]: Reached target Swaps. client # [ 7.527863] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 7.508582] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 7.511774] systemd[1]: Listening on Process Core Dump Socket. worker2 # [ 7.514042] systemd[1]: Listening on Credential Encryption/Decryption. worker2 # [ 7.516514] systemd[1]: Listening on Factory Reset Management. worker2 # [ 7.517783] systemd[1]: Listening on Hostname Service Socket. client # [ 7.538886] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 7.522223] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 7.524498] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 7.527444] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 7.529065] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 7.526056] systemd[1]: Starting Journal Service... worker2 # [ 7.530781] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.533212] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 7.555946] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 7.539673] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 7.536085] systemd[1]: Starting Load Kernel Modules... worker2 # [ 7.541090] systemd[1]: Listening on udev Control Socket. worker2 # [ 7.542759] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 7.546515] systemd[1]: Mounting Huge Pages File System... client # [ 7.565619] systemd[1]: Starting Create List of Static Device Nodes... client # [ 7.567982] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 7.549601] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 7.554774] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 7.511434] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 7.563277] systemd[1]: Mounting Kernel Debug File System... lb # [ 7.512839] systemd[1]: Stopped initrd-switch-root.service. lb # [ 7.516628] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 7.586535] systemd[1]: Mounting Kernel Configuration File System... lb # [ 7.521183] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.566545] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 7.591430] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 7.523302] systemd[1]: Created slice User and Session Slice. worker1 # [ 7.569177] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.575197] systemd[1]: Mounting Kernel Trace File System... lb # [ 7.524615] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 7.526325] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 7.528500] systemd[1]: Expecting device /dev/hvc0... lb # [ 7.530372] systemd[1]: Expecting device /dev/ttyAMA0... client # [ 7.599906] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 7.531447] systemd[1]: Reached target Local Encrypted Volumes. lb # [ 7.532867] systemd[1]: Stopped target initrd-fs.target. client # [ 7.602828] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 7.535064] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 7.586798] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 7.536977] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 7.583170] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 7.588055] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 7.538858] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 7.540112] systemd[1]: Reached target Path Units. lb # [ 7.541416] systemd[1]: Reached target Remote File Systems. lb # [ 7.542687] systemd[1]: Reached target Slice Units. worker1 # [ 7.588488] systemd[1]: Listening on Journal Log Access Socket. lb # [ 7.543936] systemd[1]: Reached target Swaps. lb # [ 7.547426] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 7.550638] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.596211] systemd[1]: Mounted Huge Pages File System. lb # [ 7.553009] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 7.555490] systemd[1]: Listening on Factory Reset Management. worker2 # [ 7.606862] systemd[1]: Mounting Kernel Configuration File System... lb # [ 7.556776] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 7.608389] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.601730] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 7.602219] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 7.602655] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 7.603140] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 7.611501] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 7.561713] systemd[1]: Starting Journal Log Access Socket... lb # [ 7.563398] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.608050] systemd[1]: Mounted FUSE Control File System. worker2 # [ 7.613872] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 7.566273] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 7.567892] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 7.639317] systemd[1]: Mounting FUSE Control File System... lb # [ 7.569580] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.573209] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 7.643147] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.579571] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.581033] systemd[1]: Listening on udev Control Socket. lb # [ 7.582660] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 7.631715] systemd[1]: Mounting FUSE Control File System... worker2 # [ 7.632176] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.586713] systemd[1]: Mounting Huge Pages File System... worker1 # [ 7.635456] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 7.593794] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 7.597745] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 7.642902] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.673564] systemd[1]: Starting Journal Service... worker2 # [ 7.661101] systemd[1]: Starting Journal Service... lb # [ 7.613390] systemd[1]: Mounting Kernel Trace File System... client # [ 7.685957] systemd[1]: Starting Load Kernel Modules... worker2 # [ 7.676426] systemd[1]: Starting Load Kernel Modules... lb # [ 7.629022] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 7.631405] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 7.702580] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 7.648720] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.694604] EXT4-fs (vda): re-mounted 9c69db1f-c800-4716-bd16-1a58065274da. lb # [ 7.653115] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 7.706398] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 7.659172] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.703430] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 7.704315] systemd[1]: Listening on Disk Image Download Service Socket. client # [ 7.731848] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 7.707777] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 7.732305] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.715807] systemd[1]: Starting Remount Root and Kernel File Systems... lb # [ 7.665388] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.717981] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.714861] systemd[1]: Starting Load/Save OS Random Seed... client # [ 7.743300] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 7.717452] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.728120] systemd[1]: Starting Coldplug All udev Devices... client # [ 7.747121] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 7.723129] systemd[1]: Finished Load Kernel Modules. worker1 # [ 7.729808] systemd[1]: Starting Firewall... client # [ 7.756864] systemd[1]: Mounted Huge Pages File System. worker2 # [ 7.736643] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 7.739206] systemd[1]: Mounted Huge Pages File System. worker2 # [ 7.742236] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 7.697172] systemd[1]: Mounting FUSE Control File System... worker2 # [ 7.747063] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 7.747594] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 7.748005] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 7.751739] systemd[1]: Mounted FUSE Control File System. client # [ 7.773026] systemd-journald[269]: Collecting audit messages is enabled. lb # [ 7.704413] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.750067] systemd[1]: Starting Apply Kernel Variables... client # [ 7.778125] systemd[1]: Mounted POSIX Message Queue File System. client # [ 7.785623] systemd[1]: Mounted Kernel Debug File System. client # [ 7.786871] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 7.770147] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 7.766926] systemd-journald[270]: Collecting audit messages is enabled. client # [ 7.779585] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.775805] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.796644] systemd[1]: Finished Create List of Static Device Nodes. client # [ 7.797238] systemd[1]: Started Journal Service. client # [ 7.788410] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 7.789610] systemd[1]: Mounted Kernel Configuration File System. lb # [ 7.740489] systemd[1]: Starting Journal Service... client # [ 7.804446] systemd-modules-load[270]: Module 'atkbd' is built in client # [ 7.805540] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.806695] systemd-modules-load[270]: Module 'loop' is built in lb # [ 7.752795] systemd[1]: Starting Load Kernel Modules... client # [ 7.818896] systemd[1]: Finished Load Kernel Modules. lb # [ 7.771920] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 7.785334] systemd[1]: Starting Remount Root and Kernel File Systems... lb # [ 7.789658] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.836310] systemd[1]: Started Journal Service. client # [ 7.851442] systemd[1]: Starting Apply Kernel Variables... client # [ 7.859329] systemd[1]: Mounted FUSE Control File System. worker2 # [ 7.854111] systemd[1]: Finished Load Kernel Modules. worker1 # [ 7.838763] systemd[1]: Queued start job for default target Multi-User System. worker1 # [ 7.841973] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 7.810613] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 7.863480] systemd[1]: Starting Firewall... worker1 # [ 7.857948] systemd-modules-load[271]: Module 'atkbd' is built in worker2 # [ 7.875853] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 7.879588] systemd-journald[272]: Collecting audit messages is enabled. worker2 # [ 7.882779] EXT4-fs (vda): re-mounted 80cb3d53-321a-4129-8263-8fcad6d40ced. worker1 # [ 7.866700] systemd-modules-load[271]: Module 'loop' is built in lb # [ 7.836191] systemd-journald[268]: Collecting audit messages is enabled. client # [ 7.890957] systemd-oomd[271]: No swap; memory pressure usage will be degraded client # [ 7.911639] EXT4-fs (vda): re-mounted bd498c12-dff1-4cc1-bc1d-be1cb8394cc7. lb # [ 7.843681] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 7.878355] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.903037] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 7.902062] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 7.853080] systemd[1]: Mounted Huge Pages File System. worker2 # [ 7.904788] systemd[1]: Listening on Disk Image Download Service Socket. client # [ 7.911147] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 7.856335] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 7.907071] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 7.858749] systemd[1]: Mounted Kernel Debug File System. client # [ 7.914235] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 7.895624] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 7.865615] systemd[1]: Mounted Kernel Trace File System. lb # [ 7.854041] systemd[1]: Queued start job for default target Multi-User System. client # [ 7.922755] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.923770] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 7.902506] systemd[1]: Reached target First Boot Complete. lb # [ 7.872261] systemd[1]: Started Journal Service. worker2 # [ 7.923373] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 7.923785] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.909169] systemd-oomd[272]: No swap; memory pressure usage will be degraded client # [ 7.936246] systemd[1]: Starting Load/Save OS Random Seed... client # [ 7.937109] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.921738] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 7.930262] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 7.949250] systemd[1]: Started Journal Service. worker1 # [ 7.944563] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 7.948904] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 7.960643] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.962066] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 7.970590] systemd-modules-load[273]: Module 'atkbd' is built in lb # [ 7.937052] EXT4-fs (vda): re-mounted 209df36f-c74f-438a-b883-feb5929c5341. worker2 # [ 7.980848] systemd-modules-load[273]: Module 'loop' is built in worker1 # [ 7.990324] systemd-journald[270]: Received client request to flush runtime journal. worker2 # [ 7.994146] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 8.031740] systemd-journald[269]: Received client request to flush runtime journal. client # [ 8.078468] systemd[1]: Finished Load/Save OS Random Seed. client # [ 8.079433] systemd[1]: Reached target First Boot Complete. worker1 # [ 8.058583] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 8.065647] systemd[1]: Finished Apply Kernel Variables. client # [ 8.084663] systemd[1]: Finished Apply Kernel Variables. client # [ 8.089201] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 8.080898] systemd-oomd[274]: No swap; memory pressure usage will be degraded worker2 # [ 8.092840] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 8.094027] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 8.118156] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 8.126083] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 8.116484] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 8.115200] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 8.134254] systemd-journald[272]: Received client request to flush runtime journal. worker1 # [ 8.116471] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 8.121145] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 8.180563] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 8.183799] systemd[1]: Reached target First Boot Complete. worker2 # [ 8.188656] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 8.242479] systemd-udevd[310]: Using default interface naming scheme 'v261'. worker2 # [ 8.301096] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 8.304369] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 8.305347] systemd[1]: Mounting /run/wrappers... worker2 # [ 8.312721] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 8.383577] systemd[1]: Mounted /run/wrappers. worker1 # [ 8.392478] systemd[1]: Reached target Local File Systems. worker1 # [ 8.393310] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 8.394239] systemd[1]: Starting register-nix-paths.service... worker1 # [ 8.395079] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 8.395885] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 8.423477] systemd-udevd[309]: Using default interface naming scheme 'v261'. worker1 # [ 8.432748] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 8.433722] systemd[1]: Starting Create System Files and Directories... worker2 # [ 8.458405] systemd[1]: Mounting /run/wrappers... worker1 # [ 8.462773] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 7.858997] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 8.498312] systemd-journald[268]: Received client request to flush runtime journal. worker2 # [ 8.537819] systemd[1]: Mounted /run/wrappers. worker2 # [ 8.541671] systemd[1]: Reached target Local File Systems. worker1 # [ 8.546523] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 8.555033] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 8.556256] systemd[1]: Finished Save Transient machine-id to Disk. worker2 # [ 8.567055] systemd[1]: Starting register-nix-paths.service... worker2 # [ 8.573189] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 8.580209] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 8.593622] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 8.599579] systemd[1]: Starting Create System Files and Directories... client # [ 8.620893] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 8.623249] systemd[1]: Reached target Preparation for Local File Systems. client # [ 8.630382] systemd[1]: Mounting /run/wrappers... client # [ 8.636664] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 8.583583] systemd-modules-load[269]: Module 'atkbd' is built in lb # [ 8.592764] systemd-modules-load[269]: Module 'loop' is built in lb # [ 8.593872] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 8.643444] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 8.594976] systemd[1]: Mounted Kernel Configuration File System. lb # [ 8.595971] systemd[1]: Mounted FUSE Control File System. lb # [ 8.609397] systemd[1]: Finished Load Kernel Modules. lb # [ 8.610359] systemd[1]: Starting Firewall... lb # [ 8.611175] systemd[1]: Starting Apply Kernel Variables... lb # [ 8.624620] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 8.625944] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 8.626985] systemd[1]: Listening on Disk Image Download Service Socket. lb # [ 8.636149] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 8.638354] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 8.686070] systemd[1]: Finished Create System Files and Directories. lb # [ 8.644824] systemd-oomd[271]: No swap; memory pressure usage will be degraded lb # [ 8.646422] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 8.647360] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 8.694130] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 8.704651] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 8.661970] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 8.714033] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 8.666671] systemd[1]: Finished Apply Kernel Variables. lb # [ 8.667595] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 8.674460] systemd[1]: Reached target First Boot Complete. lb # [ 8.675312] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 8.723989] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 8.680558] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 8.681490] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 8.682383] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 8.683361] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 8.752927] systemd[1]: Mounted /run/wrappers. client # [ 8.753788] systemd[1]: Reached target Local File Systems. lb # [ 8.698972] systemd[1]: Mounting /run/wrappers... lb # [ 8.699701] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 8.767408] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 8.770754] systemd[1]: Starting register-nix-paths.service... lb # [ 8.706264] systemd[1]: Mounted /run/wrappers. lb # [ 8.706961] systemd[1]: Reached target Local File Systems. lb # [ 8.707734] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 8.778584] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 8.779527] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 8.784654] systemd[1]: Starting Save Transient machine-id to Disk... lb # [ 8.722901] systemd[1]: Starting register-nix-paths.service... lb # [ 8.723775] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 8.726413] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 8.727656] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 8.804129] systemd[1]: Starting Create System Files and Directories... lb # [ 8.739848] systemd[1]: Starting Create System Files and Directories... client # [ 8.840399] systemd[1]: Finished Coldplug All udev Devices. lb # [ 8.772907] systemd-udevd[295]: Using default interface naming scheme 'v261'. worker1 # [ 8.827858] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 8.856569] systemd-udevd[299]: Using default interface naming scheme 'v261'. worker2 # [ 8.838242] systemd[1]: Finished Create System Files and Directories. worker2 # [ 8.843709] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 8.857291] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 8.899327] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 8.912527] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 8.899891] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 8.907173] systemd[1]: Starting Update is Completed... lb # [ 8.883629] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 8.897389] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 8.998515] systemd[1]: Finished Create System Files and Directories. worker2 # [ 8.988463] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 9.010225] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 9.000810] systemd[1]: Finished Update is Completed. client # [ 9.030568] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 9.031682] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 9.003358] systemd[1]: Finished Create System Files and Directories. worker2 # [ 9.066393] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 9.022170] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 9.078323] systemd[1]: Starting Update is Completed... lb # [ 9.031242] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 9.034075] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 9.145903] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 9.177959] systemd[1]: Finished Rebuild Journal Catalog. client # [ 9.188361] systemd[1]: Starting Update is Completed... worker2 # [ 9.199306] systemd[1]: Finished Update is Completed. client # [ 9.255617] systemd[1]: Finished Update is Completed. lb # [ 9.221660] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 9.254847] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 9.268989] systemd[1]: Starting Update is Completed... worker1 # [ 9.387162] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 9.389743] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.372108] systemd[1]: Finished Update is Completed. client # [ 9.535397] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 9.557855] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 9.561472] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.585432] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 9.734382] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 9.737609] systemd[1]: Finished Create SUID/SGID Wrappers. worker1 # [ 9.724937] systemd[1]: Finished Firewall. lb # [ 9.708267] systemd[1]: Finished Coldplug All udev Devices. client # [ 9.785651] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 9.874246] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. lb # [ 9.825641] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 9.863745] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 9.941181] systemd[1]: Finished Firewall. client # [ 10.037626] (udev-worker)[353]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 10.048185] (udev-worker)[353]: Network interface NamePolicy= disabled on kernel command line. client # [ 10.053319] (udev-worker)[358]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.092437] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 10.159240] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 10.126064] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 10.130047] systemd[1]: Finished Create SUID/SGID Wrappers. worker1 # [ 10.186330] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 10.144377] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 10.205172] systemd[1]: Finished register-nix-paths.service. worker1 # [ 10.207020] systemd[1]: Reached target System Initialization. worker1 # [ 10.208898] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 10.211044] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 10.213940] systemd[1]: Reached target Timer Units. worker1 # [ 10.220620] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 10.228205] systemd[1]: Listening on Nix Daemon Socket. lb # [ 10.184301] systemd[1]: Finished register-nix-paths.service. lb # [ 10.186125] systemd[1]: Reached target System Initialization. lb # [ 10.186963] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 10.187939] systemd[1]: Started niks3 garbage collection timer. client # [ 10.256843] systemd[1]: Finished register-nix-paths.service. client # [ 10.258434] systemd[1]: Reached target System Initialization. worker2 # [ 10.242589] systemd[1]: Finished Coldplug All udev Devices. client # [ 10.264324] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 10.197533] systemd[1]: Started Daily Cleanup of Temporary Directories. lb # [ 10.198527] systemd[1]: Reached target Timer Units. client # [ 10.266199] systemd[1]: Started Daily Cleanup of Temporary Directories. lb # [ 10.199262] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 10.267215] systemd[1]: Reached target Timer Units. worker1 # [ 10.244689] systemd[1]: Listening on Nix worker-protocol over gRPC. client # [ 10.267955] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 10.206351] systemd[1]: Listening on niks3 server socket. lb # [ 10.207174] systemd[1]: Listening on Nix Daemon Socket. client # [ 10.275185] systemd[1]: Listening on Nix Daemon Socket. lb # [ 10.207913] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 10.253247] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 10.279506] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 10.282236] systemd[1]: Reached target Socket Units. lb # [ 10.215066] systemd[1]: Reached target Socket Units. lb # [ 10.215850] systemd[1]: Reached target Basic System. worker1 # [ 10.261847] systemd[1]: Reached target Socket Units. client # [ 10.285556] systemd[1]: Reached target Basic System. client # [ 10.288247] systemd[1]: Started backdoor.service. lb # [ 10.220709] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 10.221746] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.267642] systemd[1]: Reached target Basic System. client # [ 10.290832] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 10.270210] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 10.227900] systemd[1]: Starting Post-Boot Actions... client # [ 10.296324] systemd[1]: Starting nftables firewall... worker1 # [ 10.277199] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.285477] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.294762] systemd[1]: Started Reset console on configuration changes. client # [ 10.321774] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.262407] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.310477] systemd[1]: Starting resolvconf update... worker2 # [ 10.318841] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 10.316495] systemd[1]: Starting D-Bus System Message Bus... client # [ 10.340118] systemd[1]: Starting Post-Boot Actions... client # [ 10.354655] systemd[1]: Started Reset console on configuration changes. lb # [ 10.292497] systemd[1]: Starting resolvconf update... worker2 # [ 10.348107] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 10.394215] systemd[1]: Starting resolvconf update... worker1 # [ 10.376439] systemd[1]: Finished Post-Boot Actions. lb # [ 10.341250] systemd[1]: Started rustfs.service. worker1 # [ 10.396783] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 10.352259] systemd[1]: Starting rustfs-bucket.service... client # [ 10.422399] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 10.405187] systemd[1]: Finished register-nix-paths.service. worker2 # [ 10.406586] systemd[1]: Reached target System Initialization. worker2 # [ 10.408371] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 10.359295] nsncd[450]: Sep 22 11:31:44.950 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.405303] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.410530] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 10.413879] systemd[1]: Reached target Timer Units. worker2 # [ 10.419921] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 10.416672] nsncd[488]: Sep 22 11:31:44.999 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.428261] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 10.429778] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.435998] systemd[1]: Starting User Login Management... worker2 # [ 10.440538] systemd[1]: Listening on Nix worker-protocol over gRPC. client # [ 10.459511] nsncd[393]: Sep 22 11:31:45.043 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 10.398617] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 10.471357] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.452734] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 10.408133] systemd[1]: Finished Post-Boot Actions. client # [ 10.479125] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.461512] systemd[1]: Reached target Socket Units. worker1 # [ 10.461446] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 10.465906] systemd[1]: Reached target Basic System. lb # [ 10.417121] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. client # connecting to host... client # [ 10.488724] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.472762] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 10.473281] dbus-broker-launch[492]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.479636] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 10.500998] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 10.480878] dbus-broker-launch[492]: NSS returned no entry for 'systemd-timesync' worker2 # [ 10.491011] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.486620] dbus-broker-launch[492]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 10.504153] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.503745] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 10.516767] systemd[1]: Starting resolvconf update... lb # [ 10.469507] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 10.470652] systemd[1]: Reached target User and Group Name Lookups. client: Guest shell says: b'Spawning backdoor root shell...\n' worker2 # [ 10.522613] systemd[1]: Starting D-Bus System Message Bus... client # [ 10.550193] systemd[1]: Starting User Login Management... client # [ 10.553984] systemd[1]: Finished Post-Boot Actions. lb # [ 10.488346] systemd[1]: Started backdoor.service. lb # [ 10.507978] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 10.558643] dbus-broker-launch[492]: Ready worker2 # [ 10.577948] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.607046] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 10.565913] systemd[1]: Starting User Login Management... lb # [ 10.568673] (udev-worker)[358]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 10.616810] systemd[1]: Reached target Host and Network Name Lookups. client # [ 10.645815] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 10.615868] systemd-logind[500]: New seat seat0. client: connected to guest root shell worker2 # [ 10.626035] nsncd[489]: Sep 22 11:31:45.195 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 10.578557] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 10.627508] systemd[1]: Started User Login Management. client: (connecting took 10.98 seconds) worker1 # [ 10.631736] systemd[1]: Starting linger-users.service... client: (finished: waiting for the VM to finish booting, in 10.98 seconds) worker2 # [ 10.639475] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.646506] systemd[1]: Starting User Login Management... worker1 # [ 10.643130] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 10.597882] (udev-worker)[362]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 10.646844] systemd[1]: Stopping Host and Network Name Lookups... worker1 # [ 10.647751] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 10.665009] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 10.669322] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 10.675326] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 10.676827] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 10.683013] dbus-broker-launch[493]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 10.681278] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 10.640230] (udev-worker)[362]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.685527] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 10.695299] dbus-broker-launch[493]: NSS returned no entry for 'systemd-timesync' worker2 # [ 10.702693] dbus-broker-launch[493]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 10.719904] systemd[1]: Started D-Bus System Message Bus. worker1 # [ 10.734401] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 10.739394] systemd[1]: Finished linger-users.service. worker2 # [ 10.774564] dbus-broker-launch[493]: Ready worker1 # [ 10.772501] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.775098] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.779324] nsncd[555]: Sep 22 11:31:45.369 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.787058] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.790967] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 10.812350] systemd[1]: Finished resolvconf update. worker1 # [ 10.816391] systemd[1]: Reached target Preparation for Network. lb # connecting to host... worker1 # [ 10.822904] systemd[1]: Starting DHCP Client... worker2 # [ 10.832390] systemd-logind[501]: New seat seat0. worker1 # [ 10.832112] systemd[1]: Starting Extra networking commands.... worker2 # [ 10.842155] systemd[1]: Started User Login Management. worker2 # [ 10.846134] systemd[1]: Starting linger-users.service... client # [ 10.869468] systemd-logind[418]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 10.877580] systemd-logind[418]: New seat seat0. client # [ 10.887424] systemd[1]: Started User Login Management. worker2 # [ 10.871172] systemd[1]: Stopped target Host and Network Name Lookups. worker2 # [ 10.876901] systemd[1]: Stopping Host and Network Name Lookups... client # [ 10.896618] dbus-broker-launch[416]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.882191] systemd[1]: Stopped target User and Group Name Lookups. client # [ 10.907665] systemd[1]: Starting linger-users.service... worker2 # [ 10.890968] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 10.887050] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 10.898953] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 10.896586] systemd[1]: Started backdoor.service. worker2 # [ 10.905450] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 10.911424] systemd[1]: Stopped Name Service Cache Daemon (nsncd). client # [ 10.930322] dbus-broker-launch[416]: NSS returned no entry for 'systemd-timesync' client # [ 10.931426] dbus-broker-launch[416]: Invalid user-name in /nix/store/6wvgzhp3xj1v1xybf1ixpbqi5gif5fbv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 10.918666] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.870255] dbus-broker-launch[479]: Looking up NSS user entry for 'systemd-timesync'... client # [ 10.952110] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 10.884982] dbus-broker-launch[479]: NSS returned no entry for 'systemd-timesync' lb # [ 10.886095] dbus-broker-launch[479]: Invalid user-name in /nix/store/4yiprls1sll47jid3xzq0m9nmi1qz2ww-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client # [ 10.960445] systemd[1]: Stopping Host and Network Name Lookups... client # [ 10.961467] systemd[1]: Stopped target User and Group Name Lookups. worker2 # [ 10.942514] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 10.962350] systemd[1]: Stopping User and Group Name Lookups... client # [ 10.963160] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 10.986352] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 10.967594] systemd[1]: linger-users.service: Deactivated successfully. client # [ 10.987292] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 10.970683] systemd[1]: Finished linger-users.service. worker2 # [ 10.992332] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. client # [ 11.011816] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 10.998420] systemd[1]: Started backdoor.service. lb # [ 10.953259] systemd[1]: Stopped target Host and Network Name Lookups. worker2 # [ 11.016532] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 11.035870] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.968568] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 10.969539] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 10.970480] systemd[1]: Stopping User and Group Name Lookups... lb # [ 10.971282] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 11.020916] nsncd[560]: Sep 22 11:31:45.601 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 11.055958] mousedev: PS/2 mouse device common for all mice worker1 # connecting to host... worker2 # [ 11.030126] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 11.035271] systemd[1]: Reached target User and Group Name Lookups. client # [ 11.063606] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 11.045428] systemd[1]: Finished resolvconf update. lb # [ 10.992706] systemd[1]: Started D-Bus System Message Bus. client # [ 11.069936] dbus-broker-launch[416]: Ready client # [ 11.070638] systemd[1]: Finished linger-users.service. lb # [ 11.004713] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 11.054321] systemd[1]: Reached target Preparation for Network. lb # [ 11.006032] systemd-logind[482]: New seat seat0. lb # [ 11.006796] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 11.066038] systemd[1]: Starting DHCP Client... worker2 # [ 11.068192] systemd[1]: Starting Extra networking commands.... lb # [ 11.022412] systemd[1]: Started User Login Management. lb # [ 11.048596] dbus-broker-launch[479]: Ready lb # [ 11.055441] systemd[1]: Starting linger-users.service... lb # [ 11.065206] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 11.142891] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 11.147393] nsncd[489]: Sep 22 11:31:45.731 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 11.126770] dhcpcd[591]: dhcpcd-10.3.2 starting worker2 # connecting to host... client # [ 11.156512] systemd[1]: Finished resolvconf update. client # [ 11.157266] systemd[1]: Reached target Host and Network Name Lookups. client # [ 11.158154] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 11.137115] dhcpcd[633]: dev: loaded udev client # [ 11.164288] systemd[1]: Starting DHCP Client... worker1 # [ 11.145139] (udev-worker)[346]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 11.158077] (udev-worker)[353]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 11.172342] (udev-worker)[353]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 11.190526] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 11.257153] systemd[1]: Finished Extra networking commands.. worker1 # [ 11.258062] systemd[1]: Reached target Network. worker1 # [ 11.264635] systemd[1]: Starting Permit User Sessions... lb # [ 11.224451] systemd[1]: linger-users.service: Deactivated successfully. lb # [ 11.231736] systemd[1]: Finished linger-users.service. lb # [ 11.238740] nsncd[560]: Sep 22 11:31:45.821 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 11.247244] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 11.320524] systemd-logind[418]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker1 # [ 11.312220] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 11.251941] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 11.257045] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 11.310496] (udev-worker)[462]: Network interface NamePolicy= disabled on kernel command line. lb # [ 11.267565] systemd[1]: Finished resolvconf update. worker2 # [ 11.327149] (udev-worker)[463]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 11.281228] systemd[1]: Starting DHCP Client... worker1 # [ 11.348399] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 11.349038] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 11.344911] dhcpcd[596]: dhcpcd-10.3.2 starting worker1 # [ 11.352054] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 11.352464] cfg80211: failed to load regulatory.db worker2 # [ 11.348976] (udev-worker)[463]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 11.357236] dhcpcd[637]: dev: loaded udev worker1 # [ 11.354928] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 11.366401] systemd[1]: Starting Address configuration of eth1... worker1 # [ 11.381305] systemd[1]: Finished Permit User Sessions. worker1 # [ 11.382143] systemd[1]: Started Getty on tty1. worker1 # [ 11.382796] systemd[1]: Reached target Login Prompts. client # [ 11.407804] dhcpcd[519]: dhcpcd-10.3.2 starting worker2 # [ 11.403766] 8021q: 802.1Q VLAN Support v1.8 client # [ 11.419430] dhcpcd[527]: dev: loaded udev client # [ 11.475080] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 11.442238] systemd[1]: Finished Extra networking commands.. worker2 # [ 11.453402] systemd[1]: Reached target Network. worker2 # [ 11.457364] systemd[1]: Starting Permit User Sessions... worker1 # [ 11.473194] 8021q: adding VLAN 0 to HW filter on device eth0 worker1 # [ 11.464379] dhcpcd[633]: eth0: waiting for carrier worker1 # [ 11.471481] dhcpcd[633]: eth0: waiting for carrier worker1 # [ 11.473866] dhcpcd[633]: eth0: carrier acquired worker1 # [ 11.490393] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 11.445491] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 11.493137] dhcpcd[633]: DUID 00:01:00:01:32:45:23:a2:52:54:00:12:34:56 worker1 # [ 11.495524] dhcpcd[633]: eth0: IAID 00:12:34:56 lb # [ 11.453167] systemd-logind[482]: Watching system buttons on /dev/input/event0 (gpio-keys) worker1 # [ 11.498244] dhcpcd[633]: eth0: adding address fe80::5054:ff:fe12:3456 worker2 # [ 11.519548] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 11.503054] network-addresses-eth1-start[657]: adding address 192.168.1.3/24... done worker1 # [ 11.514177] network-addresses-eth1-start[657]: adding address 2001:db8:1::3/64... done worker1 # [ 11.531432] systemd[1]: Finished Address configuration of eth1. worker2 # [ 11.540448] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 11.546151] systemd[1]: Starting Address configuration of eth1... worker2 # [ 11.568511] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 11.569089] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 11.571752] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 11.572104] cfg80211: failed to load regulatory.db client # [ 11.594109] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker2 # [ 11.581793] systemd[1]: Finished Permit User Sessions. worker2 # [ 11.588169] systemd[1]: Started Getty on tty1. worker2 # [ 11.588940] systemd[1]: Reached target Login Prompts. client # [ 11.664017] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 11.664571] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 11.669379] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 11.669727] cfg80211: failed to load regulatory.db lb # [ 11.615960] dhcpcd[608]: dhcpcd-10.3.2 starting worker2 # [ 11.679948] 8021q: adding VLAN 0 to HW filter on device eth1 worker2 # [ 11.685792] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 11.676206] dhcpcd[637]: eth0: waiting for carrier worker2 # [ 11.679712] dhcpcd[637]: eth0: waiting for carrier worker2 # [ 11.681942] dhcpcd[637]: eth0: carrier acquired lb # [ 11.638222] dhcpcd[624]: dev: loaded udev client # [ 11.712266] systemd[1]: Finished nftables firewall. client # [ 11.714065] systemd[1]: Reached target Preparation for Network. worker2 # [ 11.695139] network-addresses-eth1-start[655]: adding address 192.168.1.4/24... done worker1 # [ 11.695906] systemd-logind[500]: Watching system buttons on /dev/input/event0 (gpio-keys) worker2 # [ 11.700980] dhcpcd[637]: DUID 00:01:00:01:32:45:23:a2:52:54:00:12:34:56 worker2 # [ 11.701944] dhcpcd[637]: eth0: IAID 00:12:34:56 worker2 # [ 11.702581] dhcpcd[637]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 11.722696] systemd[1]: Starting Address configuration of eth1... client # [ 11.731106] systemd[1]: Starting Extra networking commands.... worker2 # [ 11.712320] network-addresses-eth1-start[655]: adding address 2001:db8:1::4/64... done lb # [ 11.673170] systemd[1]: Finished Firewall. lb # [ 11.676849] systemd[1]: Reached target Preparation for Network. worker2 # [ 11.732350] systemd[1]: Finished Address configuration of eth1. lb # [ 11.689878] systemd[1]: Starting Address configuration of eth1... lb # [ 11.698980] systemd[1]: Starting Extra networking commands.... client # [ 11.787139] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 11.775902] dhcpcd[527]: eth0: waiting for carrier worker1 # [ 11.765374] mousedev: PS/2 mouse device common for all mice client # [ 11.778311] dhcpcd[527]: libudev: received NULL device client # [ 11.779222] dhcpcd[527]: libudev: received NULL device client # [ 11.779963] dhcpcd[527]: eth0: carrier acquired lb # [ 11.726903] 8021q: 802.1Q VLAN Support v1.8 client # [ 11.806828] dhcpcd[527]: DUID 00:01:00:01:32:45:23:a2:52:54:00:12:34:56 client # [ 11.807885] dhcpcd[527]: eth0: IAID 00:12:34:56 client # [ 11.810740] dhcpcd[527]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 11.869685] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 11.807096] mousedev: PS/2 mouse device common for all mice client # [ 11.885083] network-addresses-eth1-start[547]: adding address 192.168.1.1/24... done worker2 # [ 11.877258] systemd-logind[501]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.906605] network-addresses-eth1-start[547]: adding address 2001:db8:1::1/64... done client # [ 11.931517] systemd[1]: Finished Address configuration of eth1. worker1 # [ 11.932939] systemd-logind[500]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.929475] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 11.940828] 8021q: adding VLAN 0 to HW filter on device eth1 worker2 # [ 12.025634] mousedev: PS/2 mouse device common for all mice lb # [ 11.976855] network-addresses-eth1-start[629]: adding address 192.168.1.2/24... done client # [ 12.059161] systemd[1]: Finished Extra networking commands.. lb # [ 12.004019] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 12.004593] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 12.063179] systemd[1]: Reached target Network. lb # [ 12.010077] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 12.010436] cfg80211: failed to load regulatory.db client # [ 12.072003] systemd[1]: Starting Permit User Sessions... lb # [ 12.025029] network-addresses-eth1-start[629]: adding address 2001:db8:1::2/64... done client: (finished: waiting for unit network-addresses-eth1.service, in 12.46 seconds) worker2 # [ 12.121468] systemd-logind[501]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } client # [ 12.142471] systemd[1]: Finished Permit User Sessions. lb # [ 12.078073] systemd[1]: Finished Address configuration of eth1. client # [ 12.154515] systemd[1]: Started Getty on tty1. client # [ 12.160281] systemd[1]: Reached target Login Prompts. client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.06 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb # [ 12.137143] dhcpcd[686]: /nix/store/1ki7bgdr0niilkg9jbkpcdaggbv0kb8a-openresolv-3.17.4/sbin/.resolvconf-wrapped: line 1250: kill: (636) - Operation not permitted lb # [ 12.139072] dhcpcd[686]: .resolvconf-wrapped: clearing stale lock pid 636 lb # [ 12.239303] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 12.229033] dhcpcd[624]: eth0: waiting for carrier lb # [ 12.232811] dhcpcd[624]: libudev: received NULL device lb # [ 12.233617] dhcpcd[624]: libudev: received NULL device lb # [ 12.234519] dhcpcd[624]: eth0: carrier acquired lb # [ 12.267022] dhcpcd[624]: DUID 00:01:00:01:32:45:23:a2:52:54:00:12:34:56 lb # [ 12.274499] dhcpcd[624]: eth0: IAID 00:12:34:56 lb # [ 12.275241] dhcpcd[624]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 12.315559] systemd[1]: Finished Extra networking commands.. lb # [ 12.323777] systemd[1]: Reached target Network. lb: (finished: waiting for unit network-addresses-eth1.service, in 0.20 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } lb # [ 12.340708] systemd[1]: Starting PostgreSQL Server... lb # [ 12.346391] systemd-logind[482]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker1 # [ 12.404275] dhcpcd[633]: eth0: soliciting a DHCP lease lb # [ 12.361323] systemd[1]: Starting Permit User Sessions... worker1 # [ 12.408728] dhcpcd[633]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 12.416250] dhcpcd[633]: eth0: probing address 10.0.2.15/24 client # [ 12.524560] dhcpcd[527]: eth0: soliciting a DHCP lease lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.12 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) client # [ 12.528713] dhcpcd[527]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 12.536243] dhcpcd[527]: eth0: probing address 10.0.2.15/24 lb # [ 12.482762] systemd[1]: Finished Permit User Sessions. lb # [ 12.495587] systemd[1]: Started Getty on tty1. lb # [ 12.502840] systemd[1]: Reached target Login Prompts. worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.04 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) client # [ 12.681732] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker1 # [ 12.702498] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.05 seconds) lb: waiting for unit niks3.service lb # [ 12.726441] dhcpcd[624]: eth0: soliciting a DHCP lease lb # [ 12.733123] dhcpcd[624]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 12.740363] dhcpcd[624]: eth0: probing address 10.0.2.15/24 lb # [ 12.753216] postgresql-pre-start[735]: The files belonging to this database system will be owned by user "postgres". lb # [ 12.754672] postgresql-pre-start[735]: This user must also own the server process. lb # [ 12.768125] postgresql-pre-start[735]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 12.769468] postgresql-pre-start[735]: The default database encoding has accordingly been set to "UTF8". lb # [ 12.770703] postgresql-pre-start[735]: The default text search configuration will be set to "english". lb # [ 12.771891] postgresql-pre-start[735]: Data page checksums are enabled. lb # [ 12.779634] postgresql-pre-start[735]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 12.783548] postgresql-pre-start[735]: creating subdirectories ... ok lb # [ 12.790701] postgresql-pre-start[735]: selecting dynamic shared memory implementation ... posix worker2 # [ 12.864642] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 12.951521] postgresql-pre-start[735]: selecting default "max_connections" ... 100 client # [ 13.053430] systemd[1]: Starting Virtual Console Setup... worker1 # [ 13.044141] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1 # [ 13.050809] systemd[1]: Starting Virtual Console Setup... client # [ 13.100425] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1 # [ 13.089892] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 13.091011] systemd[1]: Stopped Virtual Console Setup. client # [ 13.117050] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 13.121697] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 13.098817] systemd[1]: Starting Virtual Console Setup... client # [ 13.129589] systemd[1]: Starting Virtual Console Setup... lb # [ 13.073585] postgresql-pre-start[735]: selecting default "shared_buffers" ... 128MB client # [ 13.179452] systemd-logind[418]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 13.164094] systemd-logind[500]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 13.175325] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 13.188218] systemd[1]: Starting Virtual Console Setup... worker2 # [ 13.212606] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 13.220661] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 13.221565] systemd[1]: Starting Virtual Console Setup... client # [ 13.281603] systemd-vconsole-setup[639]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 13.286418] systemd[1]: Finished Virtual Console Setup. worker1 # [ 13.263238] systemd-vconsole-setup[703]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 13.268485] systemd[1]: Finished Virtual Console Setup. worker2 # [ 13.298901] systemd-logind[501]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 13.380386] systemd-vconsole-setup[703]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 13.383908] systemd[1]: Finished Virtual Console Setup. worker2 # [ 13.415678] dhcpcd[637]: eth0: soliciting a DHCP lease worker2 # [ 13.420762] dhcpcd[637]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 13.428246] dhcpcd[637]: eth0: probing address 10.0.2.15/24 client # [ 13.530431] dhcpcd[527]: eth0: soliciting an IPv6 router client # [ 13.534843] dhcpcd[527]: eth0: Router Advertisement from fe80::2 client # [ 13.537595] dhcpcd[527]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 13.540435] dhcpcd[527]: eth0: adding route to fec0::/64 client # [ 13.542468] dhcpcd[527]: eth0: adding default route via fe80::2 worker1 # [ 13.523448] dhcpcd[633]: eth0: soliciting an IPv6 router worker1 # [ 13.527991] dhcpcd[633]: eth0: Router Advertisement from fe80::2 worker1 # [ 13.530820] dhcpcd[633]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 13.533843] dhcpcd[633]: eth0: adding route to fec0::/64 worker1 # [ 13.536481] dhcpcd[633]: eth0: adding default route via fe80::2 worker2 # [ 13.789994] dhcpcd[637]: eth0: soliciting an IPv6 router worker2 # [ 13.794735] dhcpcd[637]: eth0: Router Advertisement from fe80::2 worker2 # [ 13.797510] dhcpcd[637]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 13.800533] dhcpcd[637]: eth0: adding route to fec0::/64 worker2 # [ 13.802840] dhcpcd[637]: eth0: adding default route via fe80::2 lb # [ 13.795774] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 14.023760] dhcpcd[624]: eth0: soliciting an IPv6 router lb # [ 14.024425] dhcpcd[624]: eth0: Router Advertisement from fe80::2 lb # [ 14.024557] dhcpcd[624]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 14.024602] dhcpcd[624]: eth0: adding route to fec0::/64 lb # [ 14.024643] dhcpcd[624]: eth0: adding default route via fe80::2 lb # [ 14.305801] systemd[1]: Starting Virtual Console Setup... lb # [ 14.338151] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 14.339277] systemd[1]: Stopped Virtual Console Setup. lb # [ 14.356493] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 14.363834] systemd[1]: Starting Virtual Console Setup... lb # [ 14.455378] systemd-logind[482]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 14.638501] systemd-vconsole-setup[765]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 14.643107] systemd[1]: Finished Virtual Console Setup. lb # [ 14.993185] postgresql-pre-start[735]: selecting default time zone ... UTC lb # [ 14.996355] postgresql-pre-start[735]: creating configuration files ... ok lb # [ 15.353903] postgresql-pre-start[735]: running bootstrap script ... ok lb # [ 15.912972] postgresql-pre-start[735]: performing post-bootstrap initialization ... ok lb # [ 16.072040] postgresql-pre-start[735]: syncing data to disk ... ok lb # [ 16.074116] postgresql-pre-start[735]: initdb: warning: enabling "trust" authentication for local connections lb # [ 16.075414] postgresql-pre-start[735]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 16.077614] postgresql-pre-start[735]: Success. You can now start the database server using: lb # [ 16.078795] postgresql-pre-start[735]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 16.168256] postgres[790]: [790] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 16.171291] postgres[790]: [790] LOG: listening on IPv6 address "::1", port 5432 lb # [ 16.172683] postgres[790]: [790] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 16.174351] postgres[790]: [790] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 16.185381] postgres[799]: [799] LOG: database system was shut down at 2026-09-22 11:31:50 GMT lb # [ 16.192555] postgres[790]: [790] LOG: database system is ready to accept connections lb # [ 16.197405] systemd[1]: Started PostgreSQL Server. lb # [ 16.202160] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 16.421465] postgresql-setup-start[814]: CREATE DATABASE lb # [ 16.460573] postgresql-setup-start[824]: CREATE ROLE lb # [ 16.476649] postgresql-setup-start[826]: ALTER DATABASE lb # [ 16.482702] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 16.484646] systemd[1]: Reached target PostgreSQL. worker1 # [ 17.194844] dhcpcd[633]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 17.198176] dhcpcd[633]: eth0: adding route to 10.0.2.0/24 worker1 # [ 17.204378] dhcpcd[633]: eth0: adding default route via 10.0.2.2 worker1 # [ 17.332780] systemd[1]: Started DHCP Client. worker1 # [ 17.334718] systemd[1]: Reached target Multi-User System. worker1 # [ 17.335572] systemd[1]: Startup finished in 973ms (kernel) + 4.847s (initrd) + 11.513s (userspace) = 17.334s. client # [ 17.368680] dhcpcd[527]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 17.372630] dhcpcd[527]: eth0: adding route to 10.0.2.0/24 client # [ 17.375077] dhcpcd[527]: eth0: adding default route via 10.0.2.2 client # [ 17.510585] systemd[1]: Started DHCP Client. client # [ 17.512435] systemd[1]: Reached target Multi-User System. client # [ 17.513521] systemd[1]: Startup finished in 983ms (kernel) + 5.035s (initrd) + 11.493s (userspace) = 17.513s. lb # [ 17.541589] dhcpcd[624]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 17.547220] dhcpcd[624]: eth0: adding route to 10.0.2.0/24 lb # [ 17.552662] dhcpcd[624]: eth0: adding default route via 10.0.2.2 lb # [ 17.701246] systemd[1]: Started DHCP Client. lb # [ 17.704172] systemd[1]: Reached target Network is Online. lb # [ 17.708792] systemd[1]: Started Envoy reverse proxy. lb # [ 17.713420] systemd[1]: Started mock-oidc.service. worker2 # [ 17.882780] dhcpcd[637]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 17.886076] dhcpcd[637]: eth0: adding route to 10.0.2.0/24 worker2 # [ 17.891041] dhcpcd[637]: eth0: adding default route via 10.0.2.2 lb # [ 17.880035] mock-oidc-server[894]: Mock OIDC Server running lb # [ 17.880911] mock-oidc-server[894]: OIDC Address: 192.168.1.2:8080 lb # [ 17.881731] mock-oidc-server[894]: Issue Address: 0.0.0.0:8081 lb # [ 17.882519] mock-oidc-server[894]: Issuer: http://192.168.1.2:8080/oidc lb # [ 17.883370] mock-oidc-server[894]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 17.885048] mock-oidc-server[894]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 17.886764] mock-oidc-server[894]: Issue tokens: http://0.0.0.0:8081/issue?sub=... worker2 # [ 18.024374] systemd[1]: Started DHCP Client. worker2 # [ 18.026808] systemd[1]: Reached target Multi-User System. worker2 # [ 18.027785] systemd[1]: Startup finished in 980ms (kernel) + 4.961s (initrd) + 12.083s (userspace) = 18.026s. lb # [ 18.095275] envoy[893]: [2026-09-22 11:31:52.686][893][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120) lb # [ 18.098187] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 18.099703] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 18.102629] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 18.155304] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 18.164828] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 18.167362] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 18.170100] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 18.172740] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 18.175296] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 18.179862] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 18.183067] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 18.185748] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 18.188584] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 18.190734] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 18.194842] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 18.197294] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 18.199885] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 18.201685] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 18.203523] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 18.206840] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 18.208685] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 18.210233] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 18.214891] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 18.216876] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder lb # [ 18.219060] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 18.221067] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 18.223403] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 18.225948] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 18.228470] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 18.232934] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 18.236606] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 18.238941] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 18.243328] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 18.245721] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 18.250711] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 18.254314] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 18.256944] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 18.259853] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 18.267229] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 18.269386] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 18.271058] envoy[893]: [2026-09-22 11:31:52.688][893][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 18.273324] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 18.275012] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 18.277216] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 18.280821] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 18.283660] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 18.286349] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 18.289011] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 18.293997] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 18.295820] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 18.302619] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 18.304866] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 18.307987] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 18.312027] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 18.314637] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 18.317670] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 18.319639] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 18.322174] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 18.326640] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher worker2 # [ 18.378397] systemd[1]: Reached target Network is Online. lb # [ 18.329483] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 18.332284] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 18.333983] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 18.335646] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory worker1 # [ 18.382227] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 18.389166] systemd[1]: Starting scheduler-token.service... lb # [ 18.339456] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 18.343167] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 18.345380] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 18.348419] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 18.350144] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 18.353484] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 18.355270] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 18.362673] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 18.369708] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 18.372329] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 18.377318] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 18.378935] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 18.381138] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 18.383162] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 18.385048] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 18.387017] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 18.397822] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 18.400435] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 18.404739] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 18.409931] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 18.412827] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 18.415495] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 18.417793] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 18.420130] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 18.429012] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 18.430840] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 18.436770] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 18.445335] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header worker2 # [ 18.495207] systemd[1]: scheduler-token.service: Deactivated successfully. lb # [ 18.448140] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy worker2 # [ 18.501601] systemd[1]: Finished scheduler-token.service. worker2 # [ 18.503832] systemd[1]: scheduler-token.service: Consumed 64ms CPU time over 110ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb # [ 18.450764] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 18.466412] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 18.468994] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 18.470783] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 18.474777] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche worker2 # [ 18.528544] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 18.480179] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 18.481839] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 18.483711] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default lb # [ 18.488787] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 18.490451] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 18.492602] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 18.495294] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd lb # [ 18.500744] envoy[893]: [2026-09-22 11:31:52.689][893][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 18.502432] envoy[893]: [2026-09-22 11:31:52.730][893][info][main] [source/server/server.cc:531] HTTP header map info: lb # [ 18.503757] envoy[893]: [2026-09-22 11:31:52.730][893][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 18.519398] envoy[893]: [2026-09-22 11:31:52.730][893][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: worker1 # [ 18.569849] nix-grpc-daemon[768]: ts=2026-09-22T11:31:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" lb # [ 18.521090] envoy[893]: [2026-09-22 11:31:52.730][893][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 18.530067] envoy[893]: [2026-09-22 11:31:52.730][893][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 18.531800] envoy[893]: [2026-09-22 11:31:52.763][893][info][main] [source/server/server.cc:968] runtime: {} lb # [ 18.533174] envoy[893]: [2026-09-22 11:31:52.770][893][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. worker1 # [ 18.578270] nix-grpc-daemon[768]: ts=2026-09-22T11:31:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc lb # [ 18.534824] envoy[893]: [2026-09-22 11:31:52.770][893][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 18.536562] envoy[893]: [2026-09-22 11:31:52.771][893][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 worker1 # [ 18.580520] nix-grpc-daemon[768]: ts=2026-09-22T11:31:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 lb # [ 18.538059] envoy[893]: [2026-09-22 11:31:52.771][893][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 18.539613] envoy[893]: [2026-09-22 11:31:52.771][893][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 18.542469] envoy[893]: [2026-09-22 11:31:52.771][893][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) lb # [ 18.544184] envoy[893]: [2026-09-22 11:31:52.917][893][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) lb # [ 18.545935] envoy[893]: [2026-09-22 11:31:52.933][893][info][config] [source/server/configuration_impl.cc:164] loading stats configuration lb # [ 18.547486] envoy[893]: [2026-09-22 11:31:52.934][893][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. worker1 # [ 18.595154] nix-grpc-daemon[768]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 18.551031] envoy[893]: [2026-09-22 11:31:52.934][893][info][main] [source/server/server.cc:1077] starting main dispatch loop worker1 # [ 18.598144] nix-grpc-daemon[768]: E0000 00:00:1790076713.190330 768 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 18.603018] nix-grpc-daemon[768]: E0000 00:00:1790076713.198707 768 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 18.624372] nix-grpc-daemon[768]: ts=2026-09-22T11:31:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 18.626684] systemd[1]: Started Nix worker-protocol over gRPC. lb # [ 18.585738] envoy[893]: [2026-09-22 11:31:53.177][893][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization lb # [ 18.587459] envoy[893]: [2026-09-22 11:31:53.177][893][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized worker1 # [ 18.631924] nix-grpc-daemon[768]: E0000 00:00:1790076713.226279 809 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 18.591115] envoy[893]: [2026-09-22 11:31:53.177][893][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager lb # [ 18.595586] envoy[893]: [2026-09-22 11:31:53.186][893][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers worker1 # [ 18.664057] nix-grpc-daemon[768]: E0000 00:00:1790076713.245683 809 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.671960] nix-grpc-daemon[771]: ts=2026-09-22T11:31:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 18.669476] nix-grpc-daemon[768]: E0000 00:00:1790076713.246147 809 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 18.672459] nix-grpc-daemon[768]: E0000 00:00:1790076713.264179 809 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.681476] nix-grpc-daemon[771]: ts=2026-09-22T11:31:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 18.680210] nix-grpc-daemon[768]: ts=2026-09-22T11:31:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.684563] nix-grpc-daemon[771]: ts=2026-09-22T11:31:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 18.681926] nix-grpc-daemon[768]: ts=2026-09-22T11:31:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 18.698076] nix-grpc-daemon[771]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 18.701057] nix-grpc-daemon[771]: E0000 00:00:1790076713.284756 771 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.706132] nix-grpc-daemon[771]: E0000 00:00:1790076713.293001 771 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.724972] nix-grpc-daemon[771]: ts=2026-09-22T11:31:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 18.728046] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 18.733676] nix-grpc-daemon[771]: E0000 00:00:1790076713.319563 813 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.760054] nix-grpc-daemon[771]: E0000 00:00:1790076713.338200 813 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.770352] nix-grpc-daemon[771]: E0000 00:00:1790076713.338718 813 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.773554] nix-grpc-daemon[771]: E0000 00:00:1790076713.356705 813 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.781313] nix-grpc-daemon[771]: ts=2026-09-22T11:31:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.783128] nix-grpc-daemon[771]: ts=2026-09-22T11:31:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 18.883025] nix-grpc-daemon[768]: ts=2026-09-22T11:31:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 18.888297] nix-grpc-daemon[768]: ts=2026-09-22T11:31:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 18.984662] nix-grpc-daemon[771]: ts=2026-09-22T11:31:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.989755] nix-grpc-daemon[771]: ts=2026-09-22T11:31:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.088975] nix-grpc-daemon[768]: ts=2026-09-22T11:31:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.093999] nix-grpc-daemon[768]: ts=2026-09-22T11:31:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.190948] nix-grpc-daemon[771]: ts=2026-09-22T11:31:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.196305] nix-grpc-daemon[771]: ts=2026-09-22T11:31:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.294912] nix-grpc-daemon[768]: ts=2026-09-22T11:31:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.300085] nix-grpc-daemon[768]: ts=2026-09-22T11:31:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.397418] nix-grpc-daemon[771]: ts=2026-09-22T11:31:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.402660] nix-grpc-daemon[771]: ts=2026-09-22T11:31:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.500947] nix-grpc-daemon[768]: ts=2026-09-22T11:31:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.506026] nix-grpc-daemon[768]: ts=2026-09-22T11:31:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.603910] nix-grpc-daemon[771]: ts=2026-09-22T11:31:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.609218] nix-grpc-daemon[771]: ts=2026-09-22T11:31:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.708654] nix-grpc-daemon[768]: ts=2026-09-22T11:31:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.712686] nix-grpc-daemon[768]: ts=2026-09-22T11:31:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.812280] nix-grpc-daemon[771]: ts=2026-09-22T11:31:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.816431] nix-grpc-daemon[771]: ts=2026-09-22T11:31:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.909292] nix-grpc-daemon[768]: ts=2026-09-22T11:31:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.914414] nix-grpc-daemon[768]: ts=2026-09-22T11:31:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.013167] nix-grpc-daemon[771]: ts=2026-09-22T11:31:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.018507] nix-grpc-daemon[771]: ts=2026-09-22T11:31:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.115562] nix-grpc-daemon[768]: ts=2026-09-22T11:31:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.121024] nix-grpc-daemon[768]: ts=2026-09-22T11:31:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.219974] nix-grpc-daemon[771]: ts=2026-09-22T11:31:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.225379] nix-grpc-daemon[771]: ts=2026-09-22T11:31:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.321827] nix-grpc-daemon[768]: ts=2026-09-22T11:31:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.327169] nix-grpc-daemon[768]: ts=2026-09-22T11:31:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.426973] nix-grpc-daemon[771]: ts=2026-09-22T11:31:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.432902] nix-grpc-daemon[771]: ts=2026-09-22T11:31:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.528441] nix-grpc-daemon[768]: ts=2026-09-22T11:31:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.533874] nix-grpc-daemon[768]: ts=2026-09-22T11:31:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.634252] nix-grpc-daemon[771]: ts=2026-09-22T11:31:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.640266] nix-grpc-daemon[771]: ts=2026-09-22T11:31:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.736875] nix-grpc-daemon[768]: ts=2026-09-22T11:31:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.741147] nix-grpc-daemon[768]: ts=2026-09-22T11:31:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.842833] nix-grpc-daemon[771]: ts=2026-09-22T11:31:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.848504] nix-grpc-daemon[771]: ts=2026-09-22T11:31:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.937497] nix-grpc-daemon[768]: ts=2026-09-22T11:31:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.942969] nix-grpc-daemon[768]: ts=2026-09-22T11:31:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.049951] nix-grpc-daemon[771]: ts=2026-09-22T11:31:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.055442] nix-grpc-daemon[771]: ts=2026-09-22T11:31:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.143851] nix-grpc-daemon[768]: ts=2026-09-22T11:31:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.149215] nix-grpc-daemon[768]: ts=2026-09-22T11:31:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.256617] nix-grpc-daemon[771]: ts=2026-09-22T11:31:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.261795] nix-grpc-daemon[771]: ts=2026-09-22T11:31:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.350036] nix-grpc-daemon[768]: ts=2026-09-22T11:31:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.355265] nix-grpc-daemon[768]: ts=2026-09-22T11:31:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.463028] nix-grpc-daemon[771]: ts=2026-09-22T11:31:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.468347] nix-grpc-daemon[771]: ts=2026-09-22T11:31:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.556112] nix-grpc-daemon[768]: ts=2026-09-22T11:31:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.561317] nix-grpc-daemon[768]: ts=2026-09-22T11:31:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.669720] nix-grpc-daemon[771]: ts=2026-09-22T11:31:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.675051] nix-grpc-daemon[771]: ts=2026-09-22T11:31:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.763910] nix-grpc-daemon[768]: ts=2026-09-22T11:31:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.768127] nix-grpc-daemon[768]: ts=2026-09-22T11:31:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.877994] nix-grpc-daemon[771]: ts=2026-09-22T11:31:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.882130] nix-grpc-daemon[771]: ts=2026-09-22T11:31:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.964233] nix-grpc-daemon[768]: ts=2026-09-22T11:31:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.969240] nix-grpc-daemon[768]: ts=2026-09-22T11:31:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.078838] nix-grpc-daemon[771]: ts=2026-09-22T11:31:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.084005] nix-grpc-daemon[771]: ts=2026-09-22T11:31:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.170163] nix-grpc-daemon[768]: ts=2026-09-22T11:31:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.175332] nix-grpc-daemon[768]: ts=2026-09-22T11:31:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.285172] nix-grpc-daemon[771]: ts=2026-09-22T11:31:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.290252] nix-grpc-daemon[771]: ts=2026-09-22T11:31:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.376160] nix-grpc-daemon[768]: ts=2026-09-22T11:31:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.381131] nix-grpc-daemon[768]: ts=2026-09-22T11:31:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.491416] nix-grpc-daemon[771]: ts=2026-09-22T11:31:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.496753] nix-grpc-daemon[771]: ts=2026-09-22T11:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.582101] nix-grpc-daemon[768]: ts=2026-09-22T11:31:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.587213] nix-grpc-daemon[768]: ts=2026-09-22T11:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.697817] nix-grpc-daemon[771]: ts=2026-09-22T11:31:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.703067] nix-grpc-daemon[771]: ts=2026-09-22T11:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.789661] nix-grpc-daemon[768]: ts=2026-09-22T11:31:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.793718] nix-grpc-daemon[768]: ts=2026-09-22T11:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.905959] nix-grpc-daemon[771]: ts=2026-09-22T11:31:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.910085] nix-grpc-daemon[771]: ts=2026-09-22T11:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.990120] nix-grpc-daemon[768]: ts=2026-09-22T11:31:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.995215] nix-grpc-daemon[768]: ts=2026-09-22T11:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.106805] nix-grpc-daemon[771]: ts=2026-09-22T11:31:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.111895] nix-grpc-daemon[771]: ts=2026-09-22T11:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.196088] nix-grpc-daemon[768]: ts=2026-09-22T11:31:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.201149] nix-grpc-daemon[768]: ts=2026-09-22T11:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.312975] nix-grpc-daemon[771]: ts=2026-09-22T11:31:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.317969] nix-grpc-daemon[771]: ts=2026-09-22T11:31:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.401931] nix-grpc-daemon[768]: ts=2026-09-22T11:31:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.407037] nix-grpc-daemon[768]: ts=2026-09-22T11:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.519065] nix-grpc-daemon[771]: ts=2026-09-22T11:31:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.524965] nix-grpc-daemon[771]: ts=2026-09-22T11:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.607811] nix-grpc-daemon[768]: ts=2026-09-22T11:31:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.613615] nix-grpc-daemon[768]: ts=2026-09-22T11:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.725897] nix-grpc-daemon[771]: ts=2026-09-22T11:31:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.731005] nix-grpc-daemon[771]: ts=2026-09-22T11:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.816221] nix-grpc-daemon[768]: ts=2026-09-22T11:31:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.820241] nix-grpc-daemon[768]: ts=2026-09-22T11:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.933749] nix-grpc-daemon[771]: ts=2026-09-22T11:31:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.937896] nix-grpc-daemon[771]: ts=2026-09-22T11:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.016535] nix-grpc-daemon[768]: ts=2026-09-22T11:31:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.021542] nix-grpc-daemon[768]: ts=2026-09-22T11:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.134621] nix-grpc-daemon[771]: ts=2026-09-22T11:31:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.139674] nix-grpc-daemon[771]: ts=2026-09-22T11:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.222595] nix-grpc-daemon[768]: ts=2026-09-22T11:31:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.227986] nix-grpc-daemon[768]: ts=2026-09-22T11:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.340976] nix-grpc-daemon[771]: ts=2026-09-22T11:31:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.346003] nix-grpc-daemon[771]: ts=2026-09-22T11:31:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.428966] nix-grpc-daemon[768]: ts=2026-09-22T11:31:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.434376] nix-grpc-daemon[768]: ts=2026-09-22T11:31:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.547136] nix-grpc-daemon[771]: ts=2026-09-22T11:31:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.552397] nix-grpc-daemon[771]: ts=2026-09-22T11:31:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.635280] nix-grpc-daemon[768]: ts=2026-09-22T11:31:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.641965] nix-grpc-daemon[768]: ts=2026-09-22T11:31:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.753222] nix-grpc-daemon[771]: ts=2026-09-22T11:31:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.758252] nix-grpc-daemon[771]: ts=2026-09-22T11:31:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.843154] nix-grpc-daemon[768]: ts=2026-09-22T11:31:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.847511] nix-grpc-daemon[768]: ts=2026-09-22T11:31:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.960919] nix-grpc-daemon[771]: ts=2026-09-22T11:31:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.965560] nix-grpc-daemon[771]: ts=2026-09-22T11:31:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.043722] nix-grpc-daemon[768]: ts=2026-09-22T11:31:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.049075] nix-grpc-daemon[768]: ts=2026-09-22T11:31:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.166803] nix-grpc-daemon[771]: ts=2026-09-22T11:31:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.171973] nix-grpc-daemon[771]: ts=2026-09-22T11:31:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.249880] nix-grpc-daemon[768]: ts=2026-09-22T11:31:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.254994] nix-grpc-daemon[768]: ts=2026-09-22T11:31:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.373006] nix-grpc-daemon[771]: ts=2026-09-22T11:31:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.378019] nix-grpc-daemon[771]: ts=2026-09-22T11:31:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.455939] nix-grpc-daemon[768]: ts=2026-09-22T11:32:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.461023] nix-grpc-daemon[768]: ts=2026-09-22T11:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.579132] nix-grpc-daemon[771]: ts=2026-09-22T11:32:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.584365] nix-grpc-daemon[771]: ts=2026-09-22T11:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.661627] nix-grpc-daemon[768]: ts=2026-09-22T11:32:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.666732] nix-grpc-daemon[768]: ts=2026-09-22T11:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.785151] nix-grpc-daemon[771]: ts=2026-09-22T11:32:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.790249] nix-grpc-daemon[771]: ts=2026-09-22T11:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.869544] nix-grpc-daemon[768]: ts=2026-09-22T11:32:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.873903] nix-grpc-daemon[768]: ts=2026-09-22T11:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.992809] nix-grpc-daemon[771]: ts=2026-09-22T11:32:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.997377] nix-grpc-daemon[771]: ts=2026-09-22T11:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.070107] nix-grpc-daemon[768]: ts=2026-09-22T11:32:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.075308] nix-grpc-daemon[768]: ts=2026-09-22T11:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.198558] nix-grpc-daemon[771]: ts=2026-09-22T11:32:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.203705] nix-grpc-daemon[771]: ts=2026-09-22T11:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.276260] nix-grpc-daemon[768]: ts=2026-09-22T11:32:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.281263] nix-grpc-daemon[768]: ts=2026-09-22T11:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.404829] nix-grpc-daemon[771]: ts=2026-09-22T11:32:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.409972] nix-grpc-daemon[771]: ts=2026-09-22T11:32:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.482074] nix-grpc-daemon[768]: ts=2026-09-22T11:32:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.487197] nix-grpc-daemon[768]: ts=2026-09-22T11:32:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.611109] nix-grpc-daemon[771]: ts=2026-09-22T11:32:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.616270] nix-grpc-daemon[771]: ts=2026-09-22T11:32:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.688260] nix-grpc-daemon[768]: ts=2026-09-22T11:32:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.693343] nix-grpc-daemon[768]: ts=2026-09-22T11:32:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.817184] nix-grpc-daemon[771]: ts=2026-09-22T11:32:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.822208] nix-grpc-daemon[771]: ts=2026-09-22T11:32:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.895790] nix-grpc-daemon[768]: ts=2026-09-22T11:32:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.900102] nix-grpc-daemon[768]: ts=2026-09-22T11:32:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.024785] nix-grpc-daemon[771]: ts=2026-09-22T11:32:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.029327] nix-grpc-daemon[771]: ts=2026-09-22T11:32:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.096439] nix-grpc-daemon[768]: ts=2026-09-22T11:32:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.101560] nix-grpc-daemon[768]: ts=2026-09-22T11:32:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.230478] nix-grpc-daemon[771]: ts=2026-09-22T11:32:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.235711] nix-grpc-daemon[771]: ts=2026-09-22T11:32:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.302467] nix-grpc-daemon[768]: ts=2026-09-22T11:32:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.307635] nix-grpc-daemon[768]: ts=2026-09-22T11:32:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.436771] nix-grpc-daemon[771]: ts=2026-09-22T11:32:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.441846] nix-grpc-daemon[771]: ts=2026-09-22T11:32:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.508841] nix-grpc-daemon[768]: ts=2026-09-22T11:32:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.513837] nix-grpc-daemon[768]: ts=2026-09-22T11:32:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.643131] nix-grpc-daemon[771]: ts=2026-09-22T11:32:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.648394] nix-grpc-daemon[771]: ts=2026-09-22T11:32:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.714481] nix-grpc-daemon[768]: ts=2026-09-22T11:32:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.719743] nix-grpc-daemon[768]: ts=2026-09-22T11:32:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.849300] nix-grpc-daemon[771]: ts=2026-09-22T11:32:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.854180] nix-grpc-daemon[771]: ts=2026-09-22T11:32:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.923162] nix-grpc-daemon[768]: ts=2026-09-22T11:32:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.930910] nix-grpc-daemon[768]: ts=2026-09-22T11:32:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.056735] nix-grpc-daemon[771]: ts=2026-09-22T11:32:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.061267] nix-grpc-daemon[771]: ts=2026-09-22T11:32:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.123158] nix-grpc-daemon[768]: ts=2026-09-22T11:32:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.128682] nix-grpc-daemon[768]: ts=2026-09-22T11:32:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.262515] nix-grpc-daemon[771]: ts=2026-09-22T11:32:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.267733] nix-grpc-daemon[771]: ts=2026-09-22T11:32:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.329815] nix-grpc-daemon[768]: ts=2026-09-22T11:32:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.335089] nix-grpc-daemon[768]: ts=2026-09-22T11:32:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.468816] nix-grpc-daemon[771]: ts=2026-09-22T11:32:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.473868] nix-grpc-daemon[771]: ts=2026-09-22T11:32:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.536125] nix-grpc-daemon[768]: ts=2026-09-22T11:32:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.541487] nix-grpc-daemon[768]: ts=2026-09-22T11:32:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.675110] nix-grpc-daemon[771]: ts=2026-09-22T11:32:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.680666] nix-grpc-daemon[771]: ts=2026-09-22T11:32:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.742556] nix-grpc-daemon[768]: ts=2026-09-22T11:32:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.748175] nix-grpc-daemon[768]: ts=2026-09-22T11:32:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.881804] nix-grpc-daemon[771]: ts=2026-09-22T11:32:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.887371] nix-grpc-daemon[771]: ts=2026-09-22T11:32:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.950993] nix-grpc-daemon[768]: ts=2026-09-22T11:32:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.955293] nix-grpc-daemon[768]: ts=2026-09-22T11:32:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.090443] nix-grpc-daemon[771]: ts=2026-09-22T11:32:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.094300] nix-grpc-daemon[771]: ts=2026-09-22T11:32:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.151786] nix-grpc-daemon[768]: ts=2026-09-22T11:32:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.157290] nix-grpc-daemon[768]: ts=2026-09-22T11:32:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.291386] nix-grpc-daemon[771]: ts=2026-09-22T11:32:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.296950] nix-grpc-daemon[771]: ts=2026-09-22T11:32:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.358061] nix-grpc-daemon[768]: ts=2026-09-22T11:32:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.363097] nix-grpc-daemon[768]: ts=2026-09-22T11:32:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.497880] nix-grpc-daemon[771]: ts=2026-09-22T11:32:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.502995] nix-grpc-daemon[771]: ts=2026-09-22T11:32:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.564182] nix-grpc-daemon[768]: ts=2026-09-22T11:32:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.569273] nix-grpc-daemon[768]: ts=2026-09-22T11:32:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.704093] nix-grpc-daemon[771]: ts=2026-09-22T11:32:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.709133] nix-grpc-daemon[771]: ts=2026-09-22T11:32:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.769875] nix-grpc-daemon[768]: ts=2026-09-22T11:32:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.775087] nix-grpc-daemon[768]: ts=2026-09-22T11:32:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.909851] nix-grpc-daemon[771]: ts=2026-09-22T11:32:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.914869] nix-grpc-daemon[771]: ts=2026-09-22T11:32:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.977407] nix-grpc-daemon[768]: ts=2026-09-22T11:32:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.982172] nix-grpc-daemon[768]: ts=2026-09-22T11:32:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.117957] nix-grpc-daemon[771]: ts=2026-09-22T11:32:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.122122] nix-grpc-daemon[771]: ts=2026-09-22T11:32:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.183035] nix-grpc-daemon[768]: ts=2026-09-22T11:32:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.188307] nix-grpc-daemon[768]: ts=2026-09-22T11:32:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.318691] nix-grpc-daemon[771]: ts=2026-09-22T11:32:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.323848] nix-grpc-daemon[771]: ts=2026-09-22T11:32:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.389084] nix-grpc-daemon[768]: ts=2026-09-22T11:32:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.394274] nix-grpc-daemon[768]: ts=2026-09-22T11:32:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.524952] nix-grpc-daemon[771]: ts=2026-09-22T11:32:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.529836] nix-grpc-daemon[771]: ts=2026-09-22T11:32:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.595217] nix-grpc-daemon[768]: ts=2026-09-22T11:32:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.600626] nix-grpc-daemon[768]: ts=2026-09-22T11:32:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.731068] nix-grpc-daemon[771]: ts=2026-09-22T11:32:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.736410] nix-grpc-daemon[771]: ts=2026-09-22T11:32:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.801503] nix-grpc-daemon[768]: ts=2026-09-22T11:32:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.806409] nix-grpc-daemon[768]: ts=2026-09-22T11:32:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.937426] nix-grpc-daemon[771]: ts=2026-09-22T11:32:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.942382] nix-grpc-daemon[771]: ts=2026-09-22T11:32:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.008999] nix-grpc-daemon[768]: ts=2026-09-22T11:32:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.012943] nix-grpc-daemon[768]: ts=2026-09-22T11:32:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.144856] nix-grpc-daemon[771]: ts=2026-09-22T11:32:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.149411] nix-grpc-daemon[771]: ts=2026-09-22T11:32:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.211011] nix-grpc-daemon[768]: ts=2026-09-22T11:32:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.217925] nix-grpc-daemon[768]: ts=2026-09-22T11:32:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.350700] nix-grpc-daemon[771]: ts=2026-09-22T11:32:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.355663] nix-grpc-daemon[771]: ts=2026-09-22T11:32:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.411957] nix-grpc-daemon[768]: ts=2026-09-22T11:32:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.417032] nix-grpc-daemon[768]: ts=2026-09-22T11:32:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.556605] nix-grpc-daemon[771]: ts=2026-09-22T11:32:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.560506] nix-grpc-daemon[771]: ts=2026-09-22T11:32:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.617729] nix-grpc-daemon[768]: ts=2026-09-22T11:32:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.622830] nix-grpc-daemon[768]: ts=2026-09-22T11:32:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.761387] nix-grpc-daemon[771]: ts=2026-09-22T11:32:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.766378] nix-grpc-daemon[771]: ts=2026-09-22T11:32:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 31.765653] rustfs-bucket-start[1022]: mb s3://farm lb # [ 31.780535] systemd[1]: Finished rustfs-bucket.service. worker1 # [ 31.823513] nix-grpc-daemon[768]: ts=2026-09-22T11:32:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.828586] nix-grpc-daemon[768]: ts=2026-09-22T11:32:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 31.795891] systemd[1]: Starting niks3 server... worker2 # [ 31.967560] nix-grpc-daemon[771]: ts=2026-09-22T11:32:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.972834] nix-grpc-daemon[771]: ts=2026-09-22T11:32:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 31.963899] postgres[1042]: [1042] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 31.968162] postgres[1042]: [1042] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC worker1 # [ 32.031811] nix-grpc-daemon[768]: ts=2026-09-22T11:32:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.036072] nix-grpc-daemon[768]: ts=2026-09-22T11:32:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 32.001796] niks3-server[1033]: 2026/09/22 11:32:06 OK 20241026095416_initial_model.sql (22.17ms) lb # [ 32.006585] niks3-server[1033]: 2026/09/22 11:32:06 OK 20251210153512_drop_unused_gin_index.sql (3.58ms) lb # [ 32.014922] niks3-server[1033]: 2026/09/22 11:32:06 OK 20251218171726_add_pins.sql (10.66ms) lb # [ 32.022814] niks3-server[1033]: 2026/09/22 11:32:06 OK 20260628120000_add_object_size_and_stats.sql (7.9ms) lb # [ 32.030139] niks3-server[1033]: 2026/09/22 11:32:06 OK 20260905000000_add_claims.sql (7.29ms) lb # [ 32.035381] niks3-server[1033]: 2026/09/22 11:32:06 OK 20260920000000_drop_claims.sql (4.87ms) lb # [ 32.040160] niks3-server[1033]: 2026/09/22 11:32:06 goose: successfully migrated database to version: 20260920000000 lb # [ 32.043273] niks3-server[1033]: 2026/09/22 11:32:06 OK 1_commit_pending_closure.sql (7.97ms) lb # [ 32.045893] niks3-server[1033]: 2026/09/22 11:32:06 OK 2_object_stats_trigger.sql (2.59ms) lb # [ 32.047886] niks3-server[1033]: 2026/09/22 11:32:06 goose: up to current file version: 2 lb # [ 32.050508] niks3-server[1033]: 2026/09/22 11:32:06 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 32.079557] niks3-server[1033]: 2026/09/22 11:32:06 INFO Read proxy enabled — serving cache objects from S3 lb # [ 32.086522] niks3-server[1033]: 2026/09/22 11:32:06 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 32.087804] niks3-server[1033]: 2026/09/22 11:32:06 INFO systemd watchdog enabled interval=15s lb # [ 32.093113] niks3-server[1033]: 2026/09/22 11:32:06 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 32.094512] systemd[1]: Started niks3 server. lb # [ 32.095762] systemd[1]: Reached target Multi-User System. lb # [ 32.096762] systemd[1]: Startup finished in 978ms (kernel) + 5.020s (initrd) + 26.093s (userspace) = 32.091s. worker2 # [ 32.175849] nix-grpc-daemon[771]: ts=2026-09-22T11:32:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.180108] nix-grpc-daemon[771]: ts=2026-09-22T11:32:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.231849] nix-grpc-daemon[768]: ts=2026-09-22T11:32:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.236798] nix-grpc-daemon[768]: ts=2026-09-22T11:32:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.376316] nix-grpc-daemon[771]: ts=2026-09-22T11:32:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.381088] nix-grpc-daemon[771]: ts=2026-09-22T11:32:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.437717] nix-grpc-daemon[768]: ts=2026-09-22T11:32:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.442942] nix-grpc-daemon[768]: ts=2026-09-22T11:32:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.582160] nix-grpc-daemon[771]: ts=2026-09-22T11:32:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.587398] nix-grpc-daemon[771]: ts=2026-09-22T11:32:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.643656] nix-grpc-daemon[768]: ts=2026-09-22T11:32:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.648946] nix-grpc-daemon[768]: ts=2026-09-22T11:32:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.788184] nix-grpc-daemon[771]: ts=2026-09-22T11:32:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.793188] nix-grpc-daemon[771]: ts=2026-09-22T11:32:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.849659] nix-grpc-daemon[768]: ts=2026-09-22T11:32:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.854734] nix-grpc-daemon[768]: ts=2026-09-22T11:32:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.994153] nix-grpc-daemon[771]: ts=2026-09-22T11:32:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.999186] nix-grpc-daemon[771]: ts=2026-09-22T11:32:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: waiting for unit niks3.service, in 20.29 seconds) lb: waiting for TCP port 5751 on localhost worker1 # [ 33.056790] nix-grpc-daemon[768]: ts=2026-09-22T11:32:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.061337] nix-grpc-daemon[768]: ts=2026-09-22T11:32:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.05 seconds) lb: waiting for TCP port 50051 on localhost lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 33.153455] envoy[893]: conn peer=[::1]:51856 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=4 worker2 # [ 33.201929] nix-grpc-daemon[771]: ts=2026-09-22T11:32:07Z level=info event=scheduler_connected addr=lb:50051 lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds) worker2 # [ 33.205935] nix-grpc-daemon[771]: ts=2026-09-22T11:32:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.263461] nix-grpc-daemon[768]: ts=2026-09-22T11:32:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.273426] nix-grpc-daemon[768]: ts=2026-09-22T11:32:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.370390] systemd[1]: Starting scheduler-token.service... worker2 # [ 33.402568] nix-grpc-daemon[771]: ts=2026-09-22T11:32:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.405729] nix-grpc-daemon[771]: ts=2026-09-22T11:32:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.445254] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 33.448653] systemd[1]: Finished scheduler-token.service. worker2 # [ 33.449992] systemd[1]: scheduler-token.service: Consumed 42ms CPU time over 78ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 33.464404] nix-grpc-daemon[768]: ts=2026-09-22T11:32:08Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.469643] nix-grpc-daemon[768]: ts=2026-09-22T11:32:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.08 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2 # [ 33.606920] nix-grpc-daemon[771]: ts=2026-09-22T11:32:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.612137] nix-grpc-daemon[771]: ts=2026-09-22T11:32:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.670283] nix-grpc-daemon[768]: ts=2026-09-22T11:32:08Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.675459] nix-grpc-daemon[768]: ts=2026-09-22T11:32:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.812999] nix-grpc-daemon[771]: ts=2026-09-22T11:32:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.817742] nix-grpc-daemon[771]: ts=2026-09-22T11:32:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.876265] nix-grpc-daemon[768]: ts=2026-09-22T11:32:08Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.881282] nix-grpc-daemon[768]: ts=2026-09-22T11:32:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.018662] nix-grpc-daemon[771]: ts=2026-09-22T11:32:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.023837] nix-grpc-daemon[771]: ts=2026-09-22T11:32:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.083896] nix-grpc-daemon[768]: ts=2026-09-22T11:32:08Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.088284] nix-grpc-daemon[768]: ts=2026-09-22T11:32:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.226487] nix-grpc-daemon[771]: ts=2026-09-22T11:32:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.230624] nix-grpc-daemon[771]: ts=2026-09-22T11:32:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.284184] nix-grpc-daemon[768]: ts=2026-09-22T11:32:08Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.289201] nix-grpc-daemon[768]: ts=2026-09-22T11:32:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.427173] nix-grpc-daemon[771]: ts=2026-09-22T11:32:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.432357] nix-grpc-daemon[771]: ts=2026-09-22T11:32:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.490004] nix-grpc-daemon[768]: ts=2026-09-22T11:32:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.495111] nix-grpc-daemon[768]: ts=2026-09-22T11:32:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 34.633395] nix-grpc-daemon[771]: ts=2026-09-22T11:32:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.638409] nix-grpc-daemon[771]: ts=2026-09-22T11:32:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 34.695822] nix-grpc-daemon[768]: ts=2026-09-22T11:32:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.701155] nix-grpc-daemon[768]: ts=2026-09-22T11:32:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.839318] nix-grpc-daemon[771]: ts=2026-09-22T11:32:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.844999] nix-grpc-daemon[771]: ts=2026-09-22T11:32:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.901938] nix-grpc-daemon[768]: ts=2026-09-22T11:32:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.906995] nix-grpc-daemon[768]: ts=2026-09-22T11:32:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.045828] nix-grpc-daemon[771]: ts=2026-09-22T11:32:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.050975] nix-grpc-daemon[771]: ts=2026-09-22T11:32:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.109043] nix-grpc-daemon[768]: ts=2026-09-22T11:32:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.113753] nix-grpc-daemon[768]: ts=2026-09-22T11:32:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.253054] nix-grpc-daemon[771]: ts=2026-09-22T11:32:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.257390] nix-grpc-daemon[771]: ts=2026-09-22T11:32:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.314833] nix-grpc-daemon[768]: ts=2026-09-22T11:32:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.320267] nix-grpc-daemon[768]: ts=2026-09-22T11:32:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.458350] nix-grpc-daemon[771]: ts=2026-09-22T11:32:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.463273] nix-grpc-daemon[771]: ts=2026-09-22T11:32:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.521598] nix-grpc-daemon[768]: ts=2026-09-22T11:32:10Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.527822] nix-grpc-daemon[768]: ts=2026-09-22T11:32:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 35.664269] nix-grpc-daemon[771]: ts=2026-09-22T11:32:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.669240] nix-grpc-daemon[771]: ts=2026-09-22T11:32:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 35.728445] nix-grpc-daemon[768]: ts=2026-09-22T11:32:10Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.733342] nix-grpc-daemon[768]: ts=2026-09-22T11:32:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.870440] nix-grpc-daemon[771]: ts=2026-09-22T11:32:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.875548] nix-grpc-daemon[771]: ts=2026-09-22T11:32:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.934250] nix-grpc-daemon[768]: ts=2026-09-22T11:32:10Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.939344] nix-grpc-daemon[768]: ts=2026-09-22T11:32:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.076709] nix-grpc-daemon[771]: ts=2026-09-22T11:32:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.081928] nix-grpc-daemon[771]: ts=2026-09-22T11:32:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.141983] nix-grpc-daemon[768]: ts=2026-09-22T11:32:10Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.146001] nix-grpc-daemon[768]: ts=2026-09-22T11:32:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.284843] nix-grpc-daemon[771]: ts=2026-09-22T11:32:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.288706] nix-grpc-daemon[771]: ts=2026-09-22T11:32:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.342419] nix-grpc-daemon[768]: ts=2026-09-22T11:32:10Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.347551] nix-grpc-daemon[768]: ts=2026-09-22T11:32:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.485779] nix-grpc-daemon[771]: ts=2026-09-22T11:32:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.490737] nix-grpc-daemon[771]: ts=2026-09-22T11:32:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.548564] nix-grpc-daemon[768]: ts=2026-09-22T11:32:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.553708] nix-grpc-daemon[768]: ts=2026-09-22T11:32:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.691958] nix-grpc-daemon[771]: ts=2026-09-22T11:32:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.697307] nix-grpc-daemon[771]: ts=2026-09-22T11:32:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 36.754416] nix-grpc-daemon[768]: ts=2026-09-22T11:32:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.759628] nix-grpc-daemon[768]: ts=2026-09-22T11:32:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 36.898208] nix-grpc-daemon[771]: ts=2026-09-22T11:32:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.903528] nix-grpc-daemon[771]: ts=2026-09-22T11:32:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.960486] nix-grpc-daemon[768]: ts=2026-09-22T11:32:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.965452] nix-grpc-daemon[768]: ts=2026-09-22T11:32:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.104915] nix-grpc-daemon[771]: ts=2026-09-22T11:32:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.110165] nix-grpc-daemon[771]: ts=2026-09-22T11:32:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.167973] nix-grpc-daemon[768]: ts=2026-09-22T11:32:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.171987] nix-grpc-daemon[768]: ts=2026-09-22T11:32:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.312913] nix-grpc-daemon[771]: ts=2026-09-22T11:32:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.317007] nix-grpc-daemon[771]: ts=2026-09-22T11:32:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.368782] nix-grpc-daemon[768]: ts=2026-09-22T11:32:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.373857] nix-grpc-daemon[768]: ts=2026-09-22T11:32:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.513805] nix-grpc-daemon[771]: ts=2026-09-22T11:32:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.518891] nix-grpc-daemon[771]: ts=2026-09-22T11:32:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.574661] nix-grpc-daemon[768]: ts=2026-09-22T11:32:12Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.579644] nix-grpc-daemon[768]: ts=2026-09-22T11:32:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.721125] nix-grpc-daemon[771]: ts=2026-09-22T11:32:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.725209] nix-grpc-daemon[771]: ts=2026-09-22T11:32:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.780728] nix-grpc-daemon[768]: ts=2026-09-22T11:32:12Z level=info event=scheduler_connected addr=lb:50051 lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 37.786126] nix-grpc-daemon[768]: ts=2026-09-22T11:32:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 37.922561] nix-grpc-daemon[771]: ts=2026-09-22T11:32:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.928123] nix-grpc-daemon[771]: ts=2026-09-22T11:32:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.986982] nix-grpc-daemon[768]: ts=2026-09-22T11:32:12Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.992215] nix-grpc-daemon[768]: ts=2026-09-22T11:32:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.131865] nix-grpc-daemon[771]: ts=2026-09-22T11:32:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.140550] nix-grpc-daemon[771]: ts=2026-09-22T11:32:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.195087] nix-grpc-daemon[768]: ts=2026-09-22T11:32:12Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.199337] nix-grpc-daemon[768]: ts=2026-09-22T11:32:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.339879] nix-grpc-daemon[771]: ts=2026-09-22T11:32:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.344529] nix-grpc-daemon[771]: ts=2026-09-22T11:32:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.395562] nix-grpc-daemon[768]: ts=2026-09-22T11:32:12Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.400956] nix-grpc-daemon[768]: ts=2026-09-22T11:32:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.540541] nix-grpc-daemon[771]: ts=2026-09-22T11:32:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.545335] nix-grpc-daemon[771]: ts=2026-09-22T11:32:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.601922] nix-grpc-daemon[768]: ts=2026-09-22T11:32:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.607528] nix-grpc-daemon[768]: ts=2026-09-22T11:32:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.746443] nix-grpc-daemon[771]: ts=2026-09-22T11:32:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.751623] nix-grpc-daemon[771]: ts=2026-09-22T11:32:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.807914] nix-grpc-daemon[768]: ts=2026-09-22T11:32:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.813096] nix-grpc-daemon[768]: ts=2026-09-22T11:32:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 38.952702] nix-grpc-daemon[771]: ts=2026-09-22T11:32:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.957763] nix-grpc-daemon[771]: ts=2026-09-22T11:32:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 39.013793] nix-grpc-daemon[768]: ts=2026-09-22T11:32:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 39.018935] nix-grpc-daemon[768]: ts=2026-09-22T11:32:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 39.158901] nix-grpc-daemon[771]: ts=2026-09-22T11:32:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 39.164110] nix-grpc-daemon[771]: ts=2026-09-22T11:32:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 39.221622] nix-grpc-daemon[768]: ts=2026-09-22T11:32:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 39.225735] nix-grpc-daemon[768]: ts=2026-09-22T11:32:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 39.366727] nix-grpc-daemon[771]: ts=2026-09-22T11:32:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 39.370611] nix-grpc-daemon[771]: ts=2026-09-22T11:32:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 39.422123] nix-grpc-daemon[768]: ts=2026-09-22T11:32:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 39.427400] nix-grpc-daemon[768]: ts=2026-09-22T11:32:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 39.567595] nix-grpc-daemon[771]: ts=2026-09-22T11:32:14Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 39.573012] nix-grpc-daemon[771]: ts=2026-09-22T11:32:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 39.627995] nix-grpc-daemon[768]: ts=2026-09-22T11:32:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 39.632998] nix-grpc-daemon[768]: ts=2026-09-22T11:32:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 39.773597] nix-grpc-daemon[771]: ts=2026-09-22T11:32:14Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 39.778744] nix-grpc-daemon[771]: ts=2026-09-22T11:32:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 39.833728] nix-grpc-daemon[768]: ts=2026-09-22T11:32:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 39.838573] nix-grpc-daemon[768]: ts=2026-09-22T11:32:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 39.979683] nix-grpc-daemon[771]: ts=2026-09-22T11:32:14Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 39.984917] nix-grpc-daemon[771]: ts=2026-09-22T11:32:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 40.039129] nix-grpc-daemon[768]: ts=2026-09-22T11:32:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 40.043886] nix-grpc-daemon[768]: ts=2026-09-22T11:32:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 40.185918] nix-grpc-daemon[771]: ts=2026-09-22T11:32:14Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 40.190983] nix-grpc-daemon[771]: ts=2026-09-22T11:32:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 40.246382] nix-grpc-daemon[768]: ts=2026-09-22T11:32:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 40.250581] nix-grpc-daemon[768]: ts=2026-09-22T11:32:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 40.393812] nix-grpc-daemon[771]: ts=2026-09-22T11:32:14Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 40.397906] nix-grpc-daemon[771]: ts=2026-09-22T11:32:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 40.447036] nix-grpc-daemon[768]: ts=2026-09-22T11:32:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 40.452422] nix-grpc-daemon[768]: ts=2026-09-22T11:32:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 40.594611] nix-grpc-daemon[771]: ts=2026-09-22T11:32:15Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 40.599875] nix-grpc-daemon[771]: ts=2026-09-22T11:32:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 40.653445] nix-grpc-daemon[768]: ts=2026-09-22T11:32:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 40.659006] nix-grpc-daemon[768]: ts=2026-09-22T11:32:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 40.800667] nix-grpc-daemon[771]: ts=2026-09-22T11:32:15Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 40.805484] nix-grpc-daemon[771]: ts=2026-09-22T11:32:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 40.859620] nix-grpc-daemon[768]: ts=2026-09-22T11:32:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 40.864618] nix-grpc-daemon[768]: ts=2026-09-22T11:32:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 41.006464] nix-grpc-daemon[771]: ts=2026-09-22T11:32:15Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 41.011580] nix-grpc-daemon[771]: ts=2026-09-22T11:32:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 41.065525] nix-grpc-daemon[768]: ts=2026-09-22T11:32:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 41.070555] nix-grpc-daemon[768]: ts=2026-09-22T11:32:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 41.212608] nix-grpc-daemon[771]: ts=2026-09-22T11:32:15Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 41.217425] nix-grpc-daemon[771]: ts=2026-09-22T11:32:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 41.273797] nix-grpc-daemon[768]: ts=2026-09-22T11:32:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 41.275956] nix-grpc-daemon[768]: ts=2026-09-22T11:32:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 41.419974] nix-grpc-daemon[771]: ts=2026-09-22T11:32:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 41.424108] nix-grpc-daemon[771]: ts=2026-09-22T11:32:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 41.473598] nix-grpc-daemon[768]: ts=2026-09-22T11:32:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 41.479051] nix-grpc-daemon[768]: ts=2026-09-22T11:32:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 41.620900] nix-grpc-daemon[771]: ts=2026-09-22T11:32:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 41.625880] nix-grpc-daemon[771]: ts=2026-09-22T11:32:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 41.679668] nix-grpc-daemon[768]: ts=2026-09-22T11:32:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 41.684949] nix-grpc-daemon[768]: ts=2026-09-22T11:32:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 41.826804] nix-grpc-daemon[771]: ts=2026-09-22T11:32:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 41.831776] nix-grpc-daemon[771]: ts=2026-09-22T11:32:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 41.885743] nix-grpc-daemon[768]: ts=2026-09-22T11:32:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 41.891067] nix-grpc-daemon[768]: ts=2026-09-22T11:32:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 42.032811] nix-grpc-daemon[771]: ts=2026-09-22T11:32:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 42.037852] nix-grpc-daemon[771]: ts=2026-09-22T11:32:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 42.091801] nix-grpc-daemon[768]: ts=2026-09-22T11:32:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 42.096944] nix-grpc-daemon[768]: ts=2026-09-22T11:32:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb # [ 42.088160] niks3-server[1033]: 2026/09/22 11:32:16 INFO lead: acquired remote=192.168.1.3:44502 worker1 # [ 42.132942] nix-grpc-daemon[768]: ts=2026-09-22T11:32:16Z level=info event=scheduler_take_over worker2 # [ 42.239042] nix-grpc-daemon[771]: ts=2026-09-22T11:32:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 42.244368] nix-grpc-daemon[771]: ts=2026-09-22T11:32:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 42.299890] nix-grpc-daemon[768]: ts=2026-09-22T11:32:16Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 42.304209] nix-grpc-daemon[768]: ts=2026-09-22T11:32:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 42.447196] nix-grpc-daemon[771]: ts=2026-09-22T11:32:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 42.451419] nix-grpc-daemon[771]: ts=2026-09-22T11:32:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 42.500389] nix-grpc-daemon[768]: ts=2026-09-22T11:32:17Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 42.505535] nix-grpc-daemon[768]: ts=2026-09-22T11:32:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 42.648105] nix-grpc-daemon[771]: ts=2026-09-22T11:32:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 42.653515] nix-grpc-daemon[771]: ts=2026-09-22T11:32:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 42.706547] nix-grpc-daemon[768]: ts=2026-09-22T11:32:17Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 42.724105] nix-grpc-daemon[768]: ts=2026-09-22T11:32:17Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:43034 worker1 # [ 42.729471] nix-grpc-daemon[768]: ts=2026-09-22T11:32:17Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 42.854727] nix-grpc-daemon[771]: ts=2026-09-22T11:32:17Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 42.860132] nix-grpc-daemon[768]: ts=2026-09-22T11:32:17Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 42.865797] nix-grpc-daemon[768]: ts=2026-09-22T11:32:17Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:43034 worker1 # [ 42.871826] nix-grpc-daemon[768]: ts=2026-09-22T11:32:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 10.35 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker1 # [ 44.146149] nix-grpc-daemon[768]: ts=2026-09-22T11:32:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 44.160837] systemd[1]: Started Nix Daemon. worker1 # [ 44.333498] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 44.353916] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 87 ms (attempt 1/5) worker1 # [ 44.444801] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 174 ms (attempt 2/5) worker1 # [ 44.620782] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 279 ms (attempt 3/5) worker1 # [ 44.901987] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 431 ms (attempt 4/5) worker1 # [ 45.337619] nix-grpc-daemon[768]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 45.342954] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 45.378770] nix-grpc-daemon[771]: ts=2026-09-22T11:32:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 45.394911] systemd[1]: Started Nix Daemon. worker2 # [ 45.572888] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 45.595250] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 1/5) worker2 # [ 45.601642] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 78 ms (attempt 2/5) worker2 # [ 45.684224] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 3/5) worker2 # [ 45.777906] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 693 ms (attempt 4/5) worker2 # [ 46.474682] nix-grpc-daemon[771]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 46.475044] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 46.494596] nix-grpc-daemon[771]: ts=2026-09-22T11:32:21Z level=debug event=path_info_miss path=52ij8s6jkav0f53amdkjkhqqbw3wp27m-nixos-system-client-test error="error: path '/nix/store/52ij8s6jkav0f53amdkjkhqqbw3wp27m-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 46.509562] nix-grpc-daemon[771]: ts=2026-09-22T11:32:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 46.517430] nix-grpc-daemon[771]: ts=2026-09-22T11:32:21Z level=debug event=path_info_miss path=52ij8s6jkav0f53amdkjkhqqbw3wp27m-nixos-system-client-test error="error: path '/nix/store/52ij8s6jkav0f53amdkjkhqqbw3wp27m-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds) client: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds) worker2 # [ 46.848511] nix-grpc-daemon[771]: ts=2026-09-22T11:32:21Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 46.854776] nix-grpc-daemon[771]: ts=2026-09-22T11:32:21Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:45724 worker2 # [ 46.863294] nix-grpc-daemon[771]: ts=2026-09-22T11:32:21Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:45724 worker2 # [ 46.870919] nix-grpc-daemon[771]: ts=2026-09-22T11:32:21Z level=debug event=path_info_miss path=52ij8s6jkav0f53amdkjkhqqbw3wp27m-nixos-system-client-test error="error: path '/nix/store/52ij8s6jkav0f53amdkjkhqqbw3wp27m-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 46.885118] nix-grpc-daemon[771]: ts=2026-09-22T11:32:21Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:45724 worker2 # [ 46.892835] nix-grpc-daemon[771]: ts=2026-09-22T11:32:21Z level=debug event=path_info_miss path=52ij8s6jkav0f53amdkjkhqqbw3wp27m-nixos-system-client-test error="error: path '/nix/store/52ij8s6jkav0f53amdkjkhqqbw3wp27m-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds) worker1 # [ 47.107489] nix-grpc-daemon[768]: ts=2026-09-22T11:32:21Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 47.281616] nix-grpc-daemon[771]: ts=2026-09-22T11:32:21Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.42 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 worker2 # [ 47.442181] nix-grpc-daemon[771]: ts=2026-09-22T11:32:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45724 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 47.448150] nix-grpc-daemon[768]: ts=2026-09-22T11:32:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 47.483617] nix-grpc-daemon[768]: ts=2026-09-22T11:32:22Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=0 worker1 # [ 47.490763] nix-grpc-daemon[768]: ts=2026-09-22T11:32:22Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 47.532789] nix-grpc-daemon[768]: ts=2026-09-22T11:32:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 47.550394] nix-grpc-daemon[768]: ts=2026-09-22T11:32:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 client # copying 1 paths to 'lb:50051'... worker1 # [ 47.568094] nix-grpc-daemon[768]: ts=2026-09-22T11:32:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'... worker1 # [ 47.595360] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 47.639529] nix-grpc-daemon[768]: ts=2026-09-22T11:32:22Z level=info event=niks3_push_started pid=867 worker1 # [ 47.858505] nix-grpc-daemon[867]: time=2026-09-22T11:32:22.453Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 47.967348] nix-daemon[852]: accepted connection from pid 880, user nix-grpc-daemon (trusted) worker1 # [ 47.985065] nix-daemon[852]: reaped child process 883, status = succeeded worker1 # [ 48.086253] nix-daemon[852]: accepted connection from pid 885, user nix-grpc-daemon (trusted) worker1 # [ 48.101205] nix-daemon[852]: reaped child process 888, status = succeeded lb # [ 48.066416] niks3-server[1033]: 2026/09/22 11:32:22 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 48.129332] nix-grpc-daemon[867]: time=2026-09-22T11:32:22.723Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 48.130805] nix-grpc-daemon[867]: time=2026-09-22T11:32:22.723Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)" lb # [ 48.134454] niks3-server[1033]: 2026/09/22 11:32:22 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst lb # [ 48.153198] niks3-server[1033]: 2026/09/22 11:32:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign worker1 # [ 48.206207] nix-grpc-daemon[867]: time=2026-09-22T11:32:22.800Z level=INFO msg="Uploading 1 narinfos" lb # [ 48.162606] niks3-server[1033]: 2026/09/22 11:32:22 INFO Signed narinfos id=1 count=1 lb # [ 48.167579] niks3-server[1033]: 2026/09/22 11:32:22 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls lb # [ 48.181730] niks3-server[1033]: 2026/09/22 11:32:22 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 48.194990] niks3-server[1033]: 2026/09/22 11:32:22 INFO Completed upload id=1 worker1 # [ 48.240330] nix-grpc-daemon[867]: time=2026-09-22T11:32:22.835Z level=INFO msg="Upload complete. (377ms)" lb # [ 48.199216] niks3-server[1033]: 2026/09/22 11:32:22 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo worker1 # [ 48.252413] nix-grpc-daemon[768]: ts=2026-09-22T11:32:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 48.268767] nix-grpc-daemon[768]: ts=2026-09-22T11:32:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 48.280898] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 48.284322] nix-daemon[852]: reaped child process 865, status = succeeded client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051... worker1 # [ 48.296310] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv lb # [ 48.328156] envoy[893]: conn peer=[2001:db8:1::1]:36276 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=2403 lb # [ 48.333510] envoy[893]: conn peer=[2001:db8:1::1]:36282 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=65 lb # [ 48.339043] envoy[893]: conn peer=[2001:db8:1::1]:36286 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=18 lb # [ 48.343591] envoy[893]: conn peer=[2001:db8:1::1]:36300 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=12 client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker1 # [ 49.976859] nix-daemon[852]: reaped child process 892, status = succeeded worker1 # [ 50.086724] nix-daemon[852]: accepted connection from pid 908, user nix-grpc-daemon (trusted) worker1 # [ 50.104610] nix-daemon[852]: reaped child process 911, status = succeeded lb # [ 50.069298] niks3-server[1033]: 2026/09/22 11:32:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 50.135487] nix-grpc-daemon[867]: time=2026-09-22T11:32:24.727Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 50.138114] nix-grpc-daemon[867]: time=2026-09-22T11:32:24.727Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)" lb # [ 50.132242] niks3-server[1033]: 2026/09/22 11:32:24 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv lb # [ 50.141371] niks3-server[1033]: 2026/09/22 11:32:24 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 50.150870] niks3-server[1033]: 2026/09/22 11:32:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign worker1 # [ 50.202167] nix-grpc-daemon[867]: time=2026-09-22T11:32:24.797Z level=INFO msg="Uploading 1 narinfos" lb # [ 50.160437] niks3-server[1033]: 2026/09/22 11:32:24 INFO Signed narinfos id=2 count=1 lb # [ 50.165210] niks3-server[1033]: 2026/09/22 11:32:24 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls lb # [ 50.179443] niks3-server[1033]: 2026/09/22 11:32:24 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 50.189376] niks3-server[1033]: 2026/09/22 11:32:24 INFO Completed upload id=2 worker1 # [ 50.233788] nix-grpc-daemon[867]: time=2026-09-22T11:32:24.829Z level=INFO msg="Upload complete. (323ms)" lb # [ 50.201773] niks3-server[1033]: 2026/09/22 11:32:24 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo worker1 # [ 50.257944] nix-grpc-daemon[768]: ts=2026-09-22T11:32:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=1 worker1 # [ 50.278031] nix-grpc-daemon[768]: ts=2026-09-22T11:32:24Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=0 worker1 # [ 50.282531] nix-grpc-daemon[768]: ts=2026-09-22T11:32:24Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 50.288934] nix-daemon[852]: reaped child process 890, status = succeeded worker1 # [ 50.311021] nix-grpc-daemon[768]: ts=2026-09-22T11:32:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 50.332491] nix-grpc-daemon[768]: ts=2026-09-22T11:32:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 50.355557] nix-grpc-daemon[867]: time=2026-09-22T11:32:24.951Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'... worker1 # [ 50.371443] nix-grpc-daemon[768]: ts=2026-09-22T11:32:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 50.399075] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 50.659225] nix-daemon[852]: accepted connection from pid 917, user nix-grpc-daemon (trusted) worker1 # [ 50.697812] nix-daemon[852]: reaped child process 920, status = succeeded worker1 # [ 50.845512] nix-daemon[852]: accepted connection from pid 922, user nix-grpc-daemon (trusted) worker1 # [ 50.870954] nix-daemon[852]: reaped child process 925, status = succeeded lb # [ 50.831601] niks3-server[1033]: 2026/09/22 11:32:25 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 50.899343] nix-grpc-daemon[867]: time=2026-09-22T11:32:25.494Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 50.901405] nix-grpc-daemon[867]: time=2026-09-22T11:32:25.494Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)" lb # [ 50.891842] niks3-server[1033]: 2026/09/22 11:32:25 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst worker1 # [ 50.954823] nix-grpc-daemon[867]: time=2026-09-22T11:32:25.549Z level=INFO msg="Uploading 1 narinfos" lb # [ 50.910383] niks3-server[1033]: 2026/09/22 11:32:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign lb # [ 50.926735] niks3-server[1033]: 2026/09/22 11:32:25 INFO Signed narinfos id=3 count=1 lb # [ 50.934716] niks3-server[1033]: 2026/09/22 11:32:25 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls lb # [ 50.952449] niks3-server[1033]: 2026/09/22 11:32:25 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 50.973787] niks3-server[1033]: 2026/09/22 11:32:25 INFO Completed upload id=3 worker1 # [ 51.019030] nix-grpc-daemon[867]: time=2026-09-22T11:32:25.614Z level=INFO msg="Upload complete. (591ms)" lb # [ 50.978751] niks3-server[1033]: 2026/09/22 11:32:25 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo worker1 # [ 51.032558] nix-grpc-daemon[768]: ts=2026-09-22T11:32:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 51.049231] nix-grpc-daemon[768]: ts=2026-09-22T11:32:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 51.072462] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 51.092933] nix-daemon[852]: reaped child process 915, status = succeeded client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051... worker1 # [ 51.106963] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker1 # [ 52.396841] nix-daemon[852]: reaped child process 929, status = succeeded worker1 # [ 52.552656] nix-daemon[852]: accepted connection from pid 945, user nix-grpc-daemon (trusted) worker1 # [ 52.581238] nix-daemon[852]: reaped child process 948, status = succeeded lb # [ 52.535167] niks3-server[1033]: 2026/09/22 11:32:27 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 52.595911] nix-grpc-daemon[867]: time=2026-09-22T11:32:27.190Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 52.598505] nix-grpc-daemon[867]: time=2026-09-22T11:32:27.190Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)" lb # [ 52.594409] niks3-server[1033]: 2026/09/22 11:32:27 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv lb # [ 52.619783] niks3-server[1033]: 2026/09/22 11:32:27 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst lb # [ 52.636874] niks3-server[1033]: 2026/09/22 11:32:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign worker1 # [ 52.683042] nix-grpc-daemon[867]: time=2026-09-22T11:32:27.277Z level=INFO msg="Uploading 1 narinfos" lb # [ 52.646082] niks3-server[1033]: 2026/09/22 11:32:27 INFO Signed narinfos id=4 count=1 lb # [ 52.655900] niks3-server[1033]: 2026/09/22 11:32:27 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls lb # [ 52.673085] niks3-server[1033]: 2026/09/22 11:32:27 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete lb # [ 52.687678] niks3-server[1033]: 2026/09/22 11:32:27 INFO Completed upload id=4 worker1 # [ 52.732950] nix-grpc-daemon[867]: time=2026-09-22T11:32:27.328Z level=INFO msg="Upload complete. (403ms)" lb # [ 52.691291] niks3-server[1033]: 2026/09/22 11:32:27 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo worker1 # [ 52.764539] nix-grpc-daemon[768]: ts=2026-09-22T11:32:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=1 worker1 # [ 52.780786] nix-grpc-daemon[768]: ts=2026-09-22T11:32:27Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=0 worker1 # [ 52.793476] nix-grpc-daemon[768]: ts=2026-09-22T11:32:27Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 52.803869] nix-daemon[852]: reaped child process 927, status = succeeded worker1 # [ 52.813727] nix-grpc-daemon[768]: ts=2026-09-22T11:32:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 52.829096] nix-grpc-daemon[768]: ts=2026-09-22T11:32:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 52.850508] nix-grpc-daemon[867]: time=2026-09-22T11:32:27.445Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 52.869298] nix-grpc-daemon[768]: ts=2026-09-22T11:32:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'... worker1 # [ 52.900176] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 53.142188] nix-daemon[852]: accepted connection from pid 954, user nix-grpc-daemon (trusted) worker1 # [ 53.169357] nix-daemon[852]: reaped child process 957, status = succeeded worker1 # [ 53.321436] nix-daemon[852]: accepted connection from pid 959, user nix-grpc-daemon (trusted) worker1 # [ 53.348198] nix-daemon[852]: reaped child process 962, status = succeeded lb # [ 53.300714] niks3-server[1033]: 2026/09/22 11:32:27 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 53.362458] nix-grpc-daemon[867]: time=2026-09-22T11:32:27.957Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 53.364141] nix-grpc-daemon[867]: time=2026-09-22T11:32:27.957Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)" lb # [ 53.357008] niks3-server[1033]: 2026/09/22 11:32:27 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst worker1 # [ 53.432896] nix-grpc-daemon[867]: time=2026-09-22T11:32:28.027Z level=INFO msg="Uploading 1 narinfos" lb # [ 53.379158] niks3-server[1033]: 2026/09/22 11:32:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign lb # [ 53.393467] niks3-server[1033]: 2026/09/22 11:32:27 INFO Signed narinfos id=5 count=1 lb # [ 53.410586] niks3-server[1033]: 2026/09/22 11:32:28 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls lb # [ 53.422421] niks3-server[1033]: 2026/09/22 11:32:28 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete worker1 # [ 53.474184] nix-grpc-daemon[867]: time=2026-09-22T11:32:28.069Z level=INFO msg="Upload complete. (543ms)" lb # [ 53.430618] niks3-server[1033]: 2026/09/22 11:32:28 INFO Completed upload id=5 lb # [ 53.435851] niks3-server[1033]: 2026/09/22 11:32:28 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo worker1 # [ 53.488415] nix-grpc-daemon[768]: ts=2026-09-22T11:32:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 53.505190] nix-grpc-daemon[768]: ts=2026-09-22T11:32:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 53.527154] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 53.538038] nix-daemon[852]: reaped child process 952, status = succeeded client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051... client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 53.557821] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker1 # [ 54.424703] nix-daemon[852]: reaped child process 966, status = succeeded worker1 # [ 54.556441] nix-daemon[852]: accepted connection from pid 982, user nix-grpc-daemon (trusted) worker1 # [ 54.574665] nix-daemon[852]: reaped child process 985, status = succeeded lb # [ 54.535565] niks3-server[1033]: 2026/09/22 11:32:29 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 54.592486] nix-grpc-daemon[867]: time=2026-09-22T11:32:29.186Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 54.594381] nix-grpc-daemon[867]: time=2026-09-22T11:32:29.186Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)" lb # [ 54.577610] niks3-server[1033]: 2026/09/22 11:32:29 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst lb # [ 54.593063] niks3-server[1033]: 2026/09/22 11:32:29 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv lb # [ 54.604061] niks3-server[1033]: 2026/09/22 11:32:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign worker1 # [ 54.653893] nix-grpc-daemon[867]: time=2026-09-22T11:32:29.248Z level=INFO msg="Uploading 1 narinfos" lb # [ 54.612987] niks3-server[1033]: 2026/09/22 11:32:29 INFO Signed narinfos id=6 count=1 lb # [ 54.618699] niks3-server[1033]: 2026/09/22 11:32:29 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls lb # [ 54.629784] niks3-server[1033]: 2026/09/22 11:32:29 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete lb # [ 54.636584] niks3-server[1033]: 2026/09/22 11:32:29 INFO Completed upload id=6 worker1 # [ 54.681151] nix-grpc-daemon[867]: time=2026-09-22T11:32:29.276Z level=INFO msg="Upload complete. (305ms)" lb # [ 54.640543] niks3-server[1033]: 2026/09/22 11:32:29 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo worker1 # [ 54.697088] nix-grpc-daemon[768]: ts=2026-09-22T11:32:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=1 worker1 # [ 54.717022] nix-daemon[852]: reaped child process 964, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 7.46 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.19 seconds) client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client # copying 3 paths... client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.02 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds) worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.25 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 worker2 # [ 56.010446] nix-grpc-daemon[771]: ts=2026-09-22T11:32:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45724 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 56.025593] nix-grpc-daemon[771]: ts=2026-09-22T11:32:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45724 worker1 # [ 56.053983] nix-grpc-daemon[768]: ts=2026-09-22T11:32:30Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.30 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.16 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 1.21 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2 worker2 # [ 56.914257] nix-grpc-daemon[771]: ts=2026-09-22T11:32:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45724 client # don't know how to build these paths: client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker1 # [ 56.920854] nix-grpc-daemon[768]: ts=2026-09-22T11:32:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 56.931724] nix-grpc-daemon[867]: time=2026-09-22T11:32:31.526Z level=INFO msg="All 2 paths already cached" worker1 # [ 56.947915] nix-grpc-daemon[768]: ts=2026-09-22T11:32:31Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=0 worker1 # [ 56.956149] nix-grpc-daemon[768]: ts=2026-09-22T11:32:31Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 56.993145] nix-grpc-daemon[768]: ts=2026-09-22T11:32:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 57.009119] nix-grpc-daemon[768]: ts=2026-09-22T11:32:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 57.023803] nix-grpc-daemon[867]: time=2026-09-22T11:32:31.619Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'... worker1 # [ 57.035372] nix-grpc-daemon[768]: ts=2026-09-22T11:32:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 57.050299] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 57.203731] nix-daemon[852]: accepted connection from pid 1014, user nix-grpc-daemon (trusted) worker1 # [ 57.229673] nix-daemon[852]: reaped child process 1017, status = succeeded worker1 # [ 57.322493] nix-daemon[852]: accepted connection from pid 1019, user nix-grpc-daemon (trusted) worker1 # [ 57.340972] nix-daemon[852]: reaped child process 1022, status = succeeded lb # [ 57.295002] niks3-server[1033]: 2026/09/22 11:32:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 57.349461] nix-grpc-daemon[867]: time=2026-09-22T11:32:31.944Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 57.350940] nix-grpc-daemon[867]: time=2026-09-22T11:32:31.944Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)" lb # [ 57.324683] niks3-server[1033]: 2026/09/22 11:32:31 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst worker1 # [ 57.384467] nix-grpc-daemon[867]: time=2026-09-22T11:32:31.979Z level=INFO msg="Uploading 1 narinfos" lb # [ 57.339223] niks3-server[1033]: 2026/09/22 11:32:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign lb # [ 57.351135] niks3-server[1033]: 2026/09/22 11:32:31 INFO Signed narinfos id=7 count=1 lb # [ 57.359688] niks3-server[1033]: 2026/09/22 11:32:31 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls lb # [ 57.373747] niks3-server[1033]: 2026/09/22 11:32:31 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete worker1 # [ 57.425058] nix-grpc-daemon[867]: time=2026-09-22T11:32:32.020Z level=INFO msg="Upload complete. (357ms)" lb # [ 57.381156] niks3-server[1033]: 2026/09/22 11:32:31 INFO Completed upload id=7 lb # [ 57.385444] niks3-server[1033]: 2026/09/22 11:32:31 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo worker1 # [ 57.432790] nix-grpc-daemon[768]: ts=2026-09-22T11:32:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 57.447066] nix-grpc-daemon[768]: ts=2026-09-22T11:32:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 57.454820] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 57.460007] nix-daemon[852]: reaped child process 1012, status = succeeded client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051... worker1 # [ 57.474917] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b lb # [ 58.325833] envoy[893]: conn peer=[2001:db8:1::1]:36304 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9579 tx=4449 ms=7284 worker1 # [ 58.579431] nix-daemon[852]: reaped child process 1026, status = succeeded worker1 # [ 58.651743] nix-daemon[852]: accepted connection from pid 1042, user nix-grpc-daemon (trusted) worker1 # [ 58.674659] nix-daemon[852]: reaped child process 1045, status = succeeded lb # [ 58.330764] envoy[893]: conn peer=[2001:db8:1::1]:40624 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=481 ms=70 lb # [ 58.628334] niks3-server[1033]: 2026/09/22 11:32:33 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 58.687679] nix-grpc-daemon[867]: time=2026-09-22T11:32:33.282Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 58.693343] nix-grpc-daemon[867]: time=2026-09-22T11:32:33.285Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)" lb # [ 58.672421] niks3-server[1033]: 2026/09/22 11:32:33 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv lb # [ 58.690798] niks3-server[1033]: 2026/09/22 11:32:33 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst worker1 # [ 58.747306] nix-grpc-daemon[867]: time=2026-09-22T11:32:33.342Z level=INFO msg="Uploading 1 narinfos" lb # [ 58.703238] niks3-server[1033]: 2026/09/22 11:32:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign lb # [ 58.712060] niks3-server[1033]: 2026/09/22 11:32:33 INFO Signed narinfos id=8 count=1 lb # [ 58.714052] niks3-server[1033]: 2026/09/22 11:32:33 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls lb # [ 58.725102] niks3-server[1033]: 2026/09/22 11:32:33 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete lb # [ 58.729123] niks3-server[1033]: 2026/09/22 11:32:33 INFO Completed upload id=8 lb # [ 58.731314] niks3-server[1033]: 2026/09/22 11:32:33 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo worker1 # [ 58.774131] nix-grpc-daemon[867]: time=2026-09-22T11:32:33.369Z level=INFO msg="Upload complete. (289ms)" worker1 # [ 58.789308] nix-grpc-daemon[768]: ts=2026-09-22T11:32:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 worker1 # [ 58.805993] nix-daemon[852]: reaped child process 1024, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.13 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.65 seconds) subtest: outputs read back through any worker carry the cache signature client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2 worker2 # [ 59.391788] nix-grpc-daemon[771]: ts=2026-09-22T11:32:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45724 client # don't know how to build these paths: client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker1 # [ 59.399389] nix-grpc-daemon[768]: ts=2026-09-22T11:32:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 59.422950] nix-grpc-daemon[768]: ts=2026-09-22T11:32:34Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=0 worker2 # [ 59.431823] nix-grpc-daemon[771]: ts=2026-09-22T11:32:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 worker1 # [ 59.429111] nix-grpc-daemon[768]: ts=2026-09-22T11:32:34Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 59.448382] nix-grpc-daemon[771]: ts=2026-09-22T11:32:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45724 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'... worker2 # [ 59.464708] nix-grpc-daemon[771]: ts=2026-09-22T11:32:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 59.481314] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 59.510515] nix-grpc-daemon[771]: ts=2026-09-22T11:32:34Z level=info event=niks3_push_started pid=867 worker2 # [ 59.674520] nix-grpc-daemon[867]: time=2026-09-22T11:32:34.261Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 59.785949] nix-daemon[842]: accepted connection from pid 880, user nix-grpc-daemon (trusted) worker2 # [ 59.809716] nix-daemon[842]: reaped child process 883, status = succeeded worker2 # [ 59.910545] nix-daemon[842]: accepted connection from pid 885, user nix-grpc-daemon (trusted) worker2 # [ 59.927797] nix-daemon[842]: reaped child process 888, status = succeeded lb # [ 59.885479] niks3-server[1033]: 2026/09/22 11:32:34 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 59.943654] nix-grpc-daemon[867]: time=2026-09-22T11:32:34.530Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 59.947478] nix-grpc-daemon[867]: time=2026-09-22T11:32:34.532Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)" lb # [ 59.919300] niks3-server[1033]: 2026/09/22 11:32:34 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst lb # [ 59.932641] niks3-server[1033]: 2026/09/22 11:32:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign worker2 # [ 59.988190] nix-grpc-daemon[867]: time=2026-09-22T11:32:34.574Z level=INFO msg="Uploading 1 narinfos" lb # [ 59.945822] niks3-server[1033]: 2026/09/22 11:32:34 INFO Signed narinfos id=9 count=1 lb # [ 59.953170] niks3-server[1033]: 2026/09/22 11:32:34 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls lb # [ 59.967399] niks3-server[1033]: 2026/09/22 11:32:34 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete worker2 # [ 60.024809] nix-grpc-daemon[867]: time=2026-09-22T11:32:34.611Z level=INFO msg="Upload complete. (346ms)" lb # [ 59.978797] niks3-server[1033]: 2026/09/22 11:32:34 INFO Completed upload id=9 lb # [ 59.982610] niks3-server[1033]: 2026/09/22 11:32:34 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo worker2 # [ 60.032320] nix-grpc-daemon[771]: ts=2026-09-22T11:32:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 60.048330] nix-grpc-daemon[771]: ts=2026-09-22T11:32:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 60.056628] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 60.061669] nix-daemon[842]: reaped child process 865, status = succeeded client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051... client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker2 # [ 60.075352] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-sig worker2 # [ 61.185137] nix-daemon[842]: reaped child process 892, status = succeeded worker2 # [ 61.262982] nix-daemon[842]: accepted connection from pid 908, user nix-grpc-daemon (trusted) worker2 # [ 61.281760] nix-daemon[842]: reaped child process 911, status = succeeded lb # [ 61.239295] niks3-server[1033]: 2026/09/22 11:32:35 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 61.307344] nix-grpc-daemon[867]: time=2026-09-22T11:32:35.890Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 61.310573] nix-grpc-daemon[867]: time=2026-09-22T11:32:35.890Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)" lb # [ 61.293707] niks3-server[1033]: 2026/09/22 11:32:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign lb # [ 61.308752] niks3-server[1033]: 2026/09/22 11:32:35 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst lb # [ 61.316220] niks3-server[1033]: 2026/09/22 11:32:35 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv lb # [ 61.325981] niks3-server[1033]: 2026/09/22 11:32:35 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls lb # [ 61.333258] niks3-server[1033]: 2026/09/22 11:32:35 INFO Signed narinfos id=10 count=1 worker2 # [ 61.383142] nix-grpc-daemon[867]: time=2026-09-22T11:32:35.968Z level=INFO msg="Uploading 1 narinfos" lb # [ 61.348439] niks3-server[1033]: 2026/09/22 11:32:35 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete lb # [ 61.356160] niks3-server[1033]: 2026/09/22 11:32:35 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo lb # [ 61.362009] niks3-server[1033]: 2026/09/22 11:32:35 INFO Completed upload id=10 worker2 # [ 61.411291] nix-grpc-daemon[867]: time=2026-09-22T11:32:35.998Z level=INFO msg="Upload complete. (324ms)" worker2 # [ 61.423384] nix-grpc-daemon[771]: ts=2026-09-22T11:32:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=1 worker1 # [ 61.424805] nix-grpc-daemon[768]: ts=2026-09-22T11:32:36Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=0 worker1 # [ 61.425117] nix-grpc-daemon[768]: ts=2026-09-22T11:32:36Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 61.444757] nix-daemon[842]: reaped child process 890, status = succeeded worker2 # [ 61.473181] nix-grpc-daemon[771]: ts=2026-09-22T11:32:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 61.489457] nix-grpc-daemon[771]: ts=2026-09-22T11:32:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 61.504416] nix-grpc-daemon[867]: time=2026-09-22T11:32:36.091Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'... worker2 # [ 61.515032] nix-grpc-daemon[771]: ts=2026-09-22T11:32:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 61.529305] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 61.683224] nix-daemon[842]: accepted connection from pid 915, user nix-grpc-daemon (trusted) worker2 # [ 61.706262] nix-daemon[842]: reaped child process 918, status = succeeded worker2 # [ 61.805699] nix-daemon[842]: accepted connection from pid 920, user nix-grpc-daemon (trusted) worker2 # [ 61.823912] nix-daemon[842]: reaped child process 923, status = succeeded lb # [ 61.777311] niks3-server[1033]: 2026/09/22 11:32:36 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 61.838984] nix-grpc-daemon[867]: time=2026-09-22T11:32:36.425Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 61.840580] nix-grpc-daemon[867]: time=2026-09-22T11:32:36.425Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)" worker2 # [ 61.876758] nix-grpc-daemon[867]: time=2026-09-22T11:32:36.463Z level=INFO msg="Uploading 1 narinfos" lb # [ 61.818057] niks3-server[1033]: 2026/09/22 11:32:36 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst lb # [ 61.834808] niks3-server[1033]: 2026/09/22 11:32:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign lb # [ 61.844155] niks3-server[1033]: 2026/09/22 11:32:36 INFO Signed narinfos id=11 count=1 lb # [ 61.849716] niks3-server[1033]: 2026/09/22 11:32:36 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls lb # [ 61.858760] niks3-server[1033]: 2026/09/22 11:32:36 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete lb # [ 61.866293] niks3-server[1033]: 2026/09/22 11:32:36 INFO Completed upload id=11 worker2 # [ 61.914801] nix-grpc-daemon[867]: time=2026-09-22T11:32:36.501Z level=INFO msg="Upload complete. (369ms)" lb # [ 61.869142] niks3-server[1033]: 2026/09/22 11:32:36 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo worker2 # [ 61.924286] nix-grpc-daemon[771]: ts=2026-09-22T11:32:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 61.939793] nix-grpc-daemon[771]: ts=2026-09-22T11:32:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 61.946601] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 61.951657] nix-daemon[842]: reaped child process 913, status = succeeded client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051... worker2 # [ 61.965279] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-sig worker2 # [ 63.401200] nix-daemon[842]: reaped child process 927, status = succeeded worker2 # [ 63.491351] nix-daemon[842]: accepted connection from pid 943, user nix-grpc-daemon (trusted) worker2 # [ 63.510448] nix-daemon[842]: reaped child process 946, status = succeeded lb # [ 63.467038] niks3-server[1033]: 2026/09/22 11:32:38 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 63.525928] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.112Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 63.527444] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.112Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)" lb # [ 63.503098] niks3-server[1033]: 2026/09/22 11:32:38 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv lb # [ 63.518130] niks3-server[1033]: 2026/09/22 11:32:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign worker2 # [ 63.572576] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.159Z level=INFO msg="Uploading 1 narinfos" lb # [ 63.529472] niks3-server[1033]: 2026/09/22 11:32:38 INFO Signed narinfos id=12 count=1 lb # [ 63.535636] niks3-server[1033]: 2026/09/22 11:32:38 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst lb # [ 63.546936] niks3-server[1033]: 2026/09/22 11:32:38 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls worker2 # [ 63.601851] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.188Z level=INFO msg="Upload complete. (226ms)" lb # [ 63.553729] niks3-server[1033]: 2026/09/22 11:32:38 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete lb # [ 63.557341] niks3-server[1033]: 2026/09/22 11:32:38 INFO Completed upload id=12 lb # [ 63.561570] niks3-server[1033]: 2026/09/22 11:32:38 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo worker1 # [ 63.618251] nix-grpc-daemon[768]: ts=2026-09-22T11:32:38Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=0 worker2 # [ 63.613089] nix-grpc-daemon[771]: ts=2026-09-22T11:32:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=1 worker2 # [ 63.631325] nix-daemon[842]: reaped child process 925, status = succeeded worker1 # [ 63.624212] nix-grpc-daemon[768]: ts=2026-09-22T11:32:38Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 63.669204] nix-grpc-daemon[771]: ts=2026-09-22T11:32:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 63.686202] nix-grpc-daemon[771]: ts=2026-09-22T11:32:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 63.700531] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.287Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'... worker2 # [ 63.711628] nix-grpc-daemon[771]: ts=2026-09-22T11:32:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 63.725130] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 63.887315] nix-daemon[842]: accepted connection from pid 950, user nix-grpc-daemon (trusted) worker2 # [ 63.911386] nix-daemon[842]: reaped child process 953, status = succeeded worker2 # [ 64.006351] nix-daemon[842]: accepted connection from pid 955, user nix-grpc-daemon (trusted) worker2 # [ 64.025163] nix-daemon[842]: reaped child process 958, status = succeeded lb # [ 63.980859] niks3-server[1033]: 2026/09/22 11:32:38 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 64.038596] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.625Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 64.040156] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.625Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)" lb # [ 64.007587] niks3-server[1033]: 2026/09/22 11:32:38 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst worker2 # [ 64.068090] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.655Z level=INFO msg="Uploading 1 narinfos" lb # [ 64.019620] niks3-server[1033]: 2026/09/22 11:32:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign lb # [ 64.027309] niks3-server[1033]: 2026/09/22 11:32:38 INFO Signed narinfos id=13 count=1 lb # [ 64.033409] niks3-server[1033]: 2026/09/22 11:32:38 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls lb # [ 64.044581] niks3-server[1033]: 2026/09/22 11:32:38 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete lb # [ 64.052672] niks3-server[1033]: 2026/09/22 11:32:38 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo worker2 # [ 64.105369] nix-grpc-daemon[867]: time=2026-09-22T11:32:38.692Z level=INFO msg="Upload complete. (363ms)" lb # [ 64.057602] niks3-server[1033]: 2026/09/22 11:32:38 INFO Completed upload id=13 worker2 # [ 64.112353] nix-grpc-daemon[771]: ts=2026-09-22T11:32:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 64.127442] nix-grpc-daemon[771]: ts=2026-09-22T11:32:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 64.134016] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 64.139048] nix-daemon[842]: reaped child process 948, status = succeeded client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051... worker2 # [ 64.152197] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-sig worker2 # [ 65.387908] nix-daemon[842]: reaped child process 962, status = succeeded worker2 # [ 65.495289] nix-daemon[842]: accepted connection from pid 978, user nix-grpc-daemon (trusted) worker2 # [ 65.518410] nix-daemon[842]: reaped child process 981, status = succeeded lb # [ 65.470668] niks3-server[1033]: 2026/09/22 11:32:40 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 65.568698] nix-grpc-daemon[867]: time=2026-09-22T11:32:40.154Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 65.571460] nix-grpc-daemon[867]: time=2026-09-22T11:32:40.154Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)" lb # [ 65.554948] niks3-server[1033]: 2026/09/22 11:32:40 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst worker2 # [ 65.626516] nix-grpc-daemon[867]: time=2026-09-22T11:32:40.213Z level=INFO msg="Uploading 1 narinfos" lb # [ 65.571810] niks3-server[1033]: 2026/09/22 11:32:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign lb # [ 65.582691] niks3-server[1033]: 2026/09/22 11:32:40 INFO Signed narinfos id=14 count=1 lb # [ 65.588240] niks3-server[1033]: 2026/09/22 11:32:40 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv lb # [ 65.596941] niks3-server[1033]: 2026/09/22 11:32:40 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls lb # [ 65.607239] niks3-server[1033]: 2026/09/22 11:32:40 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 65.612259] niks3-server[1033]: 2026/09/22 11:32:40 INFO Completed upload id=14 worker2 # [ 65.660644] nix-grpc-daemon[867]: time=2026-09-22T11:32:40.247Z level=INFO msg="Upload complete. (298ms)" lb # [ 65.614853] niks3-server[1033]: 2026/09/22 11:32:40 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo worker2 # [ 65.673289] nix-grpc-daemon[771]: ts=2026-09-22T11:32:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=1 worker2 # [ 65.692875] nix-daemon[842]: reaped child process 960, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 6.54 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.18 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 66.026878] nix-grpc-daemon[768]: ts=2026-09-22T11:32:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:40846 client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked): client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 66.057850] nix-grpc-daemon[768]: ts=2026-09-22T11:32:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:40846 worker1 # [ 66.066801] nix-grpc-daemon[768]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'... worker1 # [ 66.084780] nix-grpc-daemon[768]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'... worker1 # [ 66.099120] nix-grpc-daemon[768]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.25 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker2 # [ 66.332682] nix-grpc-daemon[771]: ts=2026-09-22T11:32:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:52530 worker2 # [ 66.375078] nix-grpc-daemon[771]: ts=2026-09-22T11:32:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:52530 client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.26 seconds) client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top client # finding garbage collector roots... client # removing stale temporary roots file "/nix/var/nix/temproots/882" client # removing stale temporary roots file "/nix/var/nix/temproots/846" client # removing stale temporary roots file "/nix/var/nix/temproots/859" client # removing stale temporary roots file "/nix/var/nix/temproots/805" client # removing stale temporary roots file "/nix/var/nix/temproots/867" client # removing stale temporary roots file "/nix/var/nix/temproots/826" client # removing stale temporary roots file "/nix/var/nix/temproots/790" client # removing stale temporary roots file "/nix/var/nix/temproots/838" client # deleting specified paths... client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' client # deleting unused links... client # note: hard linking is currently saving 0.0 KiB worker2 # [ 66.728645] nix-grpc-daemon[771]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45724 worker1 # [ 66.734594] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 66.747803] nix-grpc-daemon[867]: time=2026-09-22T11:32:41.342Z level=INFO msg="All 1 paths already cached" worker2 # [ 66.760570] nix-grpc-daemon[771]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45724 worker1 # [ 66.765793] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43040 client # copying 3 paths... worker2 # [ 66.784845] nix-grpc-daemon[771]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45724 client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'... worker2 # [ 66.813018] nix-grpc-daemon[771]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 66.820425] nix-grpc-daemon[771]: ts=2026-09-22T11:32:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_out=232 worker1 # [ 66.824137] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 66.831980] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_out=176 worker1 # [ 66.840307] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 66.846900] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_out=128 client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'... client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'... client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.55 seconds) (finished: subtest: outputs read back through any worker carry the cache signature, in 7.77 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 67.027353] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: 99bb673007594c67975a9b5d2cef2bd5 client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup worker2 # [ 67.200356] nix-grpc-daemon[771]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45724 client # [ 67.227428] nix[935]: don't know how to build these paths: client # [ 67.228885] nix[935]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 67.213358] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker1 # [ 67.219481] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 67.265764] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 67.281716] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 client # [ 67.319672] nix[935]: copying 1 paths to 'lb:50051'... worker1 # [ 67.296482] nix-grpc-daemon[768]: ts=2026-09-22T11:32:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 client # [ 67.322255] nix[935]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'... worker1 # [ 67.311309] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 67.473609] nix-daemon[852]: accepted connection from pid 1065, user nix-grpc-daemon (trusted) worker1 # [ 67.494323] nix-daemon[852]: reaped child process 1068, status = succeeded worker1 # [ 67.594502] nix-daemon[852]: accepted connection from pid 1070, user nix-grpc-daemon (trusted) worker1 # [ 67.610655] nix-daemon[852]: reaped child process 1073, status = succeeded lb # [ 67.573188] niks3-server[1033]: 2026/09/22 11:32:42 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 67.627556] nix-grpc-daemon[867]: time=2026-09-22T11:32:42.222Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 67.629179] nix-grpc-daemon[867]: time=2026-09-22T11:32:42.223Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)" lb # [ 67.600633] niks3-server[1033]: 2026/09/22 11:32:42 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst lb # [ 67.617146] niks3-server[1033]: 2026/09/22 11:32:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign worker1 # [ 67.668302] nix-grpc-daemon[867]: time=2026-09-22T11:32:42.263Z level=INFO msg="Uploading 1 narinfos" lb # [ 67.628308] niks3-server[1033]: 2026/09/22 11:32:42 INFO Signed narinfos id=15 count=1 lb # [ 67.634341] niks3-server[1033]: 2026/09/22 11:32:42 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls lb # [ 67.645693] niks3-server[1033]: 2026/09/22 11:32:42 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete lb # [ 67.795783] niks3-server[1033]: 2026/09/22 11:32:42 INFO Completed upload id=15 worker1 # [ 67.841643] nix-grpc-daemon[867]: time=2026-09-22T11:32:42.436Z level=INFO msg="Upload complete. (512ms)" lb # [ 67.803373] niks3-server[1033]: 2026/09/22 11:32:42 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo worker1 # [ 67.852382] nix-grpc-daemon[768]: ts=2026-09-22T11:32:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=432 worker1 # [ 67.871870] nix-grpc-daemon[768]: ts=2026-09-22T11:32:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 67.884776] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 67.889229] nix-daemon[852]: reaped child process 1063, status = succeeded client # [ 67.925072] nix[935]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker1 # [ 67.906389] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) client # [ 68.125675] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: 94343c2d804b43c795578cc119c16dc1 client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds) client # [ 68.341261] nix[951]: don't know how to build these paths: client # [ 68.343312] nix[951]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker2 # [ 68.320757] nix-grpc-daemon[771]: ts=2026-09-22T11:32:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45724 worker1 # [ 68.329913] nix-grpc-daemon[768]: ts=2026-09-22T11:32:42Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 client # [ 68.360848] nix[951]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker1 # [ 68.337598] nix-grpc-daemon[768]: ts=2026-09-22T11:32:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 lb # [ 68.328002] envoy[893]: conn peer=[2001:db8:1::1]:40630 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1895 ms=1891 lb # [ 68.333120] envoy[893]: conn peer=[2001:db8:1::1]:40640 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9558 tx=4408 ms=6296 lb # [ 68.338146] envoy[893]: conn peer=[2001:db8:1::1]:33778 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2887 ms=200 lb # [ 68.343348] envoy[893]: conn peer=[2001:db8:1::1]:33796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=329 ms=86 lb # [ 68.348612] envoy[893]: conn peer=[2001:db8:1::1]:33788 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=409 ms=118 lb # [ 68.354252] envoy[893]: conn peer=[2001:db8:1::1]:33790 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=375 ms=103 client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.05 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker1 # [ 71.771493] nix-daemon[852]: reaped child process 1077, status = succeeded worker1 # [ 71.911824] nix-daemon[852]: accepted connection from pid 1109, user nix-grpc-daemon (trusted) worker1 # [ 71.931442] nix-daemon[852]: reaped child process 1112, status = succeeded lb # [ 71.893406] niks3-server[1033]: 2026/09/22 11:32:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 71.952146] nix-grpc-daemon[867]: time=2026-09-22T11:32:46.543Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 71.953996] nix-grpc-daemon[867]: time=2026-09-22T11:32:46.543Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)" lb # [ 71.933383] niks3-server[1033]: 2026/09/22 11:32:46 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 71.954245] niks3-server[1033]: 2026/09/22 11:32:46 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 72.010934] nix-grpc-daemon[867]: time=2026-09-22T11:32:46.606Z level=INFO msg="Uploading 1 narinfos" lb # [ 71.966544] niks3-server[1033]: 2026/09/22 11:32:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign lb # [ 71.976542] niks3-server[1033]: 2026/09/22 11:32:46 INFO Signed narinfos id=16 count=1 lb # [ 71.982218] niks3-server[1033]: 2026/09/22 11:32:46 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls lb # [ 71.995293] niks3-server[1033]: 2026/09/22 11:32:46 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 72.000352] niks3-server[1033]: 2026/09/22 11:32:46 INFO Completed upload id=16 worker1 # [ 72.045904] nix-grpc-daemon[867]: time=2026-09-22T11:32:46.641Z level=INFO msg="Upload complete. (321ms)" lb # [ 72.005257] niks3-server[1033]: 2026/09/22 11:32:46 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo worker1 # [ 72.061158] nix-grpc-daemon[768]: ts=2026-09-22T11:32:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 72.079360] nix-grpc-daemon[768]: ts=2026-09-22T11:32:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 72.086920] nix-daemon[852]: reaped child process 1075, status = succeeded client # [ 72.114733] systemd[1]: dedup2.service: Deactivated successfully. client # [ 72.119724] systemd[1]: dedup2.service: Consumed 194ms CPU time over 3.991s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic. client # [ 72.159683] systemd[1]: dedup1.service: Deactivated successfully. client # [ 72.163232] systemd[1]: dedup1.service: Consumed 175ms CPU time over 5.135s wall clock time, 41.5M memory peak, 7.5K incoming IP traffic, 8.5K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: two clients wanting the same drv build it once, in 6.13 seconds) subtest: --timeout and --max-silent-time reach the builder worker2 # [ 73.300872] nix-grpc-daemon[771]: ts=2026-09-22T11:32:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45724 worker1 # [ 73.308562] nix-grpc-daemon[768]: ts=2026-09-22T11:32:47Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=0 worker2 # [ 73.315787] nix-grpc-daemon[771]: ts=2026-09-22T11:32:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 worker1 # [ 73.308983] nix-grpc-daemon[768]: ts=2026-09-22T11:32:47Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 73.330954] nix-grpc-daemon[771]: ts=2026-09-22T11:32:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 73.349452] nix-grpc-daemon[771]: ts=2026-09-22T11:32:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 73.365261] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 73.528520] nix-daemon[842]: accepted connection from pid 1006, user nix-grpc-daemon (trusted) worker2 # [ 73.547331] nix-daemon[842]: reaped child process 1009, status = succeeded worker2 # [ 73.644750] nix-daemon[842]: accepted connection from pid 1011, user nix-grpc-daemon (trusted) worker2 # [ 73.660245] nix-daemon[842]: reaped child process 1014, status = succeeded lb # [ 73.618280] niks3-server[1033]: 2026/09/22 11:32:48 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 73.680104] nix-grpc-daemon[867]: time=2026-09-22T11:32:48.266Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 73.681569] nix-grpc-daemon[867]: time=2026-09-22T11:32:48.266Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)" lb # [ 73.650043] niks3-server[1033]: 2026/09/22 11:32:48 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst worker2 # [ 73.714430] nix-grpc-daemon[867]: time=2026-09-22T11:32:48.301Z level=INFO msg="Uploading 1 narinfos" lb # [ 73.666017] niks3-server[1033]: 2026/09/22 11:32:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign lb # [ 73.676644] niks3-server[1033]: 2026/09/22 11:32:48 INFO Signed narinfos id=17 count=1 lb # [ 73.684520] niks3-server[1033]: 2026/09/22 11:32:48 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls lb # [ 73.695370] niks3-server[1033]: 2026/09/22 11:32:48 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete worker2 # [ 73.749553] nix-grpc-daemon[867]: time=2026-09-22T11:32:48.336Z level=INFO msg="Upload complete. (368ms)" lb # [ 73.704275] niks3-server[1033]: 2026/09/22 11:32:48 INFO Completed upload id=17 lb # [ 73.707742] niks3-server[1033]: 2026/09/22 11:32:48 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo worker2 # [ 73.760304] nix-grpc-daemon[771]: ts=2026-09-22T11:32:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 73.775435] nix-grpc-daemon[771]: ts=2026-09-22T11:32:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 73.782090] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 73.786965] nix-daemon[842]: reaped child process 1004, status = succeeded worker2 # [ 73.797947] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) lb # [ 78.327453] envoy[893]: conn peer=[2001:db8:1::1]:33810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=3759 lb # [ 78.327830] envoy[893]: conn peer=[2001:db8:1::1]:33802 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1563 ms=4920 worker2 # [ 80.215279] nix-grpc-daemon[771]: ts=2026-09-22T11:32:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=0 worker2 # [ 80.246647] nix-daemon[842]: reaped child process 1016, status = succeeded worker2 # [ 80.249415] nix-daemon[842]: reaped child process 1018, status = succeeded worker1 # [ 80.453709] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 80.469608] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=0 worker1 # [ 80.478890] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 80.487626] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 80.495395] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 80.505838] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 80.521190] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 80.680860] nix-daemon[852]: accepted connection from pid 1129, user nix-grpc-daemon (trusted) worker1 # [ 80.706574] nix-daemon[852]: reaped child process 1132, status = succeeded worker1 # [ 80.798958] nix-daemon[852]: accepted connection from pid 1134, user nix-grpc-daemon (trusted) worker1 # [ 80.817172] nix-daemon[852]: reaped child process 1137, status = succeeded lb # [ 80.771139] niks3-server[1033]: 2026/09/22 11:32:55 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 80.826640] nix-grpc-daemon[867]: time=2026-09-22T11:32:55.421Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 80.828118] nix-grpc-daemon[867]: time=2026-09-22T11:32:55.421Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)" lb # [ 80.808404] niks3-server[1033]: 2026/09/22 11:32:55 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst worker1 # [ 80.865328] nix-grpc-daemon[867]: time=2026-09-22T11:32:55.460Z level=INFO msg="Uploading 1 narinfos" lb # [ 80.824245] niks3-server[1033]: 2026/09/22 11:32:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign lb # [ 80.833248] niks3-server[1033]: 2026/09/22 11:32:55 INFO Signed narinfos id=18 count=1 worker1 # [ 80.891467] nix-grpc-daemon[867]: time=2026-09-22T11:32:55.487Z level=INFO msg="Upload complete. (355ms)" lb # [ 80.839588] niks3-server[1033]: 2026/09/22 11:32:55 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls lb # [ 80.849801] niks3-server[1033]: 2026/09/22 11:32:55 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete lb # [ 80.855324] niks3-server[1033]: 2026/09/22 11:32:55 INFO Completed upload id=18 worker1 # [ 80.900307] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=424 lb # [ 80.859310] niks3-server[1033]: 2026/09/22 11:32:55 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo worker1 # [ 80.912006] nix-grpc-daemon[768]: ts=2026-09-22T11:32:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 80.920496] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 80.923399] nix-daemon[852]: reaped child process 1127, status = succeeded worker1 # [ 80.936415] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 87.553178] nix-grpc-daemon[768]: ts=2026-09-22T11:33:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=0 worker1 # [ 87.593863] nix-daemon[852]: reaped child process 1139, status = succeeded worker1 # [ 87.597076] nix-daemon[852]: reaped child process 1141, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 14.70 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 87.871556] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: 08686b9440c44176b31e9b420d663e9b client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.10 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share worker1 # [ 88.081026] nix-grpc-daemon[768]: ts=2026-09-22T11:33:02Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43040 client # [ 88.111066] nix[1023]: don't know how to build these paths: client # [ 88.112187] nix[1023]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 88.100149] nix-grpc-daemon[768]: ts=2026-09-22T11:33:02Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker1 # [ 88.106489] nix-grpc-daemon[768]: ts=2026-09-22T11:33:02Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 88.115835] nix-grpc-daemon[771]: ts=2026-09-22T11:33:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 88.137823] nix-grpc-daemon[771]: ts=2026-09-22T11:33:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45724 client # [ 88.170823] nix[1023]: copying 1 paths to 'lb:50051'... client # [ 88.175038] nix[1023]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'... worker2 # [ 88.157298] nix-grpc-daemon[771]: ts=2026-09-22T11:33:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 88.177703] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 88.345360] nix-daemon[842]: accepted connection from pid 1041, user nix-grpc-daemon (trusted) worker2 # [ 88.369281] nix-daemon[842]: reaped child process 1044, status = succeeded lb # [ 88.328364] envoy[893]: conn peer=[2001:db8:1::1]:49556 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=978 ms=6941 worker2 # [ 88.476115] nix-daemon[842]: accepted connection from pid 1046, user nix-grpc-daemon (trusted) worker2 # [ 88.496269] nix-daemon[842]: reaped child process 1049, status = succeeded lb # [ 88.333414] envoy[893]: conn peer=[2001:db8:1::1]:49562 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2989 tx=976 ms=7125 lb # [ 88.451186] niks3-server[1033]: 2026/09/22 11:33:03 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 88.516155] nix-grpc-daemon[867]: time=2026-09-22T11:33:03.102Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 88.517728] nix-grpc-daemon[867]: time=2026-09-22T11:33:03.102Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)" lb # [ 88.486247] niks3-server[1033]: 2026/09/22 11:33:03 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst lb # [ 88.507896] niks3-server[1033]: 2026/09/22 11:33:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign worker2 # [ 88.562252] nix-grpc-daemon[867]: time=2026-09-22T11:33:03.149Z level=INFO msg="Uploading 1 narinfos" lb # [ 88.516253] niks3-server[1033]: 2026/09/22 11:33:03 INFO Signed narinfos id=19 count=1 lb # [ 88.523990] niks3-server[1033]: 2026/09/22 11:33:03 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls lb # [ 88.536164] niks3-server[1033]: 2026/09/22 11:33:03 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete worker2 # [ 88.591924] nix-grpc-daemon[867]: time=2026-09-22T11:33:03.178Z level=INFO msg="Upload complete. (394ms)" lb # [ 88.544535] niks3-server[1033]: 2026/09/22 11:33:03 INFO Completed upload id=19 lb # [ 88.547774] niks3-server[1033]: 2026/09/22 11:33:03 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo worker2 # [ 88.604445] nix-grpc-daemon[771]: ts=2026-09-22T11:33:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 88.619563] nix-grpc-daemon[771]: ts=2026-09-22T11:33:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 88.626846] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 88.631684] nix-daemon[842]: reaped child process 1039, status = succeeded worker2 # [ 88.643862] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) client # [ 88.661529] nix[1023]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client # [ 88.990682] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 374f80440e844b26860911c9daf1753e client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.12 seconds) worker2 # [ 89.246319] nix-grpc-daemon[771]: ts=2026-09-22T11:33:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45724 client # [ 89.295237] nix[1039]: this derivation will be built: worker1 # [ 89.279068] nix-grpc-daemon[768]: ts=2026-09-22T11:33:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 89.304747] nix-grpc-daemon[768]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' from 'http://lb:5751'... client # [ 89.359652] nix[1039]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 89.357725] nix-grpc-daemon[768]: ts=2026-09-22T11:33:03Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker2 # [ 89.369250] nix-grpc-daemon[771]: ts=2026-09-22T11:33:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 client # [ 89.397027] nix[1039]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client: must succeed: sleep 2; systemctl kill -s INT share1 client # [ 92.385735] systemd[1]: share1.service: Sent signal SIGINT to main process 1023 (nix) on client request. client # [ 92.396518] nix[1023]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 92.409563] nix[1023]: error: interrupted by the user client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.10 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 92.454329] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 92.458499] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 92.460940] systemd[1]: share1.service: Consumed 199ms CPU time over 4.586s wall clock time, 41.4M memory peak, 6.1K incoming IP traffic, 8.2K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 0.06 seconds) client: waiting for success: ! systemctl is-active share2 worker2 # [ 92.925043] nix-daemon[842]: reaped child process 1053, status = succeeded worker2 # [ 93.038184] nix-daemon[842]: accepted connection from pid 1087, user nix-grpc-daemon (trusted) worker2 # [ 93.056961] nix-daemon[842]: reaped child process 1090, status = succeeded lb # [ 93.013926] niks3-server[1033]: 2026/09/22 11:33:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 93.076443] nix-grpc-daemon[867]: time=2026-09-22T11:33:07.661Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 93.078246] nix-grpc-daemon[867]: time=2026-09-22T11:33:07.661Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)" lb # [ 93.063482] niks3-server[1033]: 2026/09/22 11:33:07 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 93.077700] niks3-server[1033]: 2026/09/22 11:33:07 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker2 # [ 93.141012] nix-grpc-daemon[867]: time=2026-09-22T11:33:07.726Z level=INFO msg="Uploading 1 narinfos" lb # [ 93.093348] niks3-server[1033]: 2026/09/22 11:33:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign lb # [ 93.102413] niks3-server[1033]: 2026/09/22 11:33:07 INFO Signed narinfos id=20 count=1 lb # [ 93.109635] niks3-server[1033]: 2026/09/22 11:33:07 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls lb # [ 93.119833] niks3-server[1033]: 2026/09/22 11:33:07 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete lb # [ 93.126843] niks3-server[1033]: 2026/09/22 11:33:07 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo lb # [ 93.134597] niks3-server[1033]: 2026/09/22 11:33:07 INFO Completed upload id=20 worker2 # [ 93.183108] nix-grpc-daemon[867]: time=2026-09-22T11:33:07.770Z level=INFO msg="Upload complete. (317ms)" worker2 # [ 93.197121] nix-grpc-daemon[771]: ts=2026-09-22T11:33:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker2 # [ 93.212446] nix-grpc-daemon[771]: ts=2026-09-22T11:33:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=3 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker2 # [ 93.221535] nix-daemon[842]: reaped child process 1051, status = succeeded client # [ 93.289131] systemd[1]: share2.service: Deactivated successfully. client # [ 93.294230] systemd[1]: share2.service: Consumed 258ms CPU time over 4.300s wall clock time, 31.3M memory peak, 6.6K incoming IP traffic, 5.9K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.06 seconds) (finished: subtest: the first client leaving does not fail the second, in 6.10 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 94.039373] systemd[1]: Started Nix Daemon. client # [ 94.157899] nix-daemon[1086]: accepted connection from pid 1083, user root (trusted) worker2 # [ 94.286511] nix-grpc-daemon[771]: ts=2026-09-22T11:33:08Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:45724 worker1 # [ 94.286760] nix-grpc-daemon[768]: ts=2026-09-22T11:33:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 94.305513] nix-grpc-daemon[768]: ts=2026-09-22T11:33:08Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=0 worker1 # [ 94.312777] nix-grpc-daemon[768]: ts=2026-09-22T11:33:08Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 94.321886] nix-grpc-daemon[768]: ts=2026-09-22T11:33:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 94.333223] nix-grpc-daemon[768]: ts=2026-09-22T11:33:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 94.348786] nix-grpc-daemon[768]: ts=2026-09-22T11:33:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 94.365002] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 94.521115] nix-daemon[852]: accepted connection from pid 1187, user nix-grpc-daemon (trusted) worker1 # [ 94.541626] nix-daemon[852]: reaped child process 1190, status = succeeded worker1 # [ 94.640989] nix-daemon[852]: accepted connection from pid 1192, user nix-grpc-daemon (trusted) worker1 # [ 94.659457] nix-daemon[852]: reaped child process 1195, status = succeeded lb # [ 94.619105] niks3-server[1033]: 2026/09/22 11:33:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 94.673941] nix-grpc-daemon[867]: time=2026-09-22T11:33:09.268Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 94.675441] nix-grpc-daemon[867]: time=2026-09-22T11:33:09.268Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)" lb # [ 94.648695] niks3-server[1033]: 2026/09/22 11:33:09 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst lb # [ 94.661834] niks3-server[1033]: 2026/09/22 11:33:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign worker1 # [ 94.711475] nix-grpc-daemon[867]: time=2026-09-22T11:33:09.306Z level=INFO msg="Uploading 1 narinfos" lb # [ 94.669679] niks3-server[1033]: 2026/09/22 11:33:09 INFO Signed narinfos id=21 count=1 lb # [ 94.675838] niks3-server[1033]: 2026/09/22 11:33:09 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls lb # [ 94.684602] niks3-server[1033]: 2026/09/22 11:33:09 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete worker1 # [ 94.733826] nix-grpc-daemon[867]: time=2026-09-22T11:33:09.329Z level=INFO msg="Upload complete. (352ms)" lb # [ 94.691375] niks3-server[1033]: 2026/09/22 11:33:09 INFO Completed upload id=21 lb # [ 94.696097] niks3-server[1033]: 2026/09/22 11:33:09 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo worker1 # [ 94.744291] nix-grpc-daemon[768]: ts=2026-09-22T11:33:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 94.755092] nix-grpc-daemon[768]: ts=2026-09-22T11:33:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 94.760798] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 94.765536] nix-daemon[852]: reaped child process 1185, status = succeeded worker1 # [ 94.775268] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 95.811968] nix-daemon[852]: reaped child process 1199, status = succeeded worker1 # [ 95.903032] nix-daemon[852]: accepted connection from pid 1215, user nix-grpc-daemon (trusted) worker1 # [ 95.923159] nix-daemon[852]: reaped child process 1218, status = succeeded lb # [ 95.881497] niks3-server[1033]: 2026/09/22 11:33:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 95.936372] nix-grpc-daemon[867]: time=2026-09-22T11:33:10.530Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 95.937807] nix-grpc-daemon[867]: time=2026-09-22T11:33:10.530Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)" lb # [ 95.914325] niks3-server[1033]: 2026/09/22 11:33:10 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv lb # [ 95.934114] niks3-server[1033]: 2026/09/22 11:33:10 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst worker1 # [ 95.989580] nix-grpc-daemon[867]: time=2026-09-22T11:33:10.584Z level=INFO msg="Uploading 1 narinfos" lb # [ 95.945423] niks3-server[1033]: 2026/09/22 11:33:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign lb # [ 95.953633] niks3-server[1033]: 2026/09/22 11:33:10 INFO Signed narinfos id=22 count=1 lb # [ 95.958335] niks3-server[1033]: 2026/09/22 11:33:10 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls worker1 # [ 96.010414] nix-grpc-daemon[867]: time=2026-09-22T11:33:10.606Z level=INFO msg="Upload complete. (224ms)" lb # [ 95.967473] niks3-server[1033]: 2026/09/22 11:33:10 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete lb # [ 95.974176] niks3-server[1033]: 2026/09/22 11:33:10 INFO Completed upload id=22 lb # [ 95.976398] niks3-server[1033]: 2026/09/22 11:33:10 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo worker1 # [ 96.021051] nix-grpc-daemon[768]: ts=2026-09-22T11:33:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=1 worker1 # [ 96.037999] nix-daemon[852]: reaped child process 1197, status = succeeded worker1 # [ 96.065340] nix-grpc-daemon[768]: ts=2026-09-22T11:33:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 96.100458] nix-grpc-daemon[768]: ts=2026-09-22T11:33:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 96.107345] nix-grpc-daemon[768]: ts=2026-09-22T11:33:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_out=128 worker2 # [ 96.399880] nix-grpc-daemon[771]: ts=2026-09-22T11:33:10Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 96.407789] nix-grpc-daemon[771]: ts=2026-09-22T11:33:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45724 worker1 # [ 96.425223] nix-grpc-daemon[768]: ts=2026-09-22T11:33:11Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=0 worker1 # [ 96.434444] nix-grpc-daemon[768]: ts=2026-09-22T11:33:11Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 96.443051] nix-grpc-daemon[768]: ts=2026-09-22T11:33:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 96.450568] nix-grpc-daemon[768]: ts=2026-09-22T11:33:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 96.461237] nix-grpc-daemon[867]: time=2026-09-22T11:33:11.056Z level=INFO msg="All 1 paths already cached" worker1 # [ 96.470986] nix-grpc-daemon[768]: ts=2026-09-22T11:33:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 96.485025] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 96.647032] nix-daemon[852]: accepted connection from pid 1224, user nix-grpc-daemon (trusted) worker1 # [ 96.667447] nix-daemon[852]: reaped child process 1227, status = succeeded worker1 # [ 96.766831] nix-daemon[852]: accepted connection from pid 1229, user nix-grpc-daemon (trusted) worker1 # [ 96.784798] nix-daemon[852]: reaped child process 1232, status = succeeded lb # [ 96.738836] niks3-server[1033]: 2026/09/22 11:33:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 96.794488] nix-grpc-daemon[867]: time=2026-09-22T11:33:11.389Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 96.796230] nix-grpc-daemon[867]: time=2026-09-22T11:33:11.389Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)" lb # [ 96.771960] niks3-server[1033]: 2026/09/22 11:33:11 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst worker1 # [ 96.833339] nix-grpc-daemon[867]: time=2026-09-22T11:33:11.428Z level=INFO msg="Uploading 1 narinfos" lb # [ 96.789805] niks3-server[1033]: 2026/09/22 11:33:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign lb # [ 96.799565] niks3-server[1033]: 2026/09/22 11:33:11 INFO Signed narinfos id=23 count=1 lb # [ 96.806801] niks3-server[1033]: 2026/09/22 11:33:11 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls worker1 # [ 96.864084] nix-grpc-daemon[867]: time=2026-09-22T11:33:11.459Z level=INFO msg="Upload complete. (362ms)" lb # [ 96.821624] niks3-server[1033]: 2026/09/22 11:33:11 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete worker1 # [ 96.872451] nix-grpc-daemon[768]: ts=2026-09-22T11:33:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 96.830325] niks3-server[1033]: 2026/09/22 11:33:11 INFO Completed upload id=23 lb # [ 96.833360] niks3-server[1033]: 2026/09/22 11:33:11 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo worker1 # [ 96.884994] nix-grpc-daemon[768]: ts=2026-09-22T11:33:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 96.891555] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 96.897286] nix-daemon[852]: reaped child process 1222, status = succeeded worker1 # [ 96.909153] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 97.970723] nix-daemon[852]: reaped child process 1236, status = succeeded worker1 # [ 98.098205] nix-daemon[852]: accepted connection from pid 1252, user nix-grpc-daemon (trusted) worker1 # [ 98.116222] nix-daemon[852]: reaped child process 1255, status = succeeded lb # [ 98.078701] niks3-server[1033]: 2026/09/22 11:33:12 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 98.136456] nix-grpc-daemon[867]: time=2026-09-22T11:33:12.729Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 98.138252] nix-grpc-daemon[867]: time=2026-09-22T11:33:12.729Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)" lb # [ 98.119544] niks3-server[1033]: 2026/09/22 11:33:12 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst lb # [ 98.140786] niks3-server[1033]: 2026/09/22 11:33:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign worker1 # [ 98.191107] nix-grpc-daemon[867]: time=2026-09-22T11:33:12.786Z level=INFO msg="Uploading 1 narinfos" lb # [ 98.152714] niks3-server[1033]: 2026/09/22 11:33:12 INFO Signed narinfos id=24 count=1 lb # [ 98.159090] niks3-server[1033]: 2026/09/22 11:33:12 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv lb # [ 98.168260] niks3-server[1033]: 2026/09/22 11:33:12 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls lb # [ 98.176642] niks3-server[1033]: 2026/09/22 11:33:12 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete lb # [ 98.183553] niks3-server[1033]: 2026/09/22 11:33:12 INFO Completed upload id=24 worker1 # [ 98.228241] nix-grpc-daemon[867]: time=2026-09-22T11:33:12.823Z level=INFO msg="Upload complete. (308ms)" lb # [ 98.187190] niks3-server[1033]: 2026/09/22 11:33:12 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo worker1 # [ 98.244755] nix-grpc-daemon[768]: ts=2026-09-22T11:33:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=1 worker2 # [ 98.251975] nix-grpc-daemon[771]: ts=2026-09-22T11:33:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45724 worker1 # [ 98.260865] nix-daemon[852]: reaped child process 1234, status = succeeded worker2 # [ 98.262565] nix-grpc-daemon[771]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'... worker2 # [ 98.278475] nix-grpc-daemon[771]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'... worker2 # [ 98.317030] nix-grpc-daemon[771]: ts=2026-09-22T11:33:12Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 98.323866] nix-grpc-daemon[771]: ts=2026-09-22T11:33:12Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_out=176 lb # [ 98.327668] envoy[893]: conn peer=[2001:db8:1::1]:41478 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2977 tx=814 ms=4317 lb # [ 98.332815] envoy[893]: conn peer=[2001:db8:1::1]:41482 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1592 tx=1599 ms=4012 lb # [ 98.337631] envoy[893]: conn peer=[2001:db8:1::1]:35630 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3400 tx=1965 ms=1887 lb # [ 98.342289] envoy[893]: conn peer=[2001:db8:1::1]:35638 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=76 lb # [ 98.346918] envoy[893]: conn peer=[2001:db8:1::1]:35640 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3886 tx=2180 ms=1983 lb # [ 98.351713] envoy[893]: conn peer=[2001:db8:1::1]:35652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=69 worker2 # [ 98.599271] nix-grpc-daemon[771]: ts=2026-09-22T11:33:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 98.606859] nix-grpc-daemon[771]: ts=2026-09-22T11:33:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 98.614775] nix-grpc-daemon[867]: time=2026-09-22T11:33:13.199Z level=INFO msg="All 2 paths already cached" worker1 # [ 98.622455] nix-grpc-daemon[768]: ts=2026-09-22T11:33:13Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=0 worker1 # [ 98.628174] nix-grpc-daemon[768]: ts=2026-09-22T11:33:13Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 98.632653] nix-grpc-daemon[768]: ts=2026-09-22T11:33:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 98.637009] nix-grpc-daemon[768]: ts=2026-09-22T11:33:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 98.643878] nix-grpc-daemon[867]: time=2026-09-22T11:33:13.239Z level=INFO msg="All 2 paths already cached" worker1 # [ 98.653858] nix-grpc-daemon[768]: ts=2026-09-22T11:33:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 98.668613] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 98.819874] nix-daemon[852]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker1 # [ 98.835999] nix-daemon[852]: reaped child process 1264, status = succeeded worker1 # [ 98.938691] nix-daemon[852]: accepted connection from pid 1266, user nix-grpc-daemon (trusted) worker1 # [ 98.953760] nix-daemon[852]: reaped child process 1269, status = succeeded lb # [ 98.914863] niks3-server[1033]: 2026/09/22 11:33:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 98.969230] nix-grpc-daemon[867]: time=2026-09-22T11:33:13.564Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 98.970713] nix-grpc-daemon[867]: time=2026-09-22T11:33:13.564Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)" lb # [ 98.944278] niks3-server[1033]: 2026/09/22 11:33:13 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst worker1 # [ 99.000951] nix-grpc-daemon[867]: time=2026-09-22T11:33:13.596Z level=INFO msg="Uploading 1 narinfos" lb # [ 98.957783] niks3-server[1033]: 2026/09/22 11:33:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign lb # [ 98.969801] niks3-server[1033]: 2026/09/22 11:33:13 INFO Signed narinfos id=25 count=1 lb # [ 98.974818] niks3-server[1033]: 2026/09/22 11:33:13 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls worker1 # [ 99.030432] nix-grpc-daemon[867]: time=2026-09-22T11:33:13.626Z level=INFO msg="Upload complete. (347ms)" lb # [ 98.985818] niks3-server[1033]: 2026/09/22 11:33:13 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete worker1 # [ 99.036244] nix-grpc-daemon[768]: ts=2026-09-22T11:33:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 98.995631] niks3-server[1033]: 2026/09/22 11:33:13 INFO Completed upload id=25 lb # [ 98.998305] niks3-server[1033]: 2026/09/22 11:33:13 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo worker1 # [ 99.045578] nix-grpc-daemon[768]: ts=2026-09-22T11:33:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 99.055967] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 99.061419] nix-daemon[852]: reaped child process 1259, status = succeeded worker1 # [ 99.072182] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 100.323922] nix-daemon[852]: reaped child process 1273, status = succeeded worker1 # [ 100.448228] nix-daemon[852]: accepted connection from pid 1289, user nix-grpc-daemon (trusted) worker1 # [ 100.465450] nix-daemon[852]: reaped child process 1292, status = succeeded lb # [ 100.430459] niks3-server[1033]: 2026/09/22 11:33:15 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 100.484079] nix-grpc-daemon[867]: time=2026-09-22T11:33:15.078Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 100.485767] nix-grpc-daemon[867]: time=2026-09-22T11:33:15.078Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)" lb # [ 100.464561] niks3-server[1033]: 2026/09/22 11:33:15 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker1 # [ 100.525127] nix-grpc-daemon[867]: time=2026-09-22T11:33:15.120Z level=INFO msg="Uploading 1 narinfos" lb # [ 100.480788] niks3-server[1033]: 2026/09/22 11:33:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign lb # [ 100.488455] niks3-server[1033]: 2026/09/22 11:33:15 INFO Signed narinfos id=26 count=1 lb # [ 100.495221] niks3-server[1033]: 2026/09/22 11:33:15 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst lb # [ 100.509336] niks3-server[1033]: 2026/09/22 11:33:15 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls worker1 # [ 100.558242] nix-grpc-daemon[867]: time=2026-09-22T11:33:15.153Z level=INFO msg="Upload complete. (274ms)" lb # [ 100.516115] niks3-server[1033]: 2026/09/22 11:33:15 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete lb # [ 100.520083] niks3-server[1033]: 2026/09/22 11:33:15 INFO Completed upload id=26 lb # [ 100.525556] niks3-server[1033]: 2026/09/22 11:33:15 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo worker1 # [ 100.571641] nix-grpc-daemon[768]: ts=2026-09-22T11:33:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=1 worker1 # [ 100.592371] nix-grpc-daemon[768]: ts=2026-09-22T11:33:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 100.598123] nix-daemon[852]: reaped child process 1271, status = succeeded worker2 # [ 100.612646] nix-grpc-daemon[771]: ts=2026-09-22T11:33:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 100.626318] nix-grpc-daemon[771]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'... worker2 # [ 100.642047] nix-grpc-daemon[771]: ts=2026-09-22T11:33:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_out=232 client # [ 100.787485] nix-daemon[1086]: reaped child process 1088, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.91 seconds) client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.03 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.30 seconds) client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds) worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 101.744566] nix-daemon[1086]: accepted connection from pid 1169, user root (trusted) client # this derivation will be built: client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 101.874923] nix-grpc-daemon[771]: ts=2026-09-22T11:33:16Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:45724 worker1 # [ 101.875649] nix-grpc-daemon[768]: ts=2026-09-22T11:33:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 102.045571] nix-daemon[852]: accepted connection from pid 1299, user nix-grpc-daemon (trusted) worker1 # [ 102.064516] nix-daemon[852]: reaped child process 1302, status = succeeded lb # [ 102.025114] niks3-server[1033]: 2026/09/22 11:33:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 102.080296] nix-grpc-daemon[867]: time=2026-09-22T11:33:16.675Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 102.082055] nix-grpc-daemon[867]: time=2026-09-22T11:33:16.675Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)" lb # [ 102.052824] niks3-server[1033]: 2026/09/22 11:33:16 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst worker1 # [ 102.111241] nix-grpc-daemon[867]: time=2026-09-22T11:33:16.706Z level=INFO msg="Uploading 1 narinfos" lb # [ 102.067816] niks3-server[1033]: 2026/09/22 11:33:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign lb # [ 102.078196] niks3-server[1033]: 2026/09/22 11:33:16 INFO Signed narinfos id=27 count=1 lb # [ 102.084307] niks3-server[1033]: 2026/09/22 11:33:16 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls lb # [ 102.094000] niks3-server[1033]: 2026/09/22 11:33:16 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete worker1 # [ 102.143305] nix-grpc-daemon[867]: time=2026-09-22T11:33:16.738Z level=INFO msg="Upload complete. (261ms)" lb # [ 102.100385] niks3-server[1033]: 2026/09/22 11:33:16 INFO Completed upload id=27 lb # [ 102.103125] niks3-server[1033]: 2026/09/22 11:33:16 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo worker1 # [ 102.169998] nix-grpc-daemon[768]: ts=2026-09-22T11:33:16Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=0 worker1 # [ 102.176524] nix-grpc-daemon[768]: ts=2026-09-22T11:33:16Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 102.221582] nix-grpc-daemon[771]: ts=2026-09-22T11:33:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 102.236691] nix-grpc-daemon[771]: ts=2026-09-22T11:33:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45724 client # copying 2 paths to 'lb:50051'... client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'... worker2 # [ 102.254769] nix-grpc-daemon[771]: ts=2026-09-22T11:33:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'... worker2 # [ 102.269809] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 102.438792] nix-daemon[842]: accepted connection from pid 1100, user nix-grpc-daemon (trusted) worker2 # [ 102.456609] nix-daemon[842]: reaped child process 1103, status = succeeded worker2 # [ 102.555783] nix-daemon[842]: accepted connection from pid 1105, user nix-grpc-daemon (trusted) worker2 # [ 102.569838] nix-daemon[842]: reaped child process 1108, status = succeeded lb # [ 102.528532] niks3-server[1033]: 2026/09/22 11:33:17 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 102.537727] niks3-server[1033]: 2026/09/22 11:33:17 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 102.595915] nix-grpc-daemon[867]: time=2026-09-22T11:33:17.182Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 102.597554] nix-grpc-daemon[867]: time=2026-09-22T11:33:17.182Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)" worker2 # [ 102.599253] nix-grpc-daemon[867]: time=2026-09-22T11:33:17.182Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)" lb # [ 102.577854] niks3-server[1033]: 2026/09/22 11:33:17 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst lb # [ 102.594376] niks3-server[1033]: 2026/09/22 11:33:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign worker2 # [ 102.652134] nix-grpc-daemon[867]: time=2026-09-22T11:33:17.238Z level=INFO msg="Uploading 3 narinfos" lb # [ 102.607733] niks3-server[1033]: 2026/09/22 11:33:17 INFO Signed narinfos id=29 count=2 lb # [ 102.613554] niks3-server[1033]: 2026/09/22 11:33:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign lb # [ 102.619178] niks3-server[1033]: 2026/09/22 11:33:17 INFO Signed narinfos id=28 count=1 lb # [ 102.624362] niks3-server[1033]: 2026/09/22 11:33:17 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls lb # [ 102.627013] niks3-server[1033]: 2026/09/22 11:33:17 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst lb # [ 102.637975] niks3-server[1033]: 2026/09/22 11:33:17 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls lb # [ 102.644736] niks3-server[1033]: 2026/09/22 11:33:17 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 102.651631] niks3-server[1033]: 2026/09/22 11:33:17 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 102.658358] niks3-server[1033]: 2026/09/22 11:33:17 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 102.662708] niks3-server[1033]: 2026/09/22 11:33:17 INFO Completed upload id=28 lb # [ 102.666233] niks3-server[1033]: 2026/09/22 11:33:17 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete worker2 # [ 102.714795] nix-grpc-daemon[867]: time=2026-09-22T11:33:17.301Z level=INFO msg="Upload complete. (425ms)" lb # [ 102.668868] niks3-server[1033]: 2026/09/22 11:33:17 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo lb # [ 102.671587] niks3-server[1033]: 2026/09/22 11:33:17 INFO Completed upload id=29 worker2 # [ 102.724517] nix-grpc-daemon[771]: ts=2026-09-22T11:33:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=2 nar_bytes_in=984 worker2 # [ 102.740912] nix-grpc-daemon[771]: ts=2026-09-22T11:33:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 102.755266] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 102.761645] nix-daemon[842]: reaped child process 1098, status = succeeded worker2 # [ 102.778464] nix-grpc-daemon[771]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'... client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051... worker2 # [ 102.794631] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 104.041080] nix-daemon[842]: reaped child process 1113, status = succeeded worker2 # [ 104.139516] nix-daemon[842]: accepted connection from pid 1129, user nix-grpc-daemon (trusted) worker2 # [ 104.156769] nix-daemon[842]: reaped child process 1132, status = succeeded lb # [ 104.114525] niks3-server[1033]: 2026/09/22 11:33:18 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 104.174387] nix-grpc-daemon[867]: time=2026-09-22T11:33:18.759Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 104.145254] niks3-server[1033]: 2026/09/22 11:33:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign worker2 # [ 104.202691] nix-grpc-daemon[867]: time=2026-09-22T11:33:18.789Z level=INFO msg="Uploading 1 narinfos" lb # [ 104.160479] niks3-server[1033]: 2026/09/22 11:33:18 INFO Signed narinfos id=30 count=1 lb # [ 104.168812] niks3-server[1033]: 2026/09/22 11:33:18 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv lb # [ 104.182028] niks3-server[1033]: 2026/09/22 11:33:18 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls lb # [ 104.185615] niks3-server[1033]: 2026/09/22 11:33:18 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 104.191041] niks3-server[1033]: 2026/09/22 11:33:18 INFO Completed upload id=30 worker2 # [ 104.239128] nix-grpc-daemon[867]: time=2026-09-22T11:33:18.826Z level=INFO msg="Upload complete. (228ms)" lb # [ 104.195238] niks3-server[1033]: 2026/09/22 11:33:18 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo worker2 # [ 104.248961] nix-grpc-daemon[771]: ts=2026-09-22T11:33:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=1 worker2 # [ 104.264798] nix-daemon[842]: reaped child process 1110, status = succeeded client # copying 1 paths... worker2 # [ 104.298824] nix-grpc-daemon[771]: ts=2026-09-22T11:33:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45724 client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'... worker2 # [ 104.330193] nix-grpc-daemon[771]: ts=2026-09-22T11:33:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 104.337022] nix-grpc-daemon[771]: ts=2026-09-22T11:33:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_out=120 client # [ 104.478515] nix-daemon[1086]: reaped child process 1172, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.93 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 104.654316] nix-daemon[1086]: accepted connection from pid 1191, user root (trusted) client # this derivation will be built: client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker2 # [ 104.780106] nix-grpc-daemon[771]: ts=2026-09-22T11:33:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:45724 client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 104.787650] nix-grpc-daemon[771]: ts=2026-09-22T11:33:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 104.795031] nix-grpc-daemon[867]: time=2026-09-22T11:33:19.380Z level=INFO msg="All 1 paths already cached" worker1 # [ 104.807136] nix-grpc-daemon[768]: ts=2026-09-22T11:33:19Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=0 worker1 # [ 104.807484] nix-grpc-daemon[768]: ts=2026-09-22T11:33:19Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 104.807646] nix-grpc-daemon[768]: ts=2026-09-22T11:33:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 104.830398] nix-grpc-daemon[768]: ts=2026-09-22T11:33:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'... worker1 # [ 104.850668] nix-grpc-daemon[768]: ts=2026-09-22T11:33:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 104.865765] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 104.886889] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 1/5) worker1 # [ 104.973773] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 191 ms (attempt 2/5) worker1 # [ 105.166582] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 3/5) worker1 # [ 105.265401] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 393 ms (attempt 4/5) worker1 # [ 105.660863] nix-grpc-daemon[768]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 105.669245] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 105.676652] nix-grpc-daemon[768]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'... worker1 # [ 105.840233] nix-daemon[852]: accepted connection from pid 1315, user nix-grpc-daemon (trusted) worker1 # [ 105.860312] nix-daemon[852]: reaped child process 1318, status = succeeded worker1 # [ 105.961744] nix-daemon[852]: accepted connection from pid 1320, user nix-grpc-daemon (trusted) worker1 # [ 105.977766] nix-daemon[852]: reaped child process 1323, status = succeeded lb # [ 105.939408] niks3-server[1033]: 2026/09/22 11:33:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 105.994188] nix-grpc-daemon[867]: time=2026-09-22T11:33:20.589Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 105.995656] nix-grpc-daemon[867]: time=2026-09-22T11:33:20.589Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)" lb # [ 105.966486] niks3-server[1033]: 2026/09/22 11:33:20 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst lb # [ 105.981946] niks3-server[1033]: 2026/09/22 11:33:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign worker1 # [ 106.034350] nix-grpc-daemon[867]: time=2026-09-22T11:33:20.629Z level=INFO msg="Uploading 1 narinfos" lb # [ 105.995318] niks3-server[1033]: 2026/09/22 11:33:20 INFO Signed narinfos id=31 count=1 lb # [ 106.002582] niks3-server[1033]: 2026/09/22 11:33:20 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls worker1 # [ 106.060831] nix-grpc-daemon[867]: time=2026-09-22T11:33:20.656Z level=INFO msg="Upload complete. (371ms)" lb # [ 106.016559] niks3-server[1033]: 2026/09/22 11:33:20 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete worker1 # [ 106.068707] nix-grpc-daemon[768]: ts=2026-09-22T11:33:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 paths=1 nar_bytes_in=568 lb # [ 106.026796] niks3-server[1033]: 2026/09/22 11:33:20 INFO Completed upload id=31 lb # [ 106.030515] niks3-server[1033]: 2026/09/22 11:33:20 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo worker1 # [ 106.079946] nix-grpc-daemon[768]: ts=2026-09-22T11:33:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 106.090650] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 106.096358] nix-daemon[852]: reaped child process 1308, status = succeeded client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051... worker1 # [ 106.107943] nix-daemon[852]: accepted connection from pid 768, user nix-grpc-daemon (trusted) client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 107.569747] nix-daemon[852]: reaped child process 1327, status = succeeded worker1 # [ 107.701763] nix-daemon[852]: accepted connection from pid 1343, user nix-grpc-daemon (trusted) worker1 # [ 107.718104] nix-daemon[852]: reaped child process 1346, status = succeeded lb # [ 107.681839] niks3-server[1033]: 2026/09/22 11:33:22 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 107.739297] nix-grpc-daemon[867]: time=2026-09-22T11:33:22.331Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" worker1 # [ 107.771200] nix-grpc-daemon[867]: time=2026-09-22T11:33:22.366Z level=INFO msg="Uploading 1 narinfos" lb # [ 107.715010] niks3-server[1033]: 2026/09/22 11:33:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign lb # [ 107.731501] niks3-server[1033]: 2026/09/22 11:33:22 INFO Signed narinfos id=32 count=1 lb # [ 107.738059] niks3-server[1033]: 2026/09/22 11:33:22 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls lb # [ 107.751409] niks3-server[1033]: 2026/09/22 11:33:22 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 107.805055] nix-grpc-daemon[867]: time=2026-09-22T11:33:22.400Z level=INFO msg="Upload complete. (287ms)" lb # [ 107.763145] niks3-server[1033]: 2026/09/22 11:33:22 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 107.769226] niks3-server[1033]: 2026/09/22 11:33:22 INFO Completed upload id=32 lb # [ 107.772377] niks3-server[1033]: 2026/09/22 11:33:22 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo worker1 # [ 107.817142] nix-grpc-daemon[768]: ts=2026-09-22T11:33:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43040 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=1 worker1 # [ 107.837135] nix-daemon[852]: reaped child process 1325, status = succeeded client # copying 1 paths... worker2 # [ 107.867748] nix-grpc-daemon[771]: ts=2026-09-22T11:33:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 107.880941] nix-grpc-daemon[771]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'... client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'... worker2 # [ 107.924278] nix-grpc-daemon[771]: ts=2026-09-22T11:33:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 107.931409] nix-grpc-daemon[771]: ts=2026-09-22T11:33:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_out=120 client # [ 108.077995] nix-daemon[1086]: reaped child process 1194, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.60 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 108.253133] nix-daemon[1086]: accepted connection from pid 1212, user root (trusted) client # this derivation will be built: client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv lb # [ 108.329280] envoy[893]: conn peer=[2001:db8:1::1]:35662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=407 ms=92 lb # [ 108.334532] envoy[893]: conn peer=[2001:db8:1::1]:35658 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4337 tx=2389 ms=2102 lb # [ 108.340433] envoy[893]: conn peer=[2001:db8:1::1]:36600 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3879 tx=2084 ms=2519 lb # [ 108.346525] envoy[893]: conn peer=[2001:db8:1::1]:36604 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=66 worker2 # [ 108.396403] nix-grpc-daemon[771]: ts=2026-09-22T11:33:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:45724 client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... lb # [ 108.351988] envoy[893]: conn peer=[2001:db8:1::1]:36618 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3752 tx=2125 ms=3209 worker1 # [ 108.403104] nix-grpc-daemon[768]: ts=2026-09-22T11:33:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43040 lb # [ 108.357845] envoy[893]: conn peer=[2001:db8:1::1]:36634 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=69 worker1 # [ 108.411426] nix-grpc-daemon[867]: time=2026-09-22T11:33:23.007Z level=INFO msg="All 1 paths already cached" worker1 # [ 108.428764] nix-grpc-daemon[768]: ts=2026-09-22T11:33:23Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=0 worker1 # [ 108.435398] nix-grpc-daemon[768]: ts=2026-09-22T11:33:23Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 108.442773] nix-grpc-daemon[771]: ts=2026-09-22T11:33:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 108.458298] nix-grpc-daemon[771]: ts=2026-09-22T11:33:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45724 client # copying 1 paths to 'lb:50051'... worker2 # [ 108.474952] nix-grpc-daemon[867]: time=2026-09-22T11:33:23.059Z level=INFO msg="All 1 paths already cached" client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'... worker2 # [ 108.480702] nix-grpc-daemon[771]: ts=2026-09-22T11:33:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 108.495356] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 108.660276] nix-daemon[842]: accepted connection from pid 1138, user nix-grpc-daemon (trusted) worker2 # [ 108.679994] nix-daemon[842]: reaped child process 1141, status = succeeded worker2 # [ 108.776809] nix-daemon[842]: accepted connection from pid 1143, user nix-grpc-daemon (trusted) worker2 # [ 108.791617] nix-daemon[842]: reaped child process 1146, status = succeeded lb # [ 108.750219] niks3-server[1033]: 2026/09/22 11:33:23 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 108.809686] nix-grpc-daemon[867]: time=2026-09-22T11:33:23.396Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 108.811159] nix-grpc-daemon[867]: time=2026-09-22T11:33:23.396Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)" lb # [ 108.779358] niks3-server[1033]: 2026/09/22 11:33:23 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst worker2 # [ 108.843454] nix-grpc-daemon[867]: time=2026-09-22T11:33:23.430Z level=INFO msg="Uploading 1 narinfos" lb # [ 108.795914] niks3-server[1033]: 2026/09/22 11:33:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign lb # [ 108.807777] niks3-server[1033]: 2026/09/22 11:33:23 INFO Signed narinfos id=33 count=1 lb # [ 108.813345] niks3-server[1033]: 2026/09/22 11:33:23 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls worker2 # [ 108.873276] nix-grpc-daemon[867]: time=2026-09-22T11:33:23.460Z level=INFO msg="Upload complete. (360ms)" lb # [ 108.822702] niks3-server[1033]: 2026/09/22 11:33:23 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete lb # [ 108.828110] niks3-server[1033]: 2026/09/22 11:33:23 INFO Completed upload id=33 lb # [ 108.832470] niks3-server[1033]: 2026/09/22 11:33:23 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo worker2 # [ 108.881162] nix-grpc-daemon[771]: ts=2026-09-22T11:33:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_in=568 worker2 # [ 108.895155] nix-grpc-daemon[771]: ts=2026-09-22T11:33:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 108.904268] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 108.906889] nix-daemon[842]: reaped child process 1136, status = succeeded client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051... worker2 # [ 108.921649] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted) client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 109.980468] nix-daemon[842]: reaped child process 1150, status = succeeded worker2 # [ 110.078028] nix-daemon[842]: accepted connection from pid 1166, user nix-grpc-daemon (trusted) worker2 # [ 110.096622] nix-daemon[842]: reaped child process 1169, status = succeeded lb # [ 110.050487] niks3-server[1033]: 2026/09/22 11:33:24 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 110.108343] nix-grpc-daemon[867]: time=2026-09-22T11:33:24.695Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 110.083109] niks3-server[1033]: 2026/09/22 11:33:24 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls worker2 # [ 110.143544] nix-grpc-daemon[867]: time=2026-09-22T11:33:24.730Z level=INFO msg="Uploading 1 narinfos" lb # [ 110.095158] niks3-server[1033]: 2026/09/22 11:33:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign lb # [ 110.107137] niks3-server[1033]: 2026/09/22 11:33:24 INFO Signed narinfos id=34 count=1 lb # [ 110.115654] niks3-server[1033]: 2026/09/22 11:33:24 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker2 # [ 110.170495] nix-grpc-daemon[867]: time=2026-09-22T11:33:24.757Z level=INFO msg="Upload complete. (236ms)" lb # [ 110.122760] niks3-server[1033]: 2026/09/22 11:33:24 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 110.129663] niks3-server[1033]: 2026/09/22 11:33:24 INFO Completed upload id=34 lb # [ 110.131971] niks3-server[1033]: 2026/09/22 11:33:24 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo worker2 # [ 110.180982] nix-grpc-daemon[771]: ts=2026-09-22T11:33:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=1 worker2 # [ 110.196439] nix-daemon[842]: reaped child process 1148, status = succeeded client # copying 1 paths... worker1 # [ 110.223343] nix-grpc-daemon[768]: ts=2026-09-22T11:33:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43040 worker1 # [ 110.236544] nix-grpc-daemon[768]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'... client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'... worker2 # [ 110.283021] nix-grpc-daemon[771]: ts=2026-09-22T11:33:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 worker2 # [ 110.290267] nix-grpc-daemon[771]: ts=2026-09-22T11:33:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45724 duration_s=0 paths=1 nar_bytes_out=120 client # [ 110.391685] nix-daemon[1086]: reaped child process 1215, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.31 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 9.48 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds) client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds) client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.11 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds) worker2 # [ 111.359051] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 111.763161] nix-grpc-daemon[771]: ts=2026-09-22T11:33:26Z level=info event=signal_exit worker2 # [ 113.147823] nix-grpc-daemon[771]: ts=2026-09-22T11:33:27Z level=info event=scheduler_restarting worker1 # [ 113.144310] nix-grpc-daemon[768]: ts=2026-09-22T11:33:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:43034 worker1 # [ 113.144629] nix-grpc-daemon[768]: ts=2026-09-22T11:33:27Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 113.180297] nix-grpc-daemon[771]: ts=2026-09-22T11:33:27Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 113.201007] nix-daemon[842]: reaped child process 844, status = succeeded worker2 # [ 113.916641] nix-grpc-daemon[771]: E0000 00:00:1790076808.503431 816 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker2 # [ 113.949625] nix-grpc-daemon[771]: E0000 00:00:1790076808.536530 816 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 115.148971] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 115.154135] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 115.160574] systemd[1]: nix-grpc-daemon.service: Consumed 3.387s CPU time over 1min 36.637s wall clock time, 58.6M memory peak, 88.9K incoming IP traffic, 233.4K outgoing IP traffic. worker2 # [ 115.176465] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 115.183948] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 115.186611] systemd[1]: nix-grpc-daemon.socket: Consumed 632us CPU time over 1min 44.772s wall clock time, 256K memory peak, 101.3K incoming IP traffic, 88.6K outgoing IP traffic. lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2 worker1 # [ 115.499798] nix-grpc-daemon[768]: ts=2026-09-22T11:33:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:43040 client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker1 # [ 115.506857] nix-grpc-daemon[768]: ts=2026-09-22T11:33:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:43040 client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.26 seconds) worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds) worker2 # [ 115.626121] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 115.695406] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 116.147897] systemd[1]: Starting scheduler-token.service... worker2 # [ 116.238547] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 116.249977] systemd[1]: Finished scheduler-token.service. worker2 # [ 116.251856] systemd[1]: scheduler-token.service: Consumed 61ms CPU time over 94ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 116.258160] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 116.383587] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 116.385940] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 116.389752] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 116.396056] nix-grpc-daemon[1193]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 116.397488] nix-grpc-daemon[1193]: E0000 00:00:1790076810.981250 1193 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 116.440104] nix-grpc-daemon[1193]: E0000 00:00:1790076810.987462 1193 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 116.442417] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 116.448262] nix-grpc-daemon[1193]: E0000 00:00:1790076810.997876 1235 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 116.451088] nix-grpc-daemon[1193]: E0000 00:00:1790076811.015954 1235 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 116.453627] nix-grpc-daemon[1193]: E0000 00:00:1790076811.016081 1235 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 116.455973] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 116.456839] nix-grpc-daemon[1193]: E0000 00:00:1790076811.034582 1235 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 116.464247] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 116.462597] nix-grpc-daemon[768]: ts=2026-09-22T11:33:31Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:43034 worker1 # [ 116.468510] nix-grpc-daemon[768]: ts=2026-09-22T11:33:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 116.671743] nix-grpc-daemon[768]: ts=2026-09-22T11:33:31Z level=info event=signal_exit lb # [ 117.090031] niks3-server[1033]: 2026/09/22 11:33:31 INFO lead: released remote=192.168.1.3:44502 worker1 # [ 118.134502] nix-grpc-daemon[768]: ts=2026-09-22T11:33:32Z level=info event=scheduler_restarting worker2 # [ 118.141011] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:32Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 118.146324] nix-grpc-daemon[768]: ts=2026-09-22T11:33:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:43034 worker1 # [ 118.151623] nix-grpc-daemon[768]: ts=2026-09-22T11:33:32Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 118.156578] nix-grpc-daemon[768]: ts=2026-09-22T11:33:32Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 118.162011] nix-grpc-daemon[768]: ts=2026-09-22T11:33:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:43034 worker1 # [ 118.167123] nix-grpc-daemon[768]: ts=2026-09-22T11:33:32Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 118.194476] nix-daemon[852]: reaped child process 854, status = succeeded worker2 # [ 118.340888] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 118.346092] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 118.370808] nix-grpc-daemon[768]: ts=2026-09-22T11:33:32Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" lb # [ 118.326989] envoy[893]: conn peer=[2001:db8:1::1]:36656 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=25 worker2 # [ 118.547037] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 118.552292] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 118.753359] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 118.758491] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 118.854130] nix-grpc-daemon[768]: E0000 00:00:1790076813.449390 813 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true worker1 # [ 118.860749] nix-grpc-daemon[768]: E0000 00:00:1790076813.449718 813 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 118.959468] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 118.964698] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 118.332079] envoy[893]: conn peer=[2001:db8:1::1]:36648 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3753 tx=2126 ms=1923 lb # [ 118.332237] envoy[893]: conn peer=[2001:db8:1::4]:57598 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=91452 tx=4637 ms=94380 lb # [ 118.332342] envoy[893]: conn peer=[2001:db8:1::1]:35366 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=22 lb # [ 118.332458] envoy[893]: conn peer=[2001:db8:1::3]:57748 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=18184 tx=4910 ms=99672 worker1 # [ 119.092477] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 119.099647] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 119.107782] systemd[1]: nix-grpc-daemon.service: Consumed 5.212s CPU time over 1min 40.715s wall clock time, 70.5M memory peak, 123K incoming IP traffic, 248.1K outgoing IP traffic. worker1 # [ 119.123532] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 119.129601] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 119.131464] systemd[1]: nix-grpc-daemon.socket: Consumed 646us CPU time over 1min 48.916s wall clock time, 256K memory peak, 131.9K incoming IP traffic, 113.5K outgoing IP traffic. lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 119.167373] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 119.171644] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2 worker2 # [ 119.368433] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 119.373286] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 119.448980] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 119.462896] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) worker2 # [ 119.490905] nix-grpc-daemon[1193]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 1/5) worker2 # [ 119.519864] nix-grpc-daemon[1193]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 69 ms (attempt 2/5) worker2 # [ 119.574235] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 119.579431] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 119.590729] nix-grpc-daemon[1193]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 3/5) worker2 # [ 119.632887] nix-grpc-daemon[1193]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 616 ms (attempt 4/5) worker2 # [ 119.780347] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 119.785096] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 119.986123] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 119.991200] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 120.193921] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 120.198019] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 120.251495] nix-grpc-daemon[1193]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 120.258582] nix-grpc-daemon[1193]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 120.281427] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 120.299136] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) worker2 # [ 120.329338] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=niks3_push_started pid=1253 worker2 # [ 120.396811] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 120.400901] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 120.487792] nix-grpc-daemon[1253]: time=2026-09-22T11:33:35.074Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 120.592143] nix-daemon[842]: accepted connection from pid 1266, user nix-grpc-daemon (trusted) worker2 # [ 120.602489] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 120.605322] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 120.614004] nix-daemon[842]: reaped child process 1269, status = succeeded lb # [ 120.573566] niks3-server[1033]: 2026/09/22 11:33:35 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 120.582148] niks3-server[1033]: 2026/09/22 11:33:35 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 120.639002] nix-grpc-daemon[1253]: time=2026-09-22T11:33:35.226Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 120.643497] nix-grpc-daemon[1253]: time=2026-09-22T11:33:35.227Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)" worker2 # [ 120.645980] nix-grpc-daemon[1253]: time=2026-09-22T11:33:35.227Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)" lb # [ 120.636348] niks3-server[1033]: 2026/09/22 11:33:35 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst lb # [ 120.655876] niks3-server[1033]: 2026/09/22 11:33:35 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 120.667227] niks3-server[1033]: 2026/09/22 11:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign worker2 # [ 120.722619] nix-grpc-daemon[1253]: time=2026-09-22T11:33:35.307Z level=INFO msg="Uploading 3 narinfos" lb # [ 120.675788] niks3-server[1033]: 2026/09/22 11:33:35 INFO Signed narinfos id=35 count=1 lb # [ 120.680204] niks3-server[1033]: 2026/09/22 11:33:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 120.688816] niks3-server[1033]: 2026/09/22 11:33:35 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls lb # [ 120.691661] niks3-server[1033]: 2026/09/22 11:33:35 INFO Signed narinfos id=36 count=2 lb # [ 120.698419] niks3-server[1033]: 2026/09/22 11:33:35 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls lb # [ 120.706702] niks3-server[1033]: 2026/09/22 11:33:35 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 120.713988] niks3-server[1033]: 2026/09/22 11:33:35 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 120.722149] niks3-server[1033]: 2026/09/22 11:33:35 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 120.729359] niks3-server[1033]: 2026/09/22 11:33:35 INFO Completed upload id=35 worker2 # [ 120.778350] nix-grpc-daemon[1253]: time=2026-09-22T11:33:35.365Z level=INFO msg="Upload complete. (286ms)" lb # [ 120.730861] niks3-server[1033]: 2026/09/22 11:33:35 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo lb # [ 120.733945] niks3-server[1033]: 2026/09/22 11:33:35 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete lb # [ 120.737577] niks3-server[1033]: 2026/09/22 11:33:35 INFO Completed upload id=36 worker2 # [ 120.788375] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 120.805883] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 120.813085] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 120.818601] nix-daemon[842]: reaped child process 1251, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.61 seconds) worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds) worker1 # [ 120.922918] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 120.933325] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 121.012138] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 121.017101] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 121.074996] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 121.077760] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 121.081551] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 121.088071] nix-grpc-daemon[1372]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 121.089500] nix-grpc-daemon[1372]: E0000 00:00:1790076815.682418 1372 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 121.059502] niks3-server[1033]: 2026/09/22 11:33:35 INFO lead: acquired remote=192.168.1.3:57752 lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 121.136104] nix-grpc-daemon[1372]: E0000 00:00:1790076815.688703 1372 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 121.138419] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 121.144731] nix-grpc-daemon[1372]: E0000 00:00:1790076815.699345 1414 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 121.147539] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=scheduler_take_over worker1 # [ 121.148892] nix-grpc-daemon[1372]: E0000 00:00:1790076815.720161 1414 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 121.151211] nix-grpc-daemon[1372]: E0000 00:00:1790076815.720301 1414 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 121.153696] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 121.154598] nix-grpc-daemon[1372]: E0000 00:00:1790076815.739652 1414 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 121.161933] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 121.166959] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:59724 worker1 # [ 121.168843] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.07 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.82 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 121.219318] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 121.223407] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 121.226767] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:59724 worker1 # [ 121.230730] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb # [ 121.280734] systemd[1]: Stopping niks3 server... worker1 # [ 121.325556] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:35Z level=info event=scheduler_lock_lost status=200 worker2 # [ 121.330343] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:35Z level=info event=scheduler_lock_lost status=200 lb # [ 121.285082] niks3-server[1033]: 2026/09/22 11:33:35 INFO Shutdown signal received, draining in-flight requests timeout=10s lb # [ 121.286677] niks3-server[1033]: 2026/09/22 11:33:35 INFO lead: released remote=192.168.1.3:57752 lb # [ 125.961685] systemd[1]: niks3.service: Deactivated successfully. lb # [ 125.966800] systemd[1]: Stopped niks3 server. lb # [ 125.971282] systemd[1]: niks3.service: Consumed 687ms CPU time over 1min 34.170s wall clock time, 27.6M memory peak, 99K incoming IP traffic, 141.7K outgoing IP traffic. lb # [ 125.996493] systemd[1]: Starting niks3 server... lb # [ 126.108564] niks3-server[1169]: 2026/09/22 11:33:40 goose: no migrations to run. current version: 20260920000000 lb # [ 126.117174] niks3-server[1169]: 2026/09/22 11:33:40 OK 1_commit_pending_closure.sql (6.64ms) lb # [ 126.121155] niks3-server[1169]: 2026/09/22 11:33:40 OK 2_object_stats_trigger.sql (3.57ms) lb # [ 126.122454] niks3-server[1169]: 2026/09/22 11:33:40 goose: up to current file version: 2 lb # [ 126.124708] niks3-server[1169]: 2026/09/22 11:33:40 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 126.136975] niks3-server[1169]: 2026/09/22 11:33:40 INFO Read proxy enabled — serving cache objects from S3 lb # [ 126.145172] niks3-server[1169]: 2026/09/22 11:33:40 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 126.147223] niks3-server[1169]: 2026/09/22 11:33:40 INFO systemd watchdog enabled interval=15s lb # [ 126.150709] niks3-server[1169]: 2026/09/22 11:33:40 INFO Starting HTTP server address=0.0.0.0:5751 lb: waiting for unit niks3.service lb # [ 126.153020] niks3-server[1169]: 2026/09/22 11:33:40 INFO lead: acquired remote=192.168.1.3:57766 lb # [ 126.154252] systemd[1]: Started niks3 server. lb: (finished: waiting for unit niks3.service, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2 worker1 # [ 126.703743] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:59736 worker1 # [ 126.717948] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted) client # don't know how to build these paths: client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 126.748959] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57062 worker1 # [ 126.792807] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:41Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=0 worker1 # [ 126.799481] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:41Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 126.845537] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59736 worker1 # [ 126.862305] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 1/5) worker1 # [ 126.913369] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 2/5) worker1 # [ 126.997261] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 104 ms (attempt 3/5) worker1 # [ 127.103317] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 244 ms (attempt 4/5) worker1 # [ 127.349247] nix-grpc-daemon[1372]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 127.356688] nix-grpc-daemon[1372]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 127.374617] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:59736 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'... worker1 # [ 127.390244] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59736 worker1 # [ 127.404354] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted) worker1 # [ 127.433412] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:42Z level=info event=niks3_push_started pid=1444 worker1 # [ 127.591715] nix-grpc-daemon[1444]: time=2026-09-22T11:33:42.186Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 127.696742] nix-daemon[852]: accepted connection from pid 1456, user nix-grpc-daemon (trusted) worker1 # [ 127.713966] nix-daemon[852]: reaped child process 1459, status = succeeded worker1 # [ 127.814707] nix-daemon[852]: accepted connection from pid 1461, user nix-grpc-daemon (trusted) worker1 # [ 127.832790] nix-daemon[852]: reaped child process 1464, status = succeeded lb # [ 127.787569] niks3-server[1169]: 2026/09/22 11:33:42 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 127.847992] nix-grpc-daemon[1444]: time=2026-09-22T11:33:42.442Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 127.849499] nix-grpc-daemon[1444]: time=2026-09-22T11:33:42.442Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)" lb # [ 127.850100] niks3-server[1169]: 2026/09/22 11:33:42 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst lb # [ 127.863546] niks3-server[1169]: 2026/09/22 11:33:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign worker1 # [ 127.920822] nix-grpc-daemon[1444]: time=2026-09-22T11:33:42.515Z level=INFO msg="Uploading 1 narinfos" lb # [ 127.879925] niks3-server[1169]: 2026/09/22 11:33:42 INFO Signed narinfos id=37 count=1 lb # [ 127.894666] niks3-server[1169]: 2026/09/22 11:33:42 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete lb # [ 127.910967] niks3-server[1169]: 2026/09/22 11:33:42 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls lb # [ 127.924656] niks3-server[1169]: 2026/09/22 11:33:42 INFO Completed upload id=37 worker1 # [ 127.970102] nix-grpc-daemon[1444]: time=2026-09-22T11:33:42.565Z level=INFO msg="Upload complete. (374ms)" lb # [ 127.938078] niks3-server[1169]: 2026/09/22 11:33:42 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo worker1 # [ 127.982148] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59736 duration_s=0 paths=1 nar_bytes_in=544 worker1 # [ 128.002468] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59736 worker1 # [ 128.010938] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted) worker1 # [ 128.018564] nix-daemon[852]: reaped child process 1442, status = succeeded client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051... worker1 # [ 128.030542] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted) client # farm-a> node-a: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv lb # [ 128.325701] envoy[893]: conn peer=[2001:db8:1::1]:35382 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1005 tx=305 ms=1357 client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker1 # [ 129.262407] nix-daemon[852]: reaped child process 1468, status = succeeded worker1 # [ 129.372068] nix-daemon[852]: accepted connection from pid 1484, user nix-grpc-daemon (trusted) worker1 # [ 129.388410] nix-daemon[852]: reaped child process 1487, status = succeeded lb # [ 129.350111] niks3-server[1169]: 2026/09/22 11:33:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 129.412293] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.005Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 129.413798] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.005Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)" lb # [ 129.395340] niks3-server[1169]: 2026/09/22 11:33:43 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv lb # [ 129.408993] niks3-server[1169]: 2026/09/22 11:33:43 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst worker1 # [ 129.466543] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.061Z level=INFO msg="Uploading 1 narinfos" lb # [ 129.421236] niks3-server[1169]: 2026/09/22 11:33:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign lb # [ 129.429581] niks3-server[1169]: 2026/09/22 11:33:44 INFO Signed narinfos id=38 count=1 lb # [ 129.434722] niks3-server[1169]: 2026/09/22 11:33:44 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls lb # [ 129.444735] niks3-server[1169]: 2026/09/22 11:33:44 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete lb # [ 129.453803] niks3-server[1169]: 2026/09/22 11:33:44 INFO Completed upload id=38 worker1 # [ 129.498532] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.094Z level=INFO msg="Upload complete. (255ms)" lb # [ 129.457169] niks3-server[1169]: 2026/09/22 11:33:44 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo worker1 # [ 129.520607] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59736 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 worker1 # [ 129.536358] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=0 worker1 # [ 129.539913] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 129.545071] nix-daemon[852]: reaped child process 1466, status = succeeded worker1 # [ 129.565333] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59736 worker1 # [ 129.581066] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:59736 worker1 # [ 129.596435] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.191Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'... worker1 # [ 129.606581] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59736 worker1 # [ 129.620187] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted) worker1 # [ 129.772126] nix-daemon[852]: accepted connection from pid 1493, user nix-grpc-daemon (trusted) worker1 # [ 129.789508] nix-daemon[852]: reaped child process 1496, status = succeeded worker1 # [ 129.890703] nix-daemon[852]: accepted connection from pid 1498, user nix-grpc-daemon (trusted) worker1 # [ 129.905962] nix-daemon[852]: reaped child process 1501, status = succeeded lb # [ 129.867130] niks3-server[1169]: 2026/09/22 11:33:44 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 129.924203] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.519Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 129.925657] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.519Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)" lb # [ 129.897458] niks3-server[1169]: 2026/09/22 11:33:44 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst lb # [ 129.910316] niks3-server[1169]: 2026/09/22 11:33:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign worker1 # [ 129.964991] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.560Z level=INFO msg="Uploading 1 narinfos" lb # [ 129.924592] niks3-server[1169]: 2026/09/22 11:33:44 INFO Signed narinfos id=39 count=1 lb # [ 129.927149] niks3-server[1169]: 2026/09/22 11:33:44 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls lb # [ 129.939546] niks3-server[1169]: 2026/09/22 11:33:44 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete lb # [ 129.947849] niks3-server[1169]: 2026/09/22 11:33:44 INFO Completed upload id=39 worker1 # [ 129.992566] nix-grpc-daemon[1444]: time=2026-09-22T11:33:44.588Z level=INFO msg="Upload complete. (356ms)" lb # [ 129.951523] niks3-server[1169]: 2026/09/22 11:33:44 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo worker1 # [ 130.004258] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59736 duration_s=0 paths=1 nar_bytes_in=656 worker1 # [ 130.014751] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59736 worker1 # [ 130.023956] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted) worker1 # [ 130.026511] nix-daemon[852]: reaped child process 1491, status = succeeded client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051... client # farm-b> node-a: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker1 # [ 130.041041] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker1 # [ 131.085366] nix-daemon[852]: reaped child process 1505, status = succeeded worker1 # [ 131.199171] nix-daemon[852]: accepted connection from pid 1521, user nix-grpc-daemon (trusted) worker1 # [ 131.216874] nix-daemon[852]: reaped child process 1524, status = succeeded lb # [ 131.176578] niks3-server[1169]: 2026/09/22 11:33:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 131.232370] nix-grpc-daemon[1444]: time=2026-09-22T11:33:45.827Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 131.233860] nix-grpc-daemon[1444]: time=2026-09-22T11:33:45.827Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)" lb # [ 131.218914] niks3-server[1169]: 2026/09/22 11:33:45 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst lb # [ 131.231696] niks3-server[1169]: 2026/09/22 11:33:45 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv lb # [ 131.239332] niks3-server[1169]: 2026/09/22 11:33:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign worker1 # [ 131.289955] nix-grpc-daemon[1444]: time=2026-09-22T11:33:45.885Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.248441] niks3-server[1169]: 2026/09/22 11:33:45 INFO Signed narinfos id=40 count=1 lb # [ 131.255736] niks3-server[1169]: 2026/09/22 11:33:45 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls lb # [ 131.268074] niks3-server[1169]: 2026/09/22 11:33:45 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete lb # [ 131.274805] niks3-server[1169]: 2026/09/22 11:33:45 INFO Completed upload id=40 worker1 # [ 131.319434] nix-grpc-daemon[1444]: time=2026-09-22T11:33:45.915Z level=INFO msg="Upload complete. (268ms)" lb # [ 131.278477] niks3-server[1169]: 2026/09/22 11:33:45 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo worker1 # [ 131.338107] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59736 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 worker1 # [ 131.356319] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:45Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=0 worker1 # [ 131.360677] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:45Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 131.365109] nix-daemon[852]: reaped child process 1503, status = succeeded worker1 # [ 131.381282] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59736 worker1 # [ 131.396277] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:59736 worker1 # [ 131.411972] nix-grpc-daemon[1444]: time=2026-09-22T11:33:46.007Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'... worker1 # [ 131.423644] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59736 worker1 # [ 131.437570] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted) worker1 # [ 131.588085] nix-daemon[852]: accepted connection from pid 1530, user nix-grpc-daemon (trusted) worker1 # [ 131.611955] nix-daemon[852]: reaped child process 1533, status = succeeded worker1 # [ 131.703279] nix-daemon[852]: accepted connection from pid 1535, user nix-grpc-daemon (trusted) worker1 # [ 131.721564] nix-daemon[852]: reaped child process 1538, status = succeeded lb # [ 131.674639] niks3-server[1169]: 2026/09/22 11:33:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 131.728832] nix-grpc-daemon[1444]: time=2026-09-22T11:33:46.324Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 131.730294] nix-grpc-daemon[1444]: time=2026-09-22T11:33:46.324Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)" lb # [ 131.701109] niks3-server[1169]: 2026/09/22 11:33:46 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst worker1 # [ 131.758024] nix-grpc-daemon[1444]: time=2026-09-22T11:33:46.353Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.714599] niks3-server[1169]: 2026/09/22 11:33:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign lb # [ 131.726360] niks3-server[1169]: 2026/09/22 11:33:46 INFO Signed narinfos id=41 count=1 lb # [ 131.733074] niks3-server[1169]: 2026/09/22 11:33:46 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls lb # [ 131.743462] niks3-server[1169]: 2026/09/22 11:33:46 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete lb # [ 131.750812] niks3-server[1169]: 2026/09/22 11:33:46 INFO Completed upload id=41 worker1 # [ 131.794819] nix-grpc-daemon[1444]: time=2026-09-22T11:33:46.390Z level=INFO msg="Upload complete. (341ms)" lb # [ 131.752968] niks3-server[1169]: 2026/09/22 11:33:46 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo worker1 # [ 131.804509] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59736 duration_s=0 paths=1 nar_bytes_in=784 worker1 # [ 131.816186] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59736 worker1 # [ 131.823957] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted) worker1 # [ 131.832181] nix-daemon[852]: reaped child process 1528, status = succeeded worker1 # [ 131.837951] nix-daemon[852]: accepted connection from pid 1372, user nix-grpc-daemon (trusted) client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051... client # farm-top> node-a: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker1 # [ 133.268138] nix-daemon[852]: reaped child process 1542, status = succeeded worker1 # [ 133.367021] nix-daemon[852]: accepted connection from pid 1558, user nix-grpc-daemon (trusted) worker1 # [ 133.385717] nix-daemon[852]: reaped child process 1561, status = succeeded lb # [ 133.346256] niks3-server[1169]: 2026/09/22 11:33:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 133.403635] nix-grpc-daemon[1444]: time=2026-09-22T11:33:47.997Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 133.405254] nix-grpc-daemon[1444]: time=2026-09-22T11:33:47.997Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)" lb # [ 133.377605] niks3-server[1169]: 2026/09/22 11:33:47 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv lb # [ 133.399640] niks3-server[1169]: 2026/09/22 11:33:47 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst worker1 # [ 133.453044] nix-grpc-daemon[1444]: time=2026-09-22T11:33:48.048Z level=INFO msg="Uploading 1 narinfos" lb # [ 133.410841] niks3-server[1169]: 2026/09/22 11:33:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign lb # [ 133.419030] niks3-server[1169]: 2026/09/22 11:33:47 INFO Signed narinfos id=42 count=1 lb # [ 133.423699] niks3-server[1169]: 2026/09/22 11:33:48 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls lb # [ 133.431173] niks3-server[1169]: 2026/09/22 11:33:48 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 133.436789] niks3-server[1169]: 2026/09/22 11:33:48 INFO Completed upload id=42 worker1 # [ 133.481439] nix-grpc-daemon[1444]: time=2026-09-22T11:33:48.077Z level=INFO msg="Upload complete. (231ms)" lb # [ 133.439438] niks3-server[1169]: 2026/09/22 11:33:48 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo worker1 # [ 133.493089] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59736 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 worker1 # [ 133.512248] nix-daemon[852]: reaped child process 1540, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 7.07 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 12.51 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1 # [ 133.995656] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 134.118041] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:48Z level=info event=signal_exit lb # [ 134.148203] niks3-server[1169]: 2026/09/22 11:33:48 INFO lead: released remote=192.168.1.3:57766 worker2 # [ 135.199106] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:49Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 135.199088] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:49Z level=info event=scheduler_restarting worker1 # [ 135.204443] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:49Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 135.208867] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:59724 worker1 # [ 135.213830] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:49Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 135.218390] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:59724 worker1 # [ 135.228164] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:49Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 135.246617] nix-daemon[852]: reaped child process 1434, status = succeeded worker2 # [ 135.405095] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:49Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 135.410521] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:49Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 135.414357] nix-grpc-daemon[1372]: ts=2026-09-22T11:33:50Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 135.611542] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 135.616823] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 135.817665] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 135.822567] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 136.023631] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 136.028927] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 136.150882] nix-grpc-daemon[1372]: E0000 00:00:1790076830.746189 1417 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true worker1 # [ 136.151206] nix-grpc-daemon[1372]: E0000 00:00:1790076830.746473 1417 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 136.191842] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_take_over lb # [ 136.143698] niks3-server[1169]: 2026/09/22 11:33:50 INFO lead: acquired remote=192.168.1.4:49430 worker2 # [ 136.231468] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 136.235667] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 136.355980] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 136.363279] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 136.369399] systemd[1]: nix-grpc-daemon.service: Consumed 1.415s CPU time over 15.425s wall clock time, 38.2M memory peak, 35.4K incoming IP traffic, 56.4K outgoing IP traffic. worker1 # [ 136.388889] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 136.391214] systemd[1]: Closed Nix worker-protocol over gRPC. worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker1 # [ 136.397003] systemd[1]: nix-grpc-daemon.socket: Consumed 984us CPU time over 15.471s wall clock time, 256K memory peak, 36.8K incoming IP traffic, 27.8K outgoing IP traffic. worker2 # [ 136.432474] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 136.442276] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker2 # [ 136.640121] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:51Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 136.667078] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:51Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 136.671753] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:52586 worker2 # [ 136.677347] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2 client # don't know how to build these paths: client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker2 # [ 137.820583] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 137.827121] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 137.853223] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=0 worker2 # [ 137.860360] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 137.901288] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 137.916890] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57062 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'... worker2 # [ 137.933205] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 137.949935] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) worker2 # [ 138.097087] nix-daemon[842]: accepted connection from pid 1292, user nix-grpc-daemon (trusted) worker2 # [ 138.115333] nix-daemon[842]: reaped child process 1295, status = succeeded worker2 # [ 138.216563] nix-daemon[842]: accepted connection from pid 1297, user nix-grpc-daemon (trusted) worker2 # [ 138.231416] nix-daemon[842]: reaped child process 1300, status = succeeded lb # [ 138.191299] niks3-server[1169]: 2026/09/22 11:33:52 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 138.251591] nix-grpc-daemon[1253]: time=2026-09-22T11:33:52.837Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 138.253558] nix-grpc-daemon[1253]: time=2026-09-22T11:33:52.837Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)" lb # [ 138.227643] niks3-server[1169]: 2026/09/22 11:33:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign worker2 # [ 138.286679] nix-grpc-daemon[1253]: time=2026-09-22T11:33:52.873Z level=INFO msg="Uploading 1 narinfos" lb # [ 138.242018] niks3-server[1169]: 2026/09/22 11:33:52 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst lb # [ 138.255409] niks3-server[1169]: 2026/09/22 11:33:52 INFO Signed narinfos id=43 count=1 worker2 # [ 138.310534] nix-grpc-daemon[1253]: time=2026-09-22T11:33:52.897Z level=INFO msg="Upload complete. (345ms)" lb # [ 138.263040] niks3-server[1169]: 2026/09/22 11:33:52 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls lb # [ 138.270131] niks3-server[1169]: 2026/09/22 11:33:52 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete lb # [ 138.274117] niks3-server[1169]: 2026/09/22 11:33:52 INFO Completed upload id=43 worker2 # [ 138.320983] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=0 paths=1 nar_bytes_in=520 lb # [ 138.277068] niks3-server[1169]: 2026/09/22 11:33:52 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo worker2 # [ 138.336288] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 138.347548] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) worker2 # [ 138.351764] nix-daemon[842]: reaped child process 1290, status = succeeded client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051... worker2 # [ 138.363838] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv lb # [ 138.329380] envoy[893]: conn peer=[2001:db8:1::1]:46662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9766 tx=4485 ms=6809 lb # [ 138.334452] envoy[893]: conn peer=[2001:db8:1::3]:49820 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1065 tx=425 ms=14247 client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker2 # [ 139.220854] nix-daemon[842]: reaped child process 1304, status = succeeded worker2 # [ 139.325685] nix-daemon[842]: accepted connection from pid 1320, user nix-grpc-daemon (trusted) worker2 # [ 139.343935] nix-daemon[842]: reaped child process 1323, status = succeeded lb # [ 139.295119] niks3-server[1169]: 2026/09/22 11:33:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 139.356376] nix-grpc-daemon[1253]: time=2026-09-22T11:33:53.942Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 139.357941] nix-grpc-daemon[1253]: time=2026-09-22T11:33:53.942Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)" lb # [ 139.342765] niks3-server[1169]: 2026/09/22 11:33:53 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst worker2 # [ 139.404731] nix-grpc-daemon[1253]: time=2026-09-22T11:33:53.990Z level=INFO msg="Uploading 1 narinfos" lb # [ 139.358332] niks3-server[1169]: 2026/09/22 11:33:53 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv lb # [ 139.367002] niks3-server[1169]: 2026/09/22 11:33:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign lb # [ 139.377725] niks3-server[1169]: 2026/09/22 11:33:53 INFO Signed narinfos id=44 count=1 lb # [ 139.379598] niks3-server[1169]: 2026/09/22 11:33:53 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls lb # [ 139.386051] niks3-server[1169]: 2026/09/22 11:33:53 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete worker2 # [ 139.435205] nix-grpc-daemon[1253]: time=2026-09-22T11:33:54.022Z level=INFO msg="Upload complete. (257ms)" lb # [ 139.388999] niks3-server[1169]: 2026/09/22 11:33:53 INFO Completed upload id=44 lb # [ 139.391858] niks3-server[1169]: 2026/09/22 11:33:53 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo worker2 # [ 139.454134] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 worker2 # [ 139.475184] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=0 worker2 # [ 139.481384] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 139.486156] nix-daemon[842]: reaped child process 1302, status = succeeded worker2 # [ 139.513552] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 139.529067] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 139.542894] nix-grpc-daemon[1253]: time=2026-09-22T11:33:54.129Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'... worker2 # [ 139.553469] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 139.566856] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) worker2 # [ 139.718627] nix-daemon[842]: accepted connection from pid 1329, user nix-grpc-daemon (trusted) worker2 # [ 139.734586] nix-daemon[842]: reaped child process 1332, status = succeeded worker2 # [ 139.836342] nix-daemon[842]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 139.851160] nix-daemon[842]: reaped child process 1337, status = succeeded lb # [ 139.809056] niks3-server[1169]: 2026/09/22 11:33:54 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 139.867223] nix-grpc-daemon[1253]: time=2026-09-22T11:33:54.453Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 139.868987] nix-grpc-daemon[1253]: time=2026-09-22T11:33:54.454Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)" lb # [ 139.839741] niks3-server[1169]: 2026/09/22 11:33:54 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst worker2 # [ 139.899653] nix-grpc-daemon[1253]: time=2026-09-22T11:33:54.486Z level=INFO msg="Uploading 1 narinfos" lb # [ 139.854329] niks3-server[1169]: 2026/09/22 11:33:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign lb # [ 139.865583] niks3-server[1169]: 2026/09/22 11:33:54 INFO Signed narinfos id=45 count=1 lb # [ 139.870808] niks3-server[1169]: 2026/09/22 11:33:54 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls lb # [ 139.882154] niks3-server[1169]: 2026/09/22 11:33:54 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete lb # [ 139.888866] niks3-server[1169]: 2026/09/22 11:33:54 INFO Completed upload id=45 worker2 # [ 139.937829] nix-grpc-daemon[1253]: time=2026-09-22T11:33:54.524Z level=INFO msg="Upload complete. (355ms)" lb # [ 139.892685] niks3-server[1169]: 2026/09/22 11:33:54 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo worker2 # [ 139.948278] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=0 paths=1 nar_bytes_in=640 worker2 # [ 139.958478] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 139.967934] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) worker2 # [ 139.971085] nix-daemon[842]: reaped child process 1327, status = succeeded client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051... worker2 # [ 139.984474] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker2 # [ 141.064158] nix-daemon[842]: reaped child process 1341, status = succeeded worker2 # [ 141.163918] nix-daemon[842]: accepted connection from pid 1357, user nix-grpc-daemon (trusted) worker2 # [ 141.184534] nix-daemon[842]: reaped child process 1360, status = succeeded lb # [ 141.135158] niks3-server[1169]: 2026/09/22 11:33:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 141.196112] nix-grpc-daemon[1253]: time=2026-09-22T11:33:55.781Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 141.198376] nix-grpc-daemon[1253]: time=2026-09-22T11:33:55.782Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)" lb # [ 141.175773] niks3-server[1169]: 2026/09/22 11:33:55 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst worker2 # [ 141.237272] nix-grpc-daemon[1253]: time=2026-09-22T11:33:55.824Z level=INFO msg="Uploading 1 narinfos" lb # [ 141.190547] niks3-server[1169]: 2026/09/22 11:33:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign lb # [ 141.200778] niks3-server[1169]: 2026/09/22 11:33:55 INFO Signed narinfos id=46 count=1 lb # [ 141.205661] niks3-server[1169]: 2026/09/22 11:33:55 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv lb # [ 141.214978] niks3-server[1169]: 2026/09/22 11:33:55 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls lb # [ 141.219640] niks3-server[1169]: 2026/09/22 11:33:55 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete worker2 # [ 141.267958] nix-grpc-daemon[1253]: time=2026-09-22T11:33:55.854Z level=INFO msg="Upload complete. (273ms)" lb # [ 141.222664] niks3-server[1169]: 2026/09/22 11:33:55 INFO Completed upload id=46 lb # [ 141.225078] niks3-server[1169]: 2026/09/22 11:33:55 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo worker2 # [ 141.286195] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 worker2 # [ 141.302631] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:55Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=0 worker2 # [ 141.306649] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:55Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 141.311103] nix-daemon[842]: reaped child process 1339, status = succeeded worker2 # [ 141.329173] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 141.340627] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 141.354972] nix-grpc-daemon[1253]: time=2026-09-22T11:33:55.941Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'... worker2 # [ 141.366234] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 141.379716] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) worker2 # [ 141.529057] nix-daemon[842]: accepted connection from pid 1366, user nix-grpc-daemon (trusted) worker2 # [ 141.547292] nix-daemon[842]: reaped child process 1369, status = succeeded worker2 # [ 141.648140] nix-daemon[842]: accepted connection from pid 1371, user nix-grpc-daemon (trusted) worker2 # [ 141.662147] nix-daemon[842]: reaped child process 1374, status = succeeded lb # [ 141.621036] niks3-server[1169]: 2026/09/22 11:33:56 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 141.679432] nix-grpc-daemon[1253]: time=2026-09-22T11:33:56.266Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 141.681066] nix-grpc-daemon[1253]: time=2026-09-22T11:33:56.266Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)" lb # [ 141.648675] niks3-server[1169]: 2026/09/22 11:33:56 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst worker2 # [ 141.710776] nix-grpc-daemon[1253]: time=2026-09-22T11:33:56.297Z level=INFO msg="Uploading 1 narinfos" lb # [ 141.661636] niks3-server[1169]: 2026/09/22 11:33:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign lb # [ 141.671408] niks3-server[1169]: 2026/09/22 11:33:56 INFO Signed narinfos id=47 count=1 lb # [ 141.679892] niks3-server[1169]: 2026/09/22 11:33:56 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls lb # [ 141.692607] niks3-server[1169]: 2026/09/22 11:33:56 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete worker2 # [ 141.749245] nix-grpc-daemon[1253]: time=2026-09-22T11:33:56.336Z level=INFO msg="Upload complete. (354ms)" lb # [ 141.702472] niks3-server[1169]: 2026/09/22 11:33:56 INFO Completed upload id=47 lb # [ 141.705828] niks3-server[1169]: 2026/09/22 11:33:56 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo worker2 # [ 141.756372] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=0 paths=1 nar_bytes_in=768 worker2 # [ 141.770989] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 141.776953] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) worker2 # [ 141.781479] nix-daemon[842]: reaped child process 1364, status = succeeded client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051... worker2 # [ 141.793549] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker2 # [ 143.095447] nix-daemon[842]: reaped child process 1378, status = succeeded worker2 # [ 143.165216] nix-daemon[842]: accepted connection from pid 1394, user nix-grpc-daemon (trusted) worker2 # [ 143.184913] nix-daemon[842]: reaped child process 1397, status = succeeded lb # [ 143.135489] niks3-server[1169]: 2026/09/22 11:33:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 143.195469] nix-grpc-daemon[1253]: time=2026-09-22T11:33:57.780Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 143.197145] nix-grpc-daemon[1253]: time=2026-09-22T11:33:57.780Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)" lb # [ 143.167030] niks3-server[1169]: 2026/09/22 11:33:57 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv lb # [ 143.181997] niks3-server[1169]: 2026/09/22 11:33:57 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst worker2 # [ 143.244743] nix-grpc-daemon[1253]: time=2026-09-22T11:33:57.831Z level=INFO msg="Uploading 1 narinfos" lb # [ 143.197214] niks3-server[1169]: 2026/09/22 11:33:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign lb # [ 143.204733] niks3-server[1169]: 2026/09/22 11:33:57 INFO Signed narinfos id=48 count=1 lb # [ 143.209830] niks3-server[1169]: 2026/09/22 11:33:57 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls lb # [ 143.217356] niks3-server[1169]: 2026/09/22 11:33:57 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete worker2 # [ 143.270147] nix-grpc-daemon[1253]: time=2026-09-22T11:33:57.857Z level=INFO msg="Upload complete. (266ms)" lb # [ 143.223544] niks3-server[1169]: 2026/09/22 11:33:57 INFO Completed upload id=48 lb # [ 143.226592] niks3-server[1169]: 2026/09/22 11:33:57 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo worker2 # [ 143.285102] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 worker2 # [ 143.299658] nix-daemon[842]: reaped child process 1376, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.73 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.19 seconds) worker1 # [ 143.587225] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 143.598164] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 143.738690] nix-grpc-daemon[1579]: ts=2026-09-22T11:33:58Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 143.740978] nix-grpc-daemon[1579]: ts=2026-09-22T11:33:58Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 143.744760] nix-grpc-daemon[1579]: ts=2026-09-22T11:33:58Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 143.751240] nix-grpc-daemon[1579]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 143.765910] nix-grpc-daemon[1579]: E0000 00:00:1790076838.344712 1579 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 143.792244] nix-grpc-daemon[1579]: E0000 00:00:1790076838.346778 1579 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 143.798858] nix-grpc-daemon[1579]: ts=2026-09-22T11:33:58Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 143.801096] nix-grpc-daemon[1579]: E0000 00:00:1790076838.356910 1620 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 143.803552] nix-grpc-daemon[1579]: E0000 00:00:1790076838.374810 1620 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 143.805872] nix-grpc-daemon[1579]: E0000 00:00:1790076838.375041 1620 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 143.808187] nix-grpc-daemon[1579]: E0000 00:00:1790076838.393460 1620 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 143.813369] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 143.815900] nix-grpc-daemon[1579]: ts=2026-09-22T11:33:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 143.821196] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:58Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:52586 worker2 # [ 143.826273] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:58Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 10.38 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs client # [ 144.181143] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: 39fc12e5b1b44548a33753522390304c client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds) client # [ 144.381657] nix[1339]: don't know how to build these paths: client # [ 144.383331] nix[1339]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker2 # [ 144.361315] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 144.375182] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:58Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=0 worker2 # [ 144.381908] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:58Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 144.421940] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 144.438172] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57062 client # [ 144.471812] nix[1339]: copying 1 paths to 'lb:50051'... worker2 # [ 144.452789] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 client # [ 144.474481] nix[1339]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'... worker2 # [ 144.467479] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) worker2 # [ 144.625682] nix-daemon[842]: accepted connection from pid 1410, user nix-grpc-daemon (trusted) worker2 # [ 144.652767] nix-daemon[842]: reaped child process 1413, status = succeeded worker2 # [ 144.741112] nix-daemon[842]: accepted connection from pid 1415, user nix-grpc-daemon (trusted) worker2 # [ 144.757483] nix-daemon[842]: reaped child process 1418, status = succeeded lb # [ 144.716701] niks3-server[1169]: 2026/09/22 11:33:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 144.775364] nix-grpc-daemon[1253]: time=2026-09-22T11:33:59.361Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 144.777038] nix-grpc-daemon[1253]: time=2026-09-22T11:33:59.361Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)" worker2 # [ 144.810772] nix-grpc-daemon[1253]: time=2026-09-22T11:33:59.397Z level=INFO msg="Uploading 1 narinfos" lb # [ 144.750779] niks3-server[1169]: 2026/09/22 11:33:59 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst lb # [ 144.765897] niks3-server[1169]: 2026/09/22 11:33:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign lb # [ 144.774664] niks3-server[1169]: 2026/09/22 11:33:59 INFO Signed narinfos id=49 count=1 lb # [ 144.782384] niks3-server[1169]: 2026/09/22 11:33:59 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls worker2 # [ 144.835517] nix-grpc-daemon[1253]: time=2026-09-22T11:33:59.422Z level=INFO msg="Upload complete. (349ms)" lb # [ 144.790598] niks3-server[1169]: 2026/09/22 11:33:59 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete lb # [ 144.795865] niks3-server[1169]: 2026/09/22 11:33:59 INFO Completed upload id=49 worker2 # [ 144.844235] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 144.799580] niks3-server[1169]: 2026/09/22 11:33:59 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo worker2 # [ 144.854919] nix-grpc-daemon[1193]: ts=2026-09-22T11:33:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 worker2 # [ 144.861613] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) worker2 # [ 144.866581] nix-daemon[842]: reaped child process 1408, status = succeeded client # [ 144.895765] nix[1339]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051... worker2 # [ 144.878086] nix-daemon[842]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker1 # [ 146.952843] systemd-journald[270]: Received client request to rotate journal, rotating. worker1 # [ 146.971834] systemd-journald[270]: Vacuuming done, freed 0B of archived journals from /var/log/journal/b4b129e142a041e1939b13b11d6a8e9d. worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.08 seconds) worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker2 # [ 147.022433] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds) client: must succeed: sleep 5.0 worker2 # [ 147.422988] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:02Z level=info event=draining lb # [ 148.330137] envoy[893]: conn peer=[2001:db8:1::1]:59030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9619 tx=4418 ms=5476 client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker2 # [ 152.587541] nix-daemon[842]: reaped child process 1422, status = succeeded worker2 # [ 152.667994] nix-daemon[842]: accepted connection from pid 1463, user nix-grpc-daemon (trusted) worker2 # [ 152.687396] nix-daemon[842]: reaped child process 1466, status = succeeded lb # [ 152.643182] niks3-server[1169]: 2026/09/22 11:34:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 152.704465] nix-grpc-daemon[1253]: time=2026-09-22T11:34:07.288Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 152.706159] nix-grpc-daemon[1253]: time=2026-09-22T11:34:07.288Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)" lb # [ 152.676330] niks3-server[1169]: 2026/09/22 11:34:07 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv lb # [ 152.693827] niks3-server[1169]: 2026/09/22 11:34:07 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst worker2 # [ 152.755519] nix-grpc-daemon[1253]: time=2026-09-22T11:34:07.342Z level=INFO msg="Uploading 1 narinfos" lb # [ 152.706863] niks3-server[1169]: 2026/09/22 11:34:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign lb # [ 152.715479] niks3-server[1169]: 2026/09/22 11:34:07 INFO Signed narinfos id=50 count=1 lb # [ 152.720986] niks3-server[1169]: 2026/09/22 11:34:07 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls lb # [ 152.729503] niks3-server[1169]: 2026/09/22 11:34:07 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 152.734354] niks3-server[1169]: 2026/09/22 11:34:07 INFO Completed upload id=50 worker2 # [ 152.782540] nix-grpc-daemon[1253]: time=2026-09-22T11:34:07.369Z level=INFO msg="Upload complete. (285ms)" lb # [ 152.737398] niks3-server[1169]: 2026/09/22 11:34:07 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo worker2 # [ 152.797093] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57062 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 worker2 # [ 152.813862] nix-daemon[842]: reaped child process 1420, status = succeeded client # [ 152.845853] systemd[1]: rs.service: Deactivated successfully. client # [ 152.850935] systemd[1]: rs.service: Consumed 190ms CPU time over 8.665s wall clock time, 41.2M memory peak, 7.4K incoming IP traffic, 8.4K outgoing IP traffic. worker2 # [ 153.425130] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:08Z level=info event=signal_exit lb # [ 154.149847] niks3-server[1169]: 2026/09/22 11:34:08 INFO lead: released remote=192.168.1.4:49430 worker1 # [ 155.195113] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:09Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 155.202354] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:09Z level=info event=scheduler_restarting worker2 # [ 155.207665] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:09Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 155.212421] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:52586 worker2 # [ 155.217479] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:09Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 155.223064] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:52586 worker2 # [ 155.228248] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:09Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 155.251372] nix-daemon[842]: reaped child process 1245, status = succeeded worker1 # [ 155.394723] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 155.400282] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 155.419276] nix-grpc-daemon[1193]: ts=2026-09-22T11:34:10Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 155.508183] nix-grpc-daemon[1193]: E0000 00:00:1790076850.094817 1238 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker2 # [ 155.514794] nix-grpc-daemon[1193]: E0000 00:00:1790076850.095117 1238 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.26 seconds) client: waiting for success: ! systemctl is-active rs worker1 # [ 155.601096] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 155.606281] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1 # [ 155.807034] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 155.811879] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 155.966821] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 155.975080] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 155.980648] systemd[1]: nix-grpc-daemon.service: Consumed 2.087s CPU time over 39.712s wall clock time, 39.2M memory peak, 59.1K incoming IP traffic, 132.1K outgoing IP traffic. worker2 # [ 155.997804] systemd[1]: Starting scheduler-token.service... worker1 # [ 156.012689] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 156.017772] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 156.074486] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 156.079203] systemd[1]: Finished scheduler-token.service. worker2 # [ 156.084813] systemd[1]: scheduler-token.service: Consumed 46ms CPU time over 80ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 156.091271] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 156.214499] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:10Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 156.216760] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:10Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 156.220512] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:10Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 156.220254] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 156.227107] nix-grpc-daemon[1476]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 156.224254] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 156.186644] niks3-server[1169]: 2026/09/22 11:34:10 INFO lead: acquired remote=192.168.1.4:35872 worker2 # [ 156.242399] nix-grpc-daemon[1476]: E0000 00:00:1790076850.812027 1476 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 156.267700] nix-grpc-daemon[1476]: E0000 00:00:1790076850.814073 1476 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 156.270675] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:10Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 156.272533] nix-grpc-daemon[1476]: E0000 00:00:1790076850.821522 1517 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 156.274832] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:10Z level=info event=scheduler_take_over worker2 # [ 156.278962] nix-grpc-daemon[1476]: E0000 00:00:1790076850.839263 1517 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 156.283389] nix-grpc-daemon[1476]: E0000 00:00:1790076850.839406 1517 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 156.285892] nix-grpc-daemon[1476]: E0000 00:00:1790076850.853911 1517 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 156.288153] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 156.288953] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 156.290207] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 156.420813] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 156.425791] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 156.484854] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 156.489740] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 156.626422] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 156.631562] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 156.690901] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 156.696270] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 156.832449] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 156.837549] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 156.897105] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 156.902169] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 157.038422] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 157.048939] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:34352 worker2 # [ 157.054271] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 157.103285] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 157.116138] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 157.121163] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:34352 worker2 # [ 157.126460] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:11Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.90 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid client # [ 158.089317] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: e13b8894d668482296c5d29903872344 client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds) worker2 # [ 158.286559] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34356 worker2 # [ 158.300884] nix-daemon[842]: accepted connection from pid 1476, user nix-grpc-daemon (trusted) client # [ 158.337175] nix[1375]: don't know how to build these paths: client # [ 158.338593] nix[1375]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker2 # [ 158.333053] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:12Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker2 # [ 158.339130] nix-grpc-daemon[1476]: ts=2026-09-22T11:34:12Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=1 lb # [ 158.336370] envoy[893]: conn peer=[2001:db8:1::1]:50652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1542 ms=8447 lb # [ 158.341505] envoy[893]: conn peer=[2001:db8:1::4]:50414 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=18744 tx=1341 ms=38948 worker1 # [ 158.401207] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48346 worker1 # [ 158.415925] nix-daemon[852]: accepted connection from pid 1579, user nix-grpc-daemon (trusted) worker1 # [ 158.442310] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 1/5) worker1 # [ 158.470358] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 163 ms (attempt 2/5) worker1 # [ 158.635267] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 313 ms (attempt 3/5) worker1 # [ 158.950108] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 225 ms (attempt 4/5) worker1 # [ 159.177323] nix-grpc-daemon[1579]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 159.182975] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 159.201801] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48346 client # [ 159.235712] nix[1375]: copying 1 paths to 'lb:50051'... client # [ 159.241496] nix[1375]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'... worker1 # [ 159.218325] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48346 worker1 # [ 159.231957] nix-daemon[852]: accepted connection from pid 1579, user nix-grpc-daemon (trusted) worker1 # [ 159.272278] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:13Z level=info event=niks3_push_started pid=1702 worker1 # [ 159.424591] nix-grpc-daemon[1702]: time=2026-09-22T11:34:14.019Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 159.530392] nix-daemon[852]: accepted connection from pid 1715, user nix-grpc-daemon (trusted) worker1 # [ 159.552650] nix-daemon[852]: reaped child process 1718, status = succeeded worker1 # [ 159.645794] nix-daemon[852]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker1 # [ 159.662951] nix-daemon[852]: reaped child process 1723, status = succeeded lb # [ 159.622008] niks3-server[1169]: 2026/09/22 11:34:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 159.679100] nix-grpc-daemon[1702]: time=2026-09-22T11:34:14.273Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 159.680913] nix-grpc-daemon[1702]: time=2026-09-22T11:34:14.273Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)" lb # [ 159.658121] niks3-server[1169]: 2026/09/22 11:34:14 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst worker1 # [ 159.718559] nix-grpc-daemon[1702]: time=2026-09-22T11:34:14.314Z level=INFO msg="Uploading 1 narinfos" lb # [ 159.673654] niks3-server[1169]: 2026/09/22 11:34:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign lb # [ 159.684538] niks3-server[1169]: 2026/09/22 11:34:14 INFO Signed narinfos id=51 count=1 lb # [ 159.691220] niks3-server[1169]: 2026/09/22 11:34:14 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls lb # [ 159.705492] niks3-server[1169]: 2026/09/22 11:34:14 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete lb # [ 159.716789] niks3-server[1169]: 2026/09/22 11:34:14 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo lb # [ 159.741258] niks3-server[1169]: 2026/09/22 11:34:14 INFO Completed upload id=51 worker1 # [ 159.786767] nix-grpc-daemon[1702]: time=2026-09-22T11:34:14.382Z level=INFO msg="Upload complete. (357ms)" worker1 # [ 159.799046] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48346 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 159.816774] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48346 worker1 # [ 159.826794] nix-daemon[852]: accepted connection from pid 1579, user nix-grpc-daemon (trusted) worker1 # [ 159.835664] nix-daemon[852]: reaped child process 1697, status = succeeded client # [ 159.865923] nix[1375]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... worker1 # [ 159.846015] nix-daemon[852]: accepted connection from pid 1579, user nix-grpc-daemon (trusted) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker2 # [ 162.046059] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1476 (nix-grpc-daemon) on client request. worker2 # [ 162.063707] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. worker1 # [ 162.077423] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:16Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))" lb # [ 162.029399] niks3-server[1169]: 2026/09/22 11:34:16 INFO lead: released remote=192.168.1.4:35872 client # [ 162.101481] nix[1375]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker2 # [ 162.092602] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker2 # [ 162.094878] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. worker2 # [ 162.112733] systemd[1]: nix-grpc-daemon.service: Consumed 221ms CPU time over 5.991s wall clock time, 14.3M memory peak, 9.3K incoming IP traffic, 14.8K outgoing IP traffic. worker2 # [ 162.122951] systemd[1]: Starting scheduler-token.service... worker2 # [ 162.124320] nix-daemon[842]: reaped child process 1535, status = succeeded worker2 # [ 162.175443] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 162.179822] systemd[1]: Finished scheduler-token.service. worker2 # [ 162.183947] systemd[1]: scheduler-token.service: Consumed 30ms CPU time over 64ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 162.190059] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker2 # [ 162.192509] systemd[1]: Starting scheduler-token.service... worker2 # [ 162.244213] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 162.248268] systemd[1]: Finished scheduler-token.service. worker2 # [ 162.249366] systemd[1]: scheduler-token.service: Consumed 28ms CPU time over 57ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 162.254745] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 162.277568] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 162.359570] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 162.361907] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 162.365599] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 162.370991] nix-grpc-daemon[1569]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 162.334723] niks3-server[1169]: 2026/09/22 11:34:16 INFO lead: acquired remote=192.168.1.4:56536 worker2 # [ 162.388085] nix-grpc-daemon[1569]: E0000 00:00:1790076856.957215 1569 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.40 seconds) client: must succeed: sleep 5.0 worker2 # [ 162.416157] nix-grpc-daemon[1569]: E0000 00:00:1790076856.960736 1569 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 162.418419] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 162.424077] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:16Z level=info event=scheduler_take_over worker2 # [ 162.425637] nix-grpc-daemon[1569]: E0000 00:00:1790076856.971383 1608 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 162.428173] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:35084 worker2 # [ 162.429698] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=1 worker2 # [ 162.434299] nix-grpc-daemon[1569]: E0000 00:00:1790076856.990836 1608 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 162.439018] nix-grpc-daemon[1569]: E0000 00:00:1790076856.991023 1608 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 162.441956] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 162.443247] nix-grpc-daemon[1569]: E0000 00:00:1790076857.010316 1608 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 162.446021] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 162.449119] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:17Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 162.450380] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:17Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:35084 worker2 # [ 162.452057] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker1 # [ 167.676322] nix-daemon[852]: reaped child process 1727, status = succeeded client: waiting for success: ! systemctl is-active mid worker1 # [ 167.771304] nix-daemon[852]: accepted connection from pid 1764, user nix-grpc-daemon (trusted) worker1 # [ 167.787338] nix-daemon[852]: reaped child process 1767, status = succeeded lb # [ 167.749221] niks3-server[1169]: 2026/09/22 11:34:22 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 167.807904] nix-grpc-daemon[1702]: time=2026-09-22T11:34:22.401Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 167.810478] nix-grpc-daemon[1702]: time=2026-09-22T11:34:22.401Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)" lb # [ 167.786685] niks3-server[1169]: 2026/09/22 11:34:22 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv lb # [ 167.818633] niks3-server[1169]: 2026/09/22 11:34:22 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst lb # [ 167.832197] niks3-server[1169]: 2026/09/22 11:34:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign lb # [ 167.839752] niks3-server[1169]: 2026/09/22 11:34:22 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls lb # [ 167.846958] niks3-server[1169]: 2026/09/22 11:34:22 INFO Signed narinfos id=52 count=1 worker1 # [ 167.893790] nix-grpc-daemon[1702]: time=2026-09-22T11:34:22.487Z level=INFO msg="Uploading 1 narinfos" lb # [ 167.858348] niks3-server[1169]: 2026/09/22 11:34:22 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 167.869665] niks3-server[1169]: 2026/09/22 11:34:22 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo lb # [ 167.880423] niks3-server[1169]: 2026/09/22 11:34:22 INFO Completed upload id=52 worker1 # [ 167.925736] nix-grpc-daemon[1702]: time=2026-09-22T11:34:22.521Z level=INFO msg="Upload complete. (263ms)" worker1 # [ 167.940181] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48346 duration_s=8 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 worker1 # [ 167.957930] nix-daemon[852]: reaped child process 1725, status = succeeded client # [ 167.995733] systemd[1]: mid.service: Deactivated successfully. client # [ 167.999888] systemd[1]: mid.service: Consumed 206ms CPU time over 9.911s wall clock time, 39.5M memory peak, 8K incoming IP traffic, 8.8K outgoing IP traffic. lb # [ 168.336632] envoy[893]: conn peer=[2001:db8:1::4]:33604 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4102 tx=385 ms=5809 lb # [ 168.342187] envoy[893]: conn peer=[2001:db8:1::1]:53770 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3078 tx=1629 ms=9674 client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.87 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr client # [ 168.928848] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: b1299f6e02d54b6e80bd52a70e2d707d client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds) worker2 # [ 169.086923] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 169.099202] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) client # [ 169.134143] nix[1411]: don't know how to build these paths: client # [ 169.135401] nix[1411]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker2 # [ 169.125353] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:23Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=0 worker2 # [ 169.130062] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:23Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 169.173181] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 169.188606] nix-grpc-daemon[1569]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 1/5) worker2 # [ 169.276848] nix-grpc-daemon[1569]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 2/5) worker2 # [ 169.327484] nix-grpc-daemon[1569]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 160 ms (attempt 3/5) worker2 # [ 169.496776] nix-grpc-daemon[1569]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 549 ms (attempt 4/5) worker2 # [ 170.048302] nix-grpc-daemon[1569]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 170.054582] nix-grpc-daemon[1569]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 170.073043] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35100 client # [ 170.108225] nix[1411]: copying 1 paths to 'lb:50051'... client # [ 170.108535] nix[1411]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'... worker2 # [ 170.091544] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 170.105755] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 170.133654] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:24Z level=info event=niks3_push_started pid=1643 worker2 # [ 170.300676] nix-grpc-daemon[1643]: time=2026-09-22T11:34:24.885Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 170.407671] nix-daemon[842]: accepted connection from pid 1660, user nix-grpc-daemon (trusted) worker2 # [ 170.422340] nix-daemon[842]: reaped child process 1663, status = succeeded worker2 # [ 170.523193] nix-daemon[842]: accepted connection from pid 1665, user nix-grpc-daemon (trusted) worker2 # [ 170.538039] nix-daemon[842]: reaped child process 1668, status = succeeded lb # [ 170.497258] niks3-server[1169]: 2026/09/22 11:34:25 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 170.560068] nix-grpc-daemon[1643]: time=2026-09-22T11:34:25.146Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 170.561550] nix-grpc-daemon[1643]: time=2026-09-22T11:34:25.146Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)" lb # [ 170.541861] niks3-server[1169]: 2026/09/22 11:34:25 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst worker2 # [ 170.603844] nix-grpc-daemon[1643]: time=2026-09-22T11:34:25.190Z level=INFO msg="Uploading 1 narinfos" lb # [ 170.556980] niks3-server[1169]: 2026/09/22 11:34:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign lb # [ 170.566784] niks3-server[1169]: 2026/09/22 11:34:25 INFO Signed narinfos id=53 count=1 lb # [ 170.573446] niks3-server[1169]: 2026/09/22 11:34:25 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls lb # [ 170.584271] niks3-server[1169]: 2026/09/22 11:34:25 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete lb # [ 170.590109] niks3-server[1169]: 2026/09/22 11:34:25 INFO Completed upload id=53 worker2 # [ 170.638291] nix-grpc-daemon[1643]: time=2026-09-22T11:34:25.225Z level=INFO msg="Upload complete. (339ms)" lb # [ 170.592914] niks3-server[1169]: 2026/09/22 11:34:25 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo worker2 # [ 170.649286] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 170.662911] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 170.671979] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 170.674464] nix-daemon[842]: reaped child process 1641, status = succeeded worker2 # [ 170.686505] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) client # [ 170.705170] nix[1411]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051... client: must succeed: systemctl kill -s INT intr client # [ 172.645372] systemd[1]: intr.service: Sent signal SIGINT to main process 1411 (nix) on client request. worker2 # [ 172.636245] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:27Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv client # [ 172.655406] nix[1411]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 172.665590] nix[1411]: error: interrupted by the user client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds) client # [ 172.687817] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 172.692459] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 172.694313] systemd[1]: intr.service: Consumed 164ms CPU time over 3.761s wall clock time, 41.5M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic. worker2 # [ 172.714325] nix-daemon[842]: reaped child process 1670, status = succeeded worker2 # [ 172.716245] nix-daemon[1672]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 172.724527] nix-daemon[842]: reaped child process 1672, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 3.88 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw client # [ 172.833112] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: 395ecb4ffa5a47fc8823dd4a094cb39b client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds) worker2 # [ 173.006756] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:35100 client # [ 173.033153] nix[1432]: don't know how to build these paths: client # [ 173.034645] nix[1432]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker2 # [ 173.024243] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:27Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=0 worker2 # [ 173.031670] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:27Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 173.075435] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48346 worker1 # [ 173.092364] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48346 client # [ 173.128515] nix[1432]: copying 1 paths to 'lb:50051'... worker1 # [ 173.107149] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48346 client # [ 173.132867] nix[1432]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'... worker1 # [ 173.122872] nix-daemon[852]: accepted connection from pid 1579, user nix-grpc-daemon (trusted) worker1 # [ 173.291384] nix-daemon[852]: accepted connection from pid 1795, user nix-grpc-daemon (trusted) worker1 # [ 173.311037] nix-daemon[852]: reaped child process 1798, status = succeeded worker1 # [ 173.408114] nix-daemon[852]: accepted connection from pid 1800, user nix-grpc-daemon (trusted) worker1 # [ 173.423368] nix-daemon[852]: reaped child process 1803, status = succeeded lb # [ 173.386174] niks3-server[1169]: 2026/09/22 11:34:27 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 173.443229] nix-grpc-daemon[1702]: time=2026-09-22T11:34:28.037Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 173.444906] nix-grpc-daemon[1702]: time=2026-09-22T11:34:28.037Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)" lb # [ 173.416953] niks3-server[1169]: 2026/09/22 11:34:28 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst lb # [ 173.431988] niks3-server[1169]: 2026/09/22 11:34:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign worker1 # [ 173.482803] nix-grpc-daemon[1702]: time=2026-09-22T11:34:28.078Z level=INFO msg="Uploading 1 narinfos" lb # [ 173.443658] niks3-server[1169]: 2026/09/22 11:34:28 INFO Signed narinfos id=54 count=1 lb # [ 173.451242] niks3-server[1169]: 2026/09/22 11:34:28 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls lb # [ 173.465949] niks3-server[1169]: 2026/09/22 11:34:28 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete lb # [ 173.472923] niks3-server[1169]: 2026/09/22 11:34:28 INFO Completed upload id=54 worker1 # [ 173.517154] nix-grpc-daemon[1702]: time=2026-09-22T11:34:28.112Z level=INFO msg="Upload complete. (377ms)" lb # [ 173.476056] niks3-server[1169]: 2026/09/22 11:34:28 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo worker1 # [ 173.524257] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48346 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 173.539248] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48346 worker1 # [ 173.547016] nix-daemon[852]: accepted connection from pid 1579, user nix-grpc-daemon (trusted) worker1 # [ 173.552300] nix-daemon[852]: reaped child process 1793, status = succeeded client # [ 173.585970] nix[1432]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051... worker1 # [ 173.566615] nix-daemon[852]: accepted connection from pid 1579, user nix-grpc-daemon (trusted) worker1: must succeed: systemctl reload nix-grpc-daemon.service worker1 # [ 175.415336] systemd[1]: Reloading Nix worker-protocol over gRPC... worker1 # [ 175.446137] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.07 seconds) worker1: must succeed: pgrep -f 'read -t [9]0 x' worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2 worker2 # [ 175.733689] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:35100 client # don't know how to build these paths: client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker2 # [ 175.742768] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35100 worker1 # [ 175.766178] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:30Z level=info event=draining worker2 # [ 175.771035] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=0 worker2 # [ 175.777521] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 175.816808] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 175.836757] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35100 client # copying 1 paths to 'lb:50051'... worker2 # [ 175.853221] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'... worker2 # [ 175.876184] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 176.042548] nix-daemon[842]: accepted connection from pid 1723, user nix-grpc-daemon (trusted) worker2 # [ 176.067878] nix-daemon[842]: reaped child process 1726, status = succeeded worker2 # [ 176.159018] nix-daemon[842]: accepted connection from pid 1728, user nix-grpc-daemon (trusted) worker2 # [ 176.175223] nix-daemon[842]: reaped child process 1731, status = succeeded lb # [ 176.133188] niks3-server[1169]: 2026/09/22 11:34:30 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 176.191744] nix-grpc-daemon[1643]: time=2026-09-22T11:34:30.778Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 176.193440] nix-grpc-daemon[1643]: time=2026-09-22T11:34:30.778Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)" lb # [ 176.172294] niks3-server[1169]: 2026/09/22 11:34:30 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst lb # [ 176.190191] niks3-server[1169]: 2026/09/22 11:34:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign worker2 # [ 176.244144] nix-grpc-daemon[1643]: time=2026-09-22T11:34:30.831Z level=INFO msg="Uploading 1 narinfos" lb # [ 176.200172] niks3-server[1169]: 2026/09/22 11:34:30 INFO Signed narinfos id=55 count=1 lb # [ 176.205531] niks3-server[1169]: 2026/09/22 11:34:30 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls lb # [ 176.214240] niks3-server[1169]: 2026/09/22 11:34:30 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete worker2 # [ 176.270338] nix-grpc-daemon[1643]: time=2026-09-22T11:34:30.857Z level=INFO msg="Upload complete. (377ms)" lb # [ 176.223710] niks3-server[1169]: 2026/09/22 11:34:30 INFO Completed upload id=55 lb # [ 176.230621] niks3-server[1169]: 2026/09/22 11:34:30 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo worker2 # [ 176.280384] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 paths=1 nar_bytes_in=528 worker2 # [ 176.300447] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 176.305378] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 176.316388] nix-daemon[842]: reaped child process 1721, status = succeeded client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051... client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker2 # [ 176.325560] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker2 # [ 177.596807] nix-daemon[842]: reaped child process 1735, status = succeeded worker2 # [ 177.703911] nix-daemon[842]: accepted connection from pid 1751, user nix-grpc-daemon (trusted) worker2 # [ 177.724434] nix-daemon[842]: reaped child process 1754, status = succeeded lb # [ 177.674769] niks3-server[1169]: 2026/09/22 11:34:32 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 177.736514] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.321Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 177.738390] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.321Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)" lb # [ 177.715258] niks3-server[1169]: 2026/09/22 11:34:32 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv lb # [ 177.733707] niks3-server[1169]: 2026/09/22 11:34:32 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst worker2 # [ 177.796122] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.382Z level=INFO msg="Uploading 1 narinfos" lb # [ 177.746027] niks3-server[1169]: 2026/09/22 11:34:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign lb # [ 177.753674] niks3-server[1169]: 2026/09/22 11:34:32 INFO Signed narinfos id=56 count=1 lb # [ 177.755717] niks3-server[1169]: 2026/09/22 11:34:32 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls lb # [ 177.763684] niks3-server[1169]: 2026/09/22 11:34:32 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 177.771895] niks3-server[1169]: 2026/09/22 11:34:32 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo worker2 # [ 177.820739] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.407Z level=INFO msg="Upload complete. (283ms)" lb # [ 177.776351] niks3-server[1169]: 2026/09/22 11:34:32 INFO Completed upload id=56 worker2 # [ 177.837928] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=1 worker2 # [ 177.855111] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=0 worker2 # [ 177.859070] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 177.863228] nix-daemon[842]: reaped child process 1733, status = succeeded worker2 # [ 177.882813] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 177.894091] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 177.908345] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.495Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 177.918922] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'... worker2 # [ 177.941972] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 178.101889] nix-daemon[842]: accepted connection from pid 1760, user nix-grpc-daemon (trusted) worker2 # [ 178.124305] nix-daemon[842]: reaped child process 1763, status = succeeded worker2 # [ 178.218053] nix-daemon[842]: accepted connection from pid 1765, user nix-grpc-daemon (trusted) worker2 # [ 178.235773] nix-daemon[842]: reaped child process 1768, status = succeeded lb # [ 178.185913] niks3-server[1169]: 2026/09/22 11:34:32 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 178.244160] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.831Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 178.245654] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.831Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)" lb # [ 178.220933] niks3-server[1169]: 2026/09/22 11:34:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign worker2 # [ 178.277017] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.863Z level=INFO msg="Uploading 1 narinfos" lb # [ 178.236918] niks3-server[1169]: 2026/09/22 11:34:32 INFO Signed narinfos id=57 count=1 lb # [ 178.245893] niks3-server[1169]: 2026/09/22 11:34:32 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst lb # [ 178.259699] niks3-server[1169]: 2026/09/22 11:34:32 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls worker2 # [ 178.314278] nix-grpc-daemon[1643]: time=2026-09-22T11:34:32.901Z level=INFO msg="Upload complete. (355ms)" lb # [ 178.268850] niks3-server[1169]: 2026/09/22 11:34:32 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete worker2 # [ 178.320250] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 paths=1 nar_bytes_in=640 lb # [ 178.274164] niks3-server[1169]: 2026/09/22 11:34:32 INFO Completed upload id=57 lb # [ 178.278248] niks3-server[1169]: 2026/09/22 11:34:32 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo worker2 # [ 178.332945] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 178.338433] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 178.343204] nix-daemon[842]: reaped child process 1758, status = succeeded worker2 # [ 178.353282] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051... client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv lb # [ 178.330902] envoy[893]: conn peer=[2001:db8:1::1]:54320 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=829 ms=3575 client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker2 # [ 179.858158] nix-daemon[842]: reaped child process 1772, status = succeeded worker2 # [ 179.933800] nix-daemon[842]: accepted connection from pid 1788, user nix-grpc-daemon (trusted) worker2 # [ 179.953546] nix-daemon[842]: reaped child process 1791, status = succeeded lb # [ 179.909221] niks3-server[1169]: 2026/09/22 11:34:34 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 179.968537] nix-grpc-daemon[1643]: time=2026-09-22T11:34:34.554Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 179.970302] nix-grpc-daemon[1643]: time=2026-09-22T11:34:34.554Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)" lb # [ 179.946721] niks3-server[1169]: 2026/09/22 11:34:34 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv lb # [ 179.961122] niks3-server[1169]: 2026/09/22 11:34:34 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst lb # [ 179.972913] niks3-server[1169]: 2026/09/22 11:34:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign worker2 # [ 180.028108] nix-grpc-daemon[1643]: time=2026-09-22T11:34:34.614Z level=INFO msg="Uploading 1 narinfos" lb # [ 179.982034] niks3-server[1169]: 2026/09/22 11:34:34 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls lb # [ 179.990904] niks3-server[1169]: 2026/09/22 11:34:34 INFO Signed narinfos id=58 count=1 lb # [ 180.001522] niks3-server[1169]: 2026/09/22 11:34:34 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete lb # [ 180.008850] niks3-server[1169]: 2026/09/22 11:34:34 INFO Completed upload id=58 worker2 # [ 180.057074] nix-grpc-daemon[1643]: time=2026-09-22T11:34:34.644Z level=INFO msg="Upload complete. (292ms)" lb # [ 180.014097] niks3-server[1169]: 2026/09/22 11:34:34 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo worker2 # [ 180.070270] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=1 worker2 # [ 180.094613] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:34Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=0 worker2 # [ 180.099281] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:34Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 180.104169] nix-daemon[842]: reaped child process 1770, status = succeeded worker2 # [ 180.133574] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 180.148787] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 180.162647] nix-grpc-daemon[1643]: time=2026-09-22T11:34:34.749Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 180.173524] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'... worker2 # [ 180.196417] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 180.353533] nix-daemon[842]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker2 # [ 180.369824] nix-daemon[842]: reaped child process 1800, status = succeeded worker2 # [ 180.470225] nix-daemon[842]: accepted connection from pid 1802, user nix-grpc-daemon (trusted) worker2 # [ 180.489110] nix-daemon[842]: reaped child process 1805, status = succeeded lb # [ 180.438316] niks3-server[1169]: 2026/09/22 11:34:35 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 180.495967] nix-grpc-daemon[1643]: time=2026-09-22T11:34:35.082Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 180.497716] nix-grpc-daemon[1643]: time=2026-09-22T11:34:35.082Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)" lb # [ 180.463966] niks3-server[1169]: 2026/09/22 11:34:35 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst lb # [ 180.480582] niks3-server[1169]: 2026/09/22 11:34:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign worker2 # [ 180.535278] nix-grpc-daemon[1643]: time=2026-09-22T11:34:35.122Z level=INFO msg="Uploading 1 narinfos" lb # [ 180.493210] niks3-server[1169]: 2026/09/22 11:34:35 INFO Signed narinfos id=59 count=1 lb # [ 180.499995] niks3-server[1169]: 2026/09/22 11:34:35 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls worker2 # [ 180.562598] nix-grpc-daemon[1643]: time=2026-09-22T11:34:35.149Z level=INFO msg="Upload complete. (349ms)" lb # [ 180.514404] niks3-server[1169]: 2026/09/22 11:34:35 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete lb # [ 180.521321] niks3-server[1169]: 2026/09/22 11:34:35 INFO Completed upload id=59 worker2 # [ 180.569431] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 paths=1 nar_bytes_in=768 lb # [ 180.523799] niks3-server[1169]: 2026/09/22 11:34:35 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo worker2 # [ 180.581145] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 180.589531] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 180.592398] nix-daemon[842]: reaped child process 1795, status = succeeded client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051... worker2 # [ 180.605273] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker2 # [ 181.466885] nix-daemon[842]: reaped child process 1809, status = succeeded worker2 # [ 181.580465] nix-daemon[842]: accepted connection from pid 1825, user nix-grpc-daemon (trusted) worker2 # [ 181.598859] nix-daemon[842]: reaped child process 1828, status = succeeded lb # [ 181.555267] niks3-server[1169]: 2026/09/22 11:34:36 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 181.615907] nix-grpc-daemon[1643]: time=2026-09-22T11:34:36.201Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 181.618782] nix-grpc-daemon[1643]: time=2026-09-22T11:34:36.201Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)" lb # [ 181.595240] niks3-server[1169]: 2026/09/22 11:34:36 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst worker2 # [ 181.666430] nix-grpc-daemon[1643]: time=2026-09-22T11:34:36.253Z level=INFO msg="Uploading 1 narinfos" lb # [ 181.610263] niks3-server[1169]: 2026/09/22 11:34:36 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv lb # [ 181.622178] niks3-server[1169]: 2026/09/22 11:34:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign lb # [ 181.629003] niks3-server[1169]: 2026/09/22 11:34:36 INFO Signed narinfos id=60 count=1 lb # [ 181.630987] niks3-server[1169]: 2026/09/22 11:34:36 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls lb # [ 181.637286] niks3-server[1169]: 2026/09/22 11:34:36 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete lb # [ 181.644292] niks3-server[1169]: 2026/09/22 11:34:36 INFO Completed upload id=60 worker2 # [ 181.693511] nix-grpc-daemon[1643]: time=2026-09-22T11:34:36.280Z level=INFO msg="Upload complete. (278ms)" lb # [ 181.646677] niks3-server[1169]: 2026/09/22 11:34:36 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo worker2 # [ 181.709132] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=1 worker2 # [ 181.725389] nix-daemon[842]: reaped child process 1807, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.26 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds) worker1: must succeed: pkill -f 'read -t [9]0 x' worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds) client: waiting for success: ! systemctl is-active sw worker1 # [ 182.244982] nix-daemon[852]: reaped child process 1807, status = succeeded worker1 # [ 182.352841] nix-daemon[852]: accepted connection from pid 1848, user nix-grpc-daemon (trusted) worker1 # [ 182.374269] nix-daemon[852]: reaped child process 1851, status = succeeded lb # [ 182.327437] niks3-server[1169]: 2026/09/22 11:34:36 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 182.383235] nix-grpc-daemon[1702]: time=2026-09-22T11:34:36.977Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 182.385568] nix-grpc-daemon[1702]: time=2026-09-22T11:34:36.977Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)" lb # [ 182.359032] niks3-server[1169]: 2026/09/22 11:34:36 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst lb # [ 182.378501] niks3-server[1169]: 2026/09/22 11:34:36 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls worker1 # [ 182.432723] nix-grpc-daemon[1702]: time=2026-09-22T11:34:37.027Z level=INFO msg="Uploading 1 narinfos" lb # [ 182.391652] niks3-server[1169]: 2026/09/22 11:34:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign lb # [ 182.401039] niks3-server[1169]: 2026/09/22 11:34:36 INFO Signed narinfos id=61 count=1 lb # [ 182.410206] niks3-server[1169]: 2026/09/22 11:34:36 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker1 # [ 182.455178] nix-grpc-daemon[1702]: time=2026-09-22T11:34:37.050Z level=INFO msg="Upload complete. (270ms)" lb # [ 182.422434] niks3-server[1169]: 2026/09/22 11:34:36 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete lb # [ 182.430367] niks3-server[1169]: 2026/09/22 11:34:37 INFO Completed upload id=61 worker1 # [ 182.473165] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48346 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=1 lb # [ 182.433075] niks3-server[1169]: 2026/09/22 11:34:37 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo worker1 # [ 182.489792] nix-daemon[852]: reaped child process 1805, status = succeeded client # [ 182.523653] systemd[1]: sw.service: Deactivated successfully. client # [ 182.528205] systemd[1]: sw.service: Consumed 187ms CPU time over 9.694s wall clock time, 43.2M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic. worker1 # [ 182.767952] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:37Z level=info event=signal_exit client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds) worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service worker1 # [ 184.762237] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:39Z level=info event=scheduler_restarting worker2 # [ 184.777418] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:39Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:35084 worker2 # [ 184.782131] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:39Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 184.794231] nix-grpc-daemon[1579]: ts=2026-09-22T11:34:39Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 184.819885] nix-daemon[852]: reaped child process 1691, status = succeeded worker1 # [ 185.465384] nix-grpc-daemon[1579]: E0000 00:00:1790076880.060842 1624 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true worker1 # [ 185.474391] nix-grpc-daemon[1579]: E0000 00:00:1790076880.069892 1624 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1 # [ 186.158891] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 186.163559] systemd[1]: nix-grpc-daemon.service: Consumed 1.154s CPU time over 42.563s wall clock time, 36.4M memory peak, 26.4K incoming IP traffic, 38.1K outgoing IP traffic. worker1 # [ 186.187783] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 186.337998] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 186.340279] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 186.344149] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 186.349424] nix-grpc-daemon[1865]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 186.350901] nix-grpc-daemon[1865]: E0000 00:00:1790076880.944236 1865 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.24 seconds) lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 186.380174] nix-grpc-daemon[1865]: E0000 00:00:1790076880.950272 1865 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 186.382501] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 186.400475] nix-grpc-daemon[1865]: E0000 00:00:1790076880.960742 1905 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 186.403369] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 186.404406] nix-grpc-daemon[1865]: E0000 00:00:1790076880.980617 1905 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 186.406696] nix-grpc-daemon[1865]: E0000 00:00:1790076880.980777 1905 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 186.409207] nix-grpc-daemon[1865]: E0000 00:00:1790076880.995160 1905 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 186.417316] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 186.424625] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:41Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:35084 worker2 # [ 186.429923] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:41Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.14 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.89 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker2 # [ 187.868370] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 187.874684] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 187.920331] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:42Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=0 worker2 # [ 187.926733] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:42Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 187.933006] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:42Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=0 worker2 # [ 187.938289] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:42Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=0 worker1 # [ 187.978373] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 187.991907] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 188.016148] nix-grpc-daemon[1865]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 1/5) worker1 # [ 188.092807] nix-grpc-daemon[1865]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2 ms (attempt 2/5) worker1 # [ 188.102224] nix-grpc-daemon[1865]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 3/5) worker1 # [ 188.156726] nix-grpc-daemon[1865]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 548 ms (attempt 4/5) lb # [ 188.333535] envoy[893]: conn peer=[2001:db8:1::1]:51980 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9664 tx=4396 ms=5993 lb # [ 188.339005] envoy[893]: conn peer=[2001:db8:1::1]:51972 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1517 ms=9482 lb # [ 188.343965] envoy[893]: conn peer=[2001:db8:1::3]:37470 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2297 tx=738 ms=40970 worker1 # [ 188.707417] nix-grpc-daemon[1865]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 188.713864] nix-grpc-daemon[1865]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 188.732375] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 188.748690] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 188.762742] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 188.801915] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:43Z level=info event=niks3_push_started pid=1925 worker1 # [ 188.955718] nix-grpc-daemon[1925]: time=2026-09-22T11:34:43.549Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 189.059842] nix-daemon[852]: accepted connection from pid 1938, user nix-grpc-daemon (trusted) worker1 # [ 189.077987] nix-daemon[852]: reaped child process 1941, status = succeeded worker1 # [ 189.177238] nix-daemon[852]: accepted connection from pid 1943, user nix-grpc-daemon (trusted) worker1 # [ 189.194852] nix-daemon[852]: reaped child process 1946, status = succeeded lb # [ 189.155976] niks3-server[1169]: 2026/09/22 11:34:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 189.211536] nix-grpc-daemon[1925]: time=2026-09-22T11:34:43.806Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 189.213935] nix-grpc-daemon[1925]: time=2026-09-22T11:34:43.806Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)" lb # [ 189.193721] niks3-server[1169]: 2026/09/22 11:34:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign worker1 # [ 189.248428] nix-grpc-daemon[1925]: time=2026-09-22T11:34:43.843Z level=INFO msg="Uploading 1 narinfos" lb # [ 189.206559] niks3-server[1169]: 2026/09/22 11:34:43 INFO Signed narinfos id=62 count=1 lb # [ 189.219540] niks3-server[1169]: 2026/09/22 11:34:43 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst lb # [ 189.229259] niks3-server[1169]: 2026/09/22 11:34:43 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls worker1 # [ 189.283349] nix-grpc-daemon[1925]: time=2026-09-22T11:34:43.878Z level=INFO msg="Upload complete. (328ms)" lb # [ 189.239651] niks3-server[1169]: 2026/09/22 11:34:43 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete lb # [ 189.247661] niks3-server[1169]: 2026/09/22 11:34:43 INFO Completed upload id=62 worker1 # [ 189.293380] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 189.252267] niks3-server[1169]: 2026/09/22 11:34:43 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo worker1 # [ 189.306957] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 189.312566] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 189.316912] nix-daemon[852]: reaped child process 1923, status = succeeded worker1 # [ 189.328160] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 190.839215] nix-daemon[852]: reaped child process 1950, status = succeeded worker1 # [ 190.910695] nix-daemon[852]: accepted connection from pid 1966, user nix-grpc-daemon (trusted) worker1 # [ 190.931011] nix-daemon[852]: reaped child process 1969, status = succeeded lb # [ 190.885373] niks3-server[1169]: 2026/09/22 11:34:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 190.940463] nix-grpc-daemon[1925]: time=2026-09-22T11:34:45.535Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 190.941970] nix-grpc-daemon[1925]: time=2026-09-22T11:34:45.535Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)" lb # [ 190.917694] niks3-server[1169]: 2026/09/22 11:34:45 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv lb # [ 190.940486] niks3-server[1169]: 2026/09/22 11:34:45 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst worker1 # [ 191.004112] nix-grpc-daemon[1925]: time=2026-09-22T11:34:45.598Z level=INFO msg="Uploading 1 narinfos" lb # [ 190.952789] niks3-server[1169]: 2026/09/22 11:34:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign lb # [ 190.964580] niks3-server[1169]: 2026/09/22 11:34:45 INFO Signed narinfos id=63 count=1 lb # [ 190.967234] niks3-server[1169]: 2026/09/22 11:34:45 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls lb # [ 190.979990] niks3-server[1169]: 2026/09/22 11:34:45 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete lb # [ 190.987366] niks3-server[1169]: 2026/09/22 11:34:45 INFO Completed upload id=63 worker1 # [ 191.031869] nix-grpc-daemon[1925]: time=2026-09-22T11:34:45.627Z level=INFO msg="Upload complete. (293ms)" lb # [ 190.990712] niks3-server[1169]: 2026/09/22 11:34:45 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo worker2 # [ 191.046267] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:45Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 191.045175] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=6 outputs=1 worker2 # [ 191.059859] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:45Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=0 worker1 # [ 191.065302] nix-daemon[852]: reaped child process 1948, status = succeeded worker1 # [ 191.098274] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 191.113327] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 191.127103] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 191.140111] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 191.296330] nix-daemon[852]: accepted connection from pid 1973, user nix-grpc-daemon (trusted) worker1 # [ 191.312852] nix-daemon[852]: reaped child process 1976, status = succeeded worker1 # [ 191.414074] nix-daemon[852]: accepted connection from pid 1978, user nix-grpc-daemon (trusted) worker1 # [ 191.431903] nix-daemon[852]: reaped child process 1981, status = succeeded lb # [ 191.385394] niks3-server[1169]: 2026/09/22 11:34:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 191.438778] nix-grpc-daemon[1925]: time=2026-09-22T11:34:46.034Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 191.440421] nix-grpc-daemon[1925]: time=2026-09-22T11:34:46.034Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)" lb # [ 191.412971] niks3-server[1169]: 2026/09/22 11:34:46 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst worker1 # [ 191.471423] nix-grpc-daemon[1925]: time=2026-09-22T11:34:46.066Z level=INFO msg="Uploading 1 narinfos" lb # [ 191.426384] niks3-server[1169]: 2026/09/22 11:34:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign lb # [ 191.440468] niks3-server[1169]: 2026/09/22 11:34:46 INFO Signed narinfos id=64 count=1 lb # [ 191.443747] niks3-server[1169]: 2026/09/22 11:34:46 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls worker1 # [ 191.505701] nix-grpc-daemon[1925]: time=2026-09-22T11:34:46.101Z level=INFO msg="Upload complete. (350ms)" lb # [ 191.461121] niks3-server[1169]: 2026/09/22 11:34:46 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete lb # [ 191.466988] niks3-server[1169]: 2026/09/22 11:34:46 INFO Completed upload id=64 lb # [ 191.470514] niks3-server[1169]: 2026/09/22 11:34:46 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo worker1 # [ 191.515832] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 191.526838] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 191.534609] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 191.536894] nix-daemon[852]: reaped child process 1971, status = succeeded worker1 # [ 191.548535] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 192.629208] nix-daemon[852]: reaped child process 1985, status = succeeded worker1 # [ 192.695391] nix-daemon[852]: accepted connection from pid 2001, user nix-grpc-daemon (trusted) worker1 # [ 192.713815] nix-daemon[852]: reaped child process 2004, status = succeeded lb # [ 192.675108] niks3-server[1169]: 2026/09/22 11:34:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 192.730061] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.324Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 192.731513] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.324Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)" lb # [ 192.713397] niks3-server[1169]: 2026/09/22 11:34:47 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv lb # [ 192.725524] niks3-server[1169]: 2026/09/22 11:34:47 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst worker1 # [ 192.782704] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.377Z level=INFO msg="Uploading 1 narinfos" lb # [ 192.740566] niks3-server[1169]: 2026/09/22 11:34:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign lb # [ 192.749221] niks3-server[1169]: 2026/09/22 11:34:47 INFO Signed narinfos id=65 count=1 lb # [ 192.751731] niks3-server[1169]: 2026/09/22 11:34:47 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls lb # [ 192.764265] niks3-server[1169]: 2026/09/22 11:34:47 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete lb # [ 192.770548] niks3-server[1169]: 2026/09/22 11:34:47 INFO Completed upload id=65 worker1 # [ 192.813763] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.409Z level=INFO msg="Upload complete. (256ms)" lb # [ 192.772906] niks3-server[1169]: 2026/09/22 11:34:47 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo worker2 # [ 192.831946] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:47Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 192.828005] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=7 outputs=1 worker2 # [ 192.840070] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:47Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=0 worker1 # [ 192.847446] nix-daemon[852]: reaped child process 1983, status = succeeded worker1 # [ 192.874009] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 192.889460] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 192.904742] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 192.918490] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 193.073552] nix-daemon[852]: accepted connection from pid 2008, user nix-grpc-daemon (trusted) worker1 # [ 193.090444] nix-daemon[852]: reaped child process 2011, status = succeeded worker1 # [ 193.195400] nix-daemon[852]: accepted connection from pid 2013, user nix-grpc-daemon (trusted) worker1 # [ 193.212615] nix-daemon[852]: reaped child process 2016, status = succeeded lb # [ 193.172802] niks3-server[1169]: 2026/09/22 11:34:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 193.226142] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.821Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 193.227724] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.821Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)" worker1 # [ 193.262202] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.857Z level=INFO msg="Uploading 1 narinfos" lb # [ 193.205533] niks3-server[1169]: 2026/09/22 11:34:47 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst lb # [ 193.219941] niks3-server[1169]: 2026/09/22 11:34:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign lb # [ 193.230307] niks3-server[1169]: 2026/09/22 11:34:47 INFO Signed narinfos id=66 count=1 lb # [ 193.239786] niks3-server[1169]: 2026/09/22 11:34:47 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls lb # [ 193.250352] niks3-server[1169]: 2026/09/22 11:34:47 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete lb # [ 193.255860] niks3-server[1169]: 2026/09/22 11:34:47 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo lb # [ 193.261470] niks3-server[1169]: 2026/09/22 11:34:47 INFO Completed upload id=66 worker1 # [ 193.305537] nix-grpc-daemon[1925]: time=2026-09-22T11:34:47.901Z level=INFO msg="Upload complete. (371ms)" worker1 # [ 193.314904] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 193.330798] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 193.337590] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 193.342927] nix-daemon[852]: reaped child process 2006, status = succeeded worker1 # [ 193.354947] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 194.036346] nix-daemon[852]: reaped child process 2020, status = succeeded worker1 # [ 194.137359] nix-daemon[852]: accepted connection from pid 2036, user nix-grpc-daemon (trusted) worker1 # [ 194.154136] nix-daemon[852]: reaped child process 2039, status = succeeded lb # [ 194.117043] niks3-server[1169]: 2026/09/22 11:34:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 194.172430] nix-grpc-daemon[1925]: time=2026-09-22T11:34:48.766Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 194.174181] nix-grpc-daemon[1925]: time=2026-09-22T11:34:48.766Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)" lb # [ 194.147963] niks3-server[1169]: 2026/09/22 11:34:48 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv lb # [ 194.162029] niks3-server[1169]: 2026/09/22 11:34:48 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst worker1 # [ 194.225149] nix-grpc-daemon[1925]: time=2026-09-22T11:34:48.820Z level=INFO msg="Uploading 1 narinfos" lb # [ 194.179660] niks3-server[1169]: 2026/09/22 11:34:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign lb # [ 194.190696] niks3-server[1169]: 2026/09/22 11:34:48 INFO Signed narinfos id=67 count=1 lb # [ 194.197649] niks3-server[1169]: 2026/09/22 11:34:48 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls worker1 # [ 194.255374] nix-grpc-daemon[1925]: time=2026-09-22T11:34:48.850Z level=INFO msg="Upload complete. (286ms)" lb # [ 194.209694] niks3-server[1169]: 2026/09/22 11:34:48 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete lb # [ 194.218856] niks3-server[1169]: 2026/09/22 11:34:48 INFO Completed upload id=67 lb # [ 194.222614] niks3-server[1169]: 2026/09/22 11:34:48 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo worker2 # [ 194.269120] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:48Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 194.272243] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=8 outputs=1 worker2 # [ 194.282684] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:48Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=0 worker1 # [ 194.288665] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 194.297306] nix-daemon[852]: reaped child process 2018, status = succeeded worker1 # [ 194.305567] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 194.319515] nix-grpc-daemon[1925]: time=2026-09-22T11:34:48.915Z level=INFO msg="All 1 paths already cached" worker1 # [ 194.329276] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 194.342317] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 194.495002] nix-daemon[852]: accepted connection from pid 2043, user nix-grpc-daemon (trusted) worker1 # [ 194.515587] nix-daemon[852]: reaped child process 2046, status = succeeded worker1 # [ 194.613845] nix-daemon[852]: accepted connection from pid 2048, user nix-grpc-daemon (trusted) worker1 # [ 194.631008] nix-daemon[852]: reaped child process 2051, status = succeeded lb # [ 194.589532] niks3-server[1169]: 2026/09/22 11:34:49 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 194.642867] nix-grpc-daemon[1925]: time=2026-09-22T11:34:49.238Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 194.644391] nix-grpc-daemon[1925]: time=2026-09-22T11:34:49.238Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)" lb # [ 194.616591] niks3-server[1169]: 2026/09/22 11:34:49 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst worker1 # [ 194.675065] nix-grpc-daemon[1925]: time=2026-09-22T11:34:49.270Z level=INFO msg="Uploading 1 narinfos" lb # [ 194.630877] niks3-server[1169]: 2026/09/22 11:34:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign lb # [ 194.643408] niks3-server[1169]: 2026/09/22 11:34:49 INFO Signed narinfos id=68 count=1 lb # [ 194.648988] niks3-server[1169]: 2026/09/22 11:34:49 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls lb # [ 194.658788] niks3-server[1169]: 2026/09/22 11:34:49 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete worker1 # [ 194.710454] nix-grpc-daemon[1925]: time=2026-09-22T11:34:49.306Z level=INFO msg="Upload complete. (353ms)" lb # [ 194.665787] niks3-server[1169]: 2026/09/22 11:34:49 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo lb # [ 194.668993] niks3-server[1169]: 2026/09/22 11:34:49 INFO Completed upload id=68 worker1 # [ 194.719750] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 194.732223] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 194.739186] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 194.743855] nix-daemon[852]: reaped child process 2041, status = succeeded worker1 # [ 194.755668] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 196.246575] nix-daemon[852]: reaped child process 2055, status = succeeded worker1 # [ 196.316444] nix-daemon[852]: accepted connection from pid 2071, user nix-grpc-daemon (trusted) worker1 # [ 196.333482] nix-daemon[852]: reaped child process 2074, status = succeeded lb # [ 196.298231] niks3-server[1169]: 2026/09/22 11:34:50 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 196.356440] nix-grpc-daemon[1925]: time=2026-09-22T11:34:50.951Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 196.358232] nix-grpc-daemon[1925]: time=2026-09-22T11:34:50.951Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)" lb # [ 196.348684] niks3-server[1169]: 2026/09/22 11:34:50 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst lb # [ 196.357100] niks3-server[1169]: 2026/09/22 11:34:50 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker1 # [ 196.409801] nix-grpc-daemon[1925]: time=2026-09-22T11:34:51.005Z level=INFO msg="Uploading 1 narinfos" lb # [ 196.365946] niks3-server[1169]: 2026/09/22 11:34:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign lb # [ 196.372224] niks3-server[1169]: 2026/09/22 11:34:50 INFO Signed narinfos id=69 count=1 lb # [ 196.376945] niks3-server[1169]: 2026/09/22 11:34:50 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls lb # [ 196.384740] niks3-server[1169]: 2026/09/22 11:34:50 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 196.390320] niks3-server[1169]: 2026/09/22 11:34:50 INFO Completed upload id=69 lb # [ 196.393341] niks3-server[1169]: 2026/09/22 11:34:50 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo worker1 # [ 196.433919] nix-grpc-daemon[1925]: time=2026-09-22T11:34:51.029Z level=INFO msg="Upload complete. (264ms)" worker2 # [ 196.447875] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:51Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 196.445273] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=9 outputs=1 worker2 # [ 196.460298] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:51Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=0 worker1 # [ 196.465391] nix-daemon[852]: reaped child process 2053, status = succeeded worker1 # [ 196.500542] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 196.516885] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 196.531101] nix-grpc-daemon[1925]: time=2026-09-22T11:34:51.126Z level=INFO msg="All 1 paths already cached" worker1 # [ 196.540766] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 196.555945] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 196.713188] nix-daemon[852]: accepted connection from pid 2078, user nix-grpc-daemon (trusted) worker1 # [ 196.737578] nix-daemon[852]: reaped child process 2081, status = succeeded worker1 # [ 196.830933] nix-daemon[852]: accepted connection from pid 2083, user nix-grpc-daemon (trusted) worker1 # [ 196.845432] nix-daemon[852]: reaped child process 2086, status = succeeded lb # [ 196.807398] niks3-server[1169]: 2026/09/22 11:34:51 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 196.862087] nix-grpc-daemon[1925]: time=2026-09-22T11:34:51.457Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 196.863545] nix-grpc-daemon[1925]: time=2026-09-22T11:34:51.457Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)" lb # [ 196.834846] niks3-server[1169]: 2026/09/22 11:34:51 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst worker1 # [ 196.893452] nix-grpc-daemon[1925]: time=2026-09-22T11:34:51.488Z level=INFO msg="Uploading 1 narinfos" lb # [ 196.849830] niks3-server[1169]: 2026/09/22 11:34:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign lb # [ 196.859963] niks3-server[1169]: 2026/09/22 11:34:51 INFO Signed narinfos id=70 count=1 lb # [ 196.867412] niks3-server[1169]: 2026/09/22 11:34:51 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls lb # [ 196.877059] niks3-server[1169]: 2026/09/22 11:34:51 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete worker1 # [ 196.928411] nix-grpc-daemon[1925]: time=2026-09-22T11:34:51.524Z level=INFO msg="Upload complete. (356ms)" lb # [ 196.886671] niks3-server[1169]: 2026/09/22 11:34:51 INFO Completed upload id=70 lb # [ 196.889876] niks3-server[1169]: 2026/09/22 11:34:51 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo worker1 # [ 196.936279] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 196.950638] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 196.956898] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 196.961885] nix-daemon[852]: reaped child process 2076, status = succeeded worker1 # [ 196.973738] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 197.844260] nix-daemon[852]: reaped child process 2090, status = succeeded worker1 # [ 197.954939] nix-daemon[852]: accepted connection from pid 2106, user nix-grpc-daemon (trusted) worker1 # [ 197.978407] nix-daemon[852]: reaped child process 2109, status = succeeded lb # [ 197.938974] niks3-server[1169]: 2026/09/22 11:34:52 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 197.996603] nix-grpc-daemon[1925]: time=2026-09-22T11:34:52.589Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 197.998574] nix-grpc-daemon[1925]: time=2026-09-22T11:34:52.590Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)" lb # [ 197.987362] niks3-server[1169]: 2026/09/22 11:34:52 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv lb # [ 197.999656] niks3-server[1169]: 2026/09/22 11:34:52 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst worker1 # [ 198.056736] nix-grpc-daemon[1925]: time=2026-09-22T11:34:52.652Z level=INFO msg="Uploading 1 narinfos" lb # [ 198.011686] niks3-server[1169]: 2026/09/22 11:34:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign lb # [ 198.020341] niks3-server[1169]: 2026/09/22 11:34:52 INFO Signed narinfos id=71 count=1 lb # [ 198.026371] niks3-server[1169]: 2026/09/22 11:34:52 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls lb # [ 198.035713] niks3-server[1169]: 2026/09/22 11:34:52 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete worker1 # [ 198.085001] nix-grpc-daemon[1925]: time=2026-09-22T11:34:52.680Z level=INFO msg="Upload complete. (301ms)" lb # [ 198.042074] niks3-server[1169]: 2026/09/22 11:34:52 INFO Completed upload id=71 lb # [ 198.044810] niks3-server[1169]: 2026/09/22 11:34:52 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo worker2 # [ 198.099906] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:52Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 198.099728] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=10 outputs=1 worker2 # [ 198.112934] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:52Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=0 worker1 # [ 198.116297] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 198.129870] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 198.136330] nix-daemon[852]: reaped child process 2088, status = succeeded worker1 # [ 198.143431] nix-grpc-daemon[1925]: time=2026-09-22T11:34:52.739Z level=INFO msg="All 1 paths already cached" worker1 # [ 198.153952] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 198.167324] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 198.312228] nix-daemon[852]: accepted connection from pid 2113, user nix-grpc-daemon (trusted) worker1 # [ 198.333012] nix-daemon[852]: reaped child process 2116, status = succeeded worker1 # [ 198.436111] nix-daemon[852]: accepted connection from pid 2118, user nix-grpc-daemon (trusted) worker1 # [ 198.452813] nix-daemon[852]: reaped child process 2121, status = succeeded lb # [ 198.413416] niks3-server[1169]: 2026/09/22 11:34:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 198.469727] nix-grpc-daemon[1925]: time=2026-09-22T11:34:53.065Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 198.471199] nix-grpc-daemon[1925]: time=2026-09-22T11:34:53.065Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)" lb # [ 198.443111] niks3-server[1169]: 2026/09/22 11:34:53 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst lb # [ 198.459256] niks3-server[1169]: 2026/09/22 11:34:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign worker1 # [ 198.510325] nix-grpc-daemon[1925]: time=2026-09-22T11:34:53.105Z level=INFO msg="Uploading 1 narinfos" lb # [ 198.470735] niks3-server[1169]: 2026/09/22 11:34:53 INFO Signed narinfos id=72 count=1 lb # [ 198.476753] niks3-server[1169]: 2026/09/22 11:34:53 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls lb # [ 198.486639] niks3-server[1169]: 2026/09/22 11:34:53 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete lb # [ 198.499204] niks3-server[1169]: 2026/09/22 11:34:53 INFO Completed upload id=72 lb # [ 198.502130] niks3-server[1169]: 2026/09/22 11:34:53 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo worker1 # [ 198.542641] nix-grpc-daemon[1925]: time=2026-09-22T11:34:53.138Z level=INFO msg="Upload complete. (362ms)" worker1 # [ 198.552434] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 198.567148] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 198.573980] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 198.578919] nix-daemon[852]: reaped child process 2111, status = succeeded worker1 # [ 198.591530] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 200.259751] nix-daemon[852]: reaped child process 2125, status = succeeded worker1 # [ 200.392483] nix-daemon[852]: accepted connection from pid 2141, user nix-grpc-daemon (trusted) worker1 # [ 200.407712] nix-daemon[852]: reaped child process 2144, status = succeeded lb # [ 200.370850] niks3-server[1169]: 2026/09/22 11:34:54 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 200.427682] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.020Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 200.429559] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.020Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)" lb # [ 200.407508] niks3-server[1169]: 2026/09/22 11:34:54 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst lb # [ 200.425565] niks3-server[1169]: 2026/09/22 11:34:55 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv lb # [ 200.436774] niks3-server[1169]: 2026/09/22 11:34:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign worker1 # [ 200.486401] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.081Z level=INFO msg="Uploading 1 narinfos" lb # [ 200.444383] niks3-server[1169]: 2026/09/22 11:34:55 INFO Signed narinfos id=73 count=1 lb # [ 200.451837] niks3-server[1169]: 2026/09/22 11:34:55 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls lb # [ 200.462222] niks3-server[1169]: 2026/09/22 11:34:55 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete worker1 # [ 200.515322] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.110Z level=INFO msg="Upload complete. (311ms)" lb # [ 200.473681] niks3-server[1169]: 2026/09/22 11:34:55 INFO Completed upload id=73 lb # [ 200.478547] niks3-server[1169]: 2026/09/22 11:34:55 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo worker2 # [ 200.528692] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:55Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 200.527715] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=11 outputs=1 worker2 # [ 200.541820] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:55Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=0 worker1 # [ 200.551397] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 200.557346] nix-daemon[852]: reaped child process 2123, status = succeeded worker1 # [ 200.566150] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 200.580784] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.175Z level=INFO msg="All 2 paths already cached" worker1 # [ 200.590407] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 200.604114] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 200.748911] nix-daemon[852]: accepted connection from pid 2148, user nix-grpc-daemon (trusted) worker1 # [ 200.774509] nix-daemon[852]: reaped child process 2151, status = succeeded worker1 # [ 200.868008] nix-daemon[852]: accepted connection from pid 2153, user nix-grpc-daemon (trusted) worker1 # [ 200.881745] nix-daemon[852]: reaped child process 2156, status = succeeded lb # [ 200.844349] niks3-server[1169]: 2026/09/22 11:34:55 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 200.899341] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.494Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 200.900906] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.494Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)" lb # [ 200.872384] niks3-server[1169]: 2026/09/22 11:34:55 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst lb # [ 200.886511] niks3-server[1169]: 2026/09/22 11:34:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign worker1 # [ 200.938811] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.534Z level=INFO msg="Uploading 1 narinfos" lb # [ 200.900097] niks3-server[1169]: 2026/09/22 11:34:55 INFO Signed narinfos id=74 count=1 lb # [ 200.902771] niks3-server[1169]: 2026/09/22 11:34:55 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls worker1 # [ 200.960356] nix-grpc-daemon[1925]: time=2026-09-22T11:34:55.556Z level=INFO msg="Upload complete. (341ms)" lb # [ 200.918038] niks3-server[1169]: 2026/09/22 11:34:55 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete lb # [ 200.923444] niks3-server[1169]: 2026/09/22 11:34:55 INFO Completed upload id=74 worker1 # [ 200.968266] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 200.926859] niks3-server[1169]: 2026/09/22 11:34:55 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo worker1 # [ 200.979581] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 200.988817] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 200.992060] nix-daemon[852]: reaped child process 2146, status = succeeded worker1 # [ 201.002427] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 201.860965] nix-daemon[852]: reaped child process 2160, status = succeeded worker1 # [ 201.997979] nix-daemon[852]: accepted connection from pid 2176, user nix-grpc-daemon (trusted) worker1 # [ 202.018772] nix-daemon[852]: reaped child process 2179, status = succeeded lb # [ 201.972895] niks3-server[1169]: 2026/09/22 11:34:56 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 202.028466] nix-grpc-daemon[1925]: time=2026-09-22T11:34:56.622Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 202.030141] nix-grpc-daemon[1925]: time=2026-09-22T11:34:56.622Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)" lb # [ 202.009399] niks3-server[1169]: 2026/09/22 11:34:56 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst lb # [ 202.021909] niks3-server[1169]: 2026/09/22 11:34:56 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker1 # [ 202.080053] nix-grpc-daemon[1925]: time=2026-09-22T11:34:56.675Z level=INFO msg="Uploading 1 narinfos" lb # [ 202.029811] niks3-server[1169]: 2026/09/22 11:34:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign lb # [ 202.040259] niks3-server[1169]: 2026/09/22 11:34:56 INFO Signed narinfos id=75 count=1 lb # [ 202.046322] niks3-server[1169]: 2026/09/22 11:34:56 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls lb # [ 202.057823] niks3-server[1169]: 2026/09/22 11:34:56 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete lb # [ 202.065695] niks3-server[1169]: 2026/09/22 11:34:56 INFO Completed upload id=75 worker1 # [ 202.109682] nix-grpc-daemon[1925]: time=2026-09-22T11:34:56.705Z level=INFO msg="Upload complete. (294ms)" lb # [ 202.068415] niks3-server[1169]: 2026/09/22 11:34:56 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo worker2 # [ 202.123853] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:56Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 202.121187] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=12 outputs=1 worker1 # [ 202.142531] nix-daemon[852]: reaped child process 2158, status = succeeded worker1 # [ 202.172903] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 202.188472] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 202.201759] nix-grpc-daemon[1925]: time=2026-09-22T11:34:56.797Z level=INFO msg="All 2 paths already cached" worker1 # [ 202.213053] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 202.226682] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 202.369882] nix-daemon[852]: accepted connection from pid 2183, user nix-grpc-daemon (trusted) worker1 # [ 202.386957] nix-daemon[852]: reaped child process 2186, status = succeeded worker1 # [ 202.487267] nix-daemon[852]: accepted connection from pid 2188, user nix-grpc-daemon (trusted) worker1 # [ 202.505270] nix-daemon[852]: reaped child process 2191, status = succeeded lb # [ 202.458166] niks3-server[1169]: 2026/09/22 11:34:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 202.511500] nix-grpc-daemon[1925]: time=2026-09-22T11:34:57.106Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 202.513195] nix-grpc-daemon[1925]: time=2026-09-22T11:34:57.106Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)" lb # [ 202.484246] niks3-server[1169]: 2026/09/22 11:34:57 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst worker1 # [ 202.542944] nix-grpc-daemon[1925]: time=2026-09-22T11:34:57.138Z level=INFO msg="Uploading 1 narinfos" lb # [ 202.498644] niks3-server[1169]: 2026/09/22 11:34:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign lb # [ 202.508387] niks3-server[1169]: 2026/09/22 11:34:57 INFO Signed narinfos id=76 count=1 worker1 # [ 202.570837] nix-grpc-daemon[1925]: time=2026-09-22T11:34:57.166Z level=INFO msg="Upload complete. (328ms)" lb # [ 202.517737] niks3-server[1169]: 2026/09/22 11:34:57 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete lb # [ 202.533106] niks3-server[1169]: 2026/09/22 11:34:57 INFO Completed upload id=76 lb # [ 202.535319] niks3-server[1169]: 2026/09/22 11:34:57 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo worker1 # [ 202.577676] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 202.587238] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 202.590215] nix-daemon[852]: reaped child process 2181, status = succeeded worker1 # [ 202.596197] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) lb # [ 202.559535] niks3-server[1169]: 2026/09/22 11:34:57 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls worker1 # [ 202.605946] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 204.035105] nix-daemon[852]: reaped child process 2195, status = succeeded worker1 # [ 204.145718] nix-daemon[852]: accepted connection from pid 2211, user nix-grpc-daemon (trusted) worker1 # [ 204.163124] nix-daemon[852]: reaped child process 2214, status = succeeded lb # [ 204.126644] niks3-server[1169]: 2026/09/22 11:34:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 204.186492] nix-grpc-daemon[1925]: time=2026-09-22T11:34:58.780Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 204.187962] nix-grpc-daemon[1925]: time=2026-09-22T11:34:58.781Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)" lb # [ 204.172092] niks3-server[1169]: 2026/09/22 11:34:58 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv lb # [ 204.184708] niks3-server[1169]: 2026/09/22 11:34:58 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst worker1 # [ 204.238617] nix-grpc-daemon[1925]: time=2026-09-22T11:34:58.833Z level=INFO msg="Uploading 1 narinfos" lb # [ 204.196219] niks3-server[1169]: 2026/09/22 11:34:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign lb # [ 204.199167] niks3-server[1169]: 2026/09/22 11:34:58 INFO Signed narinfos id=77 count=1 lb # [ 204.207865] niks3-server[1169]: 2026/09/22 11:34:58 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls lb # [ 204.215790] niks3-server[1169]: 2026/09/22 11:34:58 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete lb # [ 204.221012] niks3-server[1169]: 2026/09/22 11:34:58 INFO Completed upload id=77 worker1 # [ 204.265180] nix-grpc-daemon[1925]: time=2026-09-22T11:34:58.860Z level=INFO msg="Upload complete. (244ms)" lb # [ 204.223470] niks3-server[1169]: 2026/09/22 11:34:58 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo worker2 # [ 204.278104] nix-grpc-daemon[1569]: ts=2026-09-22T11:34:58Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 204.277144] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=13 outputs=1 worker1 # [ 204.295171] nix-daemon[852]: reaped child process 2193, status = succeeded worker1 # [ 204.317984] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 204.332539] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 204.347963] nix-grpc-daemon[1925]: time=2026-09-22T11:34:58.943Z level=INFO msg="All 2 paths already cached" worker1 # [ 204.358069] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 204.372782] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 204.523177] nix-daemon[852]: accepted connection from pid 2218, user nix-grpc-daemon (trusted) worker1 # [ 204.541757] nix-daemon[852]: reaped child process 2221, status = succeeded worker1 # [ 204.639522] nix-daemon[852]: accepted connection from pid 2223, user nix-grpc-daemon (trusted) worker1 # [ 204.657654] nix-daemon[852]: reaped child process 2226, status = succeeded lb # [ 204.610796] niks3-server[1169]: 2026/09/22 11:34:59 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 204.666066] nix-grpc-daemon[1925]: time=2026-09-22T11:34:59.261Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 204.667516] nix-grpc-daemon[1925]: time=2026-09-22T11:34:59.261Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)" lb # [ 204.640046] niks3-server[1169]: 2026/09/22 11:34:59 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst worker1 # [ 204.697369] nix-grpc-daemon[1925]: time=2026-09-22T11:34:59.292Z level=INFO msg="Uploading 1 narinfos" lb # [ 204.653889] niks3-server[1169]: 2026/09/22 11:34:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign lb # [ 204.663884] niks3-server[1169]: 2026/09/22 11:34:59 INFO Signed narinfos id=78 count=1 lb # [ 204.674493] niks3-server[1169]: 2026/09/22 11:34:59 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete lb # [ 204.681518] niks3-server[1169]: 2026/09/22 11:34:59 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls worker1 # [ 204.732830] nix-grpc-daemon[1925]: time=2026-09-22T11:34:59.328Z level=INFO msg="Upload complete. (345ms)" lb # [ 204.687787] niks3-server[1169]: 2026/09/22 11:34:59 INFO Completed upload id=78 lb # [ 204.693555] niks3-server[1169]: 2026/09/22 11:34:59 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo worker1 # [ 204.740128] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 204.751843] nix-grpc-daemon[1865]: ts=2026-09-22T11:34:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 204.757289] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 204.761802] nix-daemon[852]: reaped child process 2216, status = succeeded worker1 # [ 204.773674] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted) worker1 # [ 205.832284] nix-daemon[852]: reaped child process 2230, status = succeeded worker1 # [ 205.916884] nix-daemon[852]: accepted connection from pid 2246, user nix-grpc-daemon (trusted) worker1 # [ 205.935029] nix-daemon[852]: reaped child process 2249, status = succeeded lb # [ 205.897927] niks3-server[1169]: 2026/09/22 11:35:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 205.952964] nix-grpc-daemon[1925]: time=2026-09-22T11:35:00.548Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 205.956507] nix-grpc-daemon[1925]: time=2026-09-22T11:35:00.550Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)" lb # [ 205.935612] niks3-server[1169]: 2026/09/22 11:35:00 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv lb # [ 205.948338] niks3-server[1169]: 2026/09/22 11:35:00 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst worker1 # [ 206.006462] nix-grpc-daemon[1925]: time=2026-09-22T11:35:00.601Z level=INFO msg="Uploading 1 narinfos" lb # [ 205.963095] niks3-server[1169]: 2026/09/22 11:35:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign lb # [ 205.971809] niks3-server[1169]: 2026/09/22 11:35:00 INFO Signed narinfos id=79 count=1 lb # [ 205.978499] niks3-server[1169]: 2026/09/22 11:35:00 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls lb # [ 205.987328] niks3-server[1169]: 2026/09/22 11:35:00 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete lb # [ 206.002165] niks3-server[1169]: 2026/09/22 11:35:00 INFO Completed upload id=79 lb # [ 206.006961] niks3-server[1169]: 2026/09/22 11:35:00 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo worker1 # [ 206.047345] nix-grpc-daemon[1925]: time=2026-09-22T11:35:00.642Z level=INFO msg="Upload complete. (264ms)" worker1 # [ 206.061353] nix-grpc-daemon[1865]: ts=2026-09-22T11:35:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34528 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=14 outputs=1 worker1 # [ 206.079527] nix-daemon[852]: reaped child process 2228, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.49 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 206.276852] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 2b58976f03c54afba0b3d2e33a83e78e client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2 # [ 206.463329] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:35100 client # [ 206.488973] nix[1512]: don't know how to build these paths: client # [ 206.489977] nix[1512]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1 # [ 206.470530] nix-grpc-daemon[1865]: ts=2026-09-22T11:35:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34528 client # [ 206.519008] nix[1512]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker worker2 # [ 206.499262] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:01Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code lb # [ 208.335506] envoy[893]: conn peer=[2001:db8:1::1]:50956 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27183 tx=12380 ms=18216 client # [ 236.525181] nix[1512]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu} client # [ 236.536891] nix[1512]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed client # [ 236.569243] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 236.577132] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 236.580949] systemd[1]: gpu.service: Consumed 192ms CPU time over 30.298s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.08 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.32 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.18 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' lb # [ 238.339333] envoy[893]: conn peer=[2001:db8:1::1]:42222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=30066 worker1 # [ 238.386042] nix-grpc-daemon[1865]: ts=2026-09-22T11:35:32Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2 client # don't know how to build these paths: client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker1 # [ 239.444589] nix-grpc-daemon[1865]: ts=2026-09-22T11:35:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34528 worker1 # [ 239.452175] nix-grpc-daemon[1865]: ts=2026-09-22T11:35:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34528 worker2 # [ 239.481315] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:34Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=0 worker2 # [ 239.488900] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:34Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 239.533193] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 239.549404] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35100 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'... worker2 # [ 239.565824] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 239.583292] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 239.747997] nix-daemon[842]: accepted connection from pid 1862, user nix-grpc-daemon (trusted) worker2 # [ 239.765791] nix-daemon[842]: reaped child process 1865, status = succeeded worker2 # [ 239.863602] nix-daemon[842]: accepted connection from pid 1867, user nix-grpc-daemon (trusted) worker2 # [ 239.877980] nix-daemon[842]: reaped child process 1870, status = succeeded lb # [ 239.837685] niks3-server[1169]: 2026/09/22 11:35:34 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 239.898351] nix-grpc-daemon[1643]: time=2026-09-22T11:35:34.484Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 239.899827] nix-grpc-daemon[1643]: time=2026-09-22T11:35:34.484Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)" lb # [ 239.880135] niks3-server[1169]: 2026/09/22 11:35:34 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst worker2 # [ 239.942231] nix-grpc-daemon[1643]: time=2026-09-22T11:35:34.529Z level=INFO msg="Uploading 1 narinfos" lb # [ 239.893303] niks3-server[1169]: 2026/09/22 11:35:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign lb # [ 239.903251] niks3-server[1169]: 2026/09/22 11:35:34 INFO Signed narinfos id=80 count=1 lb # [ 239.916398] niks3-server[1169]: 2026/09/22 11:35:34 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls lb # [ 239.931789] niks3-server[1169]: 2026/09/22 11:35:34 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete worker2 # [ 239.987925] nix-grpc-daemon[1643]: time=2026-09-22T11:35:34.574Z level=INFO msg="Upload complete. (387ms)" lb # [ 239.941274] niks3-server[1169]: 2026/09/22 11:35:34 INFO Completed upload id=80 lb # [ 239.946507] niks3-server[1169]: 2026/09/22 11:35:34 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo worker2 # [ 240.001099] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 240.016822] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 240.023889] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 240.029887] nix-daemon[842]: reaped child process 1860, status = succeeded client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051... client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker2 # [ 240.045455] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 240.936808] nix-daemon[842]: reaped child process 1874, status = succeeded worker2 # [ 241.020457] nix-daemon[842]: accepted connection from pid 1890, user nix-grpc-daemon (trusted) worker2 # [ 241.037316] nix-daemon[842]: reaped child process 1893, status = succeeded lb # [ 240.997689] niks3-server[1169]: 2026/09/22 11:35:35 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 241.059087] nix-grpc-daemon[1643]: time=2026-09-22T11:35:35.643Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 241.060885] nix-grpc-daemon[1643]: time=2026-09-22T11:35:35.643Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)" lb # [ 241.036072] niks3-server[1169]: 2026/09/22 11:35:35 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv lb # [ 241.046670] niks3-server[1169]: 2026/09/22 11:35:35 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst worker2 # [ 241.111651] nix-grpc-daemon[1643]: time=2026-09-22T11:35:35.698Z level=INFO msg="Uploading 1 narinfos" lb # [ 241.057569] niks3-server[1169]: 2026/09/22 11:35:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign lb # [ 241.067770] niks3-server[1169]: 2026/09/22 11:35:35 INFO Signed narinfos id=81 count=1 lb # [ 241.073403] niks3-server[1169]: 2026/09/22 11:35:35 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls lb # [ 241.084140] niks3-server[1169]: 2026/09/22 11:35:35 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete lb # [ 241.090573] niks3-server[1169]: 2026/09/22 11:35:35 INFO Completed upload id=81 worker2 # [ 241.139370] nix-grpc-daemon[1643]: time=2026-09-22T11:35:35.726Z level=INFO msg="Upload complete. (280ms)" lb # [ 241.094024] niks3-server[1169]: 2026/09/22 11:35:35 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo worker2 # [ 241.158073] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=15 outputs=1 worker2 # [ 241.176316] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:35Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=0 worker2 # [ 241.183078] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:35Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 241.189160] nix-daemon[842]: reaped child process 1872, status = succeeded worker2 # [ 241.201117] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 241.216275] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 241.230746] nix-grpc-daemon[1643]: time=2026-09-22T11:35:35.817Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'... worker2 # [ 241.241960] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 241.256134] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 241.404053] nix-daemon[842]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker2 # [ 241.425811] nix-daemon[842]: reaped child process 1902, status = succeeded worker2 # [ 241.518177] nix-daemon[842]: accepted connection from pid 1904, user nix-grpc-daemon (trusted) worker2 # [ 241.535647] nix-daemon[842]: reaped child process 1907, status = succeeded lb # [ 241.487774] niks3-server[1169]: 2026/09/22 11:35:36 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 241.546554] nix-grpc-daemon[1643]: time=2026-09-22T11:35:36.133Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 241.548075] nix-grpc-daemon[1643]: time=2026-09-22T11:35:36.133Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)" lb # [ 241.519127] niks3-server[1169]: 2026/09/22 11:35:36 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst lb # [ 241.532269] niks3-server[1169]: 2026/09/22 11:35:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign worker2 # [ 241.586190] nix-grpc-daemon[1643]: time=2026-09-22T11:35:36.173Z level=INFO msg="Uploading 1 narinfos" lb # [ 241.542878] niks3-server[1169]: 2026/09/22 11:35:36 INFO Signed narinfos id=82 count=1 lb # [ 241.547553] niks3-server[1169]: 2026/09/22 11:35:36 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls lb # [ 241.555984] niks3-server[1169]: 2026/09/22 11:35:36 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete lb # [ 241.560867] niks3-server[1169]: 2026/09/22 11:35:36 INFO Completed upload id=82 worker2 # [ 241.609049] nix-grpc-daemon[1643]: time=2026-09-22T11:35:36.196Z level=INFO msg="Upload complete. (338ms)" lb # [ 241.563122] niks3-server[1169]: 2026/09/22 11:35:36 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo worker2 # [ 241.616183] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 paths=1 nar_bytes_in=632 worker2 # [ 241.629684] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 241.639006] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 241.641406] nix-daemon[842]: reaped child process 1897, status = succeeded client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051... client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker2 # [ 241.654722] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 242.708354] nix-daemon[842]: reaped child process 1911, status = succeeded worker2 # [ 242.803302] nix-daemon[842]: accepted connection from pid 1927, user nix-grpc-daemon (trusted) worker2 # [ 242.821692] nix-daemon[842]: reaped child process 1930, status = succeeded lb # [ 242.778786] niks3-server[1169]: 2026/09/22 11:35:37 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 242.840479] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.424Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 242.842282] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.424Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)" lb # [ 242.811116] niks3-server[1169]: 2026/09/22 11:35:37 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv lb # [ 242.830065] niks3-server[1169]: 2026/09/22 11:35:37 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst worker2 # [ 242.898016] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.484Z level=INFO msg="Uploading 1 narinfos" lb # [ 242.844627] niks3-server[1169]: 2026/09/22 11:35:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign lb # [ 242.853760] niks3-server[1169]: 2026/09/22 11:35:37 INFO Signed narinfos id=83 count=1 lb # [ 242.855802] niks3-server[1169]: 2026/09/22 11:35:37 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls lb # [ 242.867520] niks3-server[1169]: 2026/09/22 11:35:37 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete lb # [ 242.872910] niks3-server[1169]: 2026/09/22 11:35:37 INFO Completed upload id=83 worker2 # [ 242.922116] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.509Z level=INFO msg="Upload complete. (258ms)" lb # [ 242.877428] niks3-server[1169]: 2026/09/22 11:35:37 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo worker2 # [ 242.937491] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=16 outputs=1 worker2 # [ 242.959950] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:37Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=0 worker2 # [ 242.965048] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:37Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 242.969987] nix-daemon[842]: reaped child process 1909, status = succeeded worker2 # [ 242.997355] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 243.012322] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 243.026068] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.613Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'... worker2 # [ 243.037312] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 243.051287] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 243.209986] nix-daemon[842]: accepted connection from pid 1936, user nix-grpc-daemon (trusted) worker2 # [ 243.225106] nix-daemon[842]: reaped child process 1939, status = succeeded worker2 # [ 243.326543] nix-daemon[842]: accepted connection from pid 1941, user nix-grpc-daemon (trusted) worker2 # [ 243.341541] nix-daemon[842]: reaped child process 1944, status = succeeded lb # [ 243.298622] niks3-server[1169]: 2026/09/22 11:35:37 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 243.356294] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.943Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 243.357812] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.943Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)" lb # [ 243.328806] niks3-server[1169]: 2026/09/22 11:35:37 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst worker2 # [ 243.389628] nix-grpc-daemon[1643]: time=2026-09-22T11:35:37.976Z level=INFO msg="Uploading 1 narinfos" lb # [ 243.342390] niks3-server[1169]: 2026/09/22 11:35:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign lb # [ 243.353422] niks3-server[1169]: 2026/09/22 11:35:37 INFO Signed narinfos id=84 count=1 lb # [ 243.363410] niks3-server[1169]: 2026/09/22 11:35:37 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls worker2 # [ 243.422419] nix-grpc-daemon[1643]: time=2026-09-22T11:35:38.009Z level=INFO msg="Upload complete. (355ms)" lb # [ 243.374960] niks3-server[1169]: 2026/09/22 11:35:37 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete lb # [ 243.379570] niks3-server[1169]: 2026/09/22 11:35:37 INFO Completed upload id=84 worker2 # [ 243.432254] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 paths=1 nar_bytes_in=760 lb # [ 243.384390] niks3-server[1169]: 2026/09/22 11:35:37 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo worker2 # [ 243.443239] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 worker2 # [ 243.452648] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 243.455614] nix-daemon[842]: reaped child process 1934, status = succeeded client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051... worker2 # [ 243.467085] nix-daemon[842]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 244.100510] nix-daemon[842]: reaped child process 1948, status = succeeded worker2 # [ 244.199302] nix-daemon[842]: accepted connection from pid 1964, user nix-grpc-daemon (trusted) worker2 # [ 244.216299] nix-daemon[842]: reaped child process 1967, status = succeeded lb # [ 244.174305] niks3-server[1169]: 2026/09/22 11:35:38 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 244.232543] nix-grpc-daemon[1643]: time=2026-09-22T11:35:38.819Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 244.235074] nix-grpc-daemon[1643]: time=2026-09-22T11:35:38.821Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)" lb # [ 244.212055] niks3-server[1169]: 2026/09/22 11:35:38 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv lb # [ 244.222845] niks3-server[1169]: 2026/09/22 11:35:38 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst worker2 # [ 244.285974] nix-grpc-daemon[1643]: time=2026-09-22T11:35:38.872Z level=INFO msg="Uploading 1 narinfos" lb # [ 244.237204] niks3-server[1169]: 2026/09/22 11:35:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign lb # [ 244.245824] niks3-server[1169]: 2026/09/22 11:35:38 INFO Signed narinfos id=85 count=1 lb # [ 244.250665] niks3-server[1169]: 2026/09/22 11:35:38 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls lb # [ 244.258483] niks3-server[1169]: 2026/09/22 11:35:38 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete lb # [ 244.263711] niks3-server[1169]: 2026/09/22 11:35:38 INFO Completed upload id=85 worker2 # [ 244.312412] nix-grpc-daemon[1643]: time=2026-09-22T11:35:38.899Z level=INFO msg="Upload complete. (234ms)" lb # [ 244.266101] niks3-server[1169]: 2026/09/22 11:35:38 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo worker2 # [ 244.325140] nix-grpc-daemon[1569]: ts=2026-09-22T11:35:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35100 duration_s=0 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=17 outputs=1 worker2 # [ 244.343508] nix-daemon[842]: reaped child process 1946, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.14 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.19 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 245.396449] nix-grpc-daemon[1865]: ts=2026-09-22T11:35:39Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 8.94 seconds) (finished: run the VM test script, in 246.76 seconds) test script finished in 247.00s cleanup kill QemuMachine (pid 49) client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) client # [2026-09-22T11:35:40Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-22T11:35:40Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-22T11:35:40Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 50) lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) lb # [2026-09-22T11:35:40Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-22T11:35:40Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-22T11:35:40Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker1 # [2026-09-22T11:35:41Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-22T11:35:41Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-22T11:35:41Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker2 # [2026-09-22T11:35:41Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-22T11:35:41Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-22T11:35:41Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.24 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh