tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm worker1 # Disk image does not exist, creating the virtualisation disk image... worker1: QEMU running (pid 51) worker1 # Formatting '/build/vm-state-worker1/tmp.AyKH72Telw', fmt=raw size=1073741824 lb: QEMU running (pid 50) worker1 # mke2fs 1.47.4 (6-Mar-2025) client # Disk image does not exist, creating the virtualisation disk image... client: QEMU running (pid 49) worker2 # Disk image does not exist, creating the virtualisation disk image... worker1 # Discarding device blocks: 0/262144 done client # Formatting '/build/vm-state-client/tmp.Jzz2aL4Ldi', fmt=raw size=1073741824 lb # Disk image does not exist, creating the virtualisation disk image... worker2: QEMU running (pid 52) worker2 # Formatting '/build/vm-state-worker2/tmp.ky45lD3lRa', fmt=raw size=1073741824 lb # Formatting '/build/vm-state-lb/tmp.X7xj1ScF2V', fmt=raw size=1073741824 worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes client # mke2fs 1.47.4 (6-Mar-2025) (finished: start all VMs, in 0.49 seconds) worker1 # Filesystem UUID: 33133b99-1616-4b97-8015-09ee09300e84 client # Discarding device blocks: 0/262144 done client: waiting for unit network-addresses-eth1.service worker1 # Superblock backups stored on blocks: client # Creating filesystem with 262144 4k blocks and 65536 inodes client: waiting for the VM to finish booting worker1 # 32768, 98304, 163840, 229376 client # Filesystem UUID: 3e628041-8142-4b90-ac7f-abf3283ad4a7 lb # mke2fs 1.47.4 (6-Mar-2025) worker2 # mke2fs 1.47.4 (6-Mar-2025) worker1 # client # Superblock backups stored on blocks: lb # Discarding device blocks: 0/262144 done worker2 # Discarding device blocks: 0/262144 done worker1 # Allocating group tables: 0/8 done client # 32768, 98304, 163840, 229376 lb # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # Writing inode tables: 0/8 done client # lb # Filesystem UUID: 753fbc79-914d-4c7c-9ce8-f4b90e9bdb0a worker2 # Filesystem UUID: afc2ae95-50d3-4000-a010-8c2d52d66043 worker1 # Creating journal (8192 blocks): done client # Allocating group tables: 0/8 done lb # Superblock backups stored on blocks: worker2 # Superblock backups stored on blocks: worker1 # Writing superblocks and filesystem accounting information: 0/8 done client # Writing inode tables: 0/8 done lb # 32768, 98304, 163840, 229376 worker2 # 32768, 98304, 163840, 229376 worker1 # client # Creating journal (8192 blocks): done lb # worker2 # worker1 # Virtualisation disk image created. client # Writing superblocks and filesystem accounting information: 0/8 done lb # Allocating group tables: 0/8 done worker2 # Allocating group tables: 0/8 done worker1 # Starting virtiofs daemons... client # lb # Writing inode tables: 0/8 done worker2 # Writing inode tables: 0/8 done worker1 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # Virtualisation disk image created. lb # Creating journal (8192 blocks): done worker2 # Creating journal (8192 blocks): done worker1 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # Starting virtiofs daemons... lb # Writing superblocks and filesystem accounting information: 0/8 done worker2 # Writing superblocks and filesystem accounting information: 0/8 done worker1 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # worker2 # worker1 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # Virtualisation disk image created. client # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # Virtualisation disk image created. worker1 # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # Starting virtiofs daemons... client # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # Starting virtiofs daemons... worker1 # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-22T15:45:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-22T15:45:11Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-22T15:45:11Z INFO virtiofsd] Client connected, servicing requests worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] KASLR enabled worker1 # [ 0.000000] random: crng init done worker1 # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] efi: UEFI not found. worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker1 # [ 0.000000] Zone ranges: worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] DMA32 empty worker1 # [ 0.000000] Normal empty worker1 # [ 0.000000] Device empty worker1 # [ 0.000000] Movable zone start for each node worker1 # [ 0.000000] Early memory node ranges worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker1 # [ 0.000000] psci: probing for conduit method from DT. worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.000000] psci: Trusted OS migration not required worker1 # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker1 # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] KASLR enabled worker1 # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] random: crng init done worker1 # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] efi: UEFI not found. worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker1 # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] Zone ranges: client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] DMA32 empty client # [ 0.000000] Normal empty client # [ 0.000000] Device empty client # [ 0.000000] Movable zone start for each node worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nkxfr43rzbz80xc8q1a00w66rjj2an3s-nixos-system-worker1-test/init regInfo=/nix/store/s2vfldvxw21afqph8awka7bg9gka3b50-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.000000] Early memory node ranges worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/s2vfldvxw21afqph8awka7bg9gka3b50-closure-info/registration", will be passed to user space. client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker2 # [ 0.000000] KASLR enabled lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] psci: probing for conduit method from DT. worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] random: crng init done client # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker2 # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker2 # [ 0.000000] efi: UEFI not found. lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.000000] KASLR enabled worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT lb # [ 0.000000] random: crng init done client # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] efi: UEFI not found. client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker2 # [ 0.000000] Zone ranges: client # [ 0.000000] Detected PIPT I-cache on CPU0 worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] DMA32 empty worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.000000] Normal empty worker1 # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] Device empty lb # [ 0.000000] Zone ranges: worker2 # [ 0.000000] Movable zone start for each node worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] Early memory node ranges worker1 # [ 0.000000] Policy zone: DMA lb # [ 0.000000] DMA32 empty client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.000000] Normal empty worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Device empty lb # [ 0.000000] Movable zone start for each node worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Early memory node ranges worker1 # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] CPU features: detected: GICv3 CPU interface lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] CPU features: detected: Spectre-v4 worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker1 # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker2 # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. lb # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] alternatives: applying boot alternatives lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled. worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker2 # [ 0.000000] Detected PIPT I-cache on CPU0 worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface lb # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/c7dwh1z1cl7zl3zmm2gm23k4b20rxnfn-nixos-system-client-test/init regInfo=/nix/store/bvnhgdmv5qvfh2jgky9d95hczimkx039-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] CPU features: detected: Spectre-v4 worker2 # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 lb # [ 0.000000] CPU features: detected: GICv3 CPU interface lb # [ 0.000000] CPU features: detected: Spectre-v4 worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/bvnhgdmv5qvfh2jgky9d95hczimkx039-closure-info/registration", will be passed to user space. worker2 # [ 0.000000] alternatives: applying boot alternatives worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker1 # [ 0.000000] GICv3: 256 SPIs implemented client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/91x7j7xpp6q86mih2akg81hmd36hcvqs-nixos-system-worker2-test/init regInfo=/nix/store/i3k5wlp1fqfgi1004a4pn264iky91pbm-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.000000] software IO TLB: area num 1. worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rwj7rgrjnik4yfrdy5nxpi16ig7dj0yx-nixos-system-lb-test/init regInfo=/nix/store/y0b8gyps1r0yyzfhbdbfwqvj69pgkv2z-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/i3k5wlp1fqfgi1004a4pn264iky91pbm-closure-info/registration", will be passed to user space. client # [ 0.000000] Fallback order for Node 0: 0 worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes client # [ 0.000000] Policy zone: DMA worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/y0b8gyps1r0yyzfhbdbfwqvj69pgkv2z-closure-info/registration", will be passed to user space. worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] software IO TLB: area num 1. worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). client # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB lb # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns client # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.000000] Policy zone: DMA lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker1 # [ 0.000034] arm-pv: using stolen time PV client # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.000000] Tracing variant of Tasks RCU enabled. lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000658] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.000000] Policy zone: DMA client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.000822] Console: colour dummy device 80x25 lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker2 # [ 0.000000] allocated 3145728 bytes of page_ext client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.000830] printk: legacy console [tty0] enabled worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups lb # [ 0.000000] allocated 3145728 bytes of page_ext worker1 # [ 0.001030] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker2 # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker1 # [ 0.001036] pid_max: default: 32768 minimum: 301 client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] rcu: RCU event tracing is enabled. lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.001110] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. lb # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.001292] landlock: Up and running. client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.001295] Yama: becoming mindful. lb # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.001982] LSM support for eBPF active worker2 # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.000000] GICv3: 256 SPIs implemented worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.002153] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.000000] Root IRQ handler: gic_handle_irq worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.002185] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000000] Tracing variant of Tasks RCU enabled. client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.004102] rcu: Hierarchical SRCU implementation. worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker1 # [ 0.004107] rcu: Max phase no-delay instances is 1000. lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.005376] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker1 # [ 0.005464] EFI services will not be available. worker1 # [ 0.005532] smp: Bringing up secondary CPUs ... worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker1 # [ 0.005540] smp: Brought up 1 node, 1 CPU worker1 # [ 0.005543] SMP: Total of 1 processors activated. lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.005548] CPU: All CPU(s) started at EL1 client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.005558] CPU features: detected: Branch Target Identification client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker2 # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented worker1 # [ 0.005562] CPU features: detected: ARMv8.4 Translation Table Level client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 lb # [ 0.000000] GICv3: 256 SPIs implemented worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.000000] GICv3: 0 Extended SPIs implemented worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.005565] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.000000] Root IRQ handler: gic_handle_irq worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.005569] CPU features: detected: Data cache clean to the PoU not required for I/D coherence client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker1 # [ 0.005573] CPU features: detected: Common not Private translations client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.005576] CPU features: detected: CRC32 instructions lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker1 # [ 0.005579] CPU features: detected: Data cache clean to Point of Deep Persistence lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker1 # [ 0.005582] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) client # [ 0.000033] arm-pv: using stolen time PV worker1 # [ 0.005586] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker1 # [ 0.005589] CPU features: detected: E0PD worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 client # [ 0.000537] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker1 # [ 0.005591] CPU features: detected: Enhanced Counter Virtualization client # [ 0.000943] Console: colour dummy device 80x25 worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 client # [ 0.000950] printk: legacy console [tty0] enabled worker1 # [ 0.005594] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.005598] CPU features: detected: Enhanced Virtualization Traps worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns client # [ 0.001142] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.005601] CPU features: detected: Fine Grained Traps worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). client # [ 0.001148] pid_max: default: 32768 minimum: 301 lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.005604] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.001224] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.001358] landlock: Up and running. worker1 # [ 0.005609] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.001360] Yama: becoming mindful. lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker1 # [ 0.005611] CPU features: detected: LSE atomic instructions worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns client # [ 0.001833] LSM support for eBPF active worker2 # [ 0.000029] arm-pv: using stolen time PV worker1 # [ 0.005614] CPU features: detected: Privileged Access Never client # [ 0.001980] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.005617] CPU features: detected: PMUv3 lb # [ 0.000031] arm-pv: using stolen time PV worker1 # [ 0.005620] CPU features: detected: RAS Extension Support worker2 # [ 0.000388] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.002013] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.000556] Console: colour dummy device 80x25 client # [ 0.003836] rcu: Hierarchical SRCU implementation. worker1 # [ 0.005622] CPU features: detected: RASv1p1 Extension Support lb # [ 0.000405] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.000563] printk: legacy console [tty0] enabled client # [ 0.003841] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.005625] CPU features: detected: Random Number Generator lb # [ 0.000574] Console: colour dummy device 80x25 client # [ 0.005104] fsl-mc MSI: its@8080000 domain created lb # [ 0.000581] printk: legacy console [tty0] enabled worker1 # [ 0.005627] CPU features: detected: Speculation barrier (SB) client # [ 0.005194] EFI services will not be available. worker2 # [ 0.000775] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) client # [ 0.005274] smp: Bringing up secondary CPUs ... worker2 # [ 0.000782] pid_max: default: 32768 minimum: 301 worker1 # [ 0.005630] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.000775] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) client # [ 0.005283] smp: Brought up 1 node, 1 CPU worker2 # [ 0.000864] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker1 # [ 0.005633] CPU features: detected: TLB range maintenance instructions lb # [ 0.000782] pid_max: default: 32768 minimum: 301 client # [ 0.005286] SMP: Total of 1 processors activated. worker2 # [ 0.000992] landlock: Up and running. client # [ 0.005291] CPU: All CPU(s) started at EL1 worker1 # [ 0.005637] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker2 # [ 0.000996] Yama: becoming mindful. lb # [ 0.000863] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.000998] landlock: Up and running. worker2 # [ 0.001422] LSM support for eBPF active client # [ 0.005302] CPU features: detected: Branch Target Identification worker1 # [ 0.005677] alternatives: applying system-wide alternatives lb # [ 0.001000] Yama: becoming mindful. lb # [ 0.001434] LSM support for eBPF active client # [ 0.005307] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.001542] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.008657] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.001565] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.001576] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.005310] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.003585] rcu: Hierarchical SRCU implementation. lb # [ 0.001598] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.008847] Memory: 1405616K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved) client # [ 0.005314] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.003590] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.009252] devtmpfs: initialized lb # [ 0.003450] rcu: Hierarchical SRCU implementation. worker2 # [ 0.004860] fsl-mc MSI: its@8080000 domain created client # [ 0.005318] CPU features: detected: Common not Private translations lb # [ 0.003455] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.010992] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.004952] EFI services will not be available. client # [ 0.005321] CPU features: detected: CRC32 instructions lb # [ 0.004709] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.005084] smp: Bringing up secondary CPUs ... worker1 # [ 0.011014] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.004805] EFI services will not be available. worker2 # [ 0.005093] smp: Brought up 1 node, 1 CPU client # [ 0.005324] CPU features: detected: Data cache clean to Point of Deep Persistence lb # [ 0.004879] smp: Bringing up secondary CPUs ... worker2 # [ 0.005096] SMP: Total of 1 processors activated. worker1 # [ 0.011185] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.005327] CPU features: detected: Data cache clean to Point of Persistence lb # [ 0.004888] smp: Brought up 1 node, 1 CPU worker2 # [ 0.005102] CPU: All CPU(s) started at EL1 lb # [ 0.004891] SMP: Total of 1 processors activated. client # [ 0.005330] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.005113] CPU features: detected: Branch Target Identification lb # [ 0.004896] CPU: All CPU(s) started at EL1 client # [ 0.005333] CPU features: detected: E0PD worker2 # [ 0.005118] CPU features: detected: ARMv8.4 Translation Table Level lb # [ 0.004907] CPU features: detected: Branch Target Identification client # [ 0.005336] CPU features: detected: Enhanced Counter Virtualization lb # [ 0.004911] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.005121] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.005339] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.005343] CPU features: detected: Enhanced Virtualization Traps lb # [ 0.004914] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.005125] CPU features: detected: Data cache clean to the PoU not required for I/D coherence client # [ 0.005345] CPU features: detected: Fine Grained Traps worker2 # [ 0.005129] CPU features: detected: Common not Private translations lb # [ 0.004918] CPU features: detected: Data cache clean to the PoU not required for I/D coherence client # [ 0.005349] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.005132] CPU features: detected: CRC32 instructions lb # [ 0.004922] CPU features: detected: Common not Private translations client # [ 0.005354] CPU features: detected: RCpc load-acquire (LDAPR) lb # [ 0.004925] CPU features: detected: CRC32 instructions worker2 # [ 0.005135] CPU features: detected: Data cache clean to Point of Deep Persistence client # [ 0.005357] CPU features: detected: LSE atomic instructions lb # [ 0.004928] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.005138] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.005360] CPU features: detected: Privileged Access Never client # [ 0.005362] CPU features: detected: PMUv3 lb # [ 0.004932] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.005141] CPU features: detected: Data independent timing control (DIT) client # [ 0.005365] CPU features: detected: RAS Extension Support worker2 # [ 0.005144] CPU features: detected: E0PD worker1 # [ 0.011189] 0 pages in range for non-PLT usage lb # [ 0.004935] CPU features: detected: Data independent timing control (DIT) client # [ 0.005368] CPU features: detected: RASv1p1 Extension Support lb # [ 0.004938] CPU features: detected: E0PD worker1 # [ 0.011190] 508272 pages in range for PLT usage worker2 # [ 0.005147] CPU features: detected: Enhanced Counter Virtualization client # [ 0.005371] CPU features: detected: Random Number Generator worker1 # [ 0.011284] pinctrl core: initialized pinctrl subsystem lb # [ 0.004940] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.005150] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker1 # [ 0.012058] DMI not present or invalid. client # [ 0.005373] CPU features: detected: Speculation barrier (SB) lb # [ 0.004944] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker2 # [ 0.005153] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.015395] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.005376] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.004947] CPU features: detected: Enhanced Virtualization Traps worker2 # [ 0.005156] CPU features: detected: Fine Grained Traps worker1 # [ 0.017852] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.005379] CPU features: detected: TLB range maintenance instructions lb # [ 0.004950] CPU features: detected: Fine Grained Traps worker2 # [ 0.005160] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.005384] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.018042] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.004954] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.005165] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.005420] alternatives: applying system-wide alternatives worker1 # [ 0.018249] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.004959] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.005168] CPU features: detected: LSE atomic instructions client # [ 0.008387] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.004962] CPU features: detected: LSE atomic instructions worker1 # [ 0.018272] audit: initializing netlink subsys (disabled) worker2 # [ 0.005171] CPU features: detected: Privileged Access Never lb # [ 0.004965] CPU features: detected: Privileged Access Never worker1 # [ 0.018807] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.005174] CPU features: detected: PMUv3 lb # [ 0.004968] CPU features: detected: PMUv3 worker2 # [ 0.005176] CPU features: detected: RAS Extension Support worker1 # [ 0.018809] thermal_sys: Registered thermal governor 'bang_bang' client # [ 0.008578] Memory: 1405632K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved) lb # [ 0.004971] CPU features: detected: RAS Extension Support client # [ 0.009023] devtmpfs: initialized worker2 # [ 0.005179] CPU features: detected: RASv1p1 Extension Support worker1 # [ 0.018813] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.004973] CPU features: detected: RASv1p1 Extension Support client # [ 0.010850] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.005182] CPU features: detected: Random Number Generator worker1 # [ 0.018816] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.004976] CPU features: detected: Random Number Generator worker2 # [ 0.005184] CPU features: detected: Speculation barrier (SB) worker1 # [ 0.018821] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.010873] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.004979] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.005187] CPU features: detected: Stage-2 Force Write-Back client # [ 0.011073] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker1 # [ 0.018847] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.004981] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.005190] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.018855] cpuidle: using governor ladder client # [ 0.011077] 0 pages in range for non-PLT usage lb # [ 0.004984] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.018861] cpuidle: using governor menu client # [ 0.011079] 508272 pages in range for PLT usage worker2 # [ 0.005194] CPU features: detected: Speculative Store Bypassing Safe (SSBS) lb # [ 0.004989] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.019054] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. client # [ 0.011178] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.005232] alternatives: applying system-wide alternatives client # [ 0.011995] DMI not present or invalid. lb # [ 0.005026] alternatives: applying system-wide alternatives worker1 # [ 0.019071] ASID allocator initialised with 65536 entries worker2 # [ 0.008191] CPU features: detected: BBM Level 2 without TLB conflict abort client # [ 0.015089] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.020307] Serial: AMBA PL011 UART driver lb # [ 0.008067] CPU features: detected: BBM Level 2 without TLB conflict abort client # [ 0.017308] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.025443] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.008372] Memory: 1405688K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved) worker1 # [ 0.025595] printk: console [ttyAMA0] enabled client # [ 0.017499] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.008794] devtmpfs: initialized lb # [ 0.008316] Memory: 1405644K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved) lb # [ 0.008681] devtmpfs: initialized worker2 # [ 0.010606] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.017696] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker1 # [ 0.146502] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.010491] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.017718] audit: initializing netlink subsys (disabled) worker1 # [ 0.146522] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.010628] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.010513] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.018263] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.146527] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.018265] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.010693] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker1 # [ 0.146531] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.018268] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.146536] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.018271] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.146540] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.146544] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.146549] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker1 # [ 0.154052] fbcon: Taking over console worker1 # [ 0.154066] ACPI: Interpreter disabled. worker1 # [ 0.155928] iommu: Default domain type: Translated worker2 # [ 0.010802] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker1 # [ 0.155938] iommu: DMA domain TLB invalidation policy: strict mode worker2 # [ 0.010807] 0 pages in range for non-PLT usage lb # [ 0.010697] 0 pages in range for non-PLT usage worker1 # [ 0.157660] SCSI subsystem initialized worker2 # [ 0.010808] 508272 pages in range for PLT usage lb # [ 0.010699] 508272 pages in range for PLT usage worker2 # [ 0.010920] pinctrl core: initialized pinctrl subsystem lb # [ 0.010792] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.011658] DMI not present or invalid. lb # [ 0.011567] DMI not present or invalid. client # [ 0.018276] thermal_sys: Registered thermal governor 'power_allocator' worker2 # [ 0.014721] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.014812] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.018301] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.016986] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.018310] cpuidle: using governor ladder lb # [ 0.017002] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.018316] cpuidle: using governor menu worker2 # [ 0.017167] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.017185] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.018492] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.017366] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.017383] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.018507] ASID allocator initialised with 65536 entries worker2 # [ 0.017389] audit: initializing netlink subsys (disabled) client # [ 0.019702] Serial: AMBA PL011 UART driver lb # [ 0.017405] audit: initializing netlink subsys (disabled) worker2 # [ 0.017975] thermal_sys: Registered thermal governor 'fair_share' lb # [ 0.017948] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.024902] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.017977] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.017950] thermal_sys: Registered thermal governor 'bang_bang' client # [ 0.025061] printk: console [ttyAMA0] enabled worker2 # [ 0.017981] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.017954] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.017984] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.017956] thermal_sys: Registered thermal governor 'user_space' client # [ 0.148049] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.165465] usbcore: registered new interface driver usbfs worker2 # [ 0.017989] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.148069] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.017963] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.165490] usbcore: registered new interface driver hub worker2 # [ 0.018015] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.165514] usbcore: registered new device driver usb client # [ 0.148074] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages lb # [ 0.017988] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.018023] cpuidle: using governor ladder worker1 # [ 0.165781] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.017997] cpuidle: using governor ladder client # [ 0.148078] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker2 # [ 0.018028] cpuidle: using governor menu lb # [ 0.018003] cpuidle: using governor menu worker1 # [ 0.165787] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti client # [ 0.148082] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.018209] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.018184] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.165797] PTP clock support registered client # [ 0.148087] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.018225] ASID allocator initialised with 65536 entries worker1 # [ 0.165841] EDAC MC: Ver: 3.0.0 lb # [ 0.018199] ASID allocator initialised with 65536 entries worker2 # [ 0.019383] Serial: AMBA PL011 UART driver lb # [ 0.019419] Serial: AMBA PL011 UART driver client # [ 0.148091] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.170329] scmi_core: SCMI protocol bus registered worker2 # [ 0.024646] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 client # [ 0.148096] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.024684] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.024773] printk: console [ttyAMA0] enabled worker1 # [ 0.171267] FPGA manager framework lb # [ 0.024838] printk: console [ttyAMA0] enabled worker1 # [ 0.172266] vgaarb: loaded worker2 # [ 0.145448] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.143799] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.172899] clocksource: Switched to clocksource arch_sys_counter client # [ 0.155529] fbcon: Taking over console worker2 # [ 0.145473] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.143819] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.155544] ACPI: Interpreter disabled. worker1 # [ 0.174260] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.145479] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.157401] iommu: Default domain type: Translated lb # [ 0.143824] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.174301] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.145483] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.143828] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.157411] iommu: DMA domain TLB invalidation policy: strict mode worker1 # [ 0.175862] netfs: FS-Cache loaded worker2 # [ 0.145488] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker1 # [ 0.175983] pnp: PnP ACPI: disabled lb # [ 0.143833] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.159096] SCSI subsystem initialized worker2 # [ 0.145493] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.143837] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.145497] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.143842] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker2 # [ 0.145501] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.143846] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker2 # [ 0.152979] fbcon: Taking over console lb # [ 0.151173] fbcon: Taking over console worker2 # [ 0.152997] ACPI: Interpreter disabled. lb # [ 0.151186] ACPI: Interpreter disabled. worker2 # [ 0.154920] iommu: Default domain type: Translated lb # [ 0.153048] iommu: Default domain type: Translated worker2 # [ 0.154932] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.153057] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.154782] SCSI subsystem initialized worker2 # [ 0.156772] SCSI subsystem initialized client # [ 0.166936] usbcore: registered new interface driver usbfs worker1 # [ 0.181820] NET: Registered PF_INET protocol family client # [ 0.166960] usbcore: registered new interface driver hub worker1 # [ 0.182090] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.166985] usbcore: registered new device driver usb client # [ 0.167305] pps_core: LinuxPPS API ver. 1 registered client # [ 0.167311] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti client # [ 0.167321] PTP clock support registered client # [ 0.167368] EDAC MC: Ver: 3.0.0 client # [ 0.172080] scmi_core: SCMI protocol bus registered client # [ 0.173046] FPGA manager framework client # [ 0.173969] vgaarb: loaded worker2 # [ 0.163846] usbcore: registered new interface driver usbfs worker2 # [ 0.163882] usbcore: registered new interface driver hub lb # [ 0.162616] usbcore: registered new interface driver usbfs client # [ 0.174639] clocksource: Switched to clocksource arch_sys_counter worker2 # [ 0.163899] usbcore: registered new device driver usb lb # [ 0.162642] usbcore: registered new interface driver hub worker2 # [ 0.164200] pps_core: LinuxPPS API ver. 1 registered client # [ 0.176083] VFS: Disk quotas dquot_6.6.0 lb # [ 0.162666] usbcore: registered new device driver usb lb # [ 0.162952] pps_core: LinuxPPS API ver. 1 registered client # [ 0.176124] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.164207] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.164218] PTP clock support registered client # [ 0.177661] netfs: FS-Cache loaded lb # [ 0.162958] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.164271] EDAC MC: Ver: 3.0.0 client # [ 0.177760] pnp: PnP ACPI: disabled lb # [ 0.162969] PTP clock support registered lb # [ 0.163009] EDAC MC: Ver: 3.0.0 worker2 # [ 0.168903] scmi_core: SCMI protocol bus registered lb # [ 0.167581] scmi_core: SCMI protocol bus registered worker2 # [ 0.169882] FPGA manager framework worker2 # [ 0.170869] vgaarb: loaded lb # [ 0.168544] FPGA manager framework lb # [ 0.169531] vgaarb: loaded worker2 # [ 0.171517] clocksource: Switched to clocksource arch_sys_counter lb # [ 0.170165] clocksource: Switched to clocksource arch_sys_counter lb # [ 0.171542] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.173829] VFS: Disk quotas dquot_6.6.0 lb # [ 0.171573] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.173864] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.173126] netfs: FS-Cache loaded lb # [ 0.173240] pnp: PnP ACPI: disabled client # [ 0.183825] NET: Registered PF_INET protocol family worker2 # [ 0.177604] netfs: FS-Cache loaded client # [ 0.184087] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.177738] pnp: PnP ACPI: disabled worker2 # [ 0.181799] NET: Registered PF_INET protocol family lb # [ 0.179107] NET: Registered PF_INET protocol family worker2 # [ 0.182064] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.179376] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.211559] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.211603] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.211632] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.211703] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.211845] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.211927] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.211962] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.211998] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.212087] NET: Registered PF_UNIX/PF_LOCAL protocol family worker1 # [ 0.212107] NET: Registered PF_XDP protocol family worker1 # [ 0.212125] PCI: CLS 0 bytes, default 64 worker1 # [ 0.212361] Trying to unpack rootfs image as initramfs... worker1 # [ 0.230441] kvm [1]: HYP mode not available client # [ 0.213660] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.213704] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.213733] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.213806] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.213950] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.214038] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.214074] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.214126] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.211982] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.214221] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.209444] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.214242] NET: Registered PF_XDP protocol family worker2 # [ 0.212034] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.214261] PCI: CLS 0 bytes, default 64 lb # [ 0.209492] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.214516] Trying to unpack rootfs image as initramfs... worker2 # [ 0.212071] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.209520] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.212151] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.209593] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.212299] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.209736] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.212414] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.209823] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.212467] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.209861] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.212505] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.209896] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.212604] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.212626] NET: Registered PF_XDP protocol family lb # [ 0.210007] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.212645] PCI: CLS 0 bytes, default 64 lb # [ 0.210027] NET: Registered PF_XDP protocol family lb # [ 0.210045] PCI: CLS 0 bytes, default 64 worker2 # [ 0.212890] Trying to unpack rootfs image as initramfs... client # [ 0.232181] kvm [1]: HYP mode not available lb # [ 0.210249] Trying to unpack rootfs image as initramfs... worker2 # [ 0.229188] kvm [1]: HYP mode not available lb # [ 0.227969] kvm [1]: HYP mode not available worker1 # [ 0.325473] Initialise system trusted keyrings worker1 # [ 0.326231] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.327564] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker1 # [ 0.328350] 9p: Installing v9fs 9p2000 file system support client # [ 0.327288] Initialise system trusted keyrings client # [ 0.328021] workingset: timestamp_bits=42 max_order=19 bucket_order=0 client # [ 0.329281] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.330066] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.326954] Initialise system trusted keyrings worker2 # [ 0.327743] workingset: timestamp_bits=42 max_order=19 bucket_order=0 lb # [ 0.326644] Initialise system trusted keyrings worker2 # [ 0.329046] squashfs: version 4.0 (2009/01/31) Phillip Lougher lb # [ 0.327405] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.357105] Key type asymmetric registered lb # [ 0.328652] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker1 # [ 0.357133] Asymmetric key parser 'x509' registered worker1 # [ 0.357215] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) lb # [ 0.329421] 9p: Installing v9fs 9p2000 file system support worker1 # [ 0.359329] io scheduler mq-deadline registered worker1 # [ 0.359340] io scheduler kyber registered worker1 # [ 0.369057] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker2 # [ 0.335587] 9p: Installing v9fs 9p2000 file system support worker1 # [ 0.370455] ledtrig-cpu: registered to indicate activity on CPUs worker1 # [ 0.370834] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker1 # [ 0.370850] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker1 # [ 0.370864] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.358848] Key type asymmetric registered client # [ 0.358875] Asymmetric key parser 'x509' registered worker1 # [ 0.370872] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.358953] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker1 # [ 0.370894] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.361134] io scheduler mq-deadline registered client # [ 0.361146] io scheduler kyber registered lb # [ 0.350103] Key type asymmetric registered worker1 # [ 0.370925] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] lb # [ 0.350130] Asymmetric key parser 'x509' registered worker1 # [ 0.370999] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker1 # [ 0.371008] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.371014] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.371019] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.355640] Key type asymmetric registered worker1 # [ 0.371024] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.355661] Asymmetric key parser 'x509' registered worker2 # [ 0.355746] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker1 # [ 0.371077] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.357929] io scheduler mq-deadline registered worker1 # [ 0.371532] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.357940] io scheduler kyber registered worker1 # [ 0.371716] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.371733] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.371763] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.358254] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker1 # [ 0.371779] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.359327] io scheduler mq-deadline registered lb # [ 0.359337] io scheduler kyber registered client # [ 0.370822] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker1 # [ 0.372232] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.372413] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.372429] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.372458] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.372226] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.372645] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker1 # [ 0.391918] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.392105] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.372662] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker1 # [ 0.392122] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.372674] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker1 # [ 0.392152] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.372683] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker1 # [ 0.392621] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.392805] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.372704] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker1 # [ 0.392821] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.372730] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker1 # [ 0.392851] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.372807] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker1 # [ 0.392871] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.372816] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.370314] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.372823] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.372828] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.372833] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.372931] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint lb # [ 0.371752] ledtrig-cpu: registered to indicate activity on CPUs lb # [ 0.372147] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.373387] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.373576] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.372164] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.373593] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.372177] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.373624] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.371709] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.373641] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.372395] ledtrig-cpu: registered to indicate activity on CPUs lb # [ 0.372186] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.372800] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.374119] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.372206] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.374306] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.372818] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker1 # [ 0.405425] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint lb # [ 0.372232] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.374322] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.405633] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.372311] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker2 # [ 0.372830] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.374352] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.405664] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.372320] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.372839] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 lb # [ 0.372326] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.406129] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.393884] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker2 # [ 0.372861] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits lb # [ 0.372332] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker1 # [ 0.406318] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.394075] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker1 # [ 0.406348] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.372887] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] lb # [ 0.372337] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.394096] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.372973] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 lb # [ 0.372397] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.406742] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.394126] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.372984] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.406924] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.372833] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.394597] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.372990] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] lb # [ 0.373020] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.407215] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.372995] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] lb # [ 0.373036] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.407416] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.373000] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker1 # [ 0.407446] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.373066] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.373083] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.373083] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.407905] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.373535] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.408094] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.373533] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.373716] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.373724] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.408124] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.373732] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.373740] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.408576] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.373771] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.373761] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.408764] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.373787] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.408794] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.374260] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.374447] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.392793] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.392980] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker2 # [ 0.374463] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.392996] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.374492] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.423391] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.393027] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.423678] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.374947] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.423696] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.375134] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] lb # [ 0.393488] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.393674] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.375149] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.423726] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.393690] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.402851] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.375180] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.424188] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.402883] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.393719] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.424371] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.375663] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.393737] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.402914] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.375850] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.424388] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.402933] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.375866] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.424418] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.403432] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.375896] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.403626] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.375915] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.403656] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.376389] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.376588] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.404119] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.404309] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.376618] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.404340] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.377124] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.377318] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.404731] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.377348] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.404915] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker2 # [ 0.377744] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.405204] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.377930] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] client # [ 0.405398] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.405428] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.378178] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.378369] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.405896] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.437111] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.378399] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.406086] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.437135] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.406129] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.378856] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.379045] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.437141] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned client # [ 0.406589] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.406261] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.406798] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.379075] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.437194] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.406487] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.406829] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.437242] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.406520] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.407294] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.437292] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned lb # [ 0.407037] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.407594] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.407229] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.407614] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.437341] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned lb # [ 0.407259] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.407644] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.437389] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.407652] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.408132] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.437437] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned client # [ 0.408317] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.407837] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker1 # [ 0.437490] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.408334] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.408093] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.437539] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned client # [ 0.408364] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.408283] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.408955] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker1 # [ 0.437587] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.408313] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.408967] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.437704] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned lb # [ 0.408765] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.408972] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.437753] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned lb # [ 0.408952] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.437775] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned client # [ 0.409018] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.408982] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.437798] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned client # [ 0.409066] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.409436] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.437820] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned lb # [ 0.409624] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.409114] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.437842] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.409654] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.409161] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.423607] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.437864] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned lb # [ 0.410119] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.423828] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.409208] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker1 # [ 0.437887] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.410441] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.423860] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.409255] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.410459] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.437914] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.424405] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.437937] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.410489] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.409302] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.424706] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.437959] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.410965] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.409349] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.424724] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.437982] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned lb # [ 0.411154] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.409396] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.424754] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.411170] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.438005] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned client # [ 0.409466] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.438026] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.411209] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.425228] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.425416] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.409513] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned lb # [ 0.411787] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker1 # [ 0.438048] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.425433] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.409536] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker1 # [ 0.438074] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.411798] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.438096] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker2 # [ 0.425463] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.409558] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned lb # [ 0.411804] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.438118] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.426049] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned client # [ 0.409580] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned lb # [ 0.411850] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.438142] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.426061] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.409601] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.438171] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.411898] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned client # [ 0.409624] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.426067] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.438181] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] client # [ 0.409646] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.411946] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.438185] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker2 # [ 0.426114] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned client # [ 0.409671] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.439018] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.411998] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.426163] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned client # [ 0.409693] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.412044] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.426210] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned client # [ 0.409716] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.412091] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.426259] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned client # [ 0.409739] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned lb # [ 0.412138] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.409762] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker2 # [ 0.426307] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned client # [ 0.409784] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.412184] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.426356] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned client # [ 0.409806] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.412231] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.426403] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.409827] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.412307] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned client # [ 0.409849] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker2 # [ 0.426451] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.412353] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.409871] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.426497] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.412375] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned client # [ 0.409892] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.426567] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned lb # [ 0.412396] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned client # [ 0.409918] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker2 # [ 0.426616] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.409928] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.412417] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.426638] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned client # [ 0.409932] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.412438] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.426660] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned lb # [ 0.412460] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.426681] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned lb # [ 0.412481] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.426703] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.412506] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.426725] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned lb # [ 0.412528] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker2 # [ 0.426747] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.412550] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.426773] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned lb # [ 0.412572] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.426796] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.412595] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker2 # [ 0.426818] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.412616] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.412637] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.426840] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned client # [ 0.466823] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.412658] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker2 # [ 0.426863] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.412679] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker2 # [ 0.426884] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.412700] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.426906] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.412721] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.426928] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker1 # [ 0.488105] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) lb # [ 0.412748] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker2 # [ 0.426949] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.412758] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker2 # [ 0.426970] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.412763] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker2 # [ 0.426991] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.413591] pci 0000:00:07.0: enabling device (0000 -> 0002) worker2 # [ 0.427018] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker2 # [ 0.427028] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker2 # [ 0.427032] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker2 # [ 0.427882] pci 0000:00:07.0: enabling device (0000 -> 0002) worker1 # [ 0.494316] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.496298] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker1 # [ 0.506384] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.508382] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) client # [ 0.493157] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) lb # [ 0.490042] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) client # [ 0.499850] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.519977] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) client # [ 0.501912] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.494051] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker1 # [ 0.522598] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.524522] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker2 # [ 0.497070] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker2 # [ 0.499119] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) client # [ 0.512140] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.499510] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.514308] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) lb # [ 0.501877] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker1 # [ 0.534360] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker1 # [ 0.536319] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker2 # [ 0.509013] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker2 # [ 0.511271] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) client # [ 0.525628] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) lb # [ 0.511462] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.514098] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) client # [ 0.528194] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.529928] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.548081] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.522662] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker2 # [ 0.525227] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) lb # [ 0.523816] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker2 # [ 0.526971] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.558103] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.527254] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) lb # [ 0.529221] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) client # [ 0.540009] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) client # [ 0.542050] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) client # [ 0.546152] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker1 # [ 0.561079] msm_serial: driver initialized worker1 # [ 0.561209] SuperH (H)SCI(F) driver initialized worker1 # [ 0.561263] STM32 USART driver initialized worker2 # [ 0.536988] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker2 # [ 0.539181] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker2 # [ 0.543009] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) lb # [ 0.539278] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.541430] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) lb # [ 0.545374] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.563964] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled client # [ 0.566708] msm_serial: driver initialized client # [ 0.566870] SuperH (H)SCI(F) driver initialized worker2 # [ 0.560778] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled client # [ 0.566923] STM32 USART driver initialized lb # [ 0.563389] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker1 # [ 0.595774] loop: module loaded worker1 # [ 0.595960] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 0.568884] msm_serial: driver initialized worker2 # [ 0.569055] SuperH (H)SCI(F) driver initialized worker1 # [ 0.596798] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.569110] STM32 USART driver initialized lb # [ 0.571563] msm_serial: driver initialized lb # [ 0.571706] SuperH (H)SCI(F) driver initialized lb # [ 0.571762] STM32 USART driver initialized worker1 # [ 0.605023] megasas: 07.734.00.00-rc1 worker1 # [ 0.605792] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker1 # [ 0.608111] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.608138] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.609755] Using buffer write method worker1 # [ 0.609827] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker1 # [ 0.611572] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.601439] loop: module loaded worker1 # [ 0.611594] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.601635] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 0.603820] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.598385] loop: module loaded worker2 # [ 0.598578] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 0.607277] megasas: 07.734.00.00-rc1 worker2 # [ 0.599378] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 0.607964] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker1 # [ 0.630282] Using buffer write method worker1 # [ 0.630317] Concatenating MTD devices: worker1 # [ 0.630321] (0): "0.flash" lb # [ 0.601202] loop: module loaded worker1 # [ 0.630324] (1): "0.flash" worker1 # [ 0.630335] into device "0.flash" lb # [ 0.601409] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 0.603682] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 0.619283] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.619329] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.621004] Using buffer write method client # [ 0.621070] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.612214] megasas: 07.734.00.00-rc1 worker2 # [ 0.613041] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.614976] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.615007] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.616812] Using buffer write method lb # [ 0.614850] megasas: 07.734.00.00-rc1 worker2 # [ 0.616894] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.615596] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.618695] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.617760] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.618726] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.617785] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.619371] Using buffer write method lb # [ 0.619436] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] client # [ 0.636532] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.636564] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.638168] Using buffer write method lb # [ 0.621056] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.638194] Concatenating MTD devices: client # [ 0.638198] (0): "0.flash" lb # [ 0.621078] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.638202] (1): "0.flash" client # [ 0.638210] into device "0.flash" worker2 # [ 0.639225] Using buffer write method worker2 # [ 0.639268] Concatenating MTD devices: worker2 # [ 0.639272] (0): "0.flash" worker2 # [ 0.639277] (1): "0.flash" worker2 # [ 0.639285] into device "0.flash" lb # [ 0.638801] Using buffer write method lb # [ 0.638835] Concatenating MTD devices: lb # [ 0.638840] (0): "0.flash" lb # [ 0.638844] (1): "0.flash" lb # [ 0.638852] into device "0.flash" worker1 # [ 0.883975] Freeing initrd memory: 26976K worker1 # [ 0.889940] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 0.893771] thunder_xcv, ver 1.0 worker1 # [ 0.893812] thunder_bgx, ver 1.0 worker1 # [ 0.893834] nicpf, ver 1.0 worker1 # [ 0.894368] e1000: Intel(R) PRO/1000 Network Driver worker1 # [ 0.894375] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 0.894399] e1000e: Intel(R) PRO/1000 Network Driver worker1 # [ 0.894407] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker1 # [ 0.894436] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 0.894442] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 0.894463] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.894469] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 0.894599] sky2: driver version 1.30 worker1 # [ 0.896203] usbcore: registered new interface driver usb-storage worker1 # [ 0.896251] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.896265] usbserial: USB Serial support registered for generic worker1 # [ 0.896848] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 0.897806] ehci-pci 0000:00:07.0: EHCI Host Controller worker1 # [ 0.897832] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.898065] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.910512] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.894309] Freeing initrd memory: 26972K worker1 # [ 0.910863] hub 1-0:1.0: USB hub found worker1 # [ 0.910886] hub 1-0:1.0: 6 ports detected worker1 # [ 0.913880] rtc-pl031 9010000.pl031: registered as rtc0 worker1 # [ 0.913910] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T15:45:13 UTC (1790091913) worker1 # [ 0.914211] i2c_dev: i2c /dev entries driver worker2 # [ 0.888616] Freeing initrd memory: 26944K client # [ 0.900313] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 0.919637] sdhci: Secure Digital Host Controller Interface driver worker1 # [ 0.919649] sdhci: Copyright(c) Pierre Ossman worker2 # [ 0.894757] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 0.919915] Synopsys Designware Multimedia Card Interface Driver worker1 # [ 0.920286] sdhci-pltfm: SDHCI platform and OF driver helper client # [ 0.904141] thunder_xcv, ver 1.0 worker1 # [ 0.924593] hid: raw HID events driver (C) Jiri Kosina lb # [ 0.894564] Freeing initrd memory: 26948K client # [ 0.904186] thunder_bgx, ver 1.0 worker1 # [ 0.924858] usbcore: registered new interface driver usbhid client # [ 0.904208] nicpf, ver 1.0 worker1 # [ 0.924864] usbhid: USB HID core driver client # [ 0.904754] e1000: Intel(R) PRO/1000 Network Driver worker2 # [ 0.898755] thunder_xcv, ver 1.0 worker2 # [ 0.898801] thunder_bgx, ver 1.0 client # [ 0.904762] e1000: Copyright (c) 1999-2006 Intel Corporation. worker2 # [ 0.898824] nicpf, ver 1.0 client # [ 0.904785] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 0.899398] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.904793] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 0.899406] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.904823] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 0.899430] e1000e: Intel(R) PRO/1000 Network Driver worker1 # [ 0.927782] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available client # [ 0.904829] igb: Copyright (c) 2007-2014 Intel Corporation. lb # [ 0.900493] tun: Universal TUN/TAP device driver, 1.6 worker2 # [ 0.899438] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 0.904855] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 0.899467] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 0.930334] drop_monitor: Initializing network drop monitor service client # [ 0.904861] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 0.899473] igb: Copyright (c) 2007-2014 Intel Corporation. client # [ 0.904991] sky2: driver version 1.30 worker1 # [ 0.930523] NET: Registered PF_INET6 protocol family worker2 # [ 0.899496] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 0.906569] usbcore: registered new interface driver usb-storage worker1 # [ 0.932400] Segment Routing with IPv6 worker2 # [ 0.899502] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. client # [ 0.907446] ehci-pci 0000:00:07.0: EHCI Host Controller worker1 # [ 0.932418] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.932447] NET: Registered PF_PACKET protocol family client # [ 0.907471] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 client # [ 0.907739] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker2 # [ 0.906223] sky2: driver version 1.30 lb # [ 0.904151] thunder_xcv, ver 1.0 worker2 # [ 0.908167] ehci-pci 0000:00:07.0: EHCI Host Controller lb # [ 0.904194] thunder_bgx, ver 1.0 lb # [ 0.904216] nicpf, ver 1.0 worker1 # [ 0.934264] 9pnet: Installing 9P2000 support worker2 # [ 0.908196] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 client # [ 0.917545] usbcore: registered new interface driver usbserial_generic lb # [ 0.904754] e1000: Intel(R) PRO/1000 Network Driver worker1 # [ 0.934312] Key type dns_resolver registered worker2 # [ 0.908454] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 client # [ 0.917565] usbserial: USB Serial support registered for generic lb # [ 0.904763] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.918286] hv_vmbus: registering driver hyperv_keyboard lb # [ 0.904786] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.918716] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.919030] hub 1-0:1.0: USB hub found lb # [ 0.904795] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 0.919087] hub 1-0:1.0: 6 ports detected lb # [ 0.904823] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 0.911563] usbcore: registered new interface driver usb-storage lb # [ 0.904829] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 0.911616] usbcore: registered new interface driver usbserial_generic lb # [ 0.904868] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 0.911630] usbserial: USB Serial support registered for generic client # [ 0.923613] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 0.904874] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 0.912226] hv_vmbus: registering driver hyperv_keyboard lb # [ 0.905008] sky2: driver version 1.30 client # [ 0.923646] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T15:45:13 UTC (1790091913) lb # [ 0.906917] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.923957] i2c_dev: i2c /dev entries driver lb # [ 0.906945] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker2 # [ 0.916041] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 0.907114] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.941289] registered taskstats version 1 worker2 # [ 0.916070] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T15:45:13 UTC (1790091913) worker1 # [ 0.941425] Loading compiled-in X.509 certificates worker2 # [ 0.916449] i2c_dev: i2c /dev entries driver worker2 # [ 0.919564] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker2 # [ 0.919851] hub 1-0:1.0: USB hub found lb # [ 0.916921] usbcore: registered new interface driver usb-storage client # [ 0.929238] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 0.919872] hub 1-0:1.0: 6 ports detected client # [ 0.929248] sdhci: Copyright(c) Pierre Ossman lb # [ 0.917024] usbcore: registered new interface driver usbserial_generic client # [ 0.929505] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.917038] usbserial: USB Serial support registered for generic client # [ 0.929875] sdhci-pltfm: SDHCI platform and OF driver helper lb # [ 0.917631] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 0.949981] Demotion targets for Node 0: null lb # [ 0.918214] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker1 # [ 0.950082] Key type .fscrypt registered lb # [ 0.918512] hub 1-0:1.0: USB hub found worker1 # [ 0.950089] Key type fscrypt-provisioning registered lb # [ 0.918532] hub 1-0:1.0: 6 ports detected client # [ 0.934036] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.950187] ima: No TPM chip found, activating TPM-bypass! client # [ 0.934275] usbcore: registered new interface driver usbhid worker1 # [ 0.950205] ima: Allocated hash algorithm: sha1 client # [ 0.934282] usbhid: USB HID core driver worker1 # [ 0.950228] ima: No architecture policies found lb # [ 0.923658] rtc-pl031 9010000.pl031: registered as rtc0 worker1 # [ 0.954190] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.924234] sdhci: Secure Digital Host Controller Interface driver lb # [ 0.923687] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T15:45:13 UTC (1790091913) worker2 # [ 0.924247] sdhci: Copyright(c) Pierre Ossman lb # [ 0.924003] i2c_dev: i2c /dev entries driver worker2 # [ 0.924524] Synopsys Designware Multimedia Card Interface Driver client # [ 0.937106] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.924898] sdhci-pltfm: SDHCI platform and OF driver helper client # [ 0.938608] drop_monitor: Initializing network drop monitor service worker2 # [ 0.926421] hid: raw HID events driver (C) Jiri Kosina client # [ 0.940588] NET: Registered PF_INET6 protocol family worker2 # [ 0.926689] usbcore: registered new interface driver usbhid worker2 # [ 0.926695] usbhid: USB HID core driver client # [ 0.941727] Segment Routing with IPv6 lb # [ 0.929353] sdhci: Secure Digital Host Controller Interface driver client # [ 0.941746] In-situ OAM (IOAM) with IPv6 lb # [ 0.929368] sdhci: Copyright(c) Pierre Ossman client # [ 0.941810] NET: Registered PF_PACKET protocol family lb # [ 0.929630] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.930005] sdhci-pltfm: SDHCI platform and OF driver helper worker2 # [ 0.932455] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.934013] drop_monitor: Initializing network drop monitor service worker2 # [ 0.934158] NET: Registered PF_INET6 protocol family client # [ 0.943631] 9pnet: Installing 9P2000 support worker2 # [ 0.937215] Segment Routing with IPv6 client # [ 0.943675] Key type dns_resolver registered lb # [ 0.934271] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.937235] In-situ OAM (IOAM) with IPv6 lb # [ 0.934528] usbcore: registered new interface driver usbhid worker2 # [ 0.937268] NET: Registered PF_PACKET protocol family lb # [ 0.934535] usbhid: USB HID core driver worker2 # [ 0.938935] 9pnet: Installing 9P2000 support worker2 # [ 0.938986] Key type dns_resolver registered lb # [ 0.937409] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available client # [ 0.950278] registered taskstats version 1 client # [ 0.950439] Loading compiled-in X.509 certificates lb # [ 0.940081] drop_monitor: Initializing network drop monitor service lb # [ 0.940227] NET: Registered PF_INET6 protocol family lb # [ 0.942126] Segment Routing with IPv6 lb # [ 0.942157] In-situ OAM (IOAM) with IPv6 lb # [ 0.943336] NET: Registered PF_PACKET protocol family worker1 # [ 0.972069] clk: Disabling unused clocks worker1 # [ 0.972098] PM: genpd: Disabling unused power domains worker2 # [ 0.945670] registered taskstats version 1 worker2 # [ 0.945830] Loading compiled-in X.509 certificates lb # [ 0.944059] 9pnet: Installing 9P2000 support lb # [ 0.944104] Key type dns_resolver registered worker1 # [ 0.976242] Freeing unused kernel memory: 4736K client # [ 0.959116] Demotion targets for Node 0: null worker1 # [ 0.976441] Run /init as init process client # [ 0.959224] Key type .fscrypt registered client # [ 0.959230] Key type fscrypt-provisioning registered client # [ 0.959319] ima: No TPM chip found, activating TPM-bypass! client # [ 0.959338] ima: Allocated hash algorithm: sha1 client # [ 0.959360] ima: No architecture policies found client # [ 0.963560] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.954491] Demotion targets for Node 0: null worker2 # [ 0.954610] Key type .fscrypt registered worker2 # [ 0.954617] Key type fscrypt-provisioning registered lb # [ 0.950602] registered taskstats version 1 worker2 # [ 0.954721] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.950741] Loading compiled-in X.509 certificates worker2 # [ 0.954740] ima: Allocated hash algorithm: sha1 worker2 # [ 0.954763] ima: No architecture policies found worker2 # [ 0.958862] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 0.959415] Demotion targets for Node 0: null lb # [ 0.959527] Key type .fscrypt registered lb # [ 0.959534] Key type fscrypt-provisioning registered lb # [ 0.959626] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.959646] ima: Allocated hash algorithm: sha1 worker1 # [ 0.991175] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.959669] ima: No architecture policies found lb # [ 0.963721] input: gpio-keys as /devices/platform/gpio-keys/input/input0 client # [ 0.980996] clk: Disabling unused clocks client # [ 0.981027] PM: genpd: Disabling unused power domains client # [ 0.985152] Freeing unused kernel memory: 4736K client # [ 0.985347] Run /init as init process worker2 # [ 0.976648] clk: Disabling unused clocks worker2 # [ 0.976680] PM: genpd: Disabling unused power domains worker2 # [ 0.980891] Freeing unused kernel memory: 4736K worker2 # [ 0.981074] Run /init as init process lb # [ 0.982307] clk: Disabling unused clocks lb # [ 0.982339] PM: genpd: Disabling unused power domains lb # [ 0.986593] Freeing unused kernel memory: 4736K lb # [ 0.986803] Run /init as init process client # [ 1.000901] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 0.997781] systemd[1]: Successfully made /usr/ read-only. lb # [ 1.004383] systemd[1]: Successfully made /usr/ read-only. worker1 # [ 1.156981] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.166717] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 1.167588] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 1.166235] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.309239] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker1 # [ 1.326069] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.338343] systemd[1]: Detected virtualization qemu. client # [ 1.319229] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker1 # [ 1.340343] systemd[1]: Detected architecture arm64. worker1 # [ 1.342365] systemd[1]: Running in initrd. worker1 # [ 1.345048] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.347935] systemd[1]: Hostname set to . worker2 # [ 1.320131] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 lb # [ 1.318560] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 client # [ 1.335727] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.347886] systemd[1]: Detected virtualization qemu. client # [ 1.349869] systemd[1]: Detected architecture arm64. client # [ 1.351849] systemd[1]: Running in initrd. worker2 # [ 1.332982] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.345135] systemd[1]: Detected virtualization qemu. client # [ 1.354504] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.347233] systemd[1]: Detected architecture arm64. client # [ 1.357519] systemd[1]: Hostname set to . worker2 # [ 1.349298] systemd[1]: Running in initrd. lb # [ 1.339131] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.351897] systemd[1]: Initializing machine ID from random generator. lb # [ 1.351427] systemd[1]: Detected virtualization qemu. worker2 # [ 1.354708] systemd[1]: Hostname set to . lb # [ 1.353429] systemd[1]: Detected architecture arm64. lb # [ 1.355441] systemd[1]: Running in initrd. lb # [ 1.358034] systemd[1]: Initializing machine ID from random generator. lb # [ 1.361019] systemd[1]: Hostname set to . worker1 # [ 1.393268] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 client # [ 1.406930] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker2 # [ 1.403815] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 lb # [ 1.410481] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker1 # [ 1.516950] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.530732] usb 1-2: new high-speed USB device number 3 using ehci-pci worker2 # [ 1.523575] usb 1-2: new high-speed USB device number 3 using ehci-pci lb # [ 1.534229] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.661712] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.673914] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker1 # [ 1.679545] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker2 # [ 1.660577] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.671950] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.670712] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.691472] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 1.682187] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 1.687935] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 client # [ 1.698963] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 lb # [ 1.690705] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 lb # [ 1.696870] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.770599] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.775072] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.776142] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 1.777347] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.778282] systemd[1]: Reached target Path Units. worker1 # [ 1.778993] systemd[1]: Reached target Slice Units. worker1 # [ 1.779750] systemd[1]: Reached target Swaps. worker1 # [ 1.780449] systemd[1]: Reached target Timer Units. worker1 # [ 1.781535] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.782617] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.783612] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.784479] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.785469] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.785493] systemd[1]: Reached target Socket Units. worker1 # [ 1.788445] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.789542] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 1.799817] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 1.774747] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.786698] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.783253] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.785889] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.787094] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.788501] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.798567] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.789672] systemd[1]: Reached target Path Units. worker2 # [ 1.790495] systemd[1]: Reached target Slice Units. worker2 # [ 1.791327] systemd[1]: Reached target Swaps. worker2 # [ 1.792144] systemd[1]: Reached target Timer Units. client # [ 1.799912] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.793180] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 1.799961] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.821130] systemd[1]: Starting Journal Service... worker2 # [ 1.794426] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.799992] systemd[1]: Reached target Path Units. lb # [ 1.792919] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.800012] systemd[1]: Reached target Slice Units. worker2 # [ 1.795602] systemd[1]: Listening on Journal Sockets. client # [ 1.800032] systemd[1]: Reached target Swaps. worker2 # [ 1.796631] systemd[1]: Listening on udev Control Socket. lb # [ 1.794337] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.800053] systemd[1]: Reached target Timer Units. worker2 # [ 1.797668] systemd[1]: Listening on udev Kernel Socket. client # [ 1.800251] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.795953] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 1.798580] systemd[1]: Reached target Socket Units. client # [ 1.800418] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.797152] systemd[1]: Reached target Path Units. client # [ 1.800571] systemd[1]: Listening on Journal Sockets. lb # [ 1.798069] systemd[1]: Reached target Slice Units. client # [ 1.800714] systemd[1]: Listening on udev Control Socket. lb # [ 1.799098] systemd[1]: Reached target Swaps. worker2 # [ 1.801009] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.800839] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.828832] systemd[1]: Starting Load Kernel Modules... lb # [ 1.799957] systemd[1]: Reached target Timer Units. client # [ 1.800864] systemd[1]: Reached target Socket Units. worker2 # [ 1.802284] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 1.801082] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.829978] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.813442] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.814599] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 1.802546] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.802716] systemd[1]: Listening on Journal Sockets. lb # [ 1.802871] systemd[1]: Listening on udev Control Socket. lb # [ 1.803003] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.803031] systemd[1]: Reached target Socket Units. lb # [ 1.809773] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 1.811684] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.811160] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 1.825890] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 1.842612] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.819820] systemd[1]: Starting Journal Service... lb # [ 1.819319] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.829001] systemd[1]: Starting Journal Service... worker1 # [ 1.861045] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 1.862043] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 1.843941] systemd[1]: Starting Load Kernel Modules... worker2 # [ 1.844990] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.855456] systemd[1]: Starting Journal Service... lb # [ 1.846656] systemd[1]: Starting Load Kernel Modules... client # [ 1.858315] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.877344] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 1.858433] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 1.847540] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 1.854514] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.884831] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.887554] systemd-journald[72]: Collecting audit messages is disabled. client # [ 1.870683] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.866628] systemd-journald[72]: Collecting audit messages is disabled. client # [ 1.882824] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.884500] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 1.905215] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.875248] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 1.876064] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.914565] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker1 # [ 1.914802] [drm] features: -virgl +edid -resource_blob -host_visible worker1 # [ 1.914811] [drm] features: -context_init worker1 # [ 1.915515] [drm] number of scanouts: 1 worker1 # [ 1.915533] [drm] number of cap sets: 0 client # [ 1.904964] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.890595] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.891685] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.890915] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 1.892666] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.906149] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.897520] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.929533] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.933292] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.910230] systemd-journald[72]: Collecting audit messages is disabled. worker1 # [ 1.941423] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker1 # [ 1.941442] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker2 # [ 1.925481] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.961179] Console: switching to colour frame buffer device 160x50 lb # [ 1.939894] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.944153] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.946385] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.944494] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 1.951312] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 1.952920] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 1.948877] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.947067] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.968759] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.985673] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 1.958646] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.985859] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.985883] systemd[1]: Reached target Local File Systems. worker2 # [ 1.960920] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker2 # [ 1.961159] [drm] features: -virgl +edid -resource_blob -host_visible worker1 # [ 1.990051] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.961168] [drm] features: -context_init worker2 # [ 1.961871] [drm] number of scanouts: 1 worker2 # [ 1.961889] [drm] number of cap sets: 0 client # [ 1.974742] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.967351] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 lb # [ 1.967588] [drm] features: -virgl +edid -resource_blob -host_visible lb # [ 1.967599] [drm] features: -context_init lb # [ 1.968310] [drm] number of scanouts: 1 lb # [ 1.968327] [drm] number of cap sets: 0 worker1 # [ 2.001524] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device client # [ 1.981802] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 client # [ 1.982048] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 1.982058] [drm] features: -context_init worker2 # [ 1.979877] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker2 # [ 1.979893] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 client # [ 2.006906] [drm] number of scanouts: 1 client # [ 2.006942] [drm] number of cap sets: 0 lb # [ 1.990609] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic lb # [ 1.990628] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker1 # [ 2.025153] systemd[1]: Finished Load Kernel Modules. worker2 # [ 2.000089] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 2.000277] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 2.000306] systemd[1]: Reached target Local File Systems. worker1 # [ 2.029293] systemd[1]: Starting Apply Kernel Variables... client # [ 2.011391] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic client # [ 2.011412] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 client # [ 2.020158] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 2.020546] systemd[1]: Reached target Preparation for Local File Systems. client # [ 2.020576] systemd[1]: Reached target Local File Systems. worker2 # [ 2.007909] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 2.014843] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 2.015069] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 2.015097] systemd[1]: Reached target Local File Systems. lb # [ 2.018966] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 2.015449] Console: switching to colour frame buffer device 160x50 client # [ 2.031160] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 2.021581] Console: switching to colour frame buffer device 160x50 worker1 # [ 2.046948] systemd-modules-load[73]: Inserted module 'dm_mod' client # [ 2.039814] Console: switching to colour frame buffer device 160x50 worker1 # [ 2.065188] systemd[1]: Started Journal Service. worker2 # [ 2.040293] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 2.064305] systemd-modules-load[73]: Module 'virtio_balloon' is built in lb # [ 2.050868] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device client # [ 2.063284] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker2 # [ 2.056228] systemd[1]: Finished Load Kernel Modules. worker2 # [ 2.059634] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 2.072148] systemd-modules-load[73]: Module 'virtio_console' is built in client # [ 2.072239] systemd[1]: Finished Load Kernel Modules. lb # [ 2.062816] systemd[1]: Finished Load Kernel Modules. worker1 # [ 2.077328] systemd-modules-load[73]: Inserted module 'virtio_gpu' worker1 # [ 2.079509] systemd-modules-load[73]: Module 'virtio_rng' is built in client # [ 2.079213] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.055196] systemd-modules-load[74]: Inserted module 'dm_mod' worker2 # [ 2.056389] systemd-modules-load[74]: Module 'virtio_balloon' is built in lb # [ 2.069372] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.075749] systemd[1]: Started Journal Service. worker1 # [ 2.092418] systemd[1]: Starting Create System Files and Directories... worker1 # [ 2.093671] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 2.094819] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker2 # [ 2.069292] systemd-modules-load[74]: Module 'virtio_console' is built in worker2 # [ 2.071476] systemd-modules-load[74]: Inserted module 'virtio_gpu' client # [ 2.080723] systemd-modules-load[73]: Inserted module 'dm_mod' client # [ 2.100540] systemd[1]: Started Journal Service. client # [ 2.085296] systemd-modules-load[73]: Module 'virtio_balloon' is built in client # [ 2.086433] systemd-modules-load[73]: Module 'virtio_console' is built in client # [ 2.087504] systemd-modules-load[73]: Inserted module 'virtio_gpu' worker2 # [ 2.084441] systemd-modules-load[74]: Module 'virtio_rng' is built in worker2 # [ 2.085529] systemd[1]: Starting Create System Files and Directories... lb # [ 2.086148] systemd-modules-load[73]: Inserted module 'dm_mod' worker1 # [ 2.114636] systemd[1]: Finished Create System Files and Directories. lb # [ 2.102346] systemd[1]: Started Journal Service. lb # [ 2.091223] systemd-modules-load[73]: Module 'virtio_balloon' is built in client # [ 2.100497] systemd-modules-load[73]: Module 'virtio_rng' is built in lb # [ 2.096496] systemd-modules-load[73]: Module 'virtio_console' is built in lb # [ 2.097629] systemd-modules-load[73]: Inserted module 'virtio_gpu' lb # [ 2.098632] systemd-modules-load[73]: Module 'virtio_rng' is built in client # [ 2.108145] systemd[1]: Starting Create System Files and Directories... worker2 # [ 2.102610] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 2.131328] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.117253] systemd-udevd[78]: Using default interface naming scheme 'v261'. lb # [ 2.117008] systemd[1]: Starting Create System Files and Directories... lb # [ 2.119065] systemd-udevd[79]: Using default interface naming scheme 'v261'. client # [ 2.131511] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker2 # [ 2.128471] systemd[1]: Finished Create System Files and Directories. lb # [ 2.132069] systemd[1]: Finished Apply Kernel Variables. client # [ 2.144591] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.142748] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.150083] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 2.160944] systemd[1]: Finished Create System Files and Directories. worker1 # [ 2.192663] systemd[1]: Starting Virtual Console Setup... lb # [ 2.167352] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 2.177855] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.203719] systemd[1]: Starting Virtual Console Setup... worker1 # [ 2.244521] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.248003] systemd[1]: Finished Virtual Console Setup. client # [ 2.236135] systemd[1]: Starting Virtual Console Setup... lb # [ 2.232404] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.256467] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.259956] systemd[1]: Finished Virtual Console Setup. lb # [ 2.276491] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.284476] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.279140] systemd[1]: Finished Virtual Console Setup. client # [ 2.287744] systemd[1]: Finished Virtual Console Setup. worker1 # [ 2.865099] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 2.867060] systemd[1]: Reached target System Initialization. worker1 # [ 2.872804] systemd[1]: Reached target Basic System. worker2 # [ 2.861101] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 2.863145] systemd[1]: Reached target System Initialization. worker2 # [ 2.869099] systemd[1]: Reached target Basic System. client # [ 2.893600] systemd[1]: Finished Coldplug All udev Devices. client # [ 2.896155] systemd[1]: Reached target System Initialization. client # [ 2.897061] systemd[1]: Reached target Basic System. lb # [ 2.900178] systemd[1]: Finished Coldplug All udev Devices. lb # [ 2.902066] systemd[1]: Reached target System Initialization. lb # [ 2.902934] systemd[1]: Reached target Basic System. worker1 # [ 3.015615] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 3.036752] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 3.015076] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.052086] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.040538] (udev-worker)[91]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 3.040312] (udev-worker)[104]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 3.044541] (udev-worker)[104]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.059464] (udev-worker)[99]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 3.065696] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.078261] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.077843] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.086392] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.109915] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 3.112141] systemd[1]: Reached target Initrd Root Device. worker1 # [ 3.116490] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 3.100752] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.105489] systemd[1]: Reached target Initrd Root Device. worker2 # [ 3.110667] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 3.165824] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 3.176164] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 3.178689] systemd[1]: Mounting /sysroot... lb # [ 3.156133] systemd[1]: Found device /dev/disk/by-label/nixos. lb # [ 3.161060] systemd[1]: Reached target Initrd Root Device. client # [ 3.172885] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.167976] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 3.168106] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 3.178629] systemd[1]: Reached target Initrd Root Device. worker2 # [ 3.173844] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. client # [ 3.181956] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 3.179837] systemd[1]: Mounting /sysroot... worker1 # [ 3.228807] EXT4-fs (vda): mounted filesystem 33133b99-1616-4b97-8015-09ee09300e84 r/w with ordered data mode. Quota mode: none. worker1 # [ 3.219604] systemd[1]: Mounted /sysroot. worker1 # [ 3.221528] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.228138] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.207310] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 3.214823] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 3.217028] systemd[1]: Mounting /sysroot... client # [ 3.230518] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.238394] EXT4-fs (vda): mounted filesystem afc2ae95-50d3-4000-a010-8c2d52d66043 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.228988] systemd[1]: Mounted /sysroot. worker2 # [ 3.230788] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.255927] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 3.238160] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.236079] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 3.244325] systemd[1]: Mounting /sysroot... worker1 # [ 3.264097] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 3.268070] systemd[1]: Reloading... lb # [ 3.274853] EXT4-fs (vda): mounted filesystem 753fbc79-914d-4c7c-9ce8-f4b90e9bdb0a r/w with ordered data mode. Quota mode: none. worker2 # [ 3.263863] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.266551] systemd[1]: Mounted /sysroot. lb # [ 3.269414] systemd[1]: Reached target Initrd Root File System. client # [ 3.294502] EXT4-fs (vda): mounted filesystem 3e628041-8142-4b90-ac7f-abf3283ad4a7 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.271964] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.280869] systemd[1]: Mounted /sysroot. worker2 # [ 3.274759] systemd[1]: Reloading... lb # [ 3.273955] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 3.282270] systemd[1]: Reached target Initrd Root File System. client # [ 3.286306] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.300132] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 3.312247] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.306929] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.311115] systemd[1]: Reloading... client # [ 3.318206] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.322982] systemd[1]: Reloading... worker1 # [ 3.476421] systemd[1]: Reloading finished in 211 ms. worker2 # [ 3.480106] systemd[1]: Reloading finished in 208 ms. worker1 # [ 3.512544] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... worker1 # [ 3.517483] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... worker1 # [ 3.522081] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.525378] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.527212] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.508572] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.512748] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... worker2 # [ 3.519259] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.522263] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.522164] systemd[1]: Reloading finished in 212 ms. worker2 # [ 3.523885] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.532112] systemd[1]: Reloading finished in 206 ms. lb # [ 3.549365] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... lb # [ 3.553823] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... client # [ 3.566148] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... lb # [ 3.560901] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.563873] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 3.571145] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... lb # [ 3.566187] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.581219] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 3.582262] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 3.584111] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.830667] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.843656] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.857122] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.886391] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.869267] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.865815] systemd[1]: Mounting /sysroot/nix/.ro-store... worker2 # [ 3.866727] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 3.878355] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.877783] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 3.882375] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.891539] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.911465] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 3.886499] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.914289] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 3.894399] systemd[1]: Mounting /sysroot/run... worker2 # [ 3.899120] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.914410] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 3.911487] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.966090] fuse: init (API version 7.45) worker2 # [ 3.925309] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.935126] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.954553] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 3.930404] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.982212] virtiofs virtio6: discovered new tag: nix-store worker1 # [ 3.982986] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 3.974982] systemd[1]: Mounted /sysroot/run. worker1 # [ 3.999832] virtiofs virtio7: discovered new tag: shared worker1 # [ 4.000635] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker1 # [ 4.008741] virtiofs virtio8: discovered new tag: xchg client # [ 3.995935] fuse: init (API version 7.45) worker1 # [ 4.014456] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 3.981744] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 3.998457] fuse: init (API version 7.45) client # [ 4.013252] virtiofs virtio6: discovered new tag: nix-store client # [ 4.014044] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 4.017518] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 4.019743] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 3.995676] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 3.998574] systemd[1]: Mounted /sysroot/run. lb # [ 4.015450] fuse: init (API version 7.45) worker2 # [ 4.016618] virtiofs virtio6: discovered new tag: nix-store worker2 # [ 4.017463] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 4.029697] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 4.028376] virtiofs virtio7: discovered new tag: shared client # [ 4.029190] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker1 # [ 4.033918] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 4.035940] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 4.010647] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 4.012327] systemd[1]: Mounted /sysroot/run. lb # [ 4.024082] virtiofs virtio6: discovered new tag: nix-store client # [ 4.036791] virtiofs virtio8: discovered new tag: xchg lb # [ 4.024924] virtiofs virtio6: virtio_fs_setup_dax: No cache capability client # [ 4.037526] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 4.023523] systemd[1]: Mounted /sysroot/run. worker2 # [ 4.033243] virtiofs virtio7: discovered new tag: shared worker2 # [ 4.034028] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 4.024691] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 4.034527] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 4.039969] virtiofs virtio7: discovered new tag: shared lb # [ 4.040710] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 4.030621] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 4.045135] virtiofs virtio8: discovered new tag: xchg client # [ 4.038764] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 4.045885] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 4.040677] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 4.047280] virtiofs virtio8: discovered new tag: xchg lb # [ 4.048019] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker2 # [ 4.046787] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 4.056286] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 4.053536] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 4.057079] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.058446] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 4.059181] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 4.063199] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 4.063978] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.066690] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 4.075736] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.069583] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 4.077259] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.072492] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.381681] (udev-worker)[91]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker2 # [ 4.389398] (udev-worker)[91]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.423887] (udev-worker)[91]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.432210] (udev-worker)[91]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker2 # [ 4.420094] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.426118] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.426942] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.427690] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.460357] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.464171] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.465380] systemd[1]: Stopping Virtual Console Setup... worker1 # [ 4.468096] systemd[1]: Starting Virtual Console Setup... lb # [ 4.448323] (udev-worker)[100]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker2 # [ 4.450390] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.452673] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.452980] (udev-worker)[100]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker2 # [ 4.456201] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.482115] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.483708] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.488194] systemd[1]: Starting Virtual Console Setup... client # [ 4.479229] (udev-worker)[91]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory lb # [ 4.476117] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.477480] systemd[1]: Stopped Virtual Console Setup. client # [ 4.485237] (udev-worker)[91]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.505776] systemd-vconsole-setup[153]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 4.480166] systemd-vconsole-setup[156]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 4.481324] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.483260] systemd[1]: Finished Virtual Console Setup. worker1 # [ 4.509095] systemd[1]: Finished Virtual Console Setup. lb # [ 4.484163] systemd[1]: Starting Virtual Console Setup... lb # [ 4.493552] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.495604] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.501580] systemd[1]: Starting Virtual Console Setup... client # [ 4.513180] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.516159] systemd[1]: Stopped Virtual Console Setup. client # [ 4.517646] systemd[1]: Stopping Virtual Console Setup... client # [ 4.520150] systemd[1]: Starting Virtual Console Setup... lb # [ 4.528573] systemd-vconsole-setup[155]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 4.531779] systemd[1]: Finished Virtual Console Setup. client # [ 4.548803] systemd-vconsole-setup[152]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 4.552237] systemd[1]: Finished Virtual Console Setup. worker1 # [ 4.833203] systemd[1]: Mounting /sysroot/nix/store... client # [ 4.871099] systemd[1]: Mounting /sysroot/nix/store... lb # [ 4.867128] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 4.869013] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.899406] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.903398] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.905743] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.913504] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.930353] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.956753] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 4.938640] systemd[1]: Mounted /sysroot/nix/store. lb # [ 4.933669] systemd[1]: Reached target Initrd File Systems. client # [ 4.941774] systemd[1]: Reached target Initrd File Systems. lb # [ 4.938810] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.964431] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 4.946513] systemd[1]: Starting Find NixOS closure... worker2 # [ 4.939708] systemd[1]: Mounted /sysroot/nix/store. worker2 # [ 4.944337] systemd[1]: Reached target Initrd File Systems. worker2 # [ 4.948162] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.975744] systemd[1]: Finished Find NixOS closure. lb # [ 4.948415] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.956329] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 4.978750] systemd[1]: Reached target Initrd Default Target. worker2 # [ 4.953885] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 4.984613] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 5.008920] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 5.012465] systemd[1]: Stopped target Basic System. worker1 # [ 5.013624] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 5.016350] systemd[1]: Stopped target Path Units. worker1 # [ 5.018857] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 4.999543] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 5.024184] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 4.998479] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 5.005048] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 5.025767] systemd[1]: Stopped target Slice Units. worker1 # [ 5.026768] systemd[1]: Stopped target Socket Units. worker1 # [ 5.027713] systemd[1]: Stopped target System Initialization. lb # [ 5.004397] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 5.005027] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 5.035329] systemd[1]: Stopped target Swaps. client # [ 5.016123] systemd[1]: Finished Find NixOS closure. lb # [ 5.008263] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 5.037573] systemd[1]: Stopped target Timer Units. client # [ 5.019028] systemd[1]: Reached target Initrd Default Target. client # [ 5.021086] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 5.040961] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 5.043830] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 5.020359] systemd[1]: Finished Find NixOS closure. lb # [ 5.019961] systemd[1]: Finished Find NixOS closure. lb # [ 5.022898] systemd[1]: Reached target Initrd Default Target. worker2 # [ 5.023502] systemd[1]: Reached target Initrd Default Target. worker1 # [ 5.050499] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 5.024712] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 5.051698] systemd[1]: Stopped Find NixOS closure. worker2 # [ 5.028444] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 5.058989] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 5.059943] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 5.062147] systemd[1]: Stopped Apply Kernel Variables. worker1 # [ 5.064583] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 5.067404] systemd[1]: Stopped Load Kernel Modules. client # [ 5.052460] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 5.072478] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. client # [ 5.053792] systemd[1]: Stopped target Basic System. worker1 # [ 5.075070] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 5.056336] systemd[1]: Stopped target Initrd Root Device. lb # [ 5.049534] systemd[1]: Stopped target Initrd Default Target. client # [ 5.059289] systemd[1]: Stopped target Path Units. lb # [ 5.052580] systemd[1]: Stopped target Basic System. lb # [ 5.053468] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 5.079795] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 5.060743] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 5.081336] systemd[1]: Stopped Create System Files and Directories. worker1 # [ 5.082209] systemd[1]: Stopped target Local File Systems. lb # [ 5.056304] systemd[1]: Stopped target Path Units. worker1 # [ 5.082998] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 5.057191] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 5.064457] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 5.083940] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 5.058996] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 5.085516] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 5.086310] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 5.060153] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.060971] systemd[1]: Stopped target Basic System. client # [ 5.068195] systemd[1]: Stopped target Slice Units. worker1 # [ 5.087332] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 5.069157] systemd[1]: Stopped target Socket Units. worker1 # [ 5.088432] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 5.089152] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 5.090043] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 5.064433] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 5.090932] systemd[1]: systemd-udevd.service: Deactivated successfully. client # [ 5.071814] systemd[1]: Stopped target System Initialization. worker2 # [ 5.065640] systemd[1]: Stopped target Path Units. lb # [ 5.063978] systemd[1]: Stopped target Slice Units. worker1 # [ 5.091826] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 5.067309] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 5.077589] systemd[1]: Stopped target Swaps. worker2 # [ 5.070745] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 5.078395] systemd[1]: Stopped target Timer Units. client # [ 5.079216] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 5.072547] systemd[1]: Stopped target Socket Units. lb # [ 5.073375] systemd[1]: Stopped target System Initialization. lb # [ 5.074284] systemd[1]: Stopped target Swaps. worker1 # [ 5.100234] systemd[1]: systemd-udevd.service: Consumed 1.413s CPU time over 3.097s wall clock time, 22.2M memory peak. lb # [ 5.075025] systemd[1]: Stopped target Timer Units. worker2 # [ 5.076343] systemd[1]: Stopped target Slice Units. lb # [ 5.075842] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 5.077322] systemd[1]: Stopped target Socket Units. worker2 # [ 5.078204] systemd[1]: Stopped target System Initialization. client # [ 5.084992] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 5.104186] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 5.079216] systemd[1]: Stopped target Swaps. worker1 # [ 5.105318] systemd[1]: Closed udev Control Socket. client # [ 5.085949] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 5.087076] systemd[1]: Stopped Find NixOS closure. worker1 # [ 5.106151] systemd[1]: Starting Cleanup udev Database... client # [ 5.087921] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 5.108181] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 5.079991] systemd[1]: Stopped target Timer Units. worker1 # [ 5.112289] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 5.113170] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 5.114264] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 5.095750] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 5.089530] systemd[1]: dbus.socket: Deactivated successfully. client # [ 5.097058] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 5.090558] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 5.116245] systemd[1]: kmod-static-nodes.service: Deactivated successfully. client # [ 5.097865] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 5.090932] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 5.091559] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 5.098917] systemd[1]: Stopped Load Kernel Modules. lb # [ 5.091954] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 5.099684] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 5.120363] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 5.121314] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 5.122292] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 5.099476] systemd[1]: Stopped Find NixOS closure. worker2 # [ 5.101257] systemd[1]: Stopped Find NixOS closure. client # [ 5.107839] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 5.101526] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 5.102165] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 5.103195] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 5.104146] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 5.112361] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 5.105549] systemd[1]: Stopped Apply Kernel Variables. lb # [ 5.107384] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 5.108287] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 5.109432] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 5.110629] systemd[1]: Stopped Load Kernel Modules. client # [ 5.117406] systemd[1]: Stopped Create System Files and Directories. lb # [ 5.110760] systemd[1]: Stopped Load Kernel Modules. client # [ 5.118454] systemd[1]: Stopped target Local File Systems. worker2 # [ 5.111470] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 5.111524] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. client # [ 5.120326] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 5.116325] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 5.124327] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 5.117517] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 5.125419] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 5.144485] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 5.118502] systemd[1]: Stopped Create System Files and Directories. client # [ 5.126191] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 5.119340] systemd[1]: Stopped target Local File Systems. worker1 # [ 5.146063] systemd[1]: Finished Cleanup udev Database. client # [ 5.128706] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 5.148559] systemd[1]: Reached target Switch Root. lb # [ 5.122145] systemd[1]: Stopped target Preparation for Local File Systems. worker2 # [ 5.123365] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 5.130536] systemd[1]: Stopped Virtual Console Setup. lb # [ 5.123118] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 5.124238] systemd[1]: Stopped Coldplug All udev Devices. client # [ 5.132193] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 5.125013] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 5.126090] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker1 # [ 5.152436] systemd[1]: Starting NixOS Activation... lb # [ 5.126038] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 5.133433] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 5.127297] systemd[1]: Stopped Create System Files and Directories. lb # [ 5.127025] systemd[1]: Stopped Virtual Console Setup. client # [ 5.134421] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 5.127754] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.135344] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 5.130206] systemd[1]: Stopped target Local File Systems. client # [ 5.136429] systemd[1]: systemd-udevd.service: Consumed 1.439s CPU time over 3.097s wall clock time, 22.2M memory peak. client # [ 5.137833] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 5.138809] systemd[1]: Closed udev Control Socket. worker2 # [ 5.132160] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 5.139515] systemd[1]: Starting Cleanup udev Database... lb # [ 5.132286] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 5.134170] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 5.134022] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.136397] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 5.136431] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 5.144344] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 5.139177] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 5.140542] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 5.140308] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 5.141587] systemd[1]: Stopped Virtual Console Setup. client # [ 5.148202] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 5.141301] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 5.142318] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.149277] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 5.143223] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. lb # [ 5.142275] systemd[1]: systemd-udevd.service: Consumed 1.403s CPU time over 3.096s wall clock time, 22.3M memory peak. worker2 # [ 5.144206] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 5.145150] systemd[1]: Stopped Rule-based Manager for Device Events and Files. client # [ 5.152227] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 5.146134] systemd[1]: systemd-udevd.service: Consumed 1.382s CPU time over 3.124s wall clock time, 22.6M memory peak. client # [ 5.153267] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 5.147538] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 5.148098] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 5.156160] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 5.149135] systemd[1]: Closed udev Control Socket. lb # [ 5.149839] systemd[1]: Starting Cleanup udev Database... client # [ 5.157064] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 5.150624] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 5.160169] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 5.156512] systemd[1]: Closed udev Control Socket. worker2 # [ 5.157305] systemd[1]: Starting Cleanup udev Database... lb # [ 5.156517] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 5.158185] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 5.157606] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 5.159236] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 5.158722] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 5.159710] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 5.164188] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 5.164480] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 5.165329] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 5.166302] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 5.167246] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 5.172439] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.173452] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 5.186715] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 5.182416] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 5.190143] systemd[1]: Finished Cleanup udev Database. client # [ 5.190973] systemd[1]: Reached target Switch Root. client # [ 5.192604] systemd[1]: Starting NixOS Activation... lb # [ 5.185806] systemd[1]: Finished Cleanup udev Database. lb # [ 5.186622] systemd[1]: Reached target Switch Root. lb # [ 5.188353] systemd[1]: Starting NixOS Activation... worker2 # [ 5.199279] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 5.204686] systemd[1]: Finished Cleanup udev Database. worker2 # [ 5.205510] systemd[1]: Reached target Switch Root. worker2 # [ 5.206226] systemd[1]: Starting NixOS Activation... worker1 # [ 5.237793] initrd-nixos-activation-start[176]: booting system configuration /nix/store/nkxfr43rzbz80xc8q1a00w66rjj2an3s-nixos-system-worker1-test worker1 # [ 5.269774] initrd-nixos-activation-start[176]: running activation script... lb # [ 5.271846] initrd-nixos-activation-start[178]: booting system configuration /nix/store/rwj7rgrjnik4yfrdy5nxpi16ig7dj0yx-nixos-system-lb-test client # [ 5.279299] initrd-nixos-activation-start[176]: booting system configuration /nix/store/c7dwh1z1cl7zl3zmm2gm23k4b20rxnfn-nixos-system-client-test worker2 # [ 5.283768] initrd-nixos-activation-start[179]: booting system configuration /nix/store/91x7j7xpp6q86mih2akg81hmd36hcvqs-nixos-system-worker2-test lb # [ 5.304493] initrd-nixos-activation-start[178]: running activation script... client # [ 5.313426] initrd-nixos-activation-start[176]: running activation script... worker2 # [ 5.316276] initrd-nixos-activation-start[179]: running activation script... worker1 # [ 5.507832] initrd-nixos-activation-start[199]: setting up /etc... lb # [ 5.559514] initrd-nixos-activation-start[201]: setting up /etc... client # [ 5.571945] initrd-nixos-activation-start[199]: setting up /etc... worker2 # [ 5.567123] initrd-nixos-activation-start[202]: setting up /etc... worker1 # [ 5.628436] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 5.631204] systemd[1]: Finished NixOS Activation. worker1 # [ 5.636247] systemd[1]: Starting Switch Root... worker1 # [ 5.651603] systemd[1]: Switching root. lb # [ 5.678761] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 5.681588] systemd[1]: Finished NixOS Activation. lb # [ 5.682768] systemd[1]: Starting Switch Root... client # [ 5.691433] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 5.687531] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 5.690495] systemd[1]: Finished NixOS Activation. worker2 # [ 5.691648] systemd[1]: Starting Switch Root... client # [ 5.699979] systemd[1]: Finished NixOS Activation. client # [ 5.700910] systemd[1]: Starting Switch Root... lb # [ 5.703016] systemd[1]: Switching root. client # [ 5.719440] systemd[1]: Switching root. worker2 # [ 5.715637] systemd[1]: Switching root. worker1 # [ 5.843096] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). lb # [ 5.887999] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). client # [ 5.900337] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker2 # [ 5.905426] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker1 # [ 6.366298] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 6.379177] systemd[1]: Detected virtualization qemu. worker1 # [ 6.382545] systemd[1]: Detected architecture arm64. worker1 # [ 6.386683] systemd[1]: Detected first boot. worker1 # [ 6.391756] systemd[1]: Initializing machine ID from random generator. client # [ 6.429220] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 6.434832] systemd[1]: Detected virtualization qemu. lb # [ 6.416446] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 6.425522] systemd[1]: Detected virtualization qemu. client # [ 6.436613] systemd[1]: Detected architecture arm64. lb # [ 6.426924] systemd[1]: Detected architecture arm64. lb # [ 6.428270] systemd[1]: Detected first boot. client # [ 6.439716] systemd[1]: Detected first boot. lb # [ 6.431591] systemd[1]: Initializing machine ID from random generator. client # [ 6.444330] systemd[1]: Initializing machine ID from random generator. worker2 # [ 6.438800] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 6.451385] systemd[1]: Detected virtualization qemu. worker2 # [ 6.455011] systemd[1]: Detected architecture arm64. worker2 # [ 6.459255] systemd[1]: Detected first boot. worker2 # [ 6.465637] systemd[1]: Initializing machine ID from random generator. worker1 # [ 6.716415] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.689574] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.767554] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 6.791887] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.882781] systemd[1]: Applying preset policy. worker1 # [ 6.915993] systemd[1]: Applying preset policy. client # [ 6.955834] systemd[1]: Applying preset policy. worker2 # [ 7.024141] systemd[1]: Applying preset policy. worker1 # [ 7.183910] systemd[1]: Populated /etc with preset unit settings. lb # [ 7.166210] systemd[1]: Populated /etc with preset unit settings. client # [ 7.214523] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 7.287686] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 7.413432] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 7.415106] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 7.418358] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 7.422531] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.424565] systemd[1]: Created slice User and Session Slice. worker1 # [ 7.425990] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 7.426387] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 7.426679] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.426922] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 7.427164] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 7.427447] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 7.427682] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 7.427908] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 7.428144] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 7.428388] systemd[1]: Reached target Path Units. worker1 # [ 7.428620] systemd[1]: Reached target Remote File Systems. worker1 # [ 7.428849] systemd[1]: Reached target Slice Units. lb # [ 7.408166] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 7.438033] systemd[1]: Reached target Swaps. lb # [ 7.409467] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 7.441347] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 7.413909] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 7.444420] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.446730] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 7.418486] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.449187] systemd[1]: Listening on Factory Reset Management. lb # [ 7.420404] systemd[1]: Created slice User and Session Slice. worker1 # [ 7.450404] systemd[1]: Listening on Hostname Service Socket. lb # [ 7.421685] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 7.424447] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 7.454934] systemd[1]: Starting Journal Log Access Socket... client # [ 7.437701] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 7.426771] systemd[1]: Expecting device /dev/hvc0... client # [ 7.439270] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 7.457201] systemd[1]: Listening on Journal Audit Socket. lb # [ 7.428629] systemd[1]: Expecting device /dev/ttyAMA0... lb # [ 7.430602] systemd[1]: Reached target Local Encrypted Volumes. client # [ 7.443452] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 7.460950] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 7.432566] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 7.461582] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 7.434520] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 7.461889] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.447462] systemd[1]: Created slice Slice /system/getty. lb # [ 7.435595] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 7.462138] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 7.437231] systemd[1]: Reached target Virtual Machines and Containers. client # [ 7.449324] systemd[1]: Created slice User and Session Slice. lb # [ 7.439048] systemd[1]: Reached target Path Units. lb # [ 7.441356] systemd[1]: Reached target Remote File Systems. client # [ 7.451834] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 7.471348] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.442589] systemd[1]: Reached target Slice Units. client # [ 7.454242] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 7.472706] systemd[1]: Listening on udev Control Socket. lb # [ 7.444002] systemd[1]: Reached target Swaps. worker1 # [ 7.474321] systemd[1]: Listening on udev Varlink Socket. client # [ 7.456609] systemd[1]: Expecting device /dev/hvc0... lb # [ 7.447067] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 7.458518] systemd[1]: Expecting device /dev/ttyAMA0... client # [ 7.460477] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 7.477940] systemd[1]: Mounting Huge Pages File System... lb # [ 7.450143] systemd[1]: Listening on Process Core Dump Socket. client # [ 7.461617] systemd[1]: Stopped target initrd-fs.target. lb # [ 7.452428] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 7.463329] systemd[1]: Stopped target initrd-root-fs.target. client # [ 7.463650] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.463912] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 7.454858] systemd[1]: Listening on Factory Reset Management. client # [ 7.464180] systemd[1]: Reached target Path Units. client # [ 7.464423] systemd[1]: Reached target Remote File Systems. lb # [ 7.456063] systemd[1]: Listening on Hostname Service Socket. client # [ 7.464656] systemd[1]: Reached target Slice Units. client # [ 7.464888] systemd[1]: Reached target Swaps. worker1 # [ 7.485610] systemd[1]: Mounting POSIX Message Queue File System... client # [ 7.471429] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 7.460317] systemd[1]: Starting Journal Log Access Socket... lb # [ 7.462418] systemd[1]: Listening on Journal Audit Socket. client # [ 7.474514] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.494901] systemd[1]: Mounting Kernel Debug File System... client # [ 7.477706] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 7.466268] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 7.480169] systemd[1]: Listening on Factory Reset Management. lb # [ 7.468567] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 7.481426] systemd[1]: Listening on Hostname Service Socket. lb # [ 7.471135] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.485826] systemd[1]: Starting Journal Log Access Socket... lb # [ 7.474454] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 7.488143] systemd[1]: Listening on Journal Audit Socket. lb # [ 7.480261] systemd[1]: Listening on Disk Repartitioning Service Socket. client # [ 7.492053] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 7.481619] systemd[1]: Listening on udev Control Socket. client # [ 7.493641] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 7.483470] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 7.514575] systemd[1]: Mounting Kernel Trace File System... client # [ 7.495710] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.498776] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 7.488109] systemd[1]: Mounting Huge Pages File System... lb # [ 7.491862] systemd[1]: Mounting POSIX Message Queue File System... client # [ 7.504532] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 7.523245] systemd[1]: Starting Create List of Static Device Nodes... client # [ 7.505938] systemd[1]: Listening on udev Control Socket. client # [ 7.507773] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 7.526710] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 7.512424] systemd[1]: Mounting Huge Pages File System... lb # [ 7.506365] systemd[1]: Mounting Kernel Debug File System... client # [ 7.519957] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 7.540111] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 7.516943] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 7.514232] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 7.545062] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 7.518311] systemd[1]: Stopped initrd-switch-root.service. client # [ 7.531682] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 7.522753] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 7.526197] systemd[1]: Created slice Slice /system/getty. worker2 # [ 7.528428] systemd[1]: Created slice User and Session Slice. worker1 # [ 7.553267] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 7.526664] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 7.553651] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.529695] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 7.531638] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 7.530714] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 7.533256] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 7.535364] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.536518] systemd[1]: Reached target Local Encrypted Volumes. client # [ 7.546017] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 7.538016] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.540279] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 7.541396] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 7.542978] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 7.544740] systemd[1]: Reached target Path Units. worker2 # [ 7.546137] systemd[1]: Reached target Remote File Systems. worker2 # [ 7.547678] systemd[1]: Reached target Slice Units. worker2 # [ 7.549162] systemd[1]: Reached target Swaps. lb # [ 7.546810] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.574132] systemd[1]: Mounting FUSE Control File System... client # [ 7.559795] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 7.552186] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 7.574573] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.549319] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 7.555348] systemd[1]: Listening on Process Core Dump Socket. client # [ 7.563509] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 7.557696] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 7.554847] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 7.560121] systemd[1]: Listening on Factory Reset Management. worker2 # [ 7.561367] systemd[1]: Listening on Hostname Service Socket. lb # [ 7.558231] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.565760] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 7.568108] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.598766] systemd[1]: Starting Journal Service... worker2 # [ 7.571852] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 7.573948] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 7.576101] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.578388] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 7.589951] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.609714] systemd[1]: Starting Load Kernel Modules... worker2 # [ 7.583287] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 7.584722] systemd[1]: Listening on udev Control Socket. client # [ 7.594603] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 7.586339] systemd[1]: Listening on udev Varlink Socket. lb # [ 7.586554] systemd[1]: Mounting FUSE Control File System... worker2 # [ 7.590934] systemd[1]: Mounting Huge Pages File System... lb # [ 7.589463] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 7.602235] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 7.596047] systemd[1]: Mounting POSIX Message Queue File System... client # [ 7.602722] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.624479] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker1 # [ 7.634784] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 7.608003] systemd[1]: Mounting Kernel Debug File System... client # [ 7.620035] systemd[1]: Mounting FUSE Control File System... worker1 # [ 7.637066] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.621392] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.644502] systemd[1]: Starting Coldplug All udev Devices... lb # [ 7.616190] systemd[1]: Starting Journal Service... worker2 # [ 7.619677] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 7.648347] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 7.656650] systemd[1]: Mounted Huge Pages File System. worker2 # [ 7.632626] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 7.661520] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 7.634930] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 7.662009] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 7.662392] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 7.666454] systemd[1]: Mounted Kernel Configuration File System. client # [ 7.647968] systemd[1]: Starting Journal Service... worker2 # [ 7.644848] systemd[1]: Mounting Kernel Configuration File System... lb # [ 7.642026] systemd[1]: Starting Load Kernel Modules... worker1 # [ 7.672747] systemd[1]: Mounted FUSE Control File System. worker2 # [ 7.651187] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 7.661093] systemd[1]: Starting Load Kernel Modules... worker2 # [ 7.655750] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 7.656137] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 7.666715] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker1 # [ 7.693777] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 7.699231] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.689106] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 7.678815] systemd[1]: Mounting FUSE Control File System... worker2 # [ 7.679262] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.686849] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 7.702104] systemd[1]: Starting Remount Root and Kernel File Systems... lb # [ 7.687295] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.705152] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.703996] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 7.708034] systemd[1]: Starting Journal Service... client # [ 7.721935] systemd[1]: Starting Coldplug All udev Devices... lb # [ 7.713622] systemd[1]: Listening on Journal Log Access Socket. lb # [ 7.721560] systemd[1]: Mounted Huge Pages File System. client # [ 7.735991] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 7.729027] systemd[1]: Starting Load Kernel Modules... lb # [ 7.728334] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 7.728846] systemd[1]: Mounted Kernel Debug File System. lb # [ 7.729243] systemd[1]: Mounted Kernel Trace File System. client # [ 7.745467] systemd-journald[269]: Collecting audit messages is enabled. lb # [ 7.735972] systemd-journald[271]: Collecting audit messages is enabled. worker2 # [ 7.745807] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 7.742959] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 7.743596] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 7.778028] systemd[1]: Finished Load Kernel Modules. client # [ 7.761017] systemd[1]: Mounted Huge Pages File System. client # [ 7.762931] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 7.781065] EXT4-fs (vda): re-mounted 33133b99-1616-4b97-8015-09ee09300e84. worker2 # [ 7.755798] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 7.766334] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 7.757231] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.785733] systemd[1]: Starting Firewall... client # [ 7.751014] systemd[1]: Queued start job for default target Multi-User System. lb # [ 7.762778] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.756547] systemd[1]: systemd-journald.service: Deactivated successfully. worker1 # [ 7.793399] systemd-journald[269]: Collecting audit messages is enabled. worker2 # [ 7.767861] systemd[1]: Starting Coldplug All udev Devices... client # [ 7.779659] systemd[1]: Started Journal Service. lb # [ 7.757144] systemd[1]: Queued start job for default target Multi-User System. worker1 # [ 7.800581] systemd[1]: Starting Apply Kernel Variables... lb # [ 7.758797] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 7.776598] systemd[1]: Listening on Journal Log Access Socket. client # [ 7.770071] systemd-modules-load[270]: Module 'atkbd' is built in client # [ 7.771217] systemd-modules-load[270]: Module 'loop' is built in worker2 # [ 7.778970] systemd[1]: Mounted Huge Pages File System. worker2 # [ 7.781289] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 7.806163] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 7.781009] systemd[1]: Started Journal Service. worker1 # [ 7.806781] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 7.783574] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 7.807077] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 7.769889] systemd-modules-load[272]: Module 'atkbd' is built in worker2 # [ 7.786848] systemd[1]: Mounted Kernel Trace File System. client # [ 7.782557] systemd[1]: Mounted Kernel Trace File System. client # [ 7.783426] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 7.790224] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 7.790701] systemd[1]: Mounted FUSE Control File System. lb # [ 7.778507] systemd-modules-load[272]: Module 'loop' is built in client # [ 7.789198] systemd[1]: Finished Load Kernel Modules. worker1 # [ 7.825821] systemd[1]: Starting Load/Save OS Random Seed... worker1 # [ 7.826184] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.789156] systemd[1]: Finished Load Kernel Modules. worker2 # [ 7.810325] systemd[1]: Finished Create List of Static Device Nodes. client # [ 7.806181] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 7.816398] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 7.843766] systemd[1]: Started Journal Service. client # [ 7.809604] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 7.822398] EXT4-fs (vda): re-mounted 753fbc79-914d-4c7c-9ce8-f4b90e9bdb0a. worker1 # [ 7.837332] systemd[1]: Queued start job for default target Multi-User System. worker1 # [ 7.838541] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 7.821181] systemd[1]: Starting Firewall... worker1 # [ 7.850496] systemd-modules-load[270]: Module 'atkbd' is built in lb # [ 7.827781] systemd[1]: Starting Apply Kernel Variables... client # [ 7.859963] EXT4-fs (vda): re-mounted 3e628041-8142-4b90-ac7f-abf3283ad4a7. lb # [ 7.835330] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 7.864738] systemd-modules-load[270]: Module 'loop' is built in lb # [ 7.840336] systemd[1]: Listening on Disk Image Download Service Socket. client # [ 7.848593] systemd-oomd[271]: No swap; memory pressure usage will be degraded worker1 # [ 7.873290] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 7.848431] systemd-oomd[273]: No swap; memory pressure usage will be degraded client # [ 7.862149] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. lb # [ 7.856754] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 7.857744] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 7.871612] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 7.876498] systemd[1]: Mounted Kernel Configuration File System. client # [ 7.877399] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 7.885785] systemd[1]: Finished Load Kernel Modules. lb # [ 7.872961] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 7.876664] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.877945] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. client # [ 7.889798] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 7.896735] systemd[1]: Starting Firewall... client # [ 7.890843] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 7.904196] systemd[1]: Starting Apply Kernel Variables... lb # [ 7.890220] systemd[1]: Mounted FUSE Control File System. client # [ 7.900442] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 7.908932] systemd-journald[272]: Collecting audit messages is enabled. client # [ 7.901315] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.902509] systemd[1]: Mounted FUSE Control File System. worker1 # [ 7.935248] systemd-oomd[271]: No swap; memory pressure usage will be degraded worker2 # [ 7.931256] EXT4-fs (vda): re-mounted afc2ae95-50d3-4000-a010-8c2d52d66043. worker1 # [ 7.948342] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 7.949241] systemd[1]: Reached target First Boot Complete. worker2 # [ 7.944395] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 7.958473] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 7.945037] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 7.945333] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 7.956693] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 7.958816] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.982059] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 7.985053] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 7.978819] systemd[1]: Started Journal Service. client # [ 7.995695] systemd-journald[269]: Received client request to flush runtime journal. lb # [ 7.987527] systemd-journald[271]: Received client request to flush runtime journal. worker1 # [ 8.018519] systemd-journald[269]: Received client request to flush runtime journal. worker2 # [ 7.976424] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.977829] systemd[1]: systemd-journald.service: Deactivated successfully. worker2 # [ 7.993153] systemd-modules-load[273]: Module 'atkbd' is built in worker2 # [ 8.004008] systemd-modules-load[273]: Module 'loop' is built in worker2 # [ 8.012856] systemd[1]: Starting Flush Journal to Persistent Storage... worker1 # [ 8.044880] systemd[1]: Finished Apply Kernel Variables. client # [ 8.032417] systemd[1]: Finished Apply Kernel Variables. client # [ 8.033338] systemd[1]: Finished Load/Save OS Random Seed. client # [ 8.034132] systemd[1]: Reached target First Boot Complete. worker1 # [ 8.053756] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 8.041048] systemd[1]: Finished Flush Journal to Persistent Storage. lb # [ 8.048437] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 8.049737] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 8.057031] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 8.057956] systemd[1]: Reached target First Boot Complete. lb # [ 8.058751] systemd[1]: Finished Apply Kernel Variables. lb # [ 8.059495] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 8.078579] systemd-oomd[275]: No swap; memory pressure usage will be degraded worker2 # [ 8.090926] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 8.105980] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 8.108501] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 8.109480] systemd[1]: Reached target First Boot Complete. worker2 # [ 8.131987] systemd-journald[272]: Received client request to flush runtime journal. worker1 # [ 8.190784] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 8.196339] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 8.200482] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 8.183655] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 8.186368] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 8.192340] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 8.274241] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 8.276368] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 8.270845] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 8.272594] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 8.275525] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 8.319765] systemd-udevd[307]: Using default interface naming scheme 'v261'. lb # [ 8.348877] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 8.353735] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 8.357291] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 8.399068] systemd[1]: Mounting /run/wrappers... worker2 # [ 8.396368] systemd-udevd[312]: Using default interface naming scheme 'v261'. lb # [ 8.400877] systemd[1]: Mounting /run/wrappers... worker1 # [ 8.474219] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 8.492475] systemd[1]: Mounted /run/wrappers. lb # [ 8.473539] systemd[1]: Mounted /run/wrappers. worker1 # [ 8.500246] systemd[1]: Reached target Local File Systems. lb # [ 8.474368] systemd[1]: Reached target Local File Systems. lb # [ 8.481804] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 8.509668] systemd[1]: Listening on Boot Loader Control Service Socket. lb # [ 8.493915] systemd[1]: Starting register-nix-paths.service... worker1 # [ 8.520963] systemd[1]: Starting register-nix-paths.service... lb # [ 8.497818] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 8.498742] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 8.528170] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 8.505125] systemd[1]: Mounting /run/wrappers... lb # [ 8.504734] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 8.532618] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 8.516123] systemd[1]: Starting Create System Files and Directories... worker1 # [ 8.546534] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 8.550749] systemd[1]: Starting Create System Files and Directories... lb # [ 8.535423] systemd-udevd[307]: Using default interface naming scheme 'v261'. worker2 # [ 8.547831] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 8.562062] systemd[1]: Mounted /run/wrappers. worker2 # [ 8.570458] systemd[1]: Reached target Local File Systems. worker2 # [ 8.573760] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 8.579723] systemd[1]: Starting register-nix-paths.service... client # [ 8.589653] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 8.587192] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 8.598875] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 8.601782] systemd[1]: Reached target Preparation for Local File Systems. client # [ 8.605037] systemd[1]: Mounting /run/wrappers... client # [ 8.608530] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 8.604168] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 8.612839] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 8.620127] systemd[1]: Starting Create System Files and Directories... worker1 # [ 8.649221] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 8.656996] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 8.639076] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 8.652112] systemd[1]: Mounted /run/wrappers. client # [ 8.653130] systemd[1]: Reached target Local File Systems. lb # [ 8.649515] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 8.659680] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 8.662527] systemd[1]: Starting register-nix-paths.service... client # [ 8.669717] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 8.670871] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 8.678408] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 8.694348] systemd[1]: Starting Create System Files and Directories... client # [ 8.705078] systemd-udevd[300]: Using default interface naming scheme 'v261'. worker2 # [ 8.716382] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 8.728442] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 8.724996] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 8.771635] systemd[1]: Finished Create System Files and Directories. worker1 # [ 8.780941] systemd[1]: Starting Rebuild Journal Catalog... client # [ 8.764696] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 8.792420] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 8.776075] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 8.770977] systemd[1]: Finished Create System Files and Directories. lb # [ 8.787021] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 8.790575] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker2 # [ 8.818517] systemd[1]: Finished Create System Files and Directories. worker2 # [ 8.829026] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 8.836774] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 8.847505] systemd[1]: Finished Create System Files and Directories. client # [ 8.856267] systemd[1]: Starting Rebuild Journal Catalog... client # [ 8.865746] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 8.873082] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 8.899358] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 8.928090] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker1 # [ 8.961570] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 8.969412] systemd[1]: Starting Update is Completed... worker2 # [ 8.950026] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 8.968213] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 9.004835] systemd[1]: Finished Rebuild Journal Catalog. client # [ 9.013559] systemd[1]: Finished Rebuild Journal Catalog. client # [ 9.022207] systemd[1]: Starting Update is Completed... lb # [ 9.020313] systemd[1]: Starting Update is Completed... worker2 # [ 9.023869] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 9.050260] systemd[1]: Finished Update is Completed. worker2 # [ 9.026159] systemd[1]: Starting Update is Completed... client # [ 9.104750] systemd[1]: Finished Update is Completed. worker2 # [ 9.121636] systemd[1]: Finished Update is Completed. lb # [ 9.136387] systemd[1]: Finished Update is Completed. client # [ 9.284757] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 9.318451] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 9.369865] systemd[1]: Finished Coldplug All udev Devices. lb # [ 9.462039] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 9.504949] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 9.506143] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.510711] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 9.519681] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 9.533951] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 9.528462] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.579012] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 9.582651] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.617868] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. lb # [ 9.724345] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 9.766977] systemd[1]: Finished Firewall. worker2 # [ 9.763304] systemd[1]: Finished Firewall. lb # [ 9.763792] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. client # [ 9.823514] (udev-worker)[353]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 9.829641] (udev-worker)[353]: Network interface NamePolicy= disabled on kernel command line. client # [ 9.834064] (udev-worker)[352]: Network interface NamePolicy= disabled on kernel command line. lb # [ 9.834079] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 9.861304] systemd[1]: Finished Coldplug All udev Devices. lb # [ 9.842247] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.885286] systemd[1]: Finished register-nix-paths.service. lb # [ 9.887960] systemd[1]: Reached target System Initialization. lb # [ 9.889014] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 9.890017] systemd[1]: Started niks3 garbage collection timer. lb # [ 9.890855] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 9.922566] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 9.891809] systemd[1]: Reached target Timer Units. lb # [ 9.900913] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 9.906669] systemd[1]: Listening on niks3 server socket. lb # [ 9.907574] systemd[1]: Listening on Nix Daemon Socket. lb # [ 9.910362] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 9.911583] systemd[1]: Reached target Socket Units. lb # [ 9.916558] systemd[1]: Reached target Basic System. lb # [ 9.917331] systemd[1]: Started backdoor.service. lb # [ 9.924213] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 9.954474] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 9.930292] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.939670] systemd[1]: Starting Post-Boot Actions... lb # [ 9.968571] systemd[1]: Started Reset console on configuration changes. lb # [ 10.003070] systemd[1]: Starting resolvconf update... lb # [ 10.034612] systemd[1]: Started rustfs.service. worker2 # [ 10.048948] systemd[1]: Finished Coldplug All udev Devices. client # [ 10.076268] systemd[1]: Finished register-nix-paths.service. client # [ 10.077243] systemd[1]: Reached target System Initialization. client # [ 10.078063] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 10.071154] systemd[1]: Starting rustfs-bucket.service... client # [ 10.079043] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 10.079981] systemd[1]: Reached target Timer Units. client # [ 10.090599] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 10.093897] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 10.115102] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 10.097271] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 10.100545] systemd[1]: Reached target Socket Units. worker2 # [ 10.094273] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 10.105667] systemd[1]: Reached target Basic System. client # [ 10.108205] systemd[1]: Started backdoor.service. client # [ 10.108979] systemd[1]: Starting Import lastlog data into lastlog2 database... client # [ 10.128706] mousedev: PS/2 mouse device common for all mice client # [ 10.120760] systemd[1]: Starting nftables firewall... worker2 # [ 10.114804] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # connecting to host... lb # [ 10.131570] systemd[1]: Starting D-Bus System Message Bus... client # [ 10.142804] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.164112] systemd[1]: Finished register-nix-paths.service. worker1 # [ 10.168286] systemd[1]: Reached target System Initialization. client # [ 10.150122] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.171462] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 10.150899] nsncd[448]: Sep 22 15:45:22.740 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.178698] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 10.184330] systemd[1]: Reached target Timer Units. worker1 # [ 10.190180] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 10.191199] systemd[1]: Listening on Nix Daemon Socket. lb # [ 10.170966] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 10.177217] systemd[1]: Finished Post-Boot Actions. worker1 # [ 10.192005] systemd[1]: Listening on Nix worker-protocol over gRPC. client # [ 10.198046] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.221329] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 10.201925] (udev-worker)[347]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.231697] systemd[1]: Reached target Socket Units. worker1 # [ 10.235594] systemd[1]: Reached target Basic System. worker1 # [ 10.237231] systemd[1]: Starting Import lastlog data into lastlog2 database... client # [ 10.221884] systemd[1]: Starting resolvconf update... lb # [ 10.214783] (udev-worker)[338]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 10.244671] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.256482] systemd[1]: Starting Post-Boot Actions... lb # [ 10.236346] (udev-worker)[338]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.266886] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.284861] systemd[1]: Starting resolvconf update... client # connecting to host... worker1 # [ 10.293195] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 10.272205] systemd[1]: Finished register-nix-paths.service. worker2 # [ 10.273261] systemd[1]: Reached target System Initialization. worker2 # [ 10.275498] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 10.302398] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 10.277349] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 10.284154] systemd[1]: Reached target Timer Units. worker1 # [ 10.315785] systemd[1]: Started backdoor.service. worker2 # [ 10.290602] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 10.296112] systemd[1]: Reached target Host and Network Name Lookups. client # [ 10.307635] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 10.304253] systemd[1]: Listening on Nix Daemon Socket. client # [ 10.315936] systemd[1]: Finished Post-Boot Actions. lb # [ 10.308985] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.312719] systemd[1]: Listening on Nix worker-protocol over gRPC. client # [ 10.319010] nsncd[393]: Sep 22 15:45:22.910 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.321230] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 10.331255] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.356875] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.331237] systemd[1]: Reached target Socket Units. client: Guest shell says: b'Spawning backdoor root shell...\n' worker2 # [ 10.337008] systemd[1]: Reached target Basic System. worker2 # [ 10.341527] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 10.350238] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.345203] systemd[1]: Starting User Login Management... client # [ 10.368933] systemd[1]: Reached target Host and Network Name Lookups. client # [ 10.372742] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.359093] systemd[1]: Starting Post-Boot Actions... client # [ 10.387911] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 10.396220] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 10.413587] systemd[1]: Starting User Login Management... worker2 # [ 10.367555] systemd[1]: Started Reset console on configuration changes. client: connected to guest root shell worker1 # [ 10.406193] nsncd[487]: Sep 22 15:45:22.996 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client: (connecting took 10.84 seconds) worker1 # [ 10.424090] systemd[1]: Reached target Host and Network Name Lookups. client: (finished: waiting for the VM to finish booting, in 10.84 seconds) worker1 # [ 10.436367] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.378546] systemd[1]: Starting resolvconf update... worker2 # [ 10.389312] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 10.441311] systemd[1]: Starting User Login Management... lb # [ 10.424304] systemd[1]: Finished Import lastlog data into lastlog2 database. client # [ 10.442370] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 10.473086] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 10.451115] systemd[1]: Finished Post-Boot Actions. worker1 # connecting to host... worker1 # [ 10.488928] dbus-broker-launch[491]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 10.503462] dbus-broker-launch[491]: NSS returned no entry for 'systemd-timesync' worker2 # [ 10.477288] nsncd[489]: Sep 22 15:45:23.073 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.479486] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.510651] dbus-broker-launch[491]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 10.502861] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.530485] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 10.507583] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.515437] systemd[1]: Starting User Login Management... worker2 # [ 10.526467] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 10.549790] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 10.584319] dbus-broker-launch[491]: Ready lb # [ 10.562343] dbus-broker-launch[462]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.564915] dbus-broker-launch[493]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.584138] dbus-broker-launch[493]: NSS returned no entry for 'systemd-timesync' worker2 # [ 10.588564] dbus-broker-launch[493]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 10.599006] dbus-broker-launch[462]: NSS returned no entry for 'systemd-timesync' worker2 # [ 10.606749] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. lb # [ 10.605878] dbus-broker-launch[462]: Invalid user-name in /nix/store/4yiprls1sll47jid3xzq0m9nmi1qz2ww-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 10.615902] systemd[1]: Started backdoor.service. worker2 # [ 10.622427] systemd[1]: Started D-Bus System Message Bus. worker1 # [ 10.652639] systemd-logind[505]: New seat seat0. worker1 # [ 10.657818] systemd[1]: Started User Login Management. worker1 # [ 10.668305] systemd[1]: Starting linger-users.service... worker1 # [ 10.686581] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 10.687724] systemd[1]: Stopping Host and Network Name Lookups... worker1 # [ 10.700507] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 10.701451] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 10.702251] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 10.703130] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 10.703926] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 10.675406] systemd-logind[491]: New seat seat0. lb # [ 10.682272] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 10.686482] dbus-broker-launch[493]: Ready lb # [ 10.692888] systemd[1]: Started User Login Management. worker1 # [ 10.719566] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.725492] (udev-worker)[343]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 10.727672] (udev-worker)[343]: Network interface NamePolicy= disabled on kernel command line. lb # [ 10.702669] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 10.705813] systemd-logind[422]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) client # [ 10.714069] systemd-logind[422]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 10.715601] dbus-broker-launch[420]: Looking up NSS user entry for 'systemd-timesync'... lb # [ 10.712941] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 10.713923] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 10.714785] systemd[1]: Stopping User and Group Name Lookups... client # [ 10.736133] systemd-logind[422]: New seat seat0. client # [ 10.736906] systemd[1]: Started User Login Management. lb # [ 10.731380] systemd[1]: Starting linger-users.service... client # [ 10.744537] systemd[1]: Starting linger-users.service... lb # [ 10.737344] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 10.745433] dbus-broker-launch[420]: NSS returned no entry for 'systemd-timesync' worker2 # [ 10.741857] systemd-logind[502]: New seat seat0. lb # [ 10.746661] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 10.750793] systemd[1]: Started User Login Management. lb # [ 10.751912] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # connecting to host... worker2 # [ 10.757547] systemd[1]: Starting linger-users.service... client # [ 10.746393] dbus-broker-launch[420]: Invalid user-name in /nix/store/6wvgzhp3xj1v1xybf1ixpbqi5gif5fbv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 10.788634] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 10.793547] systemd[1]: Finished linger-users.service. worker1 # [ 10.804487] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 10.781922] dbus-broker-launch[462]: Ready worker1 # [ 10.810559] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.811907] nsncd[559]: Sep 22 15:45:23.409 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.821250] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.823234] (udev-worker)[435]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 10.801058] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 10.808142] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 10.809740] systemd[1]: Stopping Host and Network Name Lookups... worker2 # [ 10.810694] systemd[1]: Stopped target User and Group Name Lookups. worker2 # [ 10.811557] systemd[1]: Stopping User and Group Name Lookups... client # [ 10.825236] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 10.844636] systemd[1]: Finished resolvconf update. client # [ 10.826237] systemd[1]: Stopping Host and Network Name Lookups... worker1 # [ 10.845584] systemd[1]: Reached target Preparation for Network. client # [ 10.827070] systemd[1]: Stopped target User and Group Name Lookups. client # [ 10.827930] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 10.855354] systemd[1]: Starting DHCP Client... lb # [ 10.834619] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.861542] systemd[1]: Starting Extra networking commands.... client # [ 10.843785] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 10.840659] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 10.848325] systemd[1]: nscd.service: Deactivated successfully. client # [ 10.849212] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 10.843911] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 10.846081] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 10.847001] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 10.876290] dbus-broker-launch[420]: Ready worker2 # [ 10.876547] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 10.877600] systemd[1]: Finished linger-users.service. client # [ 10.885217] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 10.886222] systemd[1]: linger-users.service: Deactivated successfully. client # [ 10.894186] systemd[1]: Finished linger-users.service. worker2 # [ 10.924789] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.926130] nsncd[563]: Sep 22 15:45:23.524 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 10.926566] systemd[1]: linger-users.service: Deactivated successfully. lb # [ 10.934564] systemd[1]: Finished linger-users.service. worker2 # [ 10.935772] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 10.940321] systemd[1]: Finished resolvconf update. worker2 # [ 10.938783] systemd[1]: Reached target User and Group Name Lookups. lb # [ 10.957688] systemd[1]: Starting DHCP Client... worker2 # [ 10.960638] systemd[1]: Finished resolvconf update. worker2 # [ 10.961470] systemd[1]: Reached target Preparation for Network. worker2 # [ 10.969069] systemd[1]: Starting DHCP Client... worker2 # [ 10.973614] systemd[1]: Starting Extra networking commands.... client # [ 10.987902] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 10.992678] systemd[1]: Reached target Host and Network Name Lookups. client # [ 10.994055] nsncd[487]: Sep 22 15:45:23.580 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 10.990722] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 10.991744] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 10.999290] systemd[1]: Reached target User and Group Name Lookups. client # [ 11.006648] systemd[1]: Reached target User and Group Name Lookups. client # [ 11.009366] systemd[1]: Finished resolvconf update. lb # [ 11.005833] nsncd[569]: Sep 22 15:45:23.585 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 11.007897] systemd-logind[491]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.020353] systemd[1]: Starting DHCP Client... worker2 # [ 11.032150] (udev-worker)[351]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 11.072362] (udev-worker)[352]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 11.074643] (udev-worker)[352]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 11.132550] dhcpcd[596]: dhcpcd-10.3.2 starting worker1 # [ 11.144433] dhcpcd[636]: dev: loaded udev lb # [ 11.133326] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 11.199199] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 11.231624] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 11.244565] systemd[1]: Starting Address configuration of eth1... worker1 # [ 11.245587] systemd[1]: Finished Extra networking commands.. worker1 # [ 11.246442] systemd[1]: Reached target Network. client # [ 11.234044] dhcpcd[521]: dhcpcd-10.3.2 starting client # [ 11.245924] dhcpcd[528]: dev: loaded udev worker2 # [ 11.240835] dhcpcd[596]: dhcpcd-10.3.2 starting worker1 # [ 11.282548] systemd[1]: Starting Permit User Sessions... worker2 # [ 11.262407] dhcpcd[636]: dev: loaded udev lb # [ 11.279842] mousedev: PS/2 mouse device common for all mice client # [ 11.300201] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 11.320677] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker2 # [ 11.316247] 8021q: 802.1Q VLAN Support v1.8 lb # [ 11.302458] dhcpcd[605]: dhcpcd-10.3.2 starting worker1 # [ 11.357539] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 11.358083] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker1 # [ 11.361139] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 11.361504] cfg80211: failed to load regulatory.db lb # [ 11.319936] dhcpcd[622]: dev: loaded udev worker2 # [ 11.335238] systemd[1]: Finished Extra networking commands.. worker2 # [ 11.342042] systemd[1]: Reached target Network. worker2 # [ 11.342794] systemd[1]: Starting Permit User Sessions... worker1 # [ 11.370796] systemd[1]: Finished Permit User Sessions. worker1 # [ 11.378056] systemd[1]: Started Getty on tty1. worker1 # [ 11.378815] systemd[1]: Reached target Login Prompts. lb # [ 11.389023] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 11.438501] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 11.425963] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 11.448538] network-addresses-eth1-start[648]: adding address 192.168.1.3/24... done worker2 # [ 11.444094] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 11.467763] network-addresses-eth1-start[648]: adding address 2001:db8:1::3/64... done worker2 # [ 11.450570] systemd[1]: Finished Permit User Sessions. worker1 # [ 11.496145] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 11.480825] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 11.482734] dhcpcd[636]: eth0: waiting for carrier client # [ 11.481332] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 11.458235] systemd[1]: Started Getty on tty1. worker2 # [ 11.458991] systemd[1]: Reached target Login Prompts. worker1 # [ 11.486969] dhcpcd[636]: eth0: waiting for carrier client # [ 11.485450] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 11.485791] cfg80211: failed to load regulatory.db worker1 # [ 11.490815] dhcpcd[636]: eth0: carrier acquired worker1 # [ 11.498739] systemd[1]: Finished Address configuration of eth1. worker2 # [ 11.474103] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 11.484282] systemd[1]: Finished nftables firewall. worker2 # [ 11.479469] systemd[1]: Starting Address configuration of eth1... client # [ 11.488714] systemd[1]: Reached target Preparation for Network. worker1 # [ 11.510023] dhcpcd[636]: DUID 00:01:00:01:32:45:5f:14:52:54:00:12:34:56 worker1 # [ 11.511015] dhcpcd[636]: eth0: IAID 00:12:34:56 worker1 # [ 11.511662] dhcpcd[636]: eth0: adding address fe80::5054:ff:fe12:3456 worker2 # [ 11.505122] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 11.505685] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 11.500232] systemd[1]: Starting Address configuration of eth1... worker2 # [ 11.509530] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 11.509865] cfg80211: failed to load regulatory.db client # [ 11.508242] systemd[1]: Starting Extra networking commands.... worker1 # [ 11.537985] systemd-logind[505]: Watching system buttons on /dev/input/event0 (gpio-keys) lb # [ 11.511982] systemd-logind[491]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.543586] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker2 # [ 11.601995] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 11.615439] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 11.601873] dhcpcd[528]: eth0: waiting for carrier lb # [ 11.596945] systemd[1]: Finished Firewall. client # [ 11.604964] dhcpcd[528]: libudev: received NULL device worker2 # [ 11.612349] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 11.606130] dhcpcd[528]: libudev: received NULL device client # [ 11.607015] dhcpcd[528]: eth0: carrier acquired worker2 # [ 11.600642] dhcpcd[636]: eth0: waiting for carrier lb # [ 11.618457] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 11.619079] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 11.608607] dhcpcd[636]: eth0: waiting for carrier lb # [ 11.607723] systemd[1]: Reached target Preparation for Network. worker2 # [ 11.610894] dhcpcd[636]: eth0: carrier acquired worker2 # [ 11.613598] network-addresses-eth1-start[658]: adding address 192.168.1.4/24... done lb # [ 11.627705] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 11.628038] cfg80211: failed to load regulatory.db worker1 # [ 11.662841] mousedev: PS/2 mouse device common for all mice client # [ 11.629581] dhcpcd[528]: DUID 00:01:00:01:32:45:5f:14:52:54:00:12:34:56 client # [ 11.630588] dhcpcd[528]: eth0: IAID 00:12:34:56 client # [ 11.631226] dhcpcd[528]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 11.624300] systemd[1]: Starting Address configuration of eth1... worker2 # [ 11.631832] dhcpcd[636]: DUID 00:01:00:01:32:45:5f:14:52:54:00:12:34:56 worker2 # [ 11.633527] network-addresses-eth1-start[658]: adding address 2001:db8:1::4/64... done worker2 # [ 11.634648] dhcpcd[636]: eth0: IAID 00:12:34:56 worker2 # [ 11.635307] dhcpcd[636]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 11.662857] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 11.647769] systemd[1]: Starting Extra networking commands.... worker2 # [ 11.651220] systemd[1]: Finished Address configuration of eth1. worker1 # [ 11.684436] systemd-logind[505]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) client # [ 11.671050] network-addresses-eth1-start[544]: adding address 192.168.1.1/24... done worker2 # [ 11.677083] dhcpcd[636]: eth0: soliciting a DHCP lease worker2 # [ 11.685172] dhcpcd[636]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 11.695652] network-addresses-eth1-start[544]: adding address 2001:db8:1::1/64... done worker2 # [ 11.692877] dhcpcd[636]: eth0: probing address 10.0.2.15/24 worker2 # [ 11.712152] systemd-logind[502]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.726507] systemd[1]: Finished Address configuration of eth1. worker2 # [ 11.804232] mousedev: PS/2 mouse device common for all mice lb # [ 11.815036] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 11.804899] dhcpcd[622]: eth0: waiting for carrier lb # [ 11.811543] dhcpcd[622]: libudev: received NULL device lb # [ 11.813712] dhcpcd[622]: libudev: received NULL device lb # [ 11.818871] dhcpcd[622]: eth0: carrier acquired client # [ 11.832002] systemd[1]: Finished Extra networking commands.. client # [ 11.835937] systemd[1]: Reached target Network. client # [ 11.843727] systemd[1]: Starting Permit User Sessions... worker2 # [ 11.836822] systemd-logind[502]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.852285] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 11.843989] dhcpcd[622]: DUID 00:01:00:01:32:45:5f:14:52:54:00:12:34:56 lb # [ 11.848094] dhcpcd[622]: eth0: IAID 00:12:34:56 lb # [ 11.852225] dhcpcd[622]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 11.878714] network-addresses-eth1-start[641]: adding address 192.168.1.2/24... done client # [ 11.922689] systemd[1]: Finished Permit User Sessions. lb # [ 11.915104] network-addresses-eth1-start[641]: adding address 2001:db8:1::2/64... done client: (finished: waiting for unit network-addresses-eth1.service, in 12.36 seconds) client # [ 11.931472] systemd[1]: Started Getty on tty1. client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } client # [ 11.935658] systemd[1]: Reached target Login Prompts. lb # [ 11.953954] systemd[1]: Finished Address configuration of eth1. client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.06 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb # [ 12.119461] systemd[1]: Finished Extra networking commands.. lb # [ 12.126923] systemd[1]: Reached target Network. lb: (finished: waiting for unit network-addresses-eth1.service, in 0.14 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } lb # [ 12.136720] systemd[1]: Starting PostgreSQL Server... lb # [ 12.150171] systemd[1]: Starting Permit User Sessions... lb # [ 12.227327] systemd[1]: Finished Permit User Sessions. lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.10 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb # [ 12.240309] systemd[1]: Started Getty on tty1. lb # [ 12.243532] systemd[1]: Reached target Login Prompts. worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } client # [ 12.372306] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.05 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } lb # [ 12.472895] postgresql-pre-start[735]: The files belonging to this database system will be owned by user "postgres". lb # [ 12.474344] postgresql-pre-start[735]: This user must also own the server process. worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.05 seconds) lb: waiting for unit niks3.service lb # [ 12.491435] postgresql-pre-start[735]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 12.495828] postgresql-pre-start[735]: The default database encoding has accordingly been set to "UTF8". lb # [ 12.501285] postgresql-pre-start[735]: The default text search configuration will be set to "english". lb # [ 12.502550] postgresql-pre-start[735]: Data page checksums are enabled. lb # [ 12.503425] postgresql-pre-start[735]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 12.516746] postgresql-pre-start[735]: creating subdirectories ... ok lb # [ 12.517658] postgresql-pre-start[735]: selecting dynamic shared memory implementation ... posix worker1 # [ 12.564460] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 12.667893] postgresql-pre-start[735]: selecting default "max_connections" ... 100 client # [ 12.690524] systemd[1]: Starting Virtual Console Setup... worker2 # [ 12.706487] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client # [ 12.744427] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 12.749169] systemd[1]: Stopped Virtual Console Setup. client # [ 12.756349] systemd[1]: Starting Virtual Console Setup... client # [ 12.781662] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 12.822996] postgresql-pre-start[735]: selecting default "shared_buffers" ... 128MB client # [ 12.847243] systemd-logind[422]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 12.875571] dhcpcd[528]: eth0: soliciting a DHCP lease client # [ 12.884555] dhcpcd[528]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 12.888201] dhcpcd[528]: eth0: probing address 10.0.2.15/24 worker1 # [ 12.915191] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1 # [ 12.932234] systemd[1]: Starting Virtual Console Setup... worker1 # [ 12.953683] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 12.935051] systemd-vconsole-setup[638]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 12.939143] systemd[1]: Finished Virtual Console Setup. worker1 # [ 12.962064] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 12.962884] systemd[1]: Starting Virtual Console Setup... worker1 # [ 13.039491] systemd-logind[505]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 13.065910] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 13.076511] systemd[1]: Starting Virtual Console Setup... worker1 # [ 13.120559] systemd-vconsole-setup[700]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 13.126035] systemd[1]: Finished Virtual Console Setup. worker2 # [ 13.100132] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 13.106655] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 13.110483] systemd[1]: Starting Virtual Console Setup... client # [ 13.188567] dhcpcd[528]: eth0: soliciting an IPv6 router client # [ 13.191700] dhcpcd[528]: eth0: Router Advertisement from fe80::2 client # [ 13.193629] dhcpcd[528]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 13.195545] dhcpcd[528]: eth0: adding route to fec0::/64 client # [ 13.197090] dhcpcd[528]: eth0: adding default route via fe80::2 worker2 # [ 13.196478] systemd-logind[502]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 13.278413] systemd-vconsole-setup[704]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 13.283661] systemd[1]: Finished Virtual Console Setup. lb # [ 13.317200] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker1 # [ 13.337694] dhcpcd[636]: eth0: soliciting an IPv6 router worker1 # [ 13.340916] dhcpcd[636]: eth0: Router Advertisement from fe80::2 worker1 # [ 13.343541] dhcpcd[636]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 13.346531] dhcpcd[636]: eth0: adding route to fec0::/64 worker1 # [ 13.349445] dhcpcd[636]: eth0: adding default route via fe80::2 worker1 # [ 13.430515] dhcpcd[636]: eth0: soliciting a DHCP lease worker1 # [ 13.434485] dhcpcd[636]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 13.437865] dhcpcd[636]: eth0: probing address 10.0.2.15/24 lb # [ 13.550303] dhcpcd[622]: eth0: soliciting a DHCP lease lb # [ 13.556681] dhcpcd[622]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 13.560239] dhcpcd[622]: eth0: probing address 10.0.2.15/24 lb # [ 13.711023] systemd[1]: Starting Virtual Console Setup... worker2 # [ 13.710808] dhcpcd[636]: eth0: soliciting an IPv6 router worker2 # [ 13.714738] dhcpcd[636]: eth0: Router Advertisement from fe80::2 worker2 # [ 13.717618] dhcpcd[636]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 13.720529] dhcpcd[636]: eth0: adding route to fec0::/64 worker2 # [ 13.722970] dhcpcd[636]: eth0: adding default route via fe80::2 lb # [ 13.775269] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 13.798000] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 13.806960] systemd[1]: Stopped Virtual Console Setup. lb # [ 13.826533] systemd[1]: Starting Virtual Console Setup... lb # [ 13.898339] systemd-logind[491]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 14.119618] systemd-vconsole-setup[761]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 14.124320] systemd[1]: Finished Virtual Console Setup. lb # [ 14.175160] dhcpcd[622]: eth0: soliciting an IPv6 router lb # [ 14.177030] dhcpcd[622]: eth0: Router Advertisement from fe80::2 lb # [ 14.177869] dhcpcd[622]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 14.178767] dhcpcd[622]: eth0: adding route to fec0::/64 lb # [ 14.179505] dhcpcd[622]: eth0: adding default route via fe80::2 lb # [ 14.563179] postgresql-pre-start[735]: selecting default time zone ... UTC lb # [ 14.566826] postgresql-pre-start[735]: creating configuration files ... ok lb # [ 14.857237] postgresql-pre-start[735]: running bootstrap script ... ok lb # [ 15.607077] postgresql-pre-start[735]: performing post-bootstrap initialization ... ok lb # [ 15.779320] postgresql-pre-start[735]: syncing data to disk ... ok lb # [ 15.781906] postgresql-pre-start[735]: initdb: warning: enabling "trust" authentication for local connections lb # [ 15.783218] postgresql-pre-start[735]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 15.785464] postgresql-pre-start[735]: Success. You can now start the database server using: lb # [ 15.786678] postgresql-pre-start[735]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 15.892378] postgres[791]: [791] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 15.895805] postgres[791]: [791] LOG: listening on IPv6 address "::1", port 5432 lb # [ 15.897278] postgres[791]: [791] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 15.899177] postgres[791]: [791] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 15.914449] postgres[800]: [800] LOG: database system was shut down at 2026-09-22 15:45:28 GMT lb # [ 15.919943] postgres[791]: [791] LOG: database system is ready to accept connections lb # [ 15.926170] systemd[1]: Started PostgreSQL Server. lb # [ 15.935681] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 16.169878] postgresql-setup-start[811]: CREATE DATABASE lb # [ 16.252467] postgresql-setup-start[824]: CREATE ROLE lb # [ 16.273159] postgresql-setup-start[827]: ALTER DATABASE lb # [ 16.279994] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 16.282073] systemd[1]: Reached target PostgreSQL. worker2 # [ 16.505766] dhcpcd[636]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 16.512855] dhcpcd[636]: eth0: adding route to 10.0.2.0/24 worker2 # [ 16.515299] dhcpcd[636]: eth0: adding default route via 10.0.2.2 worker2 # [ 16.720947] systemd[1]: Started DHCP Client. worker2 # [ 16.723491] systemd[1]: Reached target Multi-User System. worker2 # [ 16.725100] systemd[1]: Startup finished in 970ms (kernel) + 5.029s (initrd) + 10.723s (userspace) = 16.723s. client # [ 17.478066] dhcpcd[528]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 17.483487] dhcpcd[528]: eth0: adding route to 10.0.2.0/24 client # [ 17.489370] dhcpcd[528]: eth0: adding default route via 10.0.2.2 client # [ 17.657095] systemd[1]: Started DHCP Client. client # [ 17.660362] systemd[1]: Reached target Multi-User System. client # [ 17.662012] systemd[1]: Startup finished in 971ms (kernel) + 5.017s (initrd) + 11.673s (userspace) = 17.661s. worker1 # [ 17.753584] dhcpcd[636]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 17.759188] dhcpcd[636]: eth0: adding route to 10.0.2.0/24 worker1 # [ 17.765630] dhcpcd[636]: eth0: adding default route via 10.0.2.2 lb # [ 17.840163] dhcpcd[622]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 17.841753] dhcpcd[622]: eth0: adding route to 10.0.2.0/24 lb # [ 17.842751] dhcpcd[622]: eth0: adding default route via 10.0.2.2 worker1 # [ 17.947940] systemd[1]: Started DHCP Client. worker1 # [ 17.950642] systemd[1]: Reached target Multi-User System. worker1 # [ 17.951512] systemd[1]: Startup finished in 964ms (kernel) + 4.970s (initrd) + 12.014s (userspace) = 17.949s. lb # [ 17.937756] systemd[1]: Started DHCP Client. lb # [ 17.940981] systemd[1]: Reached target Network is Online. lb # [ 17.946327] systemd[1]: Started Envoy reverse proxy. lb # [ 17.956799] systemd[1]: Started mock-oidc.service. lb # [ 18.126635] mock-oidc-server[895]: Mock OIDC Server running lb # [ 18.130482] mock-oidc-server[895]: OIDC Address: 192.168.1.2:8080 lb # [ 18.132177] mock-oidc-server[895]: Issue Address: 0.0.0.0:8081 lb # [ 18.133010] mock-oidc-server[895]: Issuer: http://192.168.1.2:8080/oidc lb # [ 18.133909] mock-oidc-server[895]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 18.135041] mock-oidc-server[895]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 18.142437] mock-oidc-server[895]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 18.393834] envoy[894]: [2026-09-22 15:45:30.983][894][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120) lb # [ 18.396676] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 18.398162] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 18.399969] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 18.402754] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 18.408699] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 18.439058] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 18.443278] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 18.445887] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 18.449003] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 18.451144] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 18.455406] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 18.458972] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 18.461862] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 18.468299] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 18.470102] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 18.480313] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 18.482056] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 18.490706] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 18.493421] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 18.498537] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 18.502023] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 18.504541] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 18.509761] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 18.514469] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 18.517073] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 18.518973] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 18.523733] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 18.527642] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 18.530323] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 18.533113] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 18.538100] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 18.541733] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme worker1 # [ 18.569791] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 18.543261] systemd[1]: Reached target Network is Online. lb # [ 18.546925] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 18.553171] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift worker2 # [ 18.556550] systemd[1]: Starting scheduler-token.service... lb # [ 18.557614] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 18.559935] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 18.562024] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 18.569207] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 18.571178] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 18.573220] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 18.574861] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 18.577766] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 18.608519] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 18.616834] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 18.621446] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 18.624875] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 18.626741] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header worker2 # [ 18.628987] systemd[1]: scheduler-token.service: Deactivated successfully. lb # [ 18.629716] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 18.632171] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple worker2 # [ 18.636405] systemd[1]: Finished scheduler-token.service. lb # [ 18.634506] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory worker2 # [ 18.637804] systemd[1]: scheduler-token.service: Consumed 45ms CPU time over 75ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb # [ 18.639968] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default worker2 # [ 18.642541] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 18.642209] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 18.643868] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 18.645547] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 18.647294] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 18.653655] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 18.660348] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 18.667550] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 18.669394] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 18.671888] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 18.683158] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 18.685237] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 18.687384] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 18.689295] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 18.692585] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 18.695024] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 18.701001] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 18.706875] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality worker1 # [ 18.739541] nix-grpc-daemon[765]: ts=2026-09-22T15:45:31Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" lb # [ 18.720249] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 18.722026] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid worker1 # [ 18.748875] nix-grpc-daemon[765]: ts=2026-09-22T15:45:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc lb # [ 18.723594] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 worker1 # [ 18.751319] nix-grpc-daemon[765]: ts=2026-09-22T15:45:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 lb # [ 18.729321] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 18.731564] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 18.733899] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 18.735835] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain worker1 # [ 18.765046] nix-grpc-daemon[765]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 18.738123] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe worker1 # [ 18.768223] nix-grpc-daemon[765]: E0000 00:00:1790091931.367444 765 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 18.742523] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 18.745840] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router worker1 # [ 18.773693] nix-grpc-daemon[765]: E0000 00:00:1790091931.376260 765 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.747576] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 18.750539] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 18.753388] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd worker1 # [ 18.792451] nix-grpc-daemon[765]: ts=2026-09-22T15:45:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 18.795387] systemd[1]: Started Nix worker-protocol over gRPC. lb # [ 18.758948] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy worker1 # [ 18.800126] nix-grpc-daemon[765]: E0000 00:00:1790091931.399668 807 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 18.776792] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port worker2 # [ 18.780785] nix-grpc-daemon[772]: ts=2026-09-22T15:45:31Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" lb # [ 18.780644] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 18.782289] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 18.785457] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 18.788234] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default worker2 # [ 18.789650] nix-grpc-daemon[772]: ts=2026-09-22T15:45:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc lb # [ 18.790703] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default worker2 # [ 18.791377] nix-grpc-daemon[772]: ts=2026-09-22T15:45:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 lb # [ 18.793331] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 18.795823] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 18.804360] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name worker1 # [ 18.832070] nix-grpc-daemon[765]: E0000 00:00:1790091931.426107 807 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.806620] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default worker2 # [ 18.806801] nix-grpc-daemon[772]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 18.808520] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default worker2 # [ 18.809836] nix-grpc-daemon[772]: E0000 00:00:1790091931.405705 772 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 18.810316] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 18.812730] envoy[894]: [2026-09-22 15:45:30.986][894][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 18.814495] envoy[894]: [2026-09-22 15:45:30.987][894][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder worker2 # [ 18.815110] nix-grpc-daemon[772]: E0000 00:00:1790091931.414179 772 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 18.834437] nix-grpc-daemon[765]: E0000 00:00:1790091931.426610 807 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 18.845565] nix-grpc-daemon[765]: E0000 00:00:1790091931.446931 807 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.818898] envoy[894]: [2026-09-22 15:45:30.987][894][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 18.825651] envoy[894]: [2026-09-22 15:45:30.987][894][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret worker1 # [ 18.859122] nix-grpc-daemon[765]: ts=2026-09-22T15:45:31Z level=info event=scheduler_connected addr=lb:50051 lb # [ 18.832104] envoy[894]: [2026-09-22 15:45:30.987][894][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query worker2 # [ 18.836146] nix-grpc-daemon[772]: ts=2026-09-22T15:45:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 18.838708] systemd[1]: Started Nix worker-protocol over gRPC. lb # [ 18.835509] envoy[894]: [2026-09-22 15:45:30.987][894][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher worker1 # [ 18.867452] nix-grpc-daemon[765]: ts=2026-09-22T15:45:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 18.843436] envoy[894]: [2026-09-22 15:45:30.987][894][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 18.850525] envoy[894]: [2026-09-22 15:45:30.987][894][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 18.853189] envoy[894]: [2026-09-22 15:45:30.987][894][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 18.857424] envoy[894]: [2026-09-22 15:45:30.987][894][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case worker2 # [ 18.842053] nix-grpc-daemon[772]: E0000 00:00:1790091931.439909 813 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 18.859681] envoy[894]: [2026-09-22 15:45:30.987][894][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 18.865281] envoy[894]: [2026-09-22 15:45:31.010][894][info][main] [source/server/server.cc:531] HTTP header map info: worker2 # [ 18.872169] nix-grpc-daemon[772]: E0000 00:00:1790091931.461700 813 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.866606] envoy[894]: [2026-09-22 15:45:31.010][894][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 18.882194] envoy[894]: [2026-09-22 15:45:31.010][894][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: worker2 # [ 18.882262] nix-grpc-daemon[772]: E0000 00:00:1790091931.462225 813 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.885314] nix-grpc-daemon[772]: E0000 00:00:1790091931.480674 813 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.883666] envoy[894]: [2026-09-22 15:45:31.010][894][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 18.894836] envoy[894]: [2026-09-22 15:45:31.010][894][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 18.898365] envoy[894]: [2026-09-22 15:45:31.039][894][info][main] [source/server/server.cc:968] runtime: {} lb # [ 18.900404] envoy[894]: [2026-09-22 15:45:31.045][894][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. worker2 # [ 18.901265] nix-grpc-daemon[772]: ts=2026-09-22T15:45:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 18.902073] envoy[894]: [2026-09-22 15:45:31.045][894][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 18.903652] envoy[894]: [2026-09-22 15:45:31.045][894][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 lb # [ 18.905336] envoy[894]: [2026-09-22 15:45:31.045][894][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 18.906903] envoy[894]: [2026-09-22 15:45:31.045][894][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 18.908570] envoy[894]: [2026-09-22 15:45:31.045][894][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) lb # [ 18.910102] envoy[894]: [2026-09-22 15:45:31.127][894][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) lb # [ 18.911594] envoy[894]: [2026-09-22 15:45:31.143][894][info][config] [source/server/configuration_impl.cc:164] loading stats configuration lb # [ 18.913339] envoy[894]: [2026-09-22 15:45:31.147][894][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 18.917172] envoy[894]: [2026-09-22 15:45:31.147][894][info][main] [source/server/server.cc:1077] starting main dispatch loop lb # [ 18.918596] envoy[894]: [2026-09-22 15:45:31.383][894][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization lb # [ 18.920285] envoy[894]: [2026-09-22 15:45:31.383][894][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized lb # [ 18.922026] envoy[894]: [2026-09-22 15:45:31.383][894][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager lb # [ 18.923690] envoy[894]: [2026-09-22 15:45:31.408][894][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers worker1 # [ 19.068579] nix-grpc-daemon[765]: ts=2026-09-22T15:45:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.071936] nix-grpc-daemon[765]: ts=2026-09-22T15:45:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.102760] nix-grpc-daemon[772]: ts=2026-09-22T15:45:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.106406] nix-grpc-daemon[772]: ts=2026-09-22T15:45:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.273142] nix-grpc-daemon[765]: ts=2026-09-22T15:45:31Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.278776] nix-grpc-daemon[765]: ts=2026-09-22T15:45:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.307670] nix-grpc-daemon[772]: ts=2026-09-22T15:45:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.312990] nix-grpc-daemon[772]: ts=2026-09-22T15:45:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.479667] nix-grpc-daemon[765]: ts=2026-09-22T15:45:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.485297] nix-grpc-daemon[765]: ts=2026-09-22T15:45:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.514570] nix-grpc-daemon[772]: ts=2026-09-22T15:45:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.520108] nix-grpc-daemon[772]: ts=2026-09-22T15:45:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.685759] nix-grpc-daemon[765]: ts=2026-09-22T15:45:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.688923] nix-grpc-daemon[765]: ts=2026-09-22T15:45:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.721047] nix-grpc-daemon[772]: ts=2026-09-22T15:45:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.724351] nix-grpc-daemon[772]: ts=2026-09-22T15:45:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.891992] nix-grpc-daemon[765]: ts=2026-09-22T15:45:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.896500] nix-grpc-daemon[765]: ts=2026-09-22T15:45:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.927621] nix-grpc-daemon[772]: ts=2026-09-22T15:45:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.932222] nix-grpc-daemon[772]: ts=2026-09-22T15:45:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.092879] nix-grpc-daemon[765]: ts=2026-09-22T15:45:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.098379] nix-grpc-daemon[765]: ts=2026-09-22T15:45:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.129309] nix-grpc-daemon[772]: ts=2026-09-22T15:45:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.134865] nix-grpc-daemon[772]: ts=2026-09-22T15:45:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.299348] nix-grpc-daemon[765]: ts=2026-09-22T15:45:32Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.305925] nix-grpc-daemon[765]: ts=2026-09-22T15:45:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.336408] nix-grpc-daemon[772]: ts=2026-09-22T15:45:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.341824] nix-grpc-daemon[772]: ts=2026-09-22T15:45:32Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.506410] nix-grpc-daemon[765]: ts=2026-09-22T15:45:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.512260] nix-grpc-daemon[765]: ts=2026-09-22T15:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.543286] nix-grpc-daemon[772]: ts=2026-09-22T15:45:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.549055] nix-grpc-daemon[772]: ts=2026-09-22T15:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 20.559341] rustfs-bucket-start[930]: mb s3://farm lb # [ 20.568754] systemd[1]: Finished rustfs-bucket.service. lb # [ 20.575903] systemd[1]: Starting niks3 server... worker1 # [ 20.713400] nix-grpc-daemon[765]: ts=2026-09-22T15:45:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.718896] nix-grpc-daemon[765]: ts=2026-09-22T15:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 20.715640] postgres[942]: [942] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 20.720213] postgres[942]: [942] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC lb # [ 20.741595] niks3-server[937]: 2026/09/22 15:45:33 OK 20241026095416_initial_model.sql (14.58ms) lb # [ 20.744868] niks3-server[937]: 2026/09/22 15:45:33 OK 20251210153512_drop_unused_gin_index.sql (3.57ms) lb # [ 20.750916] niks3-server[937]: 2026/09/22 15:45:33 OK 20251218171726_add_pins.sql (5.98ms) worker2 # [ 20.750352] nix-grpc-daemon[772]: ts=2026-09-22T15:45:33Z level=info event=scheduler_connected addr=lb:50051 lb # [ 20.755457] niks3-server[937]: 2026/09/22 15:45:33 OK 20260628120000_add_object_size_and_stats.sql (4.51ms) worker2 # [ 20.755816] nix-grpc-daemon[772]: ts=2026-09-22T15:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 20.760599] niks3-server[937]: 2026/09/22 15:45:33 OK 20260905000000_add_claims.sql (5.1ms) lb # [ 20.764388] niks3-server[937]: 2026/09/22 15:45:33 OK 20260920000000_drop_claims.sql (3.69ms) lb # [ 20.766690] niks3-server[937]: 2026/09/22 15:45:33 goose: successfully migrated database to version: 20260920000000 lb # [ 20.769812] niks3-server[937]: 2026/09/22 15:45:33 OK 1_commit_pending_closure.sql (5.39ms) lb # [ 20.772316] niks3-server[937]: 2026/09/22 15:45:33 OK 2_object_stats_trigger.sql (2.36ms) lb # [ 20.774219] niks3-server[937]: 2026/09/22 15:45:33 goose: up to current file version: 2 lb # [ 20.776002] niks3-server[937]: 2026/09/22 15:45:33 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 20.799071] niks3-server[937]: 2026/09/22 15:45:33 INFO Read proxy enabled — serving cache objects from S3 lb # [ 20.803612] niks3-server[937]: 2026/09/22 15:45:33 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 20.809175] niks3-server[937]: 2026/09/22 15:45:33 INFO systemd watchdog enabled interval=15s lb # [ 20.810324] niks3-server[937]: 2026/09/22 15:45:33 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 20.812622] systemd[1]: Started niks3 server. lb # [ 20.813897] systemd[1]: Reached target Multi-User System. lb # [ 20.814654] systemd[1]: Startup finished in 977ms (kernel) + 5.004s (initrd) + 14.830s (userspace) = 20.811s. worker1 # [ 20.921479] nix-grpc-daemon[765]: ts=2026-09-22T15:45:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.926496] nix-grpc-daemon[765]: ts=2026-09-22T15:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.958875] nix-grpc-daemon[772]: ts=2026-09-22T15:45:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.963155] nix-grpc-daemon[772]: ts=2026-09-22T15:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.127407] nix-grpc-daemon[765]: ts=2026-09-22T15:45:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.132862] nix-grpc-daemon[765]: ts=2026-09-22T15:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: waiting for unit niks3.service, in 8.65 seconds) lb: waiting for TCP port 5751 on localhost worker2 # [ 21.159646] nix-grpc-daemon[772]: ts=2026-09-22T15:45:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.165213] nix-grpc-daemon[772]: ts=2026-09-22T15:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.05 seconds) lb: waiting for TCP port 50051 on localhost lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 21.288952] envoy[894]: conn peer=[::1]:33426 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=4 lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds) worker1 # [ 21.334855] nix-grpc-daemon[765]: ts=2026-09-22T15:45:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.344373] nix-grpc-daemon[765]: ts=2026-09-22T15:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.366460] nix-grpc-daemon[772]: ts=2026-09-22T15:45:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.371820] nix-grpc-daemon[772]: ts=2026-09-22T15:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.455002] systemd[1]: Starting scheduler-token.service... worker1 # [ 21.535980] nix-grpc-daemon[765]: ts=2026-09-22T15:45:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.541537] nix-grpc-daemon[765]: ts=2026-09-22T15:45:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.550241] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 21.554642] systemd[1]: Finished scheduler-token.service. worker2 # [ 21.555680] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 98ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 21.572385] nix-grpc-daemon[772]: ts=2026-09-22T15:45:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.575003] nix-grpc-daemon[772]: ts=2026-09-22T15:45:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.05 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1 # [ 21.742318] nix-grpc-daemon[765]: ts=2026-09-22T15:45:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.747758] nix-grpc-daemon[765]: ts=2026-09-22T15:45:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.776454] nix-grpc-daemon[772]: ts=2026-09-22T15:45:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.781628] nix-grpc-daemon[772]: ts=2026-09-22T15:45:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.950512] nix-grpc-daemon[765]: ts=2026-09-22T15:45:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.954709] nix-grpc-daemon[765]: ts=2026-09-22T15:45:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.984608] nix-grpc-daemon[772]: ts=2026-09-22T15:45:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.988960] nix-grpc-daemon[772]: ts=2026-09-22T15:45:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.151282] nix-grpc-daemon[765]: ts=2026-09-22T15:45:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.156782] nix-grpc-daemon[765]: ts=2026-09-22T15:45:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.185421] nix-grpc-daemon[772]: ts=2026-09-22T15:45:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.190675] nix-grpc-daemon[772]: ts=2026-09-22T15:45:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.357547] nix-grpc-daemon[765]: ts=2026-09-22T15:45:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.362774] nix-grpc-daemon[765]: ts=2026-09-22T15:45:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.391887] nix-grpc-daemon[772]: ts=2026-09-22T15:45:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.397995] nix-grpc-daemon[772]: ts=2026-09-22T15:45:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.563751] nix-grpc-daemon[765]: ts=2026-09-22T15:45:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.569088] nix-grpc-daemon[765]: ts=2026-09-22T15:45:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.597920] nix-grpc-daemon[772]: ts=2026-09-22T15:45:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.602822] nix-grpc-daemon[772]: ts=2026-09-22T15:45:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 22.769853] nix-grpc-daemon[765]: ts=2026-09-22T15:45:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.775194] nix-grpc-daemon[765]: ts=2026-09-22T15:45:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.803994] nix-grpc-daemon[772]: ts=2026-09-22T15:45:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.810625] nix-grpc-daemon[772]: ts=2026-09-22T15:45:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.977845] nix-grpc-daemon[765]: ts=2026-09-22T15:45:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.982006] nix-grpc-daemon[765]: ts=2026-09-22T15:45:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.013483] nix-grpc-daemon[772]: ts=2026-09-22T15:45:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.017593] nix-grpc-daemon[772]: ts=2026-09-22T15:45:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.178423] nix-grpc-daemon[765]: ts=2026-09-22T15:45:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.183639] nix-grpc-daemon[765]: ts=2026-09-22T15:45:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.213914] nix-grpc-daemon[772]: ts=2026-09-22T15:45:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.219133] nix-grpc-daemon[772]: ts=2026-09-22T15:45:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.384661] nix-grpc-daemon[765]: ts=2026-09-22T15:45:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.389868] nix-grpc-daemon[765]: ts=2026-09-22T15:45:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.420264] nix-grpc-daemon[772]: ts=2026-09-22T15:45:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.425347] nix-grpc-daemon[772]: ts=2026-09-22T15:45:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.590722] nix-grpc-daemon[765]: ts=2026-09-22T15:45:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.596059] nix-grpc-daemon[765]: ts=2026-09-22T15:45:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.626585] nix-grpc-daemon[772]: ts=2026-09-22T15:45:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.631961] nix-grpc-daemon[772]: ts=2026-09-22T15:45:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 23.797006] nix-grpc-daemon[765]: ts=2026-09-22T15:45:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.802198] nix-grpc-daemon[765]: ts=2026-09-22T15:45:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 23.833164] nix-grpc-daemon[772]: ts=2026-09-22T15:45:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.838619] nix-grpc-daemon[772]: ts=2026-09-22T15:45:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.005162] nix-grpc-daemon[765]: ts=2026-09-22T15:45:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.009321] nix-grpc-daemon[765]: ts=2026-09-22T15:45:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.041600] nix-grpc-daemon[772]: ts=2026-09-22T15:45:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.045797] nix-grpc-daemon[772]: ts=2026-09-22T15:45:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.205929] nix-grpc-daemon[765]: ts=2026-09-22T15:45:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.211393] nix-grpc-daemon[765]: ts=2026-09-22T15:45:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.242428] nix-grpc-daemon[772]: ts=2026-09-22T15:45:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.247747] nix-grpc-daemon[772]: ts=2026-09-22T15:45:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.411930] nix-grpc-daemon[765]: ts=2026-09-22T15:45:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.414754] nix-grpc-daemon[765]: ts=2026-09-22T15:45:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.448841] nix-grpc-daemon[772]: ts=2026-09-22T15:45:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.454043] nix-grpc-daemon[772]: ts=2026-09-22T15:45:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.618626] nix-grpc-daemon[765]: ts=2026-09-22T15:45:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.624232] nix-grpc-daemon[765]: ts=2026-09-22T15:45:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.655108] nix-grpc-daemon[772]: ts=2026-09-22T15:45:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.660534] nix-grpc-daemon[772]: ts=2026-09-22T15:45:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 24.819644] nix-grpc-daemon[765]: ts=2026-09-22T15:45:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.825207] nix-grpc-daemon[765]: ts=2026-09-22T15:45:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.861802] nix-grpc-daemon[772]: ts=2026-09-22T15:45:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.873266] nix-grpc-daemon[772]: ts=2026-09-22T15:45:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds) worker1 # [ 25.028101] nix-grpc-daemon[765]: ts=2026-09-22T15:45:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.032207] nix-grpc-daemon[765]: ts=2026-09-22T15:45:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.073068] nix-grpc-daemon[772]: ts=2026-09-22T15:45:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.077246] nix-grpc-daemon[772]: ts=2026-09-22T15:45:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.228773] nix-grpc-daemon[765]: ts=2026-09-22T15:45:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.233997] nix-grpc-daemon[765]: ts=2026-09-22T15:45:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.273742] nix-grpc-daemon[772]: ts=2026-09-22T15:45:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.278963] nix-grpc-daemon[772]: ts=2026-09-22T15:45:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.434999] nix-grpc-daemon[765]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.440566] nix-grpc-daemon[765]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.480347] nix-grpc-daemon[772]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.485789] nix-grpc-daemon[772]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.641387] nix-grpc-daemon[765]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.646747] nix-grpc-daemon[765]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.686983] nix-grpc-daemon[772]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.692379] nix-grpc-daemon[772]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.847620] nix-grpc-daemon[765]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.853085] nix-grpc-daemon[765]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.893220] nix-grpc-daemon[772]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.898544] nix-grpc-daemon[772]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 26.055748] nix-grpc-daemon[765]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.059985] nix-grpc-daemon[765]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.101531] nix-grpc-daemon[772]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.105701] nix-grpc-daemon[772]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.256692] nix-grpc-daemon[765]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.261848] nix-grpc-daemon[765]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.302349] nix-grpc-daemon[772]: ts=2026-09-22T15:45:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.307665] nix-grpc-daemon[772]: ts=2026-09-22T15:45:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.462741] nix-grpc-daemon[765]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.467930] nix-grpc-daemon[765]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.508833] nix-grpc-daemon[772]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.514169] nix-grpc-daemon[772]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.668876] nix-grpc-daemon[765]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.674045] nix-grpc-daemon[765]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.715051] nix-grpc-daemon[772]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.720352] nix-grpc-daemon[772]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.875174] nix-grpc-daemon[765]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.880499] nix-grpc-daemon[765]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.921496] nix-grpc-daemon[772]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.926621] nix-grpc-daemon[772]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 27.083332] nix-grpc-daemon[765]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.087606] nix-grpc-daemon[765]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.129583] nix-grpc-daemon[772]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.133705] nix-grpc-daemon[772]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.283699] nix-grpc-daemon[765]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.289040] nix-grpc-daemon[765]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.330548] nix-grpc-daemon[772]: ts=2026-09-22T15:45:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.335637] nix-grpc-daemon[772]: ts=2026-09-22T15:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.490070] nix-grpc-daemon[765]: ts=2026-09-22T15:45:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.495605] nix-grpc-daemon[765]: ts=2026-09-22T15:45:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.536982] nix-grpc-daemon[772]: ts=2026-09-22T15:45:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.542146] nix-grpc-daemon[772]: ts=2026-09-22T15:45:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.696555] nix-grpc-daemon[765]: ts=2026-09-22T15:45:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.702036] nix-grpc-daemon[765]: ts=2026-09-22T15:45:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.743566] nix-grpc-daemon[772]: ts=2026-09-22T15:45:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.748930] nix-grpc-daemon[772]: ts=2026-09-22T15:45:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.902967] nix-grpc-daemon[765]: ts=2026-09-22T15:45:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.912342] nix-grpc-daemon[765]: ts=2026-09-22T15:45:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.949971] nix-grpc-daemon[772]: ts=2026-09-22T15:45:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.955159] nix-grpc-daemon[772]: ts=2026-09-22T15:45:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 28.113400] nix-grpc-daemon[765]: ts=2026-09-22T15:45:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.117598] nix-grpc-daemon[765]: ts=2026-09-22T15:45:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 28.160404] nix-grpc-daemon[772]: ts=2026-09-22T15:45:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.167374] nix-grpc-daemon[772]: ts=2026-09-22T15:45:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.314322] nix-grpc-daemon[765]: ts=2026-09-22T15:45:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.319703] nix-grpc-daemon[765]: ts=2026-09-22T15:45:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.360102] nix-grpc-daemon[772]: ts=2026-09-22T15:45:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.365368] nix-grpc-daemon[772]: ts=2026-09-22T15:45:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.520628] nix-grpc-daemon[765]: ts=2026-09-22T15:45:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.525893] nix-grpc-daemon[765]: ts=2026-09-22T15:45:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.566503] nix-grpc-daemon[772]: ts=2026-09-22T15:45:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.571869] nix-grpc-daemon[772]: ts=2026-09-22T15:45:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.726772] nix-grpc-daemon[765]: ts=2026-09-22T15:45:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.732278] nix-grpc-daemon[765]: ts=2026-09-22T15:45:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.773146] nix-grpc-daemon[772]: ts=2026-09-22T15:45:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.778480] nix-grpc-daemon[772]: ts=2026-09-22T15:45:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.932987] nix-grpc-daemon[765]: ts=2026-09-22T15:45:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.938349] nix-grpc-daemon[765]: ts=2026-09-22T15:45:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.979485] nix-grpc-daemon[772]: ts=2026-09-22T15:45:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.984952] nix-grpc-daemon[772]: ts=2026-09-22T15:45:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 29.140625] nix-grpc-daemon[765]: ts=2026-09-22T15:45:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.145503] nix-grpc-daemon[765]: ts=2026-09-22T15:45:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 29.188182] nix-grpc-daemon[772]: ts=2026-09-22T15:45:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.192274] nix-grpc-daemon[772]: ts=2026-09-22T15:45:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.346288] nix-grpc-daemon[765]: ts=2026-09-22T15:45:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.351605] nix-grpc-daemon[765]: ts=2026-09-22T15:45:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.389116] nix-grpc-daemon[772]: ts=2026-09-22T15:45:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.394221] nix-grpc-daemon[772]: ts=2026-09-22T15:45:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.552708] nix-grpc-daemon[765]: ts=2026-09-22T15:45:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.558158] nix-grpc-daemon[765]: ts=2026-09-22T15:45:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.595306] nix-grpc-daemon[772]: ts=2026-09-22T15:45:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.600726] nix-grpc-daemon[772]: ts=2026-09-22T15:45:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.759025] nix-grpc-daemon[765]: ts=2026-09-22T15:45:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.764741] nix-grpc-daemon[765]: ts=2026-09-22T15:45:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.801892] nix-grpc-daemon[772]: ts=2026-09-22T15:45:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.807254] nix-grpc-daemon[772]: ts=2026-09-22T15:45:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.965451] nix-grpc-daemon[765]: ts=2026-09-22T15:45:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.971095] nix-grpc-daemon[765]: ts=2026-09-22T15:45:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.008155] nix-grpc-daemon[772]: ts=2026-09-22T15:45:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.013344] nix-grpc-daemon[772]: ts=2026-09-22T15:45:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.173818] nix-grpc-daemon[765]: ts=2026-09-22T15:45:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.178001] nix-grpc-daemon[765]: ts=2026-09-22T15:45:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 30.216202] nix-grpc-daemon[772]: ts=2026-09-22T15:45:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.220267] nix-grpc-daemon[772]: ts=2026-09-22T15:45:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 30.374741] nix-grpc-daemon[765]: ts=2026-09-22T15:45:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.380169] nix-grpc-daemon[765]: ts=2026-09-22T15:45:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.417479] nix-grpc-daemon[772]: ts=2026-09-22T15:45:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.422745] nix-grpc-daemon[772]: ts=2026-09-22T15:45:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.580975] nix-grpc-daemon[765]: ts=2026-09-22T15:45:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.586184] nix-grpc-daemon[765]: ts=2026-09-22T15:45:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.623856] nix-grpc-daemon[772]: ts=2026-09-22T15:45:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.629118] nix-grpc-daemon[772]: ts=2026-09-22T15:45:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.787162] nix-grpc-daemon[765]: ts=2026-09-22T15:45:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.792664] nix-grpc-daemon[765]: ts=2026-09-22T15:45:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.812136] nix-grpc-daemon[772]: ts=2026-09-22T15:45:43Z level=info event=scheduler_take_over lb # [ 30.815286] niks3-server[937]: 2026/09/22 15:45:43 INFO lead: acquired remote=192.168.1.4:57128 worker2 # [ 30.830058] nix-grpc-daemon[772]: ts=2026-09-22T15:45:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.835327] nix-grpc-daemon[772]: ts=2026-09-22T15:45:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.993482] nix-grpc-daemon[765]: ts=2026-09-22T15:45:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.974271] nix-grpc-daemon[772]: ts=2026-09-22T15:45:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:60336 worker2 # [ 30.980066] nix-grpc-daemon[772]: ts=2026-09-22T15:45:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 31.036253] nix-grpc-daemon[772]: ts=2026-09-22T15:45:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.050024] nix-grpc-daemon[772]: ts=2026-09-22T15:45:43Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 31.054529] nix-grpc-daemon[772]: ts=2026-09-22T15:45:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:60336 worker2 # [ 31.060188] nix-grpc-daemon[772]: ts=2026-09-22T15:45:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 10.41 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker1 # [ 32.341002] nix-grpc-daemon[765]: ts=2026-09-22T15:45:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 32.355664] systemd[1]: Started Nix Daemon. worker1 # [ 32.543413] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 32.565272] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 1/5) worker1 # [ 32.572884] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 77 ms (attempt 2/5) worker1 # [ 32.654054] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 192 ms (attempt 3/5) worker1 # [ 32.849475] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 717 ms (attempt 4/5) worker1 # [ 33.577948] nix-grpc-daemon[765]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 33.578468] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 33.593683] nix-grpc-daemon[772]: ts=2026-09-22T15:45:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 33.609269] systemd[1]: Started Nix Daemon. worker2 # [ 33.782610] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 33.804748] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 1/5) worker2 # [ 33.845906] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 69 ms (attempt 2/5) worker2 # [ 33.916697] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 3/5) worker2 # [ 34.000670] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 252 ms (attempt 4/5) worker2 # [ 34.255488] nix-grpc-daemon[772]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 34.261255] nix-grpc-daemon[772]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 34.272602] nix-grpc-daemon[772]: ts=2026-09-22T15:45:46Z level=debug event=path_info_miss path=c7dwh1z1cl7zl3zmm2gm23k4b20rxnfn-nixos-system-client-test error="error: path '/nix/store/c7dwh1z1cl7zl3zmm2gm23k4b20rxnfn-nixos-system-client-test' is required, but there is no substituter that can build it" worker1 # [ 34.309720] nix-grpc-daemon[765]: ts=2026-09-22T15:45:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 34.324895] nix-grpc-daemon[765]: ts=2026-09-22T15:45:46Z level=debug event=path_info_miss path=c7dwh1z1cl7zl3zmm2gm23k4b20rxnfn-nixos-system-client-test error="error: path '/nix/store/c7dwh1z1cl7zl3zmm2gm23k4b20rxnfn-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds) client: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds) worker1 # [ 34.649201] nix-grpc-daemon[765]: ts=2026-09-22T15:45:47Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 34.655966] nix-grpc-daemon[765]: ts=2026-09-22T15:45:47Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:41676 worker1 # [ 34.668474] nix-grpc-daemon[765]: ts=2026-09-22T15:45:47Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:41676 worker1 # [ 34.680647] nix-grpc-daemon[765]: ts=2026-09-22T15:45:47Z level=debug event=path_info_miss path=c7dwh1z1cl7zl3zmm2gm23k4b20rxnfn-nixos-system-client-test error="error: path '/nix/store/c7dwh1z1cl7zl3zmm2gm23k4b20rxnfn-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 34.662735] nix-grpc-daemon[772]: ts=2026-09-22T15:45:47Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:60348 worker2 # [ 34.674945] nix-grpc-daemon[772]: ts=2026-09-22T15:45:47Z level=debug event=path_info_miss path=c7dwh1z1cl7zl3zmm2gm23k4b20rxnfn-nixos-system-client-test error="error: path '/nix/store/c7dwh1z1cl7zl3zmm2gm23k4b20rxnfn-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds) worker2 # [ 34.959042] nix-grpc-daemon[772]: ts=2026-09-22T15:45:47Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 35.128092] nix-grpc-daemon[772]: ts=2026-09-22T15:45:47Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.12 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 worker2 # [ 35.296232] nix-grpc-daemon[772]: ts=2026-09-22T15:45:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60348 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 35.306056] nix-grpc-daemon[772]: ts=2026-09-22T15:45:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 35.340904] nix-grpc-daemon[772]: ts=2026-09-22T15:45:47Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=0 worker2 # [ 35.346794] nix-grpc-daemon[772]: ts=2026-09-22T15:45:47Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 35.420421] nix-grpc-daemon[765]: ts=2026-09-22T15:45:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 35.438324] nix-grpc-daemon[765]: ts=2026-09-22T15:45:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41676 client # copying 1 paths to 'lb:50051'... worker1 # [ 35.454896] nix-grpc-daemon[765]: ts=2026-09-22T15:45:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'... worker1 # [ 35.482143] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 35.516816] nix-grpc-daemon[765]: ts=2026-09-22T15:45:48Z level=info event=niks3_push_started pid=842 worker1 # [ 35.708105] nix-grpc-daemon[842]: time=2026-09-22T15:45:48.309Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 35.819137] nix-daemon[829]: accepted connection from pid 855, user nix-grpc-daemon (trusted) worker1 # [ 35.841016] nix-daemon[829]: reaped child process 858, status = succeeded worker1 # [ 35.946624] nix-daemon[829]: accepted connection from pid 860, user nix-grpc-daemon (trusted) worker1 # [ 35.967015] nix-daemon[829]: reaped child process 863, status = succeeded lb # [ 35.939913] niks3-server[937]: 2026/09/22 15:45:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 35.986038] nix-grpc-daemon[842]: time=2026-09-22T15:45:48.587Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 35.987576] nix-grpc-daemon[842]: time=2026-09-22T15:45:48.587Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)" lb # [ 35.990596] niks3-server[937]: 2026/09/22 15:45:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign lb # [ 36.036600] niks3-server[937]: 2026/09/22 15:45:48 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst lb # [ 36.050492] niks3-server[937]: 2026/09/22 15:45:48 INFO Signed narinfos id=1 count=1 worker1 # [ 36.078078] nix-grpc-daemon[842]: time=2026-09-22T15:45:48.680Z level=INFO msg="Uploading 1 narinfos" lb # [ 36.060922] niks3-server[937]: 2026/09/22 15:45:48 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls lb # [ 36.072584] niks3-server[937]: 2026/09/22 15:45:48 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 36.080849] niks3-server[937]: 2026/09/22 15:45:48 INFO Completed upload id=1 worker1 # [ 36.107669] nix-grpc-daemon[842]: time=2026-09-22T15:45:48.710Z level=INFO msg="Upload complete. (396ms)" lb # [ 36.090328] niks3-server[937]: 2026/09/22 15:45:48 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo worker1 # [ 36.122372] nix-grpc-daemon[765]: ts=2026-09-22T15:45:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 36.138485] nix-grpc-daemon[765]: ts=2026-09-22T15:45:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 36.149335] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 36.152852] nix-daemon[829]: reaped child process 840, status = succeeded client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051... worker1 # [ 36.166424] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker1 # [ 37.425240] nix-daemon[829]: reaped child process 867, status = succeeded worker1 # [ 37.544710] nix-daemon[829]: accepted connection from pid 883, user nix-grpc-daemon (trusted) worker1 # [ 37.561835] nix-daemon[829]: reaped child process 886, status = succeeded lb # [ 37.542820] niks3-server[937]: 2026/09/22 15:45:50 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 37.584543] nix-grpc-daemon[842]: time=2026-09-22T15:45:50.185Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 37.586796] nix-grpc-daemon[842]: time=2026-09-22T15:45:50.185Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)" lb # [ 37.593815] niks3-server[937]: 2026/09/22 15:45:50 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 37.607043] niks3-server[937]: 2026/09/22 15:45:50 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv lb # [ 37.617466] niks3-server[937]: 2026/09/22 15:45:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign lb # [ 37.624828] niks3-server[937]: 2026/09/22 15:45:50 INFO Signed narinfos id=2 count=1 worker1 # [ 37.651174] nix-grpc-daemon[842]: time=2026-09-22T15:45:50.252Z level=INFO msg="Uploading 1 narinfos" lb # [ 37.630655] niks3-server[937]: 2026/09/22 15:45:50 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls lb # [ 37.644000] niks3-server[937]: 2026/09/22 15:45:50 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 37.650907] niks3-server[937]: 2026/09/22 15:45:50 INFO Completed upload id=2 worker1 # [ 37.677235] nix-grpc-daemon[842]: time=2026-09-22T15:45:50.279Z level=INFO msg="Upload complete. (295ms)" lb # [ 37.654181] niks3-server[937]: 2026/09/22 15:45:50 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo worker1 # [ 37.689152] nix-grpc-daemon[765]: ts=2026-09-22T15:45:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=1 worker2 # [ 37.674016] nix-grpc-daemon[772]: ts=2026-09-22T15:45:50Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=0 worker2 # [ 37.679974] nix-grpc-daemon[772]: ts=2026-09-22T15:45:50Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 37.710330] nix-daemon[829]: reaped child process 865, status = succeeded worker1 # [ 37.752675] nix-grpc-daemon[765]: ts=2026-09-22T15:45:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 37.768955] nix-grpc-daemon[765]: ts=2026-09-22T15:45:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 37.784491] nix-grpc-daemon[842]: time=2026-09-22T15:45:50.387Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 37.796111] nix-grpc-daemon[765]: ts=2026-09-22T15:45:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'... worker1 # [ 37.817607] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 37.996107] nix-daemon[829]: accepted connection from pid 890, user nix-grpc-daemon (trusted) worker1 # [ 38.047998] nix-daemon[829]: reaped child process 893, status = succeeded worker1 # [ 38.134630] nix-daemon[829]: accepted connection from pid 895, user nix-grpc-daemon (trusted) worker1 # [ 38.155207] nix-daemon[829]: reaped child process 899, status = succeeded lb # [ 38.132764] niks3-server[937]: 2026/09/22 15:45:50 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 38.172754] nix-grpc-daemon[842]: time=2026-09-22T15:45:50.775Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 38.174248] nix-grpc-daemon[842]: time=2026-09-22T15:45:50.775Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)" lb # [ 38.184125] niks3-server[937]: 2026/09/22 15:45:50 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst lb # [ 38.195672] niks3-server[937]: 2026/09/22 15:45:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign worker1 # [ 38.226214] nix-grpc-daemon[842]: time=2026-09-22T15:45:50.828Z level=INFO msg="Uploading 1 narinfos" lb # [ 38.205120] niks3-server[937]: 2026/09/22 15:45:50 INFO Signed narinfos id=3 count=1 lb # [ 38.211963] niks3-server[937]: 2026/09/22 15:45:50 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls lb # [ 38.220432] niks3-server[937]: 2026/09/22 15:45:50 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 38.232604] niks3-server[937]: 2026/09/22 15:45:50 INFO Completed upload id=3 worker1 # [ 38.259185] nix-grpc-daemon[842]: time=2026-09-22T15:45:50.861Z level=INFO msg="Upload complete. (423ms)" lb # [ 38.236466] niks3-server[937]: 2026/09/22 15:45:50 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo worker1 # [ 38.272320] nix-grpc-daemon[765]: ts=2026-09-22T15:45:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 38.284840] nix-grpc-daemon[765]: ts=2026-09-22T15:45:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 38.295333] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 38.301662] nix-daemon[829]: reaped child process 888, status = succeeded client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051... worker1 # [ 38.312257] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv lb # [ 38.537438] envoy[894]: conn peer=[2001:db8:1::1]:38270 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=2023 lb # [ 38.542473] envoy[894]: conn peer=[2001:db8:1::1]:38272 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=73 lb # [ 38.547122] envoy[894]: conn peer=[2001:db8:1::1]:38274 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=11 lb # [ 38.551695] envoy[894]: conn peer=[2001:db8:1::1]:38284 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=12 client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker1 # [ 39.422902] nix-daemon[829]: reaped child process 903, status = succeeded worker1 # [ 39.504814] nix-daemon[829]: accepted connection from pid 919, user nix-grpc-daemon (trusted) worker1 # [ 39.524079] nix-daemon[829]: reaped child process 922, status = succeeded lb # [ 39.502052] niks3-server[937]: 2026/09/22 15:45:52 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 39.544358] nix-grpc-daemon[842]: time=2026-09-22T15:45:52.146Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 39.546385] nix-grpc-daemon[842]: time=2026-09-22T15:45:52.146Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)" lb # [ 39.548110] niks3-server[937]: 2026/09/22 15:45:52 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst worker1 # [ 39.589627] nix-grpc-daemon[842]: time=2026-09-22T15:45:52.192Z level=INFO msg="Uploading 1 narinfos" lb # [ 39.564370] niks3-server[937]: 2026/09/22 15:45:52 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv lb # [ 39.573055] niks3-server[937]: 2026/09/22 15:45:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign lb # [ 39.581832] niks3-server[937]: 2026/09/22 15:45:52 INFO Signed narinfos id=4 count=1 lb # [ 39.589620] niks3-server[937]: 2026/09/22 15:45:52 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls worker1 # [ 39.624081] nix-grpc-daemon[842]: time=2026-09-22T15:45:52.226Z level=INFO msg="Upload complete. (299ms)" lb # [ 39.598461] niks3-server[937]: 2026/09/22 15:45:52 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete lb # [ 39.603212] niks3-server[937]: 2026/09/22 15:45:52 INFO Completed upload id=4 lb # [ 39.607239] niks3-server[937]: 2026/09/22 15:45:52 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo worker2 # [ 39.621291] nix-grpc-daemon[772]: ts=2026-09-22T15:45:52Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=0 worker1 # [ 39.637012] nix-grpc-daemon[765]: ts=2026-09-22T15:45:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=1 worker2 # [ 39.627718] nix-grpc-daemon[772]: ts=2026-09-22T15:45:52Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 39.658291] nix-daemon[829]: reaped child process 901, status = succeeded worker1 # [ 39.699314] nix-grpc-daemon[765]: ts=2026-09-22T15:45:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 39.713579] nix-grpc-daemon[765]: ts=2026-09-22T15:45:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 39.729576] nix-grpc-daemon[842]: time=2026-09-22T15:45:52.332Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 39.740868] nix-grpc-daemon[765]: ts=2026-09-22T15:45:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'... worker1 # [ 39.763609] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 39.934315] nix-daemon[829]: accepted connection from pid 926, user nix-grpc-daemon (trusted) worker1 # [ 39.952490] nix-daemon[829]: reaped child process 929, status = succeeded worker1 # [ 40.054019] nix-daemon[829]: accepted connection from pid 931, user nix-grpc-daemon (trusted) worker1 # [ 40.071035] nix-daemon[829]: reaped child process 934, status = succeeded lb # [ 40.050372] niks3-server[937]: 2026/09/22 15:45:52 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 40.086454] nix-grpc-daemon[842]: time=2026-09-22T15:45:52.689Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 40.087932] nix-grpc-daemon[842]: time=2026-09-22T15:45:52.689Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)" lb # [ 40.077210] niks3-server[937]: 2026/09/22 15:45:52 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst worker1 # [ 40.117613] nix-grpc-daemon[842]: time=2026-09-22T15:45:52.720Z level=INFO msg="Uploading 1 narinfos" lb # [ 40.092483] niks3-server[937]: 2026/09/22 15:45:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign lb # [ 40.105037] niks3-server[937]: 2026/09/22 15:45:52 INFO Signed narinfos id=5 count=1 lb # [ 40.112452] niks3-server[937]: 2026/09/22 15:45:52 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls lb # [ 40.126495] niks3-server[937]: 2026/09/22 15:45:52 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete worker1 # [ 40.158786] nix-grpc-daemon[842]: time=2026-09-22T15:45:52.761Z level=INFO msg="Upload complete. (376ms)" lb # [ 40.135652] niks3-server[937]: 2026/09/22 15:45:52 INFO Completed upload id=5 lb # [ 40.139927] niks3-server[937]: 2026/09/22 15:45:52 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo worker1 # [ 40.168287] nix-grpc-daemon[765]: ts=2026-09-22T15:45:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 40.182688] nix-grpc-daemon[765]: ts=2026-09-22T15:45:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 40.189222] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 40.194615] nix-daemon[829]: reaped child process 924, status = succeeded client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051... client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 40.206364] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker1 # [ 41.036553] nix-daemon[829]: reaped child process 938, status = succeeded worker1 # [ 41.129782] nix-daemon[829]: accepted connection from pid 954, user nix-grpc-daemon (trusted) worker1 # [ 41.147509] nix-daemon[829]: reaped child process 957, status = succeeded lb # [ 41.129261] niks3-server[937]: 2026/09/22 15:45:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 41.170331] nix-grpc-daemon[842]: time=2026-09-22T15:45:53.771Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 41.171797] nix-grpc-daemon[842]: time=2026-09-22T15:45:53.771Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)" lb # [ 41.175869] niks3-server[937]: 2026/09/22 15:45:53 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst worker1 # [ 41.214443] nix-grpc-daemon[842]: time=2026-09-22T15:45:53.817Z level=INFO msg="Uploading 1 narinfos" lb # [ 41.189174] niks3-server[937]: 2026/09/22 15:45:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign lb # [ 41.199706] niks3-server[937]: 2026/09/22 15:45:53 INFO Signed narinfos id=6 count=1 lb # [ 41.206935] niks3-server[937]: 2026/09/22 15:45:53 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv lb # [ 41.216635] niks3-server[937]: 2026/09/22 15:45:53 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls worker1 # [ 41.253181] nix-grpc-daemon[842]: time=2026-09-22T15:45:53.856Z level=INFO msg="Upload complete. (236ms)" lb # [ 41.227710] niks3-server[937]: 2026/09/22 15:45:53 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete worker1 # [ 41.260060] nix-grpc-daemon[765]: ts=2026-09-22T15:45:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=1 lb # [ 41.233841] niks3-server[937]: 2026/09/22 15:45:53 INFO Completed upload id=6 worker1 # [ 41.267067] nix-daemon[829]: reaped child process 936, status = succeeded lb # [ 41.239061] niks3-server[937]: 2026/09/22 15:45:53 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 6.14 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.21 seconds) client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client # copying 3 paths... client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.21 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.66 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds) worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 worker2 # [ 42.470235] nix-grpc-daemon[772]: ts=2026-09-22T15:45:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60348 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 42.480433] nix-grpc-daemon[772]: ts=2026-09-22T15:45:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 42.509079] nix-grpc-daemon[772]: ts=2026-09-22T15:45:55Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.29 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.14 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 1.15 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2 worker2 # [ 43.328321] nix-grpc-daemon[772]: ts=2026-09-22T15:45:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60348 client # don't know how to build these paths: client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 43.338355] nix-grpc-daemon[772]: ts=2026-09-22T15:45:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 43.354066] nix-grpc-daemon[772]: ts=2026-09-22T15:45:55Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=0 worker2 # [ 43.360303] nix-grpc-daemon[772]: ts=2026-09-22T15:45:55Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 43.429797] nix-grpc-daemon[765]: ts=2026-09-22T15:45:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 43.442346] nix-grpc-daemon[765]: ts=2026-09-22T15:45:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 43.452955] nix-grpc-daemon[842]: time=2026-09-22T15:45:56.055Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'... worker1 # [ 43.464661] nix-grpc-daemon[765]: ts=2026-09-22T15:45:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 43.479229] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 43.641135] nix-daemon[829]: accepted connection from pid 975, user nix-grpc-daemon (trusted) worker1 # [ 43.658311] nix-daemon[829]: reaped child process 978, status = succeeded worker1 # [ 43.759984] nix-daemon[829]: accepted connection from pid 980, user nix-grpc-daemon (trusted) worker1 # [ 43.778883] nix-daemon[829]: reaped child process 983, status = succeeded lb # [ 43.754785] niks3-server[937]: 2026/09/22 15:45:56 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 43.790688] nix-grpc-daemon[842]: time=2026-09-22T15:45:56.393Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 43.792217] nix-grpc-daemon[842]: time=2026-09-22T15:45:56.393Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)" lb # [ 43.784958] niks3-server[937]: 2026/09/22 15:45:56 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst worker1 # [ 43.824765] nix-grpc-daemon[842]: time=2026-09-22T15:45:56.427Z level=INFO msg="Uploading 1 narinfos" lb # [ 43.799955] niks3-server[937]: 2026/09/22 15:45:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign lb # [ 43.810908] niks3-server[937]: 2026/09/22 15:45:56 INFO Signed narinfos id=7 count=1 lb # [ 43.817628] niks3-server[937]: 2026/09/22 15:45:56 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls worker1 # [ 43.857315] nix-grpc-daemon[842]: time=2026-09-22T15:45:56.460Z level=INFO msg="Upload complete. (360ms)" lb # [ 43.832726] niks3-server[937]: 2026/09/22 15:45:56 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete lb # [ 43.838433] niks3-server[937]: 2026/09/22 15:45:56 INFO Completed upload id=7 lb # [ 43.841766] niks3-server[937]: 2026/09/22 15:45:56 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo worker1 # [ 43.868061] nix-grpc-daemon[765]: ts=2026-09-22T15:45:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 43.877015] nix-grpc-daemon[765]: ts=2026-09-22T15:45:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 43.884756] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 43.889253] nix-daemon[829]: reaped child process 973, status = succeeded client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051... worker1 # [ 43.901017] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker1 # [ 44.999454] nix-daemon[829]: reaped child process 987, status = succeeded worker1 # [ 45.075814] nix-daemon[829]: accepted connection from pid 1004, user nix-grpc-daemon (trusted) worker1 # [ 45.096506] nix-daemon[829]: reaped child process 1007, status = succeeded lb # [ 45.077993] niks3-server[937]: 2026/09/22 15:45:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 45.112404] nix-grpc-daemon[842]: time=2026-09-22T15:45:57.714Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 45.117158] nix-grpc-daemon[842]: time=2026-09-22T15:45:57.717Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)" lb # [ 45.117040] niks3-server[937]: 2026/09/22 15:45:57 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst lb # [ 45.131214] niks3-server[937]: 2026/09/22 15:45:57 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv lb # [ 45.139830] niks3-server[937]: 2026/09/22 15:45:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign worker1 # [ 45.171901] nix-grpc-daemon[842]: time=2026-09-22T15:45:57.774Z level=INFO msg="Uploading 1 narinfos" lb # [ 45.149990] niks3-server[937]: 2026/09/22 15:45:57 INFO Signed narinfos id=8 count=1 lb # [ 45.155583] niks3-server[937]: 2026/09/22 15:45:57 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls lb # [ 45.166980] niks3-server[937]: 2026/09/22 15:45:57 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete lb # [ 45.173949] niks3-server[937]: 2026/09/22 15:45:57 INFO Completed upload id=8 worker1 # [ 45.200130] nix-grpc-daemon[842]: time=2026-09-22T15:45:57.802Z level=INFO msg="Upload complete. (289ms)" lb # [ 45.176721] niks3-server[937]: 2026/09/22 15:45:57 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo worker1 # [ 45.213239] nix-grpc-daemon[765]: ts=2026-09-22T15:45:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 worker1 # [ 45.231535] nix-daemon[829]: reaped child process 985, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.11 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.63 seconds) subtest: outputs read back through any worker carry the cache signature client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2 worker2 # [ 45.810160] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60348 client # don't know how to build these paths: client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker2 # [ 45.822431] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 45.846451] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=0 worker2 # [ 45.857020] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 45.866014] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 45.873482] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'... worker2 # [ 45.886170] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 45.901848] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 45.927252] nix-grpc-daemon[772]: ts=2026-09-22T15:45:58Z level=info event=niks3_push_started pid=910 worker2 # [ 46.093160] nix-grpc-daemon[910]: time=2026-09-22T15:45:58.690Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 46.201987] nix-daemon[856]: accepted connection from pid 922, user nix-grpc-daemon (trusted) worker2 # [ 46.223665] nix-daemon[856]: reaped child process 925, status = succeeded worker2 # [ 46.325638] nix-daemon[856]: accepted connection from pid 927, user nix-grpc-daemon (trusted) worker2 # [ 46.344879] nix-daemon[856]: reaped child process 930, status = succeeded lb # [ 46.343007] niks3-server[937]: 2026/09/22 15:45:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 46.354761] nix-grpc-daemon[910]: time=2026-09-22T15:45:58.952Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 46.356754] nix-grpc-daemon[910]: time=2026-09-22T15:45:58.953Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)" lb # [ 46.378286] niks3-server[937]: 2026/09/22 15:45:58 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst worker2 # [ 46.392078] nix-grpc-daemon[910]: time=2026-09-22T15:45:58.991Z level=INFO msg="Uploading 1 narinfos" lb # [ 46.392647] niks3-server[937]: 2026/09/22 15:45:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign lb # [ 46.404509] niks3-server[937]: 2026/09/22 15:45:58 INFO Signed narinfos id=9 count=1 lb # [ 46.410488] niks3-server[937]: 2026/09/22 15:45:58 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls worker2 # [ 46.420083] nix-grpc-daemon[910]: time=2026-09-22T15:45:59.019Z level=INFO msg="Upload complete. (327ms)" lb # [ 46.420189] niks3-server[937]: 2026/09/22 15:45:59 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete worker2 # [ 46.426300] nix-grpc-daemon[772]: ts=2026-09-22T15:45:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 46.424889] niks3-server[937]: 2026/09/22 15:45:59 INFO Completed upload id=9 lb # [ 46.431642] niks3-server[937]: 2026/09/22 15:45:59 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo worker2 # [ 46.438090] nix-grpc-daemon[772]: ts=2026-09-22T15:45:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 46.443950] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 46.448376] nix-daemon[856]: reaped child process 908, status = succeeded client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051... worker2 # [ 46.460200] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-sig worker2 # [ 47.540809] nix-daemon[856]: reaped child process 934, status = succeeded worker2 # [ 47.645355] nix-daemon[856]: accepted connection from pid 950, user nix-grpc-daemon (trusted) worker2 # [ 47.664284] nix-daemon[856]: reaped child process 953, status = succeeded lb # [ 47.670833] niks3-server[937]: 2026/09/22 15:46:00 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 47.684426] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.280Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 47.686214] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.280Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)" lb # [ 47.712530] niks3-server[937]: 2026/09/22 15:46:00 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst worker2 # [ 47.729837] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.327Z level=INFO msg="Uploading 1 narinfos" lb # [ 47.728671] niks3-server[937]: 2026/09/22 15:46:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign lb # [ 47.739006] niks3-server[937]: 2026/09/22 15:46:00 INFO Signed narinfos id=10 count=1 lb # [ 47.745361] niks3-server[937]: 2026/09/22 15:46:00 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv lb # [ 47.752135] niks3-server[937]: 2026/09/22 15:46:00 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls worker2 # [ 47.761424] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.360Z level=INFO msg="Upload complete. (291ms)" lb # [ 47.763589] niks3-server[937]: 2026/09/22 15:46:00 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete lb # [ 47.770993] niks3-server[937]: 2026/09/22 15:46:00 INFO Completed upload id=10 lb # [ 47.773837] niks3-server[937]: 2026/09/22 15:46:00 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo worker2 # [ 47.777361] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=1 worker2 # [ 47.799710] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=0 worker2 # [ 47.804661] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 47.808995] nix-daemon[856]: reaped child process 932, status = succeeded worker2 # [ 47.837720] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 47.853836] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 47.867929] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.467Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'... worker2 # [ 47.879063] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 47.893241] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 48.052122] nix-daemon[856]: accepted connection from pid 959, user nix-grpc-daemon (trusted) worker2 # [ 48.074451] nix-daemon[856]: reaped child process 962, status = succeeded worker2 # [ 48.177656] nix-daemon[856]: accepted connection from pid 964, user nix-grpc-daemon (trusted) worker2 # [ 48.198518] nix-daemon[856]: reaped child process 967, status = succeeded lb # [ 48.196343] niks3-server[937]: 2026/09/22 15:46:00 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 48.207909] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.806Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 48.209762] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.806Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)" lb # [ 48.225735] niks3-server[937]: 2026/09/22 15:46:00 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst worker2 # [ 48.249604] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.848Z level=INFO msg="Uploading 1 narinfos" lb # [ 48.241189] niks3-server[937]: 2026/09/22 15:46:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign lb # [ 48.254084] niks3-server[937]: 2026/09/22 15:46:00 INFO Signed narinfos id=11 count=1 lb # [ 48.262292] niks3-server[937]: 2026/09/22 15:46:00 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls lb # [ 48.272499] niks3-server[937]: 2026/09/22 15:46:00 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete worker2 # [ 48.282384] nix-grpc-daemon[910]: time=2026-09-22T15:46:00.881Z level=INFO msg="Upload complete. (372ms)" lb # [ 48.284374] niks3-server[937]: 2026/09/22 15:46:00 INFO Completed upload id=11 lb # [ 48.288070] niks3-server[937]: 2026/09/22 15:46:00 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo worker2 # [ 48.292355] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 48.301123] nix-grpc-daemon[772]: ts=2026-09-22T15:46:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 48.313455] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 48.319166] nix-daemon[856]: reaped child process 957, status = succeeded client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051... worker2 # [ 48.329910] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv lb # [ 48.538230] envoy[894]: conn peer=[2001:db8:1::1]:38300 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9549 tx=4382 ms=5969 lb # [ 48.543511] envoy[894]: conn peer=[2001:db8:1::1]:55730 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=62 lb # [ 48.548389] envoy[894]: conn peer=[2001:db8:1::1]:55746 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4185 tx=1914 ms=1870 client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-sig worker2 # [ 49.219811] nix-daemon[856]: reaped child process 971, status = succeeded worker2 # [ 49.284257] nix-daemon[856]: accepted connection from pid 987, user nix-grpc-daemon (trusted) worker2 # [ 49.301681] nix-daemon[856]: reaped child process 990, status = succeeded lb # [ 49.309140] niks3-server[937]: 2026/09/22 15:46:01 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 49.319710] nix-grpc-daemon[910]: time=2026-09-22T15:46:01.918Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 49.321321] nix-grpc-daemon[910]: time=2026-09-22T15:46:01.918Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)" lb # [ 49.340250] niks3-server[937]: 2026/09/22 15:46:01 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst worker2 # [ 49.360643] nix-grpc-daemon[910]: time=2026-09-22T15:46:01.959Z level=INFO msg="Uploading 1 narinfos" lb # [ 49.359743] niks3-server[937]: 2026/09/22 15:46:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign lb # [ 49.371515] niks3-server[937]: 2026/09/22 15:46:01 INFO Signed narinfos id=12 count=1 lb # [ 49.378581] niks3-server[937]: 2026/09/22 15:46:01 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv lb # [ 49.391155] niks3-server[937]: 2026/09/22 15:46:01 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls worker2 # [ 49.398090] nix-grpc-daemon[910]: time=2026-09-22T15:46:01.997Z level=INFO msg="Upload complete. (256ms)" lb # [ 49.398802] niks3-server[937]: 2026/09/22 15:46:01 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete lb # [ 49.403642] niks3-server[937]: 2026/09/22 15:46:01 INFO Completed upload id=12 lb # [ 49.407635] niks3-server[937]: 2026/09/22 15:46:01 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo worker2 # [ 49.416388] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=1 worker2 # [ 49.434416] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=0 worker2 # [ 49.438933] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 49.443826] nix-daemon[856]: reaped child process 969, status = succeeded worker2 # [ 49.473315] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 49.488532] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 49.502448] nix-grpc-daemon[910]: time=2026-09-22T15:46:02.101Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'... worker2 # [ 49.510124] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 49.524124] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 49.680046] nix-daemon[856]: accepted connection from pid 996, user nix-grpc-daemon (trusted) worker2 # [ 49.700155] nix-daemon[856]: reaped child process 999, status = succeeded worker2 # [ 49.800873] nix-daemon[856]: accepted connection from pid 1001, user nix-grpc-daemon (trusted) worker2 # [ 49.820673] nix-daemon[856]: reaped child process 1004, status = succeeded lb # [ 49.821483] niks3-server[937]: 2026/09/22 15:46:02 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 49.830730] nix-grpc-daemon[910]: time=2026-09-22T15:46:02.429Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 49.832404] nix-grpc-daemon[910]: time=2026-09-22T15:46:02.429Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)" lb # [ 49.849244] niks3-server[937]: 2026/09/22 15:46:02 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst lb # [ 49.864233] niks3-server[937]: 2026/09/22 15:46:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign worker2 # [ 49.870722] nix-grpc-daemon[910]: time=2026-09-22T15:46:02.469Z level=INFO msg="Uploading 1 narinfos" lb # [ 49.876859] niks3-server[937]: 2026/09/22 15:46:02 INFO Signed narinfos id=13 count=1 lb # [ 49.879300] niks3-server[937]: 2026/09/22 15:46:02 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls lb # [ 49.892171] niks3-server[937]: 2026/09/22 15:46:02 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete lb # [ 49.899070] niks3-server[937]: 2026/09/22 15:46:02 INFO Completed upload id=13 worker2 # [ 49.899641] nix-grpc-daemon[910]: time=2026-09-22T15:46:02.498Z level=INFO msg="Upload complete. (359ms)" lb # [ 49.901956] niks3-server[937]: 2026/09/22 15:46:02 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo worker2 # [ 49.907678] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 49.916925] nix-grpc-daemon[772]: ts=2026-09-22T15:46:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 49.929794] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 49.934502] nix-daemon[856]: reaped child process 994, status = succeeded client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051... worker2 # [ 49.948172] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-sig worker2 # [ 51.028339] nix-daemon[856]: reaped child process 1008, status = succeeded worker2 # [ 51.141020] nix-daemon[856]: accepted connection from pid 1024, user nix-grpc-daemon (trusted) worker2 # [ 51.162424] nix-daemon[856]: reaped child process 1027, status = succeeded lb # [ 51.174630] niks3-server[937]: 2026/09/22 15:46:03 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 51.190953] nix-grpc-daemon[910]: time=2026-09-22T15:46:03.789Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 51.194675] nix-grpc-daemon[910]: time=2026-09-22T15:46:03.789Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)" lb # [ 51.238119] niks3-server[937]: 2026/09/22 15:46:03 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst lb # [ 51.250631] niks3-server[937]: 2026/09/22 15:46:03 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv lb # [ 51.264188] niks3-server[937]: 2026/09/22 15:46:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign worker2 # [ 51.273319] nix-grpc-daemon[910]: time=2026-09-22T15:46:03.872Z level=INFO msg="Uploading 1 narinfos" lb # [ 51.275390] niks3-server[937]: 2026/09/22 15:46:03 INFO Signed narinfos id=14 count=1 lb # [ 51.281557] niks3-server[937]: 2026/09/22 15:46:03 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls lb # [ 51.295657] niks3-server[937]: 2026/09/22 15:46:03 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 51.304202] niks3-server[937]: 2026/09/22 15:46:03 INFO Completed upload id=14 worker2 # [ 51.304984] nix-grpc-daemon[910]: time=2026-09-22T15:46:03.904Z level=INFO msg="Upload complete. (344ms)" lb # [ 51.307299] niks3-server[937]: 2026/09/22 15:46:03 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo worker2 # [ 51.321506] nix-grpc-daemon[772]: ts=2026-09-22T15:46:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=1 worker2 # [ 51.343667] nix-daemon[856]: reaped child process 1006, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.79 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 51.750505] nix-grpc-daemon[765]: ts=2026-09-22T15:46:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:34392 client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked): client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 51.809453] nix-grpc-daemon[765]: ts=2026-09-22T15:46:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:34392 worker1 # [ 51.820155] nix-grpc-daemon[765]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'... worker1 # [ 51.843180] nix-grpc-daemon[765]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'... worker1 # [ 51.863894] nix-grpc-daemon[765]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.33 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker2 # [ 52.083360] nix-grpc-daemon[772]: ts=2026-09-22T15:46:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:46222 worker2 # [ 52.127191] nix-grpc-daemon[772]: ts=2026-09-22T15:46:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:46222 client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.30 seconds) client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top client # finding garbage collector roots... client # removing stale temporary roots file "/nix/var/nix/temproots/857" client # removing stale temporary roots file "/nix/var/nix/temproots/843" client # removing stale temporary roots file "/nix/var/nix/temproots/823" client # removing stale temporary roots file "/nix/var/nix/temproots/785" client # removing stale temporary roots file "/nix/var/nix/temproots/865" client # removing stale temporary roots file "/nix/var/nix/temproots/880" client # removing stale temporary roots file "/nix/var/nix/temproots/801" client # removing stale temporary roots file "/nix/var/nix/temproots/835" client # deleting specified paths... client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' client # deleting unused links... client # note: hard linking is currently saving 0.0 KiB worker2 # [ 52.555618] nix-grpc-daemon[772]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 52.567221] nix-grpc-daemon[772]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 52.584482] nix-grpc-daemon[910]: time=2026-09-22T15:46:05.182Z level=INFO msg="All 1 paths already cached" worker1 # [ 52.621840] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:41676 worker2 # [ 52.608936] nix-grpc-daemon[772]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:60348 client # copying 3 paths... worker1 # [ 52.655047] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:41676 client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'... worker2 # [ 52.667036] nix-grpc-daemon[772]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 52.675597] nix-grpc-daemon[772]: ts=2026-09-22T15:46:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_out=232 worker1 # [ 52.716934] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 52.725441] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_out=176 worker1 # [ 52.737051] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 52.743980] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_out=128 client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'... client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'... client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.62 seconds) (finished: subtest: outputs read back through any worker carry the cache signature, in 7.24 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 52.885845] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: 15138f67272945038baf1659223cce82 client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 53.088158] nix[932]: don't know how to build these paths: client # [ 53.090030] nix[932]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 53.106233] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41676 worker2 # [ 53.098301] nix-grpc-daemon[772]: ts=2026-09-22T15:46:05Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker2 # [ 53.105081] nix-grpc-daemon[772]: ts=2026-09-22T15:46:05Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 53.171624] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 53.193870] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41676 client # [ 53.193438] nix[932]: copying 1 paths to 'lb:50051'... worker1 # [ 53.214168] nix-grpc-daemon[765]: ts=2026-09-22T15:46:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 client # [ 53.199623] nix[932]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'... worker1 # [ 53.231231] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 53.421329] nix-daemon[829]: accepted connection from pid 1014, user nix-grpc-daemon (trusted) worker1 # [ 53.443851] nix-daemon[829]: reaped child process 1017, status = succeeded worker1 # [ 53.557515] nix-daemon[829]: accepted connection from pid 1019, user nix-grpc-daemon (trusted) worker1 # [ 53.579499] nix-daemon[829]: reaped child process 1022, status = succeeded lb # [ 53.551549] niks3-server[937]: 2026/09/22 15:46:06 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 53.593368] nix-grpc-daemon[842]: time=2026-09-22T15:46:06.195Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 53.594859] nix-grpc-daemon[842]: time=2026-09-22T15:46:06.195Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)" lb # [ 53.613074] niks3-server[937]: 2026/09/22 15:46:06 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst lb # [ 53.637330] niks3-server[937]: 2026/09/22 15:46:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign worker1 # [ 53.670248] nix-grpc-daemon[842]: time=2026-09-22T15:46:06.272Z level=INFO msg="Uploading 1 narinfos" lb # [ 53.648382] niks3-server[937]: 2026/09/22 15:46:06 INFO Signed narinfos id=15 count=1 lb # [ 53.657688] niks3-server[937]: 2026/09/22 15:46:06 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls lb # [ 53.673798] niks3-server[937]: 2026/09/22 15:46:06 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete lb # [ 53.680811] niks3-server[937]: 2026/09/22 15:46:06 INFO Completed upload id=15 worker1 # [ 53.706891] nix-grpc-daemon[842]: time=2026-09-22T15:46:06.309Z level=INFO msg="Upload complete. (452ms)" lb # [ 53.683470] niks3-server[937]: 2026/09/22 15:46:06 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo worker1 # [ 53.721011] nix-grpc-daemon[765]: ts=2026-09-22T15:46:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=432 worker1 # [ 53.732818] nix-grpc-daemon[765]: ts=2026-09-22T15:46:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 53.745771] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 53.752235] nix-daemon[829]: reaped child process 1012, status = succeeded client # [ 53.745969] nix[932]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker1 # [ 53.769089] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) client # [ 53.979412] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: 6dcbc1eb68c543ef86b95edcc76edfd2 client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds) client # [ 54.217050] nix[948]: don't know how to build these paths: client # [ 54.219121] nix[948]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker2 # [ 54.208972] nix-grpc-daemon[772]: ts=2026-09-22T15:46:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 54.222913] nix-grpc-daemon[772]: ts=2026-09-22T15:46:06Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker1 # [ 54.255642] nix-grpc-daemon[765]: ts=2026-09-22T15:46:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 client # [ 54.244696] nix[948]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.03 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker1 # [ 57.678913] nix-daemon[829]: reaped child process 1026, status = succeeded worker1 # [ 57.750337] nix-daemon[829]: accepted connection from pid 1056, user nix-grpc-daemon (trusted) worker1 # [ 57.771156] nix-daemon[829]: reaped child process 1059, status = succeeded lb # [ 57.746632] niks3-server[937]: 2026/09/22 15:46:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 57.784511] nix-grpc-daemon[842]: time=2026-09-22T15:46:10.385Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 57.786661] nix-grpc-daemon[842]: time=2026-09-22T15:46:10.385Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)" lb # [ 57.791232] niks3-server[937]: 2026/09/22 15:46:10 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 57.803635] niks3-server[937]: 2026/09/22 15:46:10 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 57.841179] nix-grpc-daemon[842]: time=2026-09-22T15:46:10.443Z level=INFO msg="Uploading 1 narinfos" lb # [ 57.814696] niks3-server[937]: 2026/09/22 15:46:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign lb # [ 57.820302] niks3-server[937]: 2026/09/22 15:46:10 INFO Signed narinfos id=16 count=1 lb # [ 57.822438] niks3-server[937]: 2026/09/22 15:46:10 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls lb # [ 57.833477] niks3-server[937]: 2026/09/22 15:46:10 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 57.841412] niks3-server[937]: 2026/09/22 15:46:10 INFO Completed upload id=16 worker1 # [ 57.866475] nix-grpc-daemon[842]: time=2026-09-22T15:46:10.469Z level=INFO msg="Upload complete. (282ms)" lb # [ 57.846174] niks3-server[937]: 2026/09/22 15:46:10 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo worker1 # [ 57.879135] nix-grpc-daemon[765]: ts=2026-09-22T15:46:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 57.899092] nix-grpc-daemon[765]: ts=2026-09-22T15:46:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 57.907371] nix-daemon[829]: reaped child process 1024, status = succeeded client # [ 57.898420] systemd[1]: dedup1.service: Deactivated successfully. client # [ 57.902850] systemd[1]: dedup1.service: Consumed 184ms CPU time over 5.016s wall clock time, 39.5M memory peak, 7.3K incoming IP traffic, 8.3K outgoing IP traffic. client # [ 57.950238] systemd[1]: dedup2.service: Deactivated successfully. client # [ 57.954150] systemd[1]: dedup2.service: Consumed 210ms CPU time over 3.974s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic. lb # [ 58.538314] envoy[894]: conn peer=[2001:db8:1::1]:55760 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9498 tx=4359 ms=5524 lb # [ 58.543288] envoy[894]: conn peer=[2001:db8:1::1]:54708 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2914 ms=248 client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds) lb # [ 58.548175] envoy[894]: conn peer=[2001:db8:1::1]:54742 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=329 ms=95 client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } lb # [ 58.553237] envoy[894]: conn peer=[2001:db8:1::1]:54726 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=116 lb # [ 58.557978] envoy[894]: conn peer=[2001:db8:1::1]:54716 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=139 lb # [ 58.562945] envoy[894]: conn peer=[2001:db8:1::1]:54758 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1563 ms=4794 lb # [ 58.568746] envoy[894]: conn peer=[2001:db8:1::1]:54762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1198 ms=3716 client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: two clients wanting the same drv build it once, in 6.04 seconds) subtest: --timeout and --max-silent-time reach the builder worker2 # [ 59.098862] nix-grpc-daemon[772]: ts=2026-09-22T15:46:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 59.115885] nix-grpc-daemon[772]: ts=2026-09-22T15:46:11Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=0 worker2 # [ 59.123102] nix-grpc-daemon[772]: ts=2026-09-22T15:46:11Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 59.132874] nix-grpc-daemon[772]: ts=2026-09-22T15:46:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 59.145150] nix-grpc-daemon[772]: ts=2026-09-22T15:46:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 59.160525] nix-grpc-daemon[772]: ts=2026-09-22T15:46:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 59.175850] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 59.333330] nix-daemon[856]: accepted connection from pid 1058, user nix-grpc-daemon (trusted) worker2 # [ 59.356093] nix-daemon[856]: reaped child process 1061, status = succeeded worker2 # [ 59.455819] nix-daemon[856]: accepted connection from pid 1063, user nix-grpc-daemon (trusted) worker2 # [ 59.474325] nix-daemon[856]: reaped child process 1066, status = succeeded lb # [ 59.482961] niks3-server[937]: 2026/09/22 15:46:12 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 59.497297] nix-grpc-daemon[910]: time=2026-09-22T15:46:12.096Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 59.498782] nix-grpc-daemon[910]: time=2026-09-22T15:46:12.096Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)" lb # [ 59.511842] niks3-server[937]: 2026/09/22 15:46:12 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst lb # [ 59.521166] niks3-server[937]: 2026/09/22 15:46:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign worker2 # [ 59.525414] nix-grpc-daemon[910]: time=2026-09-22T15:46:12.124Z level=INFO msg="Uploading 1 narinfos" lb # [ 59.525219] niks3-server[937]: 2026/09/22 15:46:12 INFO Signed narinfos id=17 count=1 lb # [ 59.532370] niks3-server[937]: 2026/09/22 15:46:12 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls lb # [ 59.540000] niks3-server[937]: 2026/09/22 15:46:12 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete worker2 # [ 59.543898] nix-grpc-daemon[910]: time=2026-09-22T15:46:12.143Z level=INFO msg="Upload complete. (351ms)" lb # [ 59.543840] niks3-server[937]: 2026/09/22 15:46:12 INFO Completed upload id=17 lb # [ 59.546989] niks3-server[937]: 2026/09/22 15:46:12 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo worker2 # [ 59.552512] nix-grpc-daemon[772]: ts=2026-09-22T15:46:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 59.560618] nix-grpc-daemon[772]: ts=2026-09-22T15:46:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 59.571433] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 59.573516] nix-daemon[856]: reaped child process 1056, status = succeeded worker2 # [ 59.584639] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 65.599864] nix-grpc-daemon[772]: ts=2026-09-22T15:46:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=0 worker2 # [ 65.633220] nix-daemon[856]: reaped child process 1068, status = succeeded worker2 # [ 65.637855] nix-daemon[856]: reaped child process 1070, status = succeeded worker1 # [ 65.904887] nix-grpc-daemon[765]: ts=2026-09-22T15:46:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41676 worker2 # [ 65.896164] nix-grpc-daemon[772]: ts=2026-09-22T15:46:18Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=0 worker1 # [ 65.927927] nix-grpc-daemon[765]: ts=2026-09-22T15:46:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker2 # [ 65.902313] nix-grpc-daemon[772]: ts=2026-09-22T15:46:18Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 65.943437] nix-grpc-daemon[765]: ts=2026-09-22T15:46:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 65.957507] nix-grpc-daemon[765]: ts=2026-09-22T15:46:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 65.973779] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 66.132867] nix-daemon[829]: accepted connection from pid 1072, user nix-grpc-daemon (trusted) worker1 # [ 66.155655] nix-daemon[829]: reaped child process 1075, status = succeeded worker1 # [ 66.256507] nix-daemon[829]: accepted connection from pid 1077, user nix-grpc-daemon (trusted) worker1 # [ 66.278486] nix-daemon[829]: reaped child process 1080, status = succeeded lb # [ 66.250915] niks3-server[937]: 2026/09/22 15:46:18 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 66.287260] nix-grpc-daemon[842]: time=2026-09-22T15:46:18.889Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 66.288934] nix-grpc-daemon[842]: time=2026-09-22T15:46:18.889Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)" lb # [ 66.285918] niks3-server[937]: 2026/09/22 15:46:18 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst lb # [ 66.303737] niks3-server[937]: 2026/09/22 15:46:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign worker1 # [ 66.335667] nix-grpc-daemon[842]: time=2026-09-22T15:46:18.938Z level=INFO msg="Uploading 1 narinfos" lb # [ 66.313994] niks3-server[937]: 2026/09/22 15:46:18 INFO Signed narinfos id=18 count=1 lb # [ 66.319996] niks3-server[937]: 2026/09/22 15:46:18 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls lb # [ 66.331783] niks3-server[937]: 2026/09/22 15:46:18 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete worker1 # [ 66.363008] nix-grpc-daemon[842]: time=2026-09-22T15:46:18.965Z level=INFO msg="Upload complete. (374ms)" lb # [ 66.339380] niks3-server[937]: 2026/09/22 15:46:18 INFO Completed upload id=18 lb # [ 66.344478] niks3-server[937]: 2026/09/22 15:46:18 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo worker1 # [ 66.372428] nix-grpc-daemon[765]: ts=2026-09-22T15:46:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 66.380433] nix-grpc-daemon[765]: ts=2026-09-22T15:46:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 66.394472] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 66.397676] nix-daemon[829]: reaped child process 1070, status = succeeded worker1 # [ 66.410077] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) lb # [ 68.538597] envoy[894]: conn peer=[2001:db8:1::1]:54778 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=954 ms=6562 worker1 # [ 72.425383] nix-grpc-daemon[765]: ts=2026-09-22T15:46:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=0 worker1 # [ 72.464798] nix-daemon[829]: reaped child process 1082, status = succeeded worker1 # [ 72.467545] nix-daemon[829]: reaped child process 1084, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 13.76 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 72.695953] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: 25a5d8b4297648e2be3083ba6a631e5b client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 72.897533] nix[1021]: don't know how to build these paths: client # [ 72.898494] nix[1021]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 72.915196] nix-grpc-daemon[765]: ts=2026-09-22T15:46:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41676 worker2 # [ 72.905214] nix-grpc-daemon[772]: ts=2026-09-22T15:46:25Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker2 # [ 72.913255] nix-grpc-daemon[772]: ts=2026-09-22T15:46:25Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 72.921159] nix-grpc-daemon[772]: ts=2026-09-22T15:46:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 72.932793] nix-grpc-daemon[772]: ts=2026-09-22T15:46:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 client # [ 72.950500] nix[1021]: copying 1 paths to 'lb:50051'... client # [ 72.953213] nix[1021]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'... worker2 # [ 72.947842] nix-grpc-daemon[772]: ts=2026-09-22T15:46:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 72.966237] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 73.118249] nix-daemon[856]: accepted connection from pid 1097, user nix-grpc-daemon (trusted) worker2 # [ 73.138464] nix-daemon[856]: reaped child process 1100, status = succeeded worker2 # [ 73.242298] nix-daemon[856]: accepted connection from pid 1102, user nix-grpc-daemon (trusted) worker2 # [ 73.262265] nix-daemon[856]: reaped child process 1105, status = succeeded lb # [ 73.260788] niks3-server[937]: 2026/09/22 15:46:25 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 73.271707] nix-grpc-daemon[910]: time=2026-09-22T15:46:25.870Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 73.273401] nix-grpc-daemon[910]: time=2026-09-22T15:46:25.870Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)" lb # [ 73.292788] niks3-server[937]: 2026/09/22 15:46:25 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst lb # [ 73.309894] niks3-server[937]: 2026/09/22 15:46:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign worker2 # [ 73.316319] nix-grpc-daemon[910]: time=2026-09-22T15:46:25.915Z level=INFO msg="Uploading 1 narinfos" lb # [ 73.317951] niks3-server[937]: 2026/09/22 15:46:25 INFO Signed narinfos id=19 count=1 lb # [ 73.325345] niks3-server[937]: 2026/09/22 15:46:25 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls worker2 # [ 73.346169] nix-grpc-daemon[910]: time=2026-09-22T15:46:25.945Z level=INFO msg="Upload complete. (365ms)" lb # [ 73.340729] niks3-server[937]: 2026/09/22 15:46:25 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete lb # [ 73.351653] niks3-server[937]: 2026/09/22 15:46:25 INFO Completed upload id=19 worker2 # [ 73.356316] nix-grpc-daemon[772]: ts=2026-09-22T15:46:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_in=432 lb # [ 73.356515] niks3-server[937]: 2026/09/22 15:46:25 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo worker2 # [ 73.368905] nix-grpc-daemon[772]: ts=2026-09-22T15:46:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 73.377443] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 73.383208] nix-daemon[856]: reaped child process 1095, status = succeeded worker2 # [ 73.394588] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) client # [ 73.399728] nix[1021]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client # [ 73.801581] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 4dfb4e9b1331469d864c6c3806bf10c6 client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds) worker1 # [ 74.037205] nix-grpc-daemon[765]: ts=2026-09-22T15:46:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41676 client # [ 74.025692] nix[1037]: don't know how to build these paths: client # [ 74.027619] nix[1037]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker2 # [ 74.033301] nix-grpc-daemon[772]: ts=2026-09-22T15:46:26Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 client # [ 74.048324] nix[1037]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... worker2 # [ 74.041029] nix-grpc-daemon[772]: ts=2026-09-22T15:46:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 client: must succeed: sleep 2; systemctl kill -s INT share1 client # [ 77.181319] systemd[1]: share1.service: Sent signal SIGINT to main process 1021 (nix) on client request. client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds) client # [ 77.190721] nix[1021]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client: waiting for success: ! systemctl is-active share1 client # [ 77.205130] nix[1021]: error: interrupted by the user client # [ 77.268326] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 77.271866] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 77.274168] systemd[1]: share1.service: Consumed 173ms CPU time over 4.575s wall clock time, 39.4M memory peak, 6.2K incoming IP traffic, 8.2K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 1.12 seconds) lb # [ 78.537824] envoy[894]: conn peer=[2001:db8:1::1]:35158 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2989 tx=951 ms=6583 lb # [ 78.542803] envoy[894]: conn peer=[2001:db8:1::1]:35650 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2990 tx=789 ms=4314 client: waiting for success: ! systemctl is-active share2 worker2 # [ 78.624514] nix-daemon[856]: reaped child process 1109, status = succeeded worker2 # [ 78.724210] nix-daemon[856]: accepted connection from pid 1145, user nix-grpc-daemon (trusted) worker2 # [ 78.742314] nix-daemon[856]: reaped child process 1148, status = succeeded lb # [ 78.748973] niks3-server[937]: 2026/09/22 15:46:31 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 78.760145] nix-grpc-daemon[910]: time=2026-09-22T15:46:31.358Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 78.761639] nix-grpc-daemon[910]: time=2026-09-22T15:46:31.358Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)" lb # [ 78.790810] niks3-server[937]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv lb # [ 78.804176] niks3-server[937]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst worker2 # [ 78.818054] nix-grpc-daemon[910]: time=2026-09-22T15:46:31.417Z level=INFO msg="Uploading 1 narinfos" lb # [ 78.819625] niks3-server[937]: 2026/09/22 15:46:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign lb # [ 78.828861] niks3-server[937]: 2026/09/22 15:46:31 INFO Signed narinfos id=20 count=1 lb # [ 78.831330] niks3-server[937]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls lb # [ 78.845365] niks3-server[937]: 2026/09/22 15:46:31 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete worker2 # [ 78.853739] nix-grpc-daemon[910]: time=2026-09-22T15:46:31.453Z level=INFO msg="Upload complete. (241ms)" lb # [ 78.854111] niks3-server[937]: 2026/09/22 15:46:31 INFO Completed upload id=20 lb # [ 78.860752] niks3-server[937]: 2026/09/22 15:46:31 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo worker2 # [ 78.867837] nix-grpc-daemon[772]: ts=2026-09-22T15:46:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker2 # [ 78.884397] nix-grpc-daemon[772]: ts=2026-09-22T15:46:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker2 # [ 78.893076] nix-daemon[856]: reaped child process 1107, status = succeeded client # [ 78.909881] systemd[1]: share2.service: Deactivated successfully. client # [ 78.915001] systemd[1]: share2.service: Consumed 190ms CPU time over 5.109s wall clock time, 33.3M memory peak, 6.2K incoming IP traffic, 5.8K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds) (finished: subtest: the first client leaving does not fail the second, in 7.11 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 79.874037] systemd[1]: Started Nix Daemon. client # [ 79.995800] nix-daemon[1089]: accepted connection from pid 1086, user root (trusted) worker2 # [ 80.142429] nix-grpc-daemon[772]: ts=2026-09-22T15:46:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:60348 worker1 # [ 80.173423] nix-grpc-daemon[765]: ts=2026-09-22T15:46:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41676 worker2 # [ 80.159273] nix-grpc-daemon[772]: ts=2026-09-22T15:46:32Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=0 worker2 # [ 80.165903] nix-grpc-daemon[772]: ts=2026-09-22T15:46:32Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 80.194628] nix-grpc-daemon[765]: ts=2026-09-22T15:46:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 80.209875] nix-grpc-daemon[765]: ts=2026-09-22T15:46:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 80.224608] nix-grpc-daemon[765]: ts=2026-09-22T15:46:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 80.239712] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 80.400949] nix-daemon[829]: accepted connection from pid 1123, user nix-grpc-daemon (trusted) worker1 # [ 80.422330] nix-daemon[829]: reaped child process 1126, status = succeeded worker1 # [ 80.518135] nix-daemon[829]: accepted connection from pid 1128, user nix-grpc-daemon (trusted) worker1 # [ 80.535660] nix-daemon[829]: reaped child process 1131, status = succeeded lb # [ 80.513805] niks3-server[937]: 2026/09/22 15:46:33 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 80.549656] nix-grpc-daemon[842]: time=2026-09-22T15:46:33.151Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 80.551127] nix-grpc-daemon[842]: time=2026-09-22T15:46:33.152Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)" lb # [ 80.539681] niks3-server[937]: 2026/09/22 15:46:33 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst lb # [ 80.553775] niks3-server[937]: 2026/09/22 15:46:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign worker1 # [ 80.586370] nix-grpc-daemon[842]: time=2026-09-22T15:46:33.189Z level=INFO msg="Uploading 1 narinfos" lb # [ 80.563877] niks3-server[937]: 2026/09/22 15:46:33 INFO Signed narinfos id=21 count=1 lb # [ 80.573867] niks3-server[937]: 2026/09/22 15:46:33 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete lb # [ 80.586706] niks3-server[937]: 2026/09/22 15:46:33 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls lb # [ 80.595799] niks3-server[937]: 2026/09/22 15:46:33 INFO Completed upload id=21 worker1 # [ 80.622369] nix-grpc-daemon[842]: time=2026-09-22T15:46:33.225Z level=INFO msg="Upload complete. (365ms)" lb # [ 80.600184] niks3-server[937]: 2026/09/22 15:46:33 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo worker1 # [ 80.632256] nix-grpc-daemon[765]: ts=2026-09-22T15:46:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 80.647956] nix-grpc-daemon[765]: ts=2026-09-22T15:46:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 80.654536] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 80.659263] nix-daemon[829]: reaped child process 1121, status = succeeded worker1 # [ 80.672137] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 82.135389] nix-daemon[829]: reaped child process 1135, status = succeeded worker1 # [ 82.251874] nix-daemon[829]: accepted connection from pid 1151, user nix-grpc-daemon (trusted) worker1 # [ 82.271730] nix-daemon[829]: reaped child process 1154, status = succeeded lb # [ 82.249892] niks3-server[937]: 2026/09/22 15:46:34 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 82.287789] nix-grpc-daemon[842]: time=2026-09-22T15:46:34.888Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 82.289556] nix-grpc-daemon[842]: time=2026-09-22T15:46:34.888Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)" lb # [ 82.287482] niks3-server[937]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst lb # [ 82.301781] niks3-server[937]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv lb # [ 82.314243] niks3-server[937]: 2026/09/22 15:46:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign worker1 # [ 82.348051] nix-grpc-daemon[842]: time=2026-09-22T15:46:34.950Z level=INFO msg="Uploading 1 narinfos" lb # [ 82.327091] niks3-server[937]: 2026/09/22 15:46:34 INFO Signed narinfos id=22 count=1 lb # [ 82.333149] niks3-server[937]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls lb # [ 82.342940] niks3-server[937]: 2026/09/22 15:46:34 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete lb # [ 82.351884] niks3-server[937]: 2026/09/22 15:46:34 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo worker1 # [ 82.384317] nix-grpc-daemon[842]: time=2026-09-22T15:46:34.987Z level=INFO msg="Upload complete. (302ms)" lb # [ 82.359568] niks3-server[937]: 2026/09/22 15:46:34 INFO Completed upload id=22 worker1 # [ 82.397264] nix-grpc-daemon[765]: ts=2026-09-22T15:46:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=1 worker1 # [ 82.418005] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 82.423561] nix-daemon[829]: reaped child process 1133, status = succeeded worker1 # [ 82.439471] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 82.446305] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_out=128 worker1 # [ 82.723281] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:41676 worker2 # [ 82.703233] nix-grpc-daemon[772]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 82.729940] nix-grpc-daemon[772]: ts=2026-09-22T15:46:35Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=0 worker2 # [ 82.736338] nix-grpc-daemon[772]: ts=2026-09-22T15:46:35Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 82.765013] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 82.779984] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 82.795165] nix-grpc-daemon[842]: time=2026-09-22T15:46:35.397Z level=INFO msg="All 1 paths already cached" worker1 # [ 82.805073] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 82.819751] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 82.980129] nix-daemon[829]: accepted connection from pid 1158, user nix-grpc-daemon (trusted) worker1 # [ 83.007786] nix-daemon[829]: reaped child process 1161, status = succeeded worker1 # [ 83.098028] nix-daemon[829]: accepted connection from pid 1163, user nix-grpc-daemon (trusted) worker1 # [ 83.115885] nix-daemon[829]: reaped child process 1166, status = succeeded lb # [ 83.094426] niks3-server[937]: 2026/09/22 15:46:35 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 83.130538] nix-grpc-daemon[842]: time=2026-09-22T15:46:35.733Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 83.132067] nix-grpc-daemon[842]: time=2026-09-22T15:46:35.733Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)" lb # [ 83.122522] niks3-server[937]: 2026/09/22 15:46:35 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst worker1 # [ 83.170749] nix-grpc-daemon[842]: time=2026-09-22T15:46:35.773Z level=INFO msg="Uploading 1 narinfos" lb # [ 83.137725] niks3-server[937]: 2026/09/22 15:46:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign lb # [ 83.150915] niks3-server[937]: 2026/09/22 15:46:35 INFO Signed narinfos id=23 count=1 lb # [ 83.159238] niks3-server[937]: 2026/09/22 15:46:35 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls worker1 # [ 83.198577] nix-grpc-daemon[842]: time=2026-09-22T15:46:35.801Z level=INFO msg="Upload complete. (361ms)" lb # [ 83.172737] niks3-server[937]: 2026/09/22 15:46:35 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete lb # [ 83.181478] niks3-server[937]: 2026/09/22 15:46:35 INFO Completed upload id=23 worker1 # [ 83.208467] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 83.185454] niks3-server[937]: 2026/09/22 15:46:35 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo worker1 # [ 83.219146] nix-grpc-daemon[765]: ts=2026-09-22T15:46:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 83.228500] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 83.230706] nix-daemon[829]: reaped child process 1156, status = succeeded worker1 # [ 83.242504] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 84.736805] nix-daemon[829]: reaped child process 1170, status = succeeded worker1 # [ 84.803069] nix-daemon[829]: accepted connection from pid 1186, user nix-grpc-daemon (trusted) worker1 # [ 84.820912] nix-daemon[829]: reaped child process 1189, status = succeeded lb # [ 84.801669] niks3-server[937]: 2026/09/22 15:46:37 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 84.836550] nix-grpc-daemon[842]: time=2026-09-22T15:46:37.439Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 84.840280] nix-grpc-daemon[842]: time=2026-09-22T15:46:37.441Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)" lb # [ 84.839120] niks3-server[937]: 2026/09/22 15:46:37 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv lb # [ 84.852665] niks3-server[937]: 2026/09/22 15:46:37 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst worker1 # [ 84.894567] nix-grpc-daemon[842]: time=2026-09-22T15:46:37.497Z level=INFO msg="Uploading 1 narinfos" lb # [ 84.869076] niks3-server[937]: 2026/09/22 15:46:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign lb # [ 84.880678] niks3-server[937]: 2026/09/22 15:46:37 INFO Signed narinfos id=24 count=1 lb # [ 84.887423] niks3-server[937]: 2026/09/22 15:46:37 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls lb # [ 84.897074] niks3-server[937]: 2026/09/22 15:46:37 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete lb # [ 84.905333] niks3-server[937]: 2026/09/22 15:46:37 INFO Completed upload id=24 worker1 # [ 84.931440] nix-grpc-daemon[842]: time=2026-09-22T15:46:37.534Z level=INFO msg="Upload complete. (276ms)" lb # [ 84.909062] niks3-server[937]: 2026/09/22 15:46:37 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo worker1 # [ 84.945117] nix-grpc-daemon[765]: ts=2026-09-22T15:46:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=1 worker1 # [ 84.964361] nix-grpc-daemon[765]: ts=2026-09-22T15:46:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 84.970588] nix-daemon[829]: reaped child process 1168, status = succeeded worker2 # [ 84.970170] nix-grpc-daemon[772]: ts=2026-09-22T15:46:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 84.985347] nix-grpc-daemon[772]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'... worker2 # [ 85.003373] nix-grpc-daemon[772]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'... worker2 # [ 85.018890] nix-grpc-daemon[772]: ts=2026-09-22T15:46:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 85.304248] nix-grpc-daemon[772]: ts=2026-09-22T15:46:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 85.311682] nix-grpc-daemon[772]: ts=2026-09-22T15:46:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 85.319297] nix-grpc-daemon[910]: time=2026-09-22T15:46:37.916Z level=INFO msg="All 2 paths already cached" worker2 # [ 85.330778] nix-grpc-daemon[772]: ts=2026-09-22T15:46:37Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=0 worker2 # [ 85.337290] nix-grpc-daemon[772]: ts=2026-09-22T15:46:37Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 85.369699] nix-grpc-daemon[765]: ts=2026-09-22T15:46:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 85.385371] nix-grpc-daemon[765]: ts=2026-09-22T15:46:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 85.399547] nix-grpc-daemon[842]: time=2026-09-22T15:46:38.002Z level=INFO msg="All 2 paths already cached" worker1 # [ 85.410096] nix-grpc-daemon[765]: ts=2026-09-22T15:46:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 85.424112] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 85.581765] nix-daemon[829]: accepted connection from pid 1193, user nix-grpc-daemon (trusted) worker1 # [ 85.598526] nix-daemon[829]: reaped child process 1196, status = succeeded worker1 # [ 85.701274] nix-daemon[829]: accepted connection from pid 1198, user nix-grpc-daemon (trusted) worker1 # [ 85.719757] nix-daemon[829]: reaped child process 1201, status = succeeded lb # [ 85.692004] niks3-server[937]: 2026/09/22 15:46:38 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 85.728223] nix-grpc-daemon[842]: time=2026-09-22T15:46:38.330Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 85.729660] nix-grpc-daemon[842]: time=2026-09-22T15:46:38.330Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)" lb # [ 85.722819] niks3-server[937]: 2026/09/22 15:46:38 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst lb # [ 85.738100] niks3-server[937]: 2026/09/22 15:46:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign worker1 # [ 85.771388] nix-grpc-daemon[842]: time=2026-09-22T15:46:38.374Z level=INFO msg="Uploading 1 narinfos" lb # [ 85.750927] niks3-server[937]: 2026/09/22 15:46:38 INFO Signed narinfos id=25 count=1 lb # [ 85.756578] niks3-server[937]: 2026/09/22 15:46:38 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls lb # [ 85.768277] niks3-server[937]: 2026/09/22 15:46:38 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete lb # [ 85.775036] niks3-server[937]: 2026/09/22 15:46:38 INFO Completed upload id=25 worker1 # [ 85.801581] nix-grpc-daemon[842]: time=2026-09-22T15:46:38.404Z level=INFO msg="Upload complete. (360ms)" lb # [ 85.778393] niks3-server[937]: 2026/09/22 15:46:38 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo worker1 # [ 85.808184] nix-grpc-daemon[765]: ts=2026-09-22T15:46:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 85.820406] nix-grpc-daemon[765]: ts=2026-09-22T15:46:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 85.828896] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 85.833817] nix-daemon[829]: reaped child process 1191, status = succeeded worker1 # [ 85.844241] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 86.501936] nix-daemon[829]: reaped child process 1205, status = succeeded worker1 # [ 86.622751] nix-daemon[829]: accepted connection from pid 1221, user nix-grpc-daemon (trusted) worker1 # [ 86.641281] nix-daemon[829]: reaped child process 1224, status = succeeded lb # [ 86.621556] niks3-server[937]: 2026/09/22 15:46:39 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 86.659212] nix-grpc-daemon[842]: time=2026-09-22T15:46:39.260Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 86.661060] nix-grpc-daemon[842]: time=2026-09-22T15:46:39.260Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)" lb # [ 86.653962] niks3-server[937]: 2026/09/22 15:46:39 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst lb # [ 86.674388] niks3-server[937]: 2026/09/22 15:46:39 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls worker1 # [ 86.710841] nix-grpc-daemon[842]: time=2026-09-22T15:46:39.313Z level=INFO msg="Uploading 1 narinfos" lb # [ 86.685924] niks3-server[937]: 2026/09/22 15:46:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign lb # [ 86.693340] niks3-server[937]: 2026/09/22 15:46:39 INFO Signed narinfos id=26 count=1 lb # [ 86.699723] niks3-server[937]: 2026/09/22 15:46:39 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv lb # [ 86.711758] niks3-server[937]: 2026/09/22 15:46:39 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete lb # [ 86.718202] niks3-server[937]: 2026/09/22 15:46:39 INFO Completed upload id=26 worker1 # [ 86.743143] nix-grpc-daemon[842]: time=2026-09-22T15:46:39.345Z level=INFO msg="Upload complete. (287ms)" lb # [ 86.721563] niks3-server[937]: 2026/09/22 15:46:39 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo worker1 # [ 86.757278] nix-grpc-daemon[765]: ts=2026-09-22T15:46:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=1 worker1 # [ 86.779704] nix-grpc-daemon[765]: ts=2026-09-22T15:46:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 86.784665] nix-daemon[829]: reaped child process 1203, status = succeeded worker1 # [ 86.793503] nix-grpc-daemon[765]: ts=2026-09-22T15:46:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 86.800073] nix-grpc-daemon[765]: ts=2026-09-22T15:46:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_out=232 client # [ 86.919795] nix-daemon[1089]: reaped child process 1091, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.20 seconds) client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.32 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.35 seconds) client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.17 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.30 seconds) worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.06 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 88.255617] nix-daemon[1089]: accepted connection from pid 1172, user root (trusted) client # this derivation will be built: client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker1 # [ 88.500230] nix-grpc-daemon[765]: ts=2026-09-22T15:46:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:41676 client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 88.482709] nix-grpc-daemon[772]: ts=2026-09-22T15:46:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 client # copying 1 paths... client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'... worker1 # [ 88.538960] nix-grpc-daemon[765]: ts=2026-09-22T15:46:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 88.563600] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) lb # [ 88.539127] envoy[894]: conn peer=[2001:db8:1::1]:35652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1197 ms=4869 lb # [ 88.544839] envoy[894]: conn peer=[2001:db8:1::1]:35666 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3417 tx=1942 ms=2336 lb # [ 88.550063] envoy[894]: conn peer=[2001:db8:1::1]:51814 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=74 lb # [ 88.554865] envoy[894]: conn peer=[2001:db8:1::1]:51824 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3888 tx=2156 ms=2379 lb # [ 88.559875] envoy[894]: conn peer=[2001:db8:1::1]:51828 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=114 lb # [ 88.565897] envoy[894]: conn peer=[2001:db8:1::1]:51842 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4296 tx=2397 ms=1531 lb # [ 88.577111] envoy[894]: conn peer=[2001:db8:1::1]:51846 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=72 worker1 # [ 88.871908] nix-daemon[829]: accepted connection from pid 1233, user nix-grpc-daemon (trusted) worker1 # [ 88.906213] nix-daemon[829]: reaped child process 1236, status = succeeded lb # [ 88.891718] niks3-server[937]: 2026/09/22 15:46:41 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 88.934427] nix-grpc-daemon[842]: time=2026-09-22T15:46:41.535Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 88.936421] nix-grpc-daemon[842]: time=2026-09-22T15:46:41.535Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)" lb # [ 88.953885] niks3-server[937]: 2026/09/22 15:46:41 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst lb # [ 88.971501] niks3-server[937]: 2026/09/22 15:46:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign worker1 # [ 89.007074] nix-grpc-daemon[842]: time=2026-09-22T15:46:41.608Z level=INFO msg="Uploading 1 narinfos" lb # [ 88.984618] niks3-server[937]: 2026/09/22 15:46:41 INFO Signed narinfos id=27 count=1 lb # [ 88.991196] niks3-server[937]: 2026/09/22 15:46:41 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls lb # [ 89.011368] niks3-server[937]: 2026/09/22 15:46:41 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete lb # [ 89.020075] niks3-server[937]: 2026/09/22 15:46:41 INFO Completed upload id=27 worker1 # [ 89.044965] nix-grpc-daemon[842]: time=2026-09-22T15:46:41.647Z level=INFO msg="Upload complete. (453ms)" lb # [ 89.023153] niks3-server[937]: 2026/09/22 15:46:41 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo worker1 # [ 89.056548] nix-grpc-daemon[765]: ts=2026-09-22T15:46:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_in=120 worker1 # [ 89.088304] nix-daemon[829]: reaped child process 1231, status = succeeded worker2 # [ 89.062278] nix-grpc-daemon[772]: ts=2026-09-22T15:46:41Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=0 worker2 # [ 89.072993] nix-grpc-daemon[772]: ts=2026-09-22T15:46:41Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 89.112462] nix-grpc-daemon[772]: ts=2026-09-22T15:46:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 89.137259] nix-grpc-daemon[772]: ts=2026-09-22T15:46:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 client # copying 2 paths to 'lb:50051'... worker2 # [ 89.168008] nix-grpc-daemon[772]: ts=2026-09-22T15:46:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'... client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'... worker2 # [ 89.196943] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 89.361675] nix-daemon[856]: accepted connection from pid 1165, user nix-grpc-daemon (trusted) worker2 # [ 89.385878] nix-daemon[856]: reaped child process 1168, status = succeeded worker2 # [ 89.485460] nix-daemon[856]: accepted connection from pid 1170, user nix-grpc-daemon (trusted) worker2 # [ 89.503988] nix-daemon[856]: reaped child process 1173, status = succeeded lb # [ 89.502574] niks3-server[937]: 2026/09/22 15:46:42 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 89.512448] niks3-server[937]: 2026/09/22 15:46:42 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 89.524394] nix-grpc-daemon[910]: time=2026-09-22T15:46:42.123Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 89.525923] nix-grpc-daemon[910]: time=2026-09-22T15:46:42.123Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)" worker2 # [ 89.527694] nix-grpc-daemon[910]: time=2026-09-22T15:46:42.123Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)" lb # [ 89.556126] niks3-server[937]: 2026/09/22 15:46:42 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst lb # [ 89.569787] niks3-server[937]: 2026/09/22 15:46:42 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst lb # [ 89.581970] niks3-server[937]: 2026/09/22 15:46:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign worker2 # [ 89.588647] nix-grpc-daemon[910]: time=2026-09-22T15:46:42.187Z level=INFO msg="Uploading 3 narinfos" lb # [ 89.590623] niks3-server[937]: 2026/09/22 15:46:42 INFO Signed narinfos id=28 count=1 lb # [ 89.596557] niks3-server[937]: 2026/09/22 15:46:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign lb # [ 89.599545] niks3-server[937]: 2026/09/22 15:46:42 INFO Signed narinfos id=29 count=2 lb # [ 89.606595] niks3-server[937]: 2026/09/22 15:46:42 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls lb # [ 89.613596] niks3-server[937]: 2026/09/22 15:46:42 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls lb # [ 89.625747] niks3-server[937]: 2026/09/22 15:46:42 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 89.635298] niks3-server[937]: 2026/09/22 15:46:42 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 89.641337] niks3-server[937]: 2026/09/22 15:46:42 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 89.644459] niks3-server[937]: 2026/09/22 15:46:42 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo lb # [ 89.648273] niks3-server[937]: 2026/09/22 15:46:42 INFO Completed upload id=28 worker2 # [ 89.649152] nix-grpc-daemon[910]: time=2026-09-22T15:46:42.248Z level=INFO msg="Upload complete. (431ms)" lb # [ 89.650435] niks3-server[937]: 2026/09/22 15:46:42 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete lb # [ 89.653811] niks3-server[937]: 2026/09/22 15:46:42 INFO Completed upload id=29 worker2 # [ 89.661437] nix-grpc-daemon[772]: ts=2026-09-22T15:46:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=2 nar_bytes_in=984 worker2 # [ 89.676704] nix-grpc-daemon[772]: ts=2026-09-22T15:46:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 89.691021] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 89.694992] nix-daemon[856]: reaped child process 1163, status = succeeded worker2 # [ 89.709603] nix-grpc-daemon[772]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'... worker2 # [ 89.722803] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051... client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 91.230899] nix-daemon[856]: reaped child process 1178, status = succeeded worker2 # [ 91.302686] nix-daemon[856]: accepted connection from pid 1194, user nix-grpc-daemon (trusted) worker2 # [ 91.320220] nix-daemon[856]: reaped child process 1197, status = succeeded lb # [ 91.324710] niks3-server[937]: 2026/09/22 15:46:43 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 91.335979] nix-grpc-daemon[910]: time=2026-09-22T15:46:43.933Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 91.356072] niks3-server[937]: 2026/09/22 15:46:43 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker2 # [ 91.368298] nix-grpc-daemon[910]: time=2026-09-22T15:46:43.967Z level=INFO msg="Uploading 1 narinfos" lb # [ 91.373219] niks3-server[937]: 2026/09/22 15:46:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign lb # [ 91.382192] niks3-server[937]: 2026/09/22 15:46:43 INFO Signed narinfos id=30 count=1 lb # [ 91.389722] niks3-server[937]: 2026/09/22 15:46:43 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 91.397988] niks3-server[937]: 2026/09/22 15:46:43 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls lb # [ 91.404715] niks3-server[937]: 2026/09/22 15:46:43 INFO Completed upload id=30 worker2 # [ 91.405765] nix-grpc-daemon[910]: time=2026-09-22T15:46:44.005Z level=INFO msg="Upload complete. (269ms)" lb # [ 91.408915] niks3-server[937]: 2026/09/22 15:46:43 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo worker2 # [ 91.417059] nix-grpc-daemon[772]: ts=2026-09-22T15:46:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=1 worker2 # [ 91.435595] nix-daemon[856]: reaped child process 1175, status = succeeded client # copying 1 paths... worker1 # [ 91.489879] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 91.503842] nix-grpc-daemon[765]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'... client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'... worker1 # [ 91.550431] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 91.557633] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_out=120 client # [ 91.665336] nix-daemon[1089]: reaped child process 1175, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.74 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 91.833400] nix-daemon[1089]: accepted connection from pid 1193, user root (trusted) client # this derivation will be built: client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 91.997665] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:41676 client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 92.006025] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 92.014670] nix-grpc-daemon[842]: time=2026-09-22T15:46:44.614Z level=INFO msg="All 1 paths already cached" worker2 # [ 91.999988] nix-grpc-daemon[772]: ts=2026-09-22T15:46:44Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=0 worker2 # [ 92.007398] nix-grpc-daemon[772]: ts=2026-09-22T15:46:44Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 92.070887] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 92.087165] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41676 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'... worker1 # [ 92.108659] nix-grpc-daemon[765]: ts=2026-09-22T15:46:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 92.124774] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 92.148734] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 1/5) worker1 # [ 92.254518] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 2/5) worker1 # [ 92.322528] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 135 ms (attempt 3/5) worker1 # [ 92.459506] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 467 ms (attempt 4/5) worker1 # [ 92.930058] nix-grpc-daemon[765]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 92.938706] nix-grpc-daemon[765]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 92.946294] nix-grpc-daemon[765]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'... worker1 # [ 93.122337] nix-daemon[829]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) worker1 # [ 93.141542] nix-daemon[829]: reaped child process 1249, status = succeeded worker1 # [ 93.247511] nix-daemon[829]: accepted connection from pid 1251, user nix-grpc-daemon (trusted) worker1 # [ 93.262227] nix-daemon[829]: reaped child process 1254, status = succeeded lb # [ 93.244812] niks3-server[937]: 2026/09/22 15:46:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 93.280991] nix-grpc-daemon[842]: time=2026-09-22T15:46:45.883Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 93.282436] nix-grpc-daemon[842]: time=2026-09-22T15:46:45.883Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)" lb # [ 93.273199] niks3-server[937]: 2026/09/22 15:46:45 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst worker1 # [ 93.312712] nix-grpc-daemon[842]: time=2026-09-22T15:46:45.915Z level=INFO msg="Uploading 1 narinfos" lb # [ 93.287254] niks3-server[937]: 2026/09/22 15:46:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign lb # [ 93.299163] niks3-server[937]: 2026/09/22 15:46:45 INFO Signed narinfos id=31 count=1 lb # [ 93.304973] niks3-server[937]: 2026/09/22 15:46:45 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls worker1 # [ 93.339254] nix-grpc-daemon[842]: time=2026-09-22T15:46:45.942Z level=INFO msg="Upload complete. (379ms)" lb # [ 93.314757] niks3-server[937]: 2026/09/22 15:46:45 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete lb # [ 93.321232] niks3-server[937]: 2026/09/22 15:46:45 INFO Completed upload id=31 worker1 # [ 93.348255] nix-grpc-daemon[765]: ts=2026-09-22T15:46:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=1 paths=1 nar_bytes_in=568 lb # [ 93.323598] niks3-server[937]: 2026/09/22 15:46:45 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo worker1 # [ 93.355158] nix-grpc-daemon[765]: ts=2026-09-22T15:46:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 93.366213] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) worker1 # [ 93.370750] nix-daemon[829]: reaped child process 1239, status = succeeded client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051... worker1 # [ 93.382404] nix-daemon[829]: accepted connection from pid 765, user nix-grpc-daemon (trusted) client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 94.689913] nix-daemon[829]: reaped child process 1258, status = succeeded worker1 # [ 94.759624] nix-daemon[829]: accepted connection from pid 1274, user nix-grpc-daemon (trusted) worker1 # [ 94.775855] nix-daemon[829]: reaped child process 1277, status = succeeded lb # [ 94.757578] niks3-server[937]: 2026/09/22 15:46:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 94.793145] nix-grpc-daemon[842]: time=2026-09-22T15:46:47.395Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 94.789861] niks3-server[937]: 2026/09/22 15:46:47 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls worker1 # [ 94.827514] nix-grpc-daemon[842]: time=2026-09-22T15:46:47.430Z level=INFO msg="Uploading 1 narinfos" lb # [ 94.802746] niks3-server[937]: 2026/09/22 15:46:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign lb # [ 94.814285] niks3-server[937]: 2026/09/22 15:46:47 INFO Signed narinfos id=32 count=1 lb # [ 94.819229] niks3-server[937]: 2026/09/22 15:46:47 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv lb # [ 94.831769] niks3-server[937]: 2026/09/22 15:46:47 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete worker1 # [ 94.856986] nix-grpc-daemon[842]: time=2026-09-22T15:46:47.459Z level=INFO msg="Upload complete. (264ms)" lb # [ 94.839749] niks3-server[937]: 2026/09/22 15:46:47 INFO Completed upload id=32 lb # [ 94.843683] niks3-server[937]: 2026/09/22 15:46:47 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo worker1 # [ 94.869202] nix-grpc-daemon[765]: ts=2026-09-22T15:46:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=1 worker1 # [ 94.884705] nix-daemon[829]: reaped child process 1256, status = succeeded client # copying 1 paths... worker2 # [ 94.893376] nix-grpc-daemon[772]: ts=2026-09-22T15:46:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 94.905796] nix-grpc-daemon[772]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'... client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'... worker1 # [ 94.975421] nix-grpc-daemon[765]: ts=2026-09-22T15:46:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 94.983840] nix-grpc-daemon[765]: ts=2026-09-22T15:46:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_out=120 client # [ 95.052681] nix-daemon[1089]: reaped child process 1196, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.39 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 95.229522] nix-daemon[1089]: accepted connection from pid 1215, user root (trusted) client # this derivation will be built: client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker2 # [ 95.368130] nix-grpc-daemon[772]: ts=2026-09-22T15:46:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:60348 client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 95.403046] nix-grpc-daemon[765]: ts=2026-09-22T15:46:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 95.412817] nix-grpc-daemon[842]: time=2026-09-22T15:46:48.015Z level=INFO msg="All 1 paths already cached" worker2 # [ 95.405010] nix-grpc-daemon[772]: ts=2026-09-22T15:46:48Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=0 worker2 # [ 95.412896] nix-grpc-daemon[772]: ts=2026-09-22T15:46:48Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 95.422773] nix-grpc-daemon[772]: ts=2026-09-22T15:46:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 95.434054] nix-grpc-daemon[772]: ts=2026-09-22T15:46:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:60348 client # copying 1 paths to 'lb:50051'... worker2 # [ 95.450658] nix-grpc-daemon[910]: time=2026-09-22T15:46:48.047Z level=INFO msg="All 1 paths already cached" client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'... worker2 # [ 95.456394] nix-grpc-daemon[772]: ts=2026-09-22T15:46:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 95.471593] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 95.622120] nix-daemon[856]: accepted connection from pid 1206, user nix-grpc-daemon (trusted) worker2 # [ 95.647190] nix-daemon[856]: reaped child process 1209, status = succeeded worker2 # [ 95.746444] nix-daemon[856]: accepted connection from pid 1211, user nix-grpc-daemon (trusted) worker2 # [ 95.768713] nix-daemon[856]: reaped child process 1214, status = succeeded lb # [ 95.771915] niks3-server[937]: 2026/09/22 15:46:48 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 95.783365] nix-grpc-daemon[910]: time=2026-09-22T15:46:48.382Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 95.785051] nix-grpc-daemon[910]: time=2026-09-22T15:46:48.382Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)" lb # [ 95.806121] niks3-server[937]: 2026/09/22 15:46:48 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst worker2 # [ 95.823718] nix-grpc-daemon[910]: time=2026-09-22T15:46:48.422Z level=INFO msg="Uploading 1 narinfos" lb # [ 95.824445] niks3-server[937]: 2026/09/22 15:46:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign lb # [ 95.834605] niks3-server[937]: 2026/09/22 15:46:48 INFO Signed narinfos id=33 count=1 lb # [ 95.839796] niks3-server[937]: 2026/09/22 15:46:48 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls worker2 # [ 95.852088] nix-grpc-daemon[910]: time=2026-09-22T15:46:48.451Z level=INFO msg="Upload complete. (364ms)" lb # [ 95.852536] niks3-server[937]: 2026/09/22 15:46:48 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete lb # [ 95.855333] niks3-server[937]: 2026/09/22 15:46:48 INFO Completed upload id=33 lb # [ 95.859076] niks3-server[937]: 2026/09/22 15:46:48 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo worker2 # [ 95.860093] nix-grpc-daemon[772]: ts=2026-09-22T15:46:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=0 paths=1 nar_bytes_in=568 worker2 # [ 95.868752] nix-grpc-daemon[772]: ts=2026-09-22T15:46:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 worker2 # [ 95.879587] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) worker2 # [ 95.884856] nix-daemon[856]: reaped child process 1204, status = succeeded client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051... worker2 # [ 95.897592] nix-daemon[856]: accepted connection from pid 772, user nix-grpc-daemon (trusted) client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 97.206699] nix-daemon[856]: reaped child process 1218, status = succeeded worker2 # [ 97.280462] nix-daemon[856]: accepted connection from pid 1234, user nix-grpc-daemon (trusted) worker2 # [ 97.302312] nix-daemon[856]: reaped child process 1237, status = succeeded lb # [ 97.302033] niks3-server[937]: 2026/09/22 15:46:49 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 97.314226] nix-grpc-daemon[910]: time=2026-09-22T15:46:49.911Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 97.335591] niks3-server[937]: 2026/09/22 15:46:49 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker2 # [ 97.346661] nix-grpc-daemon[910]: time=2026-09-22T15:46:49.945Z level=INFO msg="Uploading 1 narinfos" lb # [ 97.350452] niks3-server[937]: 2026/09/22 15:46:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign lb # [ 97.358801] niks3-server[937]: 2026/09/22 15:46:49 INFO Signed narinfos id=34 count=1 lb # [ 97.371803] niks3-server[937]: 2026/09/22 15:46:49 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 97.379518] niks3-server[937]: 2026/09/22 15:46:49 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls lb # [ 97.383481] niks3-server[937]: 2026/09/22 15:46:49 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo lb # [ 97.388253] niks3-server[937]: 2026/09/22 15:46:49 INFO Completed upload id=34 worker2 # [ 97.388600] nix-grpc-daemon[910]: time=2026-09-22T15:46:49.987Z level=INFO msg="Upload complete. (280ms)" worker2 # [ 97.401250] nix-grpc-daemon[772]: ts=2026-09-22T15:46:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60348 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=1 worker2 # [ 97.418948] nix-daemon[856]: reaped child process 1216, status = succeeded client # copying 1 paths... worker2 # [ 97.446954] nix-grpc-daemon[772]: ts=2026-09-22T15:46:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:60348 client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'... worker1 # [ 97.505917] nix-grpc-daemon[765]: ts=2026-09-22T15:46:50Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 97.518765] nix-grpc-daemon[765]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'... worker1 # [ 97.535509] nix-grpc-daemon[765]: ts=2026-09-22T15:46:50Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:41676 duration_s=0 paths=1 nar_bytes_out=120 client # [ 97.640713] nix-daemon[1089]: reaped child process 1218, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.59 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 10.59 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds) client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds) client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure lb # [ 98.536550] envoy[894]: conn peer=[2001:db8:1::1]:37746 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=68 lb # [ 98.541178] envoy[894]: conn peer=[2001:db8:1::1]:51854 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4267 tx=2117 ms=3128 lb # [ 98.545981] envoy[894]: conn peer=[2001:db8:1::1]:37748 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3773 tx=2142 ms=3002 lb # [ 98.550776] envoy[894]: conn peer=[2001:db8:1::1]:37752 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=30 lb # [ 98.555423] envoy[894]: conn peer=[2001:db8:1::1]:37762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3756 tx=2125 ms=2200 lb # [ 98.560143] envoy[894]: conn peer=[2001:db8:1::1]:37774 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=94 worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.21 seconds) worker2 # [ 98.663348] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 98.858032] nix-grpc-daemon[772]: ts=2026-09-22T15:46:51Z level=info event=signal_exit lb # [ 99.825986] niks3-server[937]: 2026/09/22 15:46:52 INFO lead: released remote=192.168.1.4:57128 lb # [ 100.809692] niks3-server[937]: 2026/09/22 15:46:53 INFO lead: acquired remote=192.168.1.3:35414 worker1 # [ 100.837730] nix-grpc-daemon[765]: ts=2026-09-22T15:46:53Z level=info event=scheduler_take_over worker1 # [ 100.853714] nix-grpc-daemon[765]: ts=2026-09-22T15:46:53Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 100.829946] nix-grpc-daemon[772]: ts=2026-09-22T15:46:53Z level=info event=scheduler_restarting worker2 # [ 100.830341] nix-grpc-daemon[772]: ts=2026-09-22T15:46:53Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 100.830468] nix-grpc-daemon[772]: ts=2026-09-22T15:46:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:60336 worker2 # [ 100.830583] nix-grpc-daemon[772]: ts=2026-09-22T15:46:53Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 100.850307] nix-grpc-daemon[772]: ts=2026-09-22T15:46:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:60336 worker2 # [ 100.855281] nix-grpc-daemon[772]: ts=2026-09-22T15:46:53Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 100.886373] nix-daemon[856]: reaped child process 858, status = succeeded worker1 # [ 101.055098] nix-grpc-daemon[765]: ts=2026-09-22T15:46:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 101.027353] nix-grpc-daemon[772]: E0000 00:00:1790092013.626389 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true worker1 # [ 101.075496] nix-grpc-daemon[765]: ts=2026-09-22T15:46:53Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:40684 worker1 # [ 101.081726] nix-grpc-daemon[765]: ts=2026-09-22T15:46:53Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 101.060661] nix-grpc-daemon[772]: E0000 00:00:1790092013.626619 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker2 # [ 101.067322] nix-grpc-daemon[772]: ts=2026-09-22T15:46:53Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 102.944570] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 102.951921] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 102.954762] systemd[1]: nix-grpc-daemon.service: Consumed 3.395s CPU time over 1min 24.305s wall clock time, 63.2M memory peak, 87.2K incoming IP traffic, 211.8K outgoing IP traffic. worker2 # [ 102.970539] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 102.978460] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 102.981015] systemd[1]: nix-grpc-daemon.socket: Consumed 712us CPU time over 1min 32.699s wall clock time, 256K memory peak, 113.2K incoming IP traffic, 95.9K outgoing IP traffic. lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2 worker1 # [ 103.315595] nix-grpc-daemon[765]: ts=2026-09-22T15:46:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41676 worker1 # [ 103.324283] nix-grpc-daemon[765]: ts=2026-09-22T15:46:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:41676 client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.26 seconds) worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds) worker2 # [ 103.422857] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 103.520963] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 103.827486] nix-grpc-daemon[765]: ts=2026-09-22T15:46:56Z level=info event=signal_exit lb # [ 103.818020] niks3-server[937]: 2026/09/22 15:46:56 INFO lead: released remote=192.168.1.3:35414 worker2 # [ 103.943143] systemd[1]: Starting scheduler-token.service... worker2 # [ 104.031818] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 104.041538] systemd[1]: Finished scheduler-token.service. worker2 # [ 104.043515] systemd[1]: scheduler-token.service: Consumed 59ms CPU time over 92ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 104.056591] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 104.194344] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:56Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 104.196695] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:56Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 104.200618] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:56Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 104.207535] nix-grpc-daemon[1261]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 104.221319] nix-grpc-daemon[1261]: E0000 00:00:1790092016.804547 1261 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 104.224833] niks3-server[937]: 2026/09/22 15:46:56 INFO lead: acquired remote=192.168.1.4:33828 worker2 # [ 104.228109] nix-grpc-daemon[1261]: E0000 00:00:1790092016.806767 1261 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 104.256164] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:56Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 104.257858] nix-grpc-daemon[1261]: E0000 00:00:1790092016.823616 1300 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 104.263571] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:56Z level=info event=scheduler_take_over worker2 # [ 104.265208] nix-grpc-daemon[1261]: E0000 00:00:1790092016.843302 1300 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 104.267647] nix-grpc-daemon[1261]: E0000 00:00:1790092016.843486 1300 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 104.273000] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 104.275875] nix-grpc-daemon[1261]: E0000 00:00:1790092016.862074 1300 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 104.278539] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 104.279812] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 104.477437] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 104.482247] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 104.683395] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 104.688818] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 104.852310] nix-grpc-daemon[765]: ts=2026-09-22T15:46:57Z level=info event=scheduler_restarting worker1 # [ 104.856242] nix-grpc-daemon[765]: ts=2026-09-22T15:46:57Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 104.860302] nix-grpc-daemon[765]: ts=2026-09-22T15:46:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:40684 worker1 # [ 104.868125] nix-grpc-daemon[765]: ts=2026-09-22T15:46:57Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 104.896115] nix-daemon[829]: reaped child process 831, status = succeeded worker2 # [ 104.889883] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 104.894931] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 104.992345] nix-grpc-daemon[765]: E0000 00:00:1790092017.594927 810 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true worker1 # [ 104.998937] nix-grpc-daemon[765]: E0000 00:00:1790092017.595204 810 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker1 # [ 105.069377] nix-grpc-daemon[765]: ts=2026-09-22T15:46:57Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 105.096001] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 105.100996] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 105.304998] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 105.319185] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 105.323817] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:49132 worker2 # [ 105.329493] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:57Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 106.905877] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 106.910901] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 106.916482] systemd[1]: nix-grpc-daemon.service: Consumed 4.744s CPU time over 1min 28.339s wall clock time, 65.1M memory peak, 102.2K incoming IP traffic, 182.6K outgoing IP traffic. worker1 # [ 106.932263] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 106.938977] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 106.941570] systemd[1]: nix-grpc-daemon.socket: Consumed 562us CPU time over 1min 36.769s wall clock time, 256K memory peak, 109.1K incoming IP traffic, 97.8K outgoing IP traffic. lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2 worker2 # [ 107.223424] nix-grpc-daemon[1261]: ts=2026-09-22T15:46:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49140 worker2 # [ 107.241190] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker2 # [ 107.267865] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 1/5) worker2 # [ 107.361523] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 152 ms (attempt 2/5) worker2 # [ 107.515644] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 3/5) worker2 # [ 107.562238] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 172 ms (attempt 4/5) worker2 # [ 107.737224] nix-grpc-daemon[1261]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 107.743851] nix-grpc-daemon[1261]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 107.767311] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49140 client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 107.783835] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker2 # [ 107.812731] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:00Z level=info event=niks3_push_started pid=1321 worker2 # [ 107.962497] nix-grpc-daemon[1321]: time=2026-09-22T15:47:00.561Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 108.074653] nix-daemon[856]: accepted connection from pid 1334, user nix-grpc-daemon (trusted) worker2 # [ 108.094495] nix-daemon[856]: reaped child process 1337, status = succeeded lb # [ 108.099119] niks3-server[937]: 2026/09/22 15:47:00 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 108.109702] niks3-server[937]: 2026/09/22 15:47:00 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 108.121396] nix-grpc-daemon[1321]: time=2026-09-22T15:47:00.718Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 108.122935] nix-grpc-daemon[1321]: time=2026-09-22T15:47:00.718Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)" worker2 # [ 108.126426] nix-grpc-daemon[1321]: time=2026-09-22T15:47:00.719Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)" lb # [ 108.157580] niks3-server[937]: 2026/09/22 15:47:00 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 108.171216] niks3-server[937]: 2026/09/22 15:47:00 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst lb # [ 108.184417] niks3-server[937]: 2026/09/22 15:47:00 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls lb # [ 108.195315] niks3-server[937]: 2026/09/22 15:47:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 108.208368] niks3-server[937]: 2026/09/22 15:47:00 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls lb # [ 108.224192] niks3-server[937]: 2026/09/22 15:47:00 INFO Signed narinfos id=36 count=2 lb # [ 108.227933] niks3-server[937]: 2026/09/22 15:47:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign worker2 # [ 108.230255] nix-grpc-daemon[1321]: time=2026-09-22T15:47:00.827Z level=INFO msg="Uploading 3 narinfos" lb # [ 108.235203] niks3-server[937]: 2026/09/22 15:47:00 INFO Signed narinfos id=35 count=1 lb # [ 108.258739] niks3-server[937]: 2026/09/22 15:47:00 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo lb # [ 108.271457] niks3-server[937]: 2026/09/22 15:47:00 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 108.279455] niks3-server[937]: 2026/09/22 15:47:00 INFO Completed upload id=35 lb # [ 108.284806] niks3-server[937]: 2026/09/22 15:47:00 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete lb # [ 108.286835] niks3-server[937]: 2026/09/22 15:47:00 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo worker2 # [ 108.293994] nix-grpc-daemon[1321]: time=2026-09-22T15:47:00.893Z level=INFO msg="Upload complete. (327ms)" lb # [ 108.296874] niks3-server[937]: 2026/09/22 15:47:00 INFO Completed upload id=36 lb # [ 108.299132] niks3-server[937]: 2026/09/22 15:47:00 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo worker2 # [ 108.308662] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49140 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 108.325714] nix-daemon[856]: reaped child process 1319, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.34 seconds) worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds) worker1 # [ 108.473170] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 108.482093] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 108.538184] envoy[894]: conn peer=[2001:db8:1::4]:46336 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=47244 tx=2772 ms=82141 lb # [ 108.543579] envoy[894]: conn peer=[2001:db8:1::1]:40618 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=29 lb # [ 108.548428] envoy[894]: conn peer=[2001:db8:1::3]:37948 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=10545 tx=3082 ms=86196 lb # [ 108.553346] envoy[894]: conn peer=[2001:db8:1::1]:40624 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1009 tx=282 ms=1099 worker1 # [ 108.627045] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 108.629465] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 108.633514] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 108.638957] nix-grpc-daemon[1301]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 108.656260] nix-grpc-daemon[1301]: E0000 00:00:1790092021.241318 1301 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 108.688327] nix-grpc-daemon[1301]: E0000 00:00:1790092021.244962 1301 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 108.691206] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 108.693183] nix-grpc-daemon[1301]: E0000 00:00:1790092021.253710 1342 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 108.695520] nix-grpc-daemon[1301]: E0000 00:00:1790092021.273597 1342 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 108.697936] nix-grpc-daemon[1301]: E0000 00:00:1790092021.273790 1342 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 108.703485] nix-grpc-daemon[1301]: E0000 00:00:1790092021.290273 1342 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 108.707882] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 108.708954] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 108.683880] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:01Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:49132 worker2 # [ 108.690084] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:01Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.15 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb # [ 109.923794] systemd[1]: Stopping niks3 server... worker1 # [ 109.951277] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:02Z level=info event=scheduler_lock_lost status=200 lb # [ 109.926823] niks3-server[937]: 2026/09/22 15:47:02 INFO Shutdown signal received, draining in-flight requests timeout=10s worker2 # [ 109.926842] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:02Z level=info event=scheduler_lock_lost status=200 lb # [ 109.926932] niks3-server[937]: 2026/09/22 15:47:02 INFO lead: released remote=192.168.1.4:33828 lb # [ 109.950791] systemd[1]: niks3.service: Deactivated successfully. lb # [ 109.952511] systemd[1]: Stopped niks3 server. lb # [ 109.954330] systemd[1]: niks3.service: Consumed 665ms CPU time over 1min 29.376s wall clock time, 27.4M memory peak, 96.5K incoming IP traffic, 139.1K outgoing IP traffic. lb # [ 109.965029] systemd[1]: Starting niks3 server... lb # [ 110.030424] niks3-server[1077]: 2026/09/22 15:47:02 goose: no migrations to run. current version: 20260920000000 lb # [ 110.035328] niks3-server[1077]: 2026/09/22 15:47:02 OK 1_commit_pending_closure.sql (6.34ms) lb # [ 110.039354] niks3-server[1077]: 2026/09/22 15:47:02 OK 2_object_stats_trigger.sql (3.67ms) lb # [ 110.041236] niks3-server[1077]: 2026/09/22 15:47:02 goose: up to current file version: 2 lb # [ 110.043176] niks3-server[1077]: 2026/09/22 15:47:02 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 110.053137] niks3-server[1077]: 2026/09/22 15:47:02 INFO Read proxy enabled — serving cache objects from S3 lb # [ 110.054444] niks3-server[1077]: 2026/09/22 15:47:02 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 110.059697] niks3-server[1077]: 2026/09/22 15:47:02 INFO systemd watchdog enabled interval=15s lb # [ 110.061576] niks3-server[1077]: 2026/09/22 15:47:02 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 110.065278] systemd[1]: Started niks3 server. lb: waiting for unit niks3.service lb: (finished: waiting for unit niks3.service, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2 worker2 # [ 110.561063] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:49140 client # don't know how to build these paths: client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 110.573496] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49140 worker2 # [ 110.604137] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=0 worker2 # [ 110.610304] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 110.656581] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49140 worker2 # [ 110.672113] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49140 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'... worker2 # [ 110.687759] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49140 worker2 # [ 110.703030] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker2 # [ 110.865844] nix-daemon[856]: accepted connection from pid 1354, user nix-grpc-daemon (trusted) worker2 # [ 110.885910] nix-daemon[856]: reaped child process 1357, status = succeeded lb # [ 110.930222] niks3-server[1077]: 2026/09/22 15:47:03 INFO lead: acquired remote=192.168.1.4:55142 worker2 # [ 110.989409] nix-daemon[856]: accepted connection from pid 1359, user nix-grpc-daemon (trusted) worker2 # [ 111.008689] nix-daemon[856]: reaped child process 1364, status = succeeded lb # [ 111.006403] niks3-server[1077]: 2026/09/22 15:47:03 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 111.036169] nix-grpc-daemon[1321]: time=2026-09-22T15:47:03.634Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 111.038099] nix-grpc-daemon[1321]: time=2026-09-22T15:47:03.634Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)" lb # [ 111.076810] niks3-server[1077]: 2026/09/22 15:47:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign lb # [ 111.092256] niks3-server[1077]: 2026/09/22 15:47:03 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst lb # [ 111.112621] niks3-server[1077]: 2026/09/22 15:47:03 INFO Signed narinfos id=37 count=1 lb # [ 111.117740] niks3-server[1077]: 2026/09/22 15:47:03 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls worker2 # [ 111.118894] nix-grpc-daemon[1321]: time=2026-09-22T15:47:03.717Z level=INFO msg="Uploading 1 narinfos" lb # [ 111.125674] niks3-server[1077]: 2026/09/22 15:47:03 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete lb # [ 111.134788] niks3-server[1077]: 2026/09/22 15:47:03 INFO Completed upload id=37 worker2 # [ 111.135838] nix-grpc-daemon[1321]: time=2026-09-22T15:47:03.735Z level=INFO msg="Upload complete. (415ms)" lb # [ 111.139899] niks3-server[1077]: 2026/09/22 15:47:03 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo worker2 # [ 111.148251] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49140 duration_s=0 paths=1 nar_bytes_in=544 worker2 # [ 111.160891] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49140 worker2 # [ 111.175555] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker2 # [ 111.181888] nix-daemon[856]: reaped child process 1352, status = succeeded client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051... worker2 # [ 111.193607] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker2 # [ 112.072084] nix-daemon[856]: reaped child process 1368, status = succeeded worker2 # [ 112.172787] nix-daemon[856]: accepted connection from pid 1384, user nix-grpc-daemon (trusted) worker2 # [ 112.191214] nix-daemon[856]: reaped child process 1387, status = succeeded lb # [ 112.198600] niks3-server[1077]: 2026/09/22 15:47:04 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 112.215128] nix-grpc-daemon[1321]: time=2026-09-22T15:47:04.811Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 112.217300] nix-grpc-daemon[1321]: time=2026-09-22T15:47:04.811Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)" lb # [ 112.236649] niks3-server[1077]: 2026/09/22 15:47:04 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst worker2 # [ 112.256072] nix-grpc-daemon[1321]: time=2026-09-22T15:47:04.855Z level=INFO msg="Uploading 1 narinfos" lb # [ 112.256721] niks3-server[1077]: 2026/09/22 15:47:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign lb # [ 112.273795] niks3-server[1077]: 2026/09/22 15:47:04 INFO Signed narinfos id=38 count=1 lb # [ 112.280971] niks3-server[1077]: 2026/09/22 15:47:04 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls lb # [ 112.290676] niks3-server[1077]: 2026/09/22 15:47:04 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv lb # [ 112.302622] niks3-server[1077]: 2026/09/22 15:47:04 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete worker2 # [ 112.303795] nix-grpc-daemon[1321]: time=2026-09-22T15:47:04.902Z level=INFO msg="Upload complete. (298ms)" lb # [ 112.308264] niks3-server[1077]: 2026/09/22 15:47:04 INFO Completed upload id=38 lb # [ 112.312688] niks3-server[1077]: 2026/09/22 15:47:04 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo worker2 # [ 112.327730] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49140 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 worker2 # [ 112.344417] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:04Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=0 worker2 # [ 112.349839] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:04Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 112.354901] nix-daemon[856]: reaped child process 1366, status = succeeded worker2 # [ 112.373670] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49140 worker2 # [ 112.390507] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49140 worker2 # [ 112.406587] nix-grpc-daemon[1321]: time=2026-09-22T15:47:05.005Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'... worker2 # [ 112.417682] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49140 worker2 # [ 112.431577] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker2 # [ 112.582457] nix-daemon[856]: accepted connection from pid 1393, user nix-grpc-daemon (trusted) worker2 # [ 112.606306] nix-daemon[856]: reaped child process 1396, status = succeeded worker2 # [ 112.703717] nix-daemon[856]: accepted connection from pid 1398, user nix-grpc-daemon (trusted) worker2 # [ 112.720479] nix-daemon[856]: reaped child process 1401, status = succeeded lb # [ 112.725619] niks3-server[1077]: 2026/09/22 15:47:05 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 112.739009] nix-grpc-daemon[1321]: time=2026-09-22T15:47:05.338Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 112.740711] nix-grpc-daemon[1321]: time=2026-09-22T15:47:05.338Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)" lb # [ 112.760381] niks3-server[1077]: 2026/09/22 15:47:05 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst lb # [ 112.776915] niks3-server[1077]: 2026/09/22 15:47:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign worker2 # [ 112.782780] nix-grpc-daemon[1321]: time=2026-09-22T15:47:05.381Z level=INFO msg="Uploading 1 narinfos" lb # [ 112.788264] niks3-server[1077]: 2026/09/22 15:47:05 INFO Signed narinfos id=39 count=1 lb # [ 112.793103] niks3-server[1077]: 2026/09/22 15:47:05 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls lb # [ 112.802525] niks3-server[1077]: 2026/09/22 15:47:05 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete lb # [ 112.809946] niks3-server[1077]: 2026/09/22 15:47:05 INFO Completed upload id=39 worker2 # [ 112.809951] nix-grpc-daemon[1321]: time=2026-09-22T15:47:05.409Z level=INFO msg="Upload complete. (362ms)" lb # [ 112.811999] niks3-server[1077]: 2026/09/22 15:47:05 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo worker2 # [ 112.820090] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49140 duration_s=0 paths=1 nar_bytes_in=656 worker2 # [ 112.835344] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49140 worker2 # [ 112.841580] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker2 # [ 112.846618] nix-daemon[856]: reaped child process 1391, status = succeeded worker2 # [ 112.856805] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051... client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker2 # [ 114.344480] nix-daemon[856]: reaped child process 1405, status = succeeded worker2 # [ 114.442830] nix-daemon[856]: accepted connection from pid 1421, user nix-grpc-daemon (trusted) worker2 # [ 114.463288] nix-daemon[856]: reaped child process 1424, status = succeeded lb # [ 114.467256] niks3-server[1077]: 2026/09/22 15:47:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 114.478785] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.077Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 114.481046] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.077Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)" lb # [ 114.506037] niks3-server[1077]: 2026/09/22 15:47:07 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst lb # [ 114.519670] niks3-server[1077]: 2026/09/22 15:47:07 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker2 # [ 114.531124] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.130Z level=INFO msg="Uploading 1 narinfos" lb # [ 114.532336] niks3-server[1077]: 2026/09/22 15:47:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign lb # [ 114.535664] niks3-server[1077]: 2026/09/22 15:47:07 INFO Signed narinfos id=40 count=1 lb # [ 114.546347] niks3-server[1077]: 2026/09/22 15:47:07 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls worker2 # [ 114.551955] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.151Z level=INFO msg="Upload complete. (283ms)" lb # [ 114.553270] niks3-server[1077]: 2026/09/22 15:47:07 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete lb # [ 114.556073] niks3-server[1077]: 2026/09/22 15:47:07 INFO Completed upload id=40 lb # [ 114.558427] niks3-server[1077]: 2026/09/22 15:47:07 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo worker2 # [ 114.572193] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49140 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 worker2 # [ 114.588524] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=0 worker2 # [ 114.593682] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 114.598129] nix-daemon[856]: reaped child process 1403, status = succeeded worker2 # [ 114.617809] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49140 worker2 # [ 114.634146] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:49140 worker2 # [ 114.648707] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.247Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'... worker2 # [ 114.660316] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49140 worker2 # [ 114.673870] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker2 # [ 114.836794] nix-daemon[856]: accepted connection from pid 1430, user nix-grpc-daemon (trusted) worker2 # [ 114.860947] nix-daemon[856]: reaped child process 1433, status = succeeded worker2 # [ 114.957174] nix-daemon[856]: accepted connection from pid 1435, user nix-grpc-daemon (trusted) worker2 # [ 114.976378] nix-daemon[856]: reaped child process 1438, status = succeeded lb # [ 114.974346] niks3-server[1077]: 2026/09/22 15:47:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 114.986273] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.585Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 114.988566] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.587Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)" lb # [ 115.008165] niks3-server[1077]: 2026/09/22 15:47:07 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst worker2 # [ 115.025175] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.624Z level=INFO msg="Uploading 1 narinfos" lb # [ 115.025075] niks3-server[1077]: 2026/09/22 15:47:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign lb # [ 115.034807] niks3-server[1077]: 2026/09/22 15:47:07 INFO Signed narinfos id=41 count=1 lb # [ 115.041582] niks3-server[1077]: 2026/09/22 15:47:07 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls lb # [ 115.054494] niks3-server[1077]: 2026/09/22 15:47:07 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete worker2 # [ 115.062169] nix-grpc-daemon[1321]: time=2026-09-22T15:47:07.661Z level=INFO msg="Upload complete. (371ms)" lb # [ 115.062691] niks3-server[1077]: 2026/09/22 15:47:07 INFO Completed upload id=41 lb # [ 115.067991] niks3-server[1077]: 2026/09/22 15:47:07 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo worker2 # [ 115.072336] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49140 duration_s=0 paths=1 nar_bytes_in=784 worker2 # [ 115.093045] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49140 worker2 # [ 115.100554] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) worker2 # [ 115.107001] nix-daemon[856]: reaped child process 1428, status = succeeded client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051... worker2 # [ 115.120603] nix-daemon[856]: accepted connection from pid 1261, user nix-grpc-daemon (trusted) client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker2 # [ 115.800686] nix-daemon[856]: reaped child process 1442, status = succeeded worker2 # [ 115.907124] nix-daemon[856]: accepted connection from pid 1458, user nix-grpc-daemon (trusted) worker2 # [ 115.927285] nix-daemon[856]: reaped child process 1461, status = succeeded lb # [ 115.934014] niks3-server[1077]: 2026/09/22 15:47:08 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 115.945833] nix-grpc-daemon[1321]: time=2026-09-22T15:47:08.543Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 115.947543] nix-grpc-daemon[1321]: time=2026-09-22T15:47:08.543Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)" lb # [ 115.970398] niks3-server[1077]: 2026/09/22 15:47:08 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst lb # [ 115.984411] niks3-server[1077]: 2026/09/22 15:47:08 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 115.995325] nix-grpc-daemon[1321]: time=2026-09-22T15:47:08.594Z level=INFO msg="Uploading 1 narinfos" lb # [ 115.997762] niks3-server[1077]: 2026/09/22 15:47:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign lb # [ 116.009992] niks3-server[1077]: 2026/09/22 15:47:08 INFO Signed narinfos id=42 count=1 lb # [ 116.018926] niks3-server[1077]: 2026/09/22 15:47:08 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls lb # [ 116.033808] niks3-server[1077]: 2026/09/22 15:47:08 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 116.038526] niks3-server[1077]: 2026/09/22 15:47:08 INFO Completed upload id=42 worker2 # [ 116.038130] nix-grpc-daemon[1321]: time=2026-09-22T15:47:08.637Z level=INFO msg="Upload complete. (305ms)" lb # [ 116.045028] niks3-server[1077]: 2026/09/22 15:47:08 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo worker2 # [ 116.051290] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49140 duration_s=0 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 worker2 # [ 116.071975] nix-daemon[856]: reaped child process 1440, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 5.75 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 6.48 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2 # [ 116.524975] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 117.224820] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:09Z level=info event=signal_exit lb # [ 117.935958] niks3-server[1077]: 2026/09/22 15:47:10 INFO lead: released remote=192.168.1.4:55142 lb # [ 118.537446] envoy[894]: conn peer=[2001:db8:1::1]:40638 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9766 tx=4492 ms=5509 worker1 # [ 118.963524] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:11Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 118.941845] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:11Z level=info event=scheduler_restarting worker2 # [ 118.948549] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:11Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 118.953477] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:49132 worker2 # [ 118.959036] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:11Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 118.963719] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:49132 worker2 # [ 118.969267] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:11Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 118.992532] nix-daemon[856]: reaped child process 1313, status = succeeded worker1 # [ 119.163230] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 119.168858] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 119.157573] nix-grpc-daemon[1261]: ts=2026-09-22T15:47:11Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 119.277804] nix-grpc-daemon[1261]: E0000 00:00:1790092031.876705 1306 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true worker2 # [ 119.284447] nix-grpc-daemon[1261]: E0000 00:00:1790092031.877014 1306 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1 # [ 119.369663] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 119.374580] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 119.575471] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 119.580652] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 119.781565] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 119.786779] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 119.989030] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 119.993152] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 120.189604] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 120.194729] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 120.244155] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 120.250923] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 120.256731] systemd[1]: nix-grpc-daemon.service: Consumed 1.703s CPU time over 16.194s wall clock time, 38.1M memory peak, 45.6K incoming IP traffic, 81.4K outgoing IP traffic. worker2 # [ 120.268471] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 120.277084] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 120.278886] systemd[1]: nix-grpc-daemon.socket: Consumed 906us CPU time over 16.858s wall clock time, 256K memory peak, 37.8K incoming IP traffic, 28.1K outgoing IP traffic. worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 120.395398] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:12Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 120.402498] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker1 # [ 120.600362] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 120.605633] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 120.806308] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 120.811564] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 120.957030] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_take_over lb # [ 120.931483] niks3-server[1077]: 2026/09/22 15:47:13 INFO lead: acquired remote=192.168.1.3:47486 worker1 # [ 121.014004] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 121.018125] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 121.214671] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 121.220059] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 121.420763] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 121.425968] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1 # [ 121.626813] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 121.632247] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 121.833159] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 121.838348] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 122.042482] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 122.049620] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:35862 worker1 # [ 122.054543] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:14Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2 worker1 # [ 122.795340] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 122.810773] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) client # don't know how to build these paths: client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 122.836406] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 122.852329] nix-grpc-daemon[1301]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 1/5) worker1 # [ 122.924773] nix-grpc-daemon[1301]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 127 ms (attempt 2/5) worker1 # [ 123.055024] nix-grpc-daemon[1301]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 12 ms (attempt 3/5) worker1 # [ 123.067662] nix-grpc-daemon[1301]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 120 ms (attempt 4/5) worker1 # [ 123.197626] nix-grpc-daemon[1301]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 123.204918] nix-grpc-daemon[1301]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 123.235155] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=0 worker1 # [ 123.242716] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 123.283744] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 123.299494] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35870 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'... worker1 # [ 123.316433] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 123.331172] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) worker1 # [ 123.368290] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:15Z level=info event=niks3_push_started pid=1380 worker1 # [ 123.519913] nix-grpc-daemon[1380]: time=2026-09-22T15:47:16.122Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 123.638218] nix-daemon[829]: accepted connection from pid 1392, user nix-grpc-daemon (trusted) worker1 # [ 123.658288] nix-daemon[829]: reaped child process 1395, status = succeeded worker1 # [ 123.764345] nix-daemon[829]: accepted connection from pid 1397, user nix-grpc-daemon (trusted) worker1 # [ 123.783782] nix-daemon[829]: reaped child process 1400, status = succeeded lb # [ 123.762027] niks3-server[1077]: 2026/09/22 15:47:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 123.800072] nix-grpc-daemon[1380]: time=2026-09-22T15:47:16.402Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 123.801610] nix-grpc-daemon[1380]: time=2026-09-22T15:47:16.402Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)" lb # [ 123.799046] niks3-server[1077]: 2026/09/22 15:47:16 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst worker1 # [ 123.839186] nix-grpc-daemon[1380]: time=2026-09-22T15:47:16.441Z level=INFO msg="Uploading 1 narinfos" lb # [ 123.813668] niks3-server[1077]: 2026/09/22 15:47:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign lb # [ 123.823041] niks3-server[1077]: 2026/09/22 15:47:16 INFO Signed narinfos id=43 count=1 lb # [ 123.830089] niks3-server[1077]: 2026/09/22 15:47:16 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls lb # [ 123.844656] niks3-server[1077]: 2026/09/22 15:47:16 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete lb # [ 123.854346] niks3-server[1077]: 2026/09/22 15:47:16 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo lb # [ 123.860453] niks3-server[1077]: 2026/09/22 15:47:16 INFO Completed upload id=43 worker1 # [ 123.886761] nix-grpc-daemon[1380]: time=2026-09-22T15:47:16.489Z level=INFO msg="Upload complete. (363ms)" worker1 # [ 123.896872] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=0 paths=1 nar_bytes_in=520 worker1 # [ 123.908674] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 123.920401] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) worker1 # [ 123.925487] nix-daemon[829]: reaped child process 1378, status = succeeded client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051... worker1 # [ 123.937612] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker1 # [ 125.625697] nix-daemon[829]: reaped child process 1404, status = succeeded worker1 # [ 125.722834] nix-daemon[829]: accepted connection from pid 1420, user nix-grpc-daemon (trusted) worker1 # [ 125.740038] nix-daemon[829]: reaped child process 1423, status = succeeded lb # [ 125.719942] niks3-server[1077]: 2026/09/22 15:47:18 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 125.759446] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.359Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 125.762221] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.359Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)" lb # [ 125.761183] niks3-server[1077]: 2026/09/22 15:47:18 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv lb # [ 125.772675] niks3-server[1077]: 2026/09/22 15:47:18 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst worker1 # [ 125.813801] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.415Z level=INFO msg="Uploading 1 narinfos" lb # [ 125.788129] niks3-server[1077]: 2026/09/22 15:47:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign lb # [ 125.796999] niks3-server[1077]: 2026/09/22 15:47:18 INFO Signed narinfos id=44 count=1 lb # [ 125.803526] niks3-server[1077]: 2026/09/22 15:47:18 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls lb # [ 125.812312] niks3-server[1077]: 2026/09/22 15:47:18 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete lb # [ 125.821833] niks3-server[1077]: 2026/09/22 15:47:18 INFO Completed upload id=44 worker1 # [ 125.847423] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.450Z level=INFO msg="Upload complete. (299ms)" lb # [ 125.827413] niks3-server[1077]: 2026/09/22 15:47:18 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo worker1 # [ 125.870883] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 worker1 # [ 125.887638] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=0 worker1 # [ 125.891741] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 125.896069] nix-daemon[829]: reaped child process 1402, status = succeeded worker1 # [ 125.914906] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 125.929785] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 125.944391] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.547Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'... worker1 # [ 125.954827] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 125.968818] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) worker1 # [ 126.118650] nix-daemon[829]: accepted connection from pid 1429, user nix-grpc-daemon (trusted) worker1 # [ 126.137342] nix-daemon[829]: reaped child process 1432, status = succeeded worker1 # [ 126.236637] nix-daemon[829]: accepted connection from pid 1434, user nix-grpc-daemon (trusted) worker1 # [ 126.252553] nix-daemon[829]: reaped child process 1437, status = succeeded lb # [ 126.231791] niks3-server[1077]: 2026/09/22 15:47:18 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 126.268090] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.870Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 126.269550] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.870Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)" lb # [ 126.263701] niks3-server[1077]: 2026/09/22 15:47:18 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst lb # [ 126.280327] niks3-server[1077]: 2026/09/22 15:47:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign worker1 # [ 126.312169] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.914Z level=INFO msg="Uploading 1 narinfos" lb # [ 126.291735] niks3-server[1077]: 2026/09/22 15:47:18 INFO Signed narinfos id=45 count=1 lb # [ 126.297550] niks3-server[1077]: 2026/09/22 15:47:18 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls lb # [ 126.308384] niks3-server[1077]: 2026/09/22 15:47:18 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete worker1 # [ 126.339758] nix-grpc-daemon[1380]: time=2026-09-22T15:47:18.942Z level=INFO msg="Upload complete. (355ms)" lb # [ 126.314462] niks3-server[1077]: 2026/09/22 15:47:18 INFO Completed upload id=45 lb # [ 126.317566] niks3-server[1077]: 2026/09/22 15:47:18 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo worker1 # [ 126.348282] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=0 paths=1 nar_bytes_in=640 worker1 # [ 126.363178] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 126.369830] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) worker1 # [ 126.374387] nix-daemon[829]: reaped child process 1427, status = succeeded client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051... worker1 # [ 126.386975] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker1 # [ 127.652467] nix-daemon[829]: reaped child process 1441, status = succeeded worker1 # [ 127.782104] nix-daemon[829]: accepted connection from pid 1457, user nix-grpc-daemon (trusted) worker1 # [ 127.803708] nix-daemon[829]: reaped child process 1460, status = succeeded lb # [ 127.783062] niks3-server[1077]: 2026/09/22 15:47:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 127.819848] nix-grpc-daemon[1380]: time=2026-09-22T15:47:20.421Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 127.821823] nix-grpc-daemon[1380]: time=2026-09-22T15:47:20.421Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)" lb # [ 127.819755] niks3-server[1077]: 2026/09/22 15:47:20 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst lb # [ 127.832998] niks3-server[1077]: 2026/09/22 15:47:20 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker1 # [ 127.872296] nix-grpc-daemon[1380]: time=2026-09-22T15:47:20.474Z level=INFO msg="Uploading 1 narinfos" lb # [ 127.847819] niks3-server[1077]: 2026/09/22 15:47:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign lb # [ 127.857291] niks3-server[1077]: 2026/09/22 15:47:20 INFO Signed narinfos id=46 count=1 lb # [ 127.863174] niks3-server[1077]: 2026/09/22 15:47:20 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls worker1 # [ 127.900393] nix-grpc-daemon[1380]: time=2026-09-22T15:47:20.501Z level=INFO msg="Upload complete. (300ms)" lb # [ 127.873733] niks3-server[1077]: 2026/09/22 15:47:20 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete lb # [ 127.878042] niks3-server[1077]: 2026/09/22 15:47:20 INFO Completed upload id=46 lb # [ 127.881262] niks3-server[1077]: 2026/09/22 15:47:20 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo worker1 # [ 127.913114] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 worker1 # [ 127.934236] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:20Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=0 worker1 # [ 127.940594] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:20Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 127.944712] nix-daemon[829]: reaped child process 1439, status = succeeded worker1 # [ 127.978867] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 127.994936] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 128.010652] nix-grpc-daemon[1380]: time=2026-09-22T15:47:20.613Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'... worker1 # [ 128.022060] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 128.036417] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) worker1 # [ 128.200616] nix-daemon[829]: accepted connection from pid 1466, user nix-grpc-daemon (trusted) worker1 # [ 128.230237] nix-daemon[829]: reaped child process 1469, status = succeeded worker1 # [ 128.330203] nix-daemon[829]: accepted connection from pid 1471, user nix-grpc-daemon (trusted) worker1 # [ 128.351658] nix-daemon[829]: reaped child process 1474, status = succeeded lb # [ 128.329029] niks3-server[1077]: 2026/09/22 15:47:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 128.364170] nix-grpc-daemon[1380]: time=2026-09-22T15:47:20.967Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 128.366160] nix-grpc-daemon[1380]: time=2026-09-22T15:47:20.968Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)" lb # [ 128.357232] niks3-server[1077]: 2026/09/22 15:47:20 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst worker1 # [ 128.398137] nix-grpc-daemon[1380]: time=2026-09-22T15:47:21.000Z level=INFO msg="Uploading 1 narinfos" lb # [ 128.371509] niks3-server[1077]: 2026/09/22 15:47:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign lb # [ 128.382730] niks3-server[1077]: 2026/09/22 15:47:20 INFO Signed narinfos id=47 count=1 lb # [ 128.390471] niks3-server[1077]: 2026/09/22 15:47:20 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls lb # [ 128.403878] niks3-server[1077]: 2026/09/22 15:47:20 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete worker1 # [ 128.436814] nix-grpc-daemon[1380]: time=2026-09-22T15:47:21.039Z level=INFO msg="Upload complete. (381ms)" lb # [ 128.411604] niks3-server[1077]: 2026/09/22 15:47:21 INFO Completed upload id=47 lb # [ 128.414047] niks3-server[1077]: 2026/09/22 15:47:21 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo worker1 # [ 128.444498] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 128.452609] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 128.462616] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) worker1 # [ 128.464270] nix-daemon[829]: reaped child process 1464, status = succeeded client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051... worker1 # [ 128.474582] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv lb # [ 128.538695] envoy[894]: conn peer=[2001:db8:1::4]:45714 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5496 tx=648 ms=14873 client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker1 # [ 129.376546] nix-daemon[829]: reaped child process 1478, status = succeeded worker1 # [ 129.448752] nix-daemon[829]: accepted connection from pid 1494, user nix-grpc-daemon (trusted) worker1 # [ 129.469182] nix-daemon[829]: reaped child process 1497, status = succeeded lb # [ 129.447242] niks3-server[1077]: 2026/09/22 15:47:22 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 129.484462] nix-grpc-daemon[1380]: time=2026-09-22T15:47:22.086Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 129.486299] nix-grpc-daemon[1380]: time=2026-09-22T15:47:22.086Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)" lb # [ 129.483023] niks3-server[1077]: 2026/09/22 15:47:22 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv lb # [ 129.497421] niks3-server[1077]: 2026/09/22 15:47:22 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst worker1 # [ 129.538504] nix-grpc-daemon[1380]: time=2026-09-22T15:47:22.140Z level=INFO msg="Uploading 1 narinfos" lb # [ 129.512435] niks3-server[1077]: 2026/09/22 15:47:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign lb # [ 129.523417] niks3-server[1077]: 2026/09/22 15:47:22 INFO Signed narinfos id=48 count=1 lb # [ 129.531044] niks3-server[1077]: 2026/09/22 15:47:22 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls worker1 # [ 129.570463] nix-grpc-daemon[1380]: time=2026-09-22T15:47:22.173Z level=INFO msg="Upload complete. (287ms)" lb # [ 129.543942] niks3-server[1077]: 2026/09/22 15:47:22 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete lb # [ 129.554138] niks3-server[1077]: 2026/09/22 15:47:22 INFO Completed upload id=48 lb # [ 129.557932] niks3-server[1077]: 2026/09/22 15:47:22 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo worker1 # [ 129.585226] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 worker1 # [ 129.603499] nix-daemon[829]: reaped child process 1476, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 7.06 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.19 seconds) worker2 # [ 129.867111] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 129.877274] systemd[1]: Starting scheduler-token.service... worker2 # [ 129.956094] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 129.963946] systemd[1]: Finished scheduler-token.service. worker2 # [ 129.966547] systemd[1]: scheduler-token.service: Consumed 44ms CPU time over 82ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 129.982305] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 130.090064] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:22Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 130.092348] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:22Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 130.096325] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:22Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 130.101624] nix-grpc-daemon[1482]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 130.103094] nix-grpc-daemon[1482]: E0000 00:00:1790092042.700198 1482 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 130.134946] nix-grpc-daemon[1482]: E0000 00:00:1790092042.706403 1482 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 130.150747] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:22Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 130.153108] nix-grpc-daemon[1482]: E0000 00:00:1790092042.716097 1524 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 130.155741] nix-grpc-daemon[1482]: E0000 00:00:1790092042.731552 1524 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 130.161228] nix-grpc-daemon[1482]: E0000 00:00:1790092042.731716 1524 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 130.165922] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 130.166773] nix-grpc-daemon[1482]: E0000 00:00:1790092042.749123 1524 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 130.169577] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:22Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 130.202735] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:22Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 130.207252] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:22Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:35862 worker1 # [ 130.212653] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:22Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 14.15 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs client # [ 130.502563] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: 6dda1e013a664e6ca11b449c6f50c90d client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds) worker2 # [ 130.700661] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50480 worker2 # [ 130.715128] nix-daemon[856]: accepted connection from pid 1482, user nix-grpc-daemon (trusted) client # [ 130.739721] nix[1342]: don't know how to build these paths: client # [ 130.742144] nix[1342]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker1 # [ 130.769911] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:23Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=0 worker1 # [ 130.777537] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:23Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 130.823271] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 130.839046] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:35870 client # [ 130.832236] nix[1342]: copying 1 paths to 'lb:50051'... client # [ 130.837732] nix[1342]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'... worker1 # [ 130.856305] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 130.870994] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) worker1 # [ 131.036157] nix-daemon[829]: accepted connection from pid 1510, user nix-grpc-daemon (trusted) worker1 # [ 131.055139] nix-daemon[829]: reaped child process 1513, status = succeeded worker1 # [ 131.154639] nix-daemon[829]: accepted connection from pid 1515, user nix-grpc-daemon (trusted) worker1 # [ 131.170061] nix-daemon[829]: reaped child process 1518, status = succeeded lb # [ 131.151335] niks3-server[1077]: 2026/09/22 15:47:23 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 131.188636] nix-grpc-daemon[1380]: time=2026-09-22T15:47:23.791Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 131.190108] nix-grpc-daemon[1380]: time=2026-09-22T15:47:23.791Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)" lb # [ 131.185851] niks3-server[1077]: 2026/09/22 15:47:23 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst worker1 # [ 131.223855] nix-grpc-daemon[1380]: time=2026-09-22T15:47:23.826Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.199520] niks3-server[1077]: 2026/09/22 15:47:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign lb # [ 131.208536] niks3-server[1077]: 2026/09/22 15:47:23 INFO Signed narinfos id=49 count=1 lb # [ 131.219335] niks3-server[1077]: 2026/09/22 15:47:23 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete worker1 # [ 131.253004] nix-grpc-daemon[1380]: time=2026-09-22T15:47:23.855Z level=INFO msg="Upload complete. (364ms)" lb # [ 131.228771] niks3-server[1077]: 2026/09/22 15:47:23 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls worker1 # [ 131.260156] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 131.236525] niks3-server[1077]: 2026/09/22 15:47:23 INFO Completed upload id=49 worker1 # [ 131.274777] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 worker1 # [ 131.280566] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) worker1 # [ 131.285461] nix-daemon[829]: reaped child process 1508, status = succeeded worker1 # [ 131.295429] nix-daemon[829]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) client # [ 131.275430] nix[1342]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker2 # [ 133.313439] systemd-journald[272]: Received client request to rotate journal, rotating. worker2 # [ 133.334677] systemd-journald[272]: Vacuuming done, freed 0B of archived journals from /var/log/journal/59011f543f9f49bda13a45f5715e3f8b. worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds) worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker1 # [ 133.415284] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds) client: must succeed: sleep 5.0 worker1 # [ 133.654305] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:26Z level=info event=draining client: (finished: must succeed: sleep 5.0, in 5.05 seconds) lb # [ 138.537897] envoy[894]: conn peer=[2001:db8:1::1]:54122 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9649 tx=4411 ms=6805 worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker1 # [ 138.733805] nix-daemon[829]: reaped child process 1522, status = succeeded worker1 # [ 138.831078] nix-daemon[829]: accepted connection from pid 1563, user nix-grpc-daemon (trusted) worker1 # [ 138.851428] nix-daemon[829]: reaped child process 1566, status = succeeded lb # [ 138.830514] niks3-server[1077]: 2026/09/22 15:47:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 138.872390] nix-grpc-daemon[1380]: time=2026-09-22T15:47:31.473Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 138.873907] nix-grpc-daemon[1380]: time=2026-09-22T15:47:31.473Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)" lb # [ 138.886574] niks3-server[1077]: 2026/09/22 15:47:31 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 138.899709] niks3-server[1077]: 2026/09/22 15:47:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign worker1 # [ 138.930685] nix-grpc-daemon[1380]: time=2026-09-22T15:47:31.533Z level=INFO msg="Uploading 1 narinfos" lb # [ 138.910252] niks3-server[1077]: 2026/09/22 15:47:31 INFO Signed narinfos id=50 count=1 lb # [ 138.914621] niks3-server[1077]: 2026/09/22 15:47:31 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv lb # [ 138.924423] niks3-server[1077]: 2026/09/22 15:47:31 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls worker1 # [ 138.950611] nix-grpc-daemon[1380]: time=2026-09-22T15:47:31.553Z level=INFO msg="Upload complete. (237ms)" lb # [ 138.928247] niks3-server[1077]: 2026/09/22 15:47:31 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 138.932180] niks3-server[1077]: 2026/09/22 15:47:31 INFO Completed upload id=50 lb # [ 138.935072] niks3-server[1077]: 2026/09/22 15:47:31 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo worker1 # [ 138.965222] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35870 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 worker1 # [ 138.985935] nix-daemon[829]: reaped child process 1520, status = succeeded client # [ 138.980579] systemd[1]: rs.service: Deactivated successfully. client # [ 138.985530] systemd[1]: rs.service: Consumed 193ms CPU time over 8.479s wall clock time, 39.5M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic. worker1 # [ 139.656659] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:32Z level=info event=signal_exit lb # [ 139.944669] niks3-server[1077]: 2026/09/22 15:47:32 INFO lead: released remote=192.168.1.3:47486 lb # [ 140.114653] niks3-server[1077]: 2026/09/22 15:47:32 INFO lead: acquired remote=192.168.1.4:45992 worker2 # [ 140.121165] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:32Z level=info event=scheduler_take_over worker2 # [ 140.939054] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:33Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 140.969609] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:33Z level=info event=scheduler_restarting worker1 # [ 140.970066] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:33Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 140.970215] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:35862 worker1 # [ 140.970342] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:33Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 140.970466] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:33Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:35862 worker1 # [ 140.970574] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:33Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 140.996975] nix-grpc-daemon[1301]: E0000 00:00:1790092053.599511 1346 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true worker1 # [ 141.003532] nix-grpc-daemon[1301]: E0000 00:00:1790092053.599767 1346 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker1 # [ 141.033835] nix-daemon[829]: reaped child process 1370, status = succeeded worker2 # [ 141.142883] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 141.147159] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 141.188366] nix-grpc-daemon[1301]: ts=2026-09-22T15:47:33Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 141.343749] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 141.361527] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:33Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 141.366036] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:33Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:40372 worker2 # [ 141.371520] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:33Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.24 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker1 # [ 142.317965] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 142.324588] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 142.327192] systemd[1]: nix-grpc-daemon.service: Consumed 1.999s CPU time over 33.842s wall clock time, 38.6M memory peak, 48.3K incoming IP traffic, 91.9K outgoing IP traffic. worker1 # [ 142.349125] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 142.502806] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 142.505097] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 142.508919] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 142.514249] nix-grpc-daemon[1573]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 142.515708] nix-grpc-daemon[1573]: E0000 00:00:1790092055.116301 1573 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 142.556267] nix-grpc-daemon[1573]: E0000 00:00:1790092055.122367 1573 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 142.559071] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 142.560974] nix-grpc-daemon[1573]: E0000 00:00:1790092055.127911 1612 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 142.563313] nix-grpc-daemon[1573]: E0000 00:00:1790092055.143706 1612 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 142.568118] nix-grpc-daemon[1573]: E0000 00:00:1790092055.143838 1612 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 142.570485] nix-grpc-daemon[1573]: E0000 00:00:1790092055.158122 1612 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 142.573651] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 142.576326] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 142.553151] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:35Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:40372 worker2 # [ 142.553431] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.93 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid client # [ 143.425088] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: 31933518b25a4b6793944f9c35751339 client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.08 seconds) client # [ 143.630113] nix[1378]: don't know how to build these paths: client # [ 143.631215] nix[1378]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker2 # [ 143.621412] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50480 worker2 # [ 143.635568] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:36Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker2 # [ 143.642267] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:36Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 143.680834] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50480 worker2 # [ 143.695875] nix-grpc-daemon[1482]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 1/5) worker2 # [ 143.787254] nix-grpc-daemon[1482]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 2/5) worker2 # [ 143.871317] nix-grpc-daemon[1482]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 3/5) worker2 # [ 144.043182] nix-grpc-daemon[1482]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 260 ms (attempt 4/5) worker2 # [ 144.306239] nix-grpc-daemon[1482]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 144.312972] nix-grpc-daemon[1482]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 144.332119] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50480 client # [ 144.352175] nix[1378]: copying 1 paths to 'lb:50051'... worker2 # [ 144.347290] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50480 client # [ 144.356399] nix[1378]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'... worker2 # [ 144.365399] nix-daemon[856]: accepted connection from pid 1482, user nix-grpc-daemon (trusted) worker2 # [ 144.398576] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:36Z level=info event=niks3_push_started pid=1610 worker2 # [ 144.547998] nix-grpc-daemon[1610]: time=2026-09-22T15:47:37.146Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 144.657342] nix-daemon[856]: accepted connection from pid 1623, user nix-grpc-daemon (trusted) worker2 # [ 144.689135] nix-daemon[856]: reaped child process 1626, status = succeeded worker2 # [ 144.810369] nix-daemon[856]: accepted connection from pid 1632, user nix-grpc-daemon (trusted) worker2 # [ 144.832544] nix-daemon[856]: reaped child process 1635, status = succeeded lb # [ 144.831413] niks3-server[1077]: 2026/09/22 15:47:37 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 144.844094] nix-grpc-daemon[1610]: time=2026-09-22T15:47:37.442Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 144.845584] nix-grpc-daemon[1610]: time=2026-09-22T15:47:37.442Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)" lb # [ 144.870023] niks3-server[1077]: 2026/09/22 15:47:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign worker2 # [ 144.883772] nix-grpc-daemon[1610]: time=2026-09-22T15:47:37.482Z level=INFO msg="Uploading 1 narinfos" lb # [ 144.885927] niks3-server[1077]: 2026/09/22 15:47:37 INFO Signed narinfos id=51 count=1 lb # [ 144.896962] niks3-server[1077]: 2026/09/22 15:47:37 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst lb # [ 144.918715] niks3-server[1077]: 2026/09/22 15:47:37 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete worker2 # [ 144.927976] nix-grpc-daemon[1610]: time=2026-09-22T15:47:37.527Z level=INFO msg="Upload complete. (375ms)" lb # [ 144.935677] niks3-server[1077]: 2026/09/22 15:47:37 INFO Completed upload id=51 worker2 # [ 144.936827] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50480 duration_s=0 paths=1 nar_bytes_in=424 lb # [ 144.941001] niks3-server[1077]: 2026/09/22 15:47:37 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo lb # [ 144.944466] niks3-server[1077]: 2026/09/22 15:47:37 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls worker2 # [ 144.953798] nix-grpc-daemon[1482]: ts=2026-09-22T15:47:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50480 worker2 # [ 144.960583] nix-daemon[856]: accepted connection from pid 1482, user nix-grpc-daemon (trusted) worker2 # [ 144.965950] nix-daemon[856]: reaped child process 1608, status = succeeded client # [ 144.983618] nix[1378]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... worker2 # [ 144.979211] nix-daemon[856]: accepted connection from pid 1482, user nix-grpc-daemon (trusted) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker2 # [ 147.313962] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1482 (nix-grpc-daemon) on client request. worker2 # [ 147.315808] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 147.323434] niks3-server[1077]: 2026/09/22 15:47:39 INFO lead: released remote=192.168.1.4:45992 client # [ 147.338789] nix[1378]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again worker2 # [ 147.336667] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker1 # [ 147.357393] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:39Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination" worker2 # [ 147.338165] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. worker2 # [ 147.339405] systemd[1]: nix-grpc-daemon.service: Consumed 600ms CPU time over 17.359s wall clock time, 34.6M memory peak, 15K incoming IP traffic, 25.5K outgoing IP traffic. client # [ 147.345024] nix[1378]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker2 # [ 147.356189] systemd[1]: Starting scheduler-token.service... worker2 # [ 147.380666] nix-daemon[856]: reaped child process 1637, status = succeeded worker2 # [ 147.381640] nix-daemon[856]: reaped child process 1541, status = succeeded worker2 # [ 147.388409] nix-daemon[1639]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 147.397337] nix-daemon[856]: reaped child process 1639, status = failed with exit code 1 worker2 # [ 147.421313] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 147.423903] systemd[1]: Finished scheduler-token.service. worker2 # [ 147.426904] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 73ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 147.430579] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker2 # [ 147.435630] systemd[1]: Starting scheduler-token.service... worker2 # [ 147.477290] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 147.480062] systemd[1]: Finished scheduler-token.service. worker2 # [ 147.480869] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 48ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 147.485369] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 147.502511] niks3-server[1077]: 2026/09/22 15:47:40 INFO lead: acquired remote=192.168.1.3:37838 worker1 # [ 147.533109] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_take_over worker1 # [ 147.557571] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 147.585342] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 147.587603] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 147.591383] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 147.596050] nix-grpc-daemon[1682]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 147.608135] nix-grpc-daemon[1682]: E0000 00:00:1790092060.195387 1682 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 147.641297] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker2 # [ 147.620067] nix-grpc-daemon[1682]: E0000 00:00:1790092060.198864 1682 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 147.622410] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.37 seconds) client: must succeed: sleep 5.0 worker2 # [ 147.650225] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 147.651482] nix-grpc-daemon[1682]: E0000 00:00:1790092060.213695 1721 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 147.654231] nix-grpc-daemon[1682]: E0000 00:00:1790092060.233445 1721 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 147.659264] nix-grpc-daemon[1682]: E0000 00:00:1790092060.233620 1721 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 147.664201] nix-grpc-daemon[1682]: E0000 00:00:1790092060.248838 1721 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 147.667049] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 147.668429] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 147.842125] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 147.847546] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" client # [ 147.843256] nix[1378]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting worker2 # [ 147.868355] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 147.874963] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 148.048407] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 148.053691] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker2 # [ 148.076060] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 148.082622] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 148.254441] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 148.259702] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker2 # [ 148.285480] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 148.295717] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves" worker1 # [ 148.462179] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 148.466337] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 148.492825] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 148.498177] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 148.539017] envoy[894]: conn peer=[2001:db8:1::1]:54124 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1517 ms=8257 lb # [ 148.544858] envoy[894]: conn peer=[2001:db8:1::3]:37168 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3358 tx=1078 ms=32475 lb # [ 148.550217] envoy[894]: conn peer=[2001:db8:1::4]:42152 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2679 tx=376 ms=17176 worker1 # [ 148.662724] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 148.667912] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 148.701284] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 148.705758] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" client # [ 148.847087] nix[1378]: scheduler at lb:50051: no healthy upstream, reconnecting worker1 # [ 148.868706] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 148.873846] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 148.902093] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 148.907430] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 149.074756] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 149.079856] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 149.108855] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 149.114473] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 149.280719] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 149.285842] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 149.315241] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 149.321771] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 149.487817] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 149.505196] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:42Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:58180 worker1 # [ 149.510290] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:42Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 149.522445] nix-grpc-daemon[1682]: ts=2026-09-22T15:47:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 149.557678] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:42Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 149.562131] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:42Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:58180 worker1 # [ 149.567504] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:42Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 150.878547] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:43Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker1 # [ 150.878854] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:43Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 150.937815] nix-grpc-daemon[1573]: ts=2026-09-22T15:47:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42214 worker1 # [ 150.954058] nix-daemon[829]: accepted connection from pid 1573, user nix-grpc-daemon (trusted) worker1 # [ 150.976164] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 1/5) worker1 # [ 151.074148] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 176 ms (attempt 2/5) worker1 # [ 151.252329] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 3/5) worker1 # [ 151.314996] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 624 ms (attempt 4/5) worker1 # [ 151.942503] nix-grpc-daemon[1573]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 151.949761] nix-grpc-daemon[1573]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 151.964909] nix-grpc-daemon[1573]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'... worker1 # [ 151.989512] nix-daemon[829]: accepted connection from pid 1573, user nix-grpc-daemon (trusted) client # [ 151.992925] nix[1378]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... worker1 # [ 152.015255] nix-daemon[829]: accepted connection from pid 1573, user nix-grpc-daemon (trusted) client: (finished: must succeed: sleep 5.0, in 5.05 seconds) client: waiting for success: ! systemctl is-active mid worker1 # [ 243.305805] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:15Z level=info event=niks3_push_started pid=1686 worker1 # [ 243.368700] nix-daemon[829]: reaped child process 1656, status = succeeded worker1 # [ 243.497080] nix-grpc-daemon[1686]: time=2026-09-22T15:49:16.099Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 243.604100] nix-daemon[829]: accepted connection from pid 1698, user nix-grpc-daemon (trusted) worker1 # [ 243.625337] nix-daemon[829]: reaped child process 1701, status = succeeded lb # [ 243.599817] niks3-server[1077]: 2026/09/22 15:49:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 243.640066] nix-grpc-daemon[1686]: time=2026-09-22T15:49:16.241Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 243.641548] nix-grpc-daemon[1686]: time=2026-09-22T15:49:16.241Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)" lb # [ 243.661951] niks3-server[1077]: 2026/09/22 15:49:16 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv lb # [ 243.679017] niks3-server[1077]: 2026/09/22 15:49:16 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst worker1 # [ 243.714852] nix-grpc-daemon[1686]: time=2026-09-22T15:49:16.316Z level=INFO msg="Uploading 1 narinfos" lb # [ 243.688542] niks3-server[1077]: 2026/09/22 15:49:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign lb # [ 243.691751] niks3-server[1077]: 2026/09/22 15:49:16 INFO Signed narinfos id=52 count=1 lb # [ 243.701677] niks3-server[1077]: 2026/09/22 15:49:16 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls lb # [ 243.708045] niks3-server[1077]: 2026/09/22 15:49:16 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 243.717977] niks3-server[1077]: 2026/09/22 15:49:16 INFO Completed upload id=52 worker1 # [ 243.747121] nix-grpc-daemon[1686]: time=2026-09-22T15:49:16.349Z level=INFO msg="Upload complete. (246ms)" lb # [ 243.723622] niks3-server[1077]: 2026/09/22 15:49:16 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo worker1 # [ 243.764454] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42214 duration_s=92 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 worker1 # [ 243.782862] nix-daemon[829]: reaped child process 1654, status = succeeded client # [ 243.777764] systemd[1]: mid.service: Deactivated successfully. client # [ 243.783149] systemd[1]: mid.service: Consumed 255ms CPU time over 1min 40.355s wall clock time, 41.4M memory peak, 4K written to disk, 9.2K incoming IP traffic, 11.2K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active mid, in 91.01 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 100.70 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr client # [ 244.092506] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: ff987b96ef5e46c093253097b9f166a8 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds) worker2 # [ 244.283377] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 244.297879] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) client # [ 244.321717] nix[1839]: don't know how to build these paths: client # [ 244.324979] nix[1839]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker1 # [ 244.357011] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:16Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=0 worker1 # [ 244.363082] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:16Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 244.378458] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 244.395100] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 1/5) worker2 # [ 244.431259] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 2/5) worker2 # [ 244.548388] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 249 ms (attempt 3/5) worker2 # [ 244.799322] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 274 ms (attempt 4/5) worker2 # [ 245.075504] nix-grpc-daemon[1682]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 245.082251] nix-grpc-daemon[1682]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 245.100772] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37710 client # [ 245.123208] nix[1839]: copying 1 paths to 'lb:50051'... worker2 # [ 245.116133] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 client # [ 245.123662] nix[1839]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'... worker2 # [ 245.130842] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) worker2 # [ 245.157703] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:17Z level=info event=niks3_push_started pid=1754 worker2 # [ 245.322550] nix-grpc-daemon[1754]: time=2026-09-22T15:49:17.921Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 245.453697] nix-daemon[856]: accepted connection from pid 1770, user nix-grpc-daemon (trusted) worker2 # [ 245.473219] nix-daemon[856]: reaped child process 1774, status = succeeded worker2 # [ 245.578195] nix-daemon[856]: accepted connection from pid 1776, user nix-grpc-daemon (trusted) worker2 # [ 245.597474] nix-daemon[856]: reaped child process 1779, status = succeeded lb # [ 245.595553] niks3-server[1077]: 2026/09/22 15:49:18 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 245.607725] nix-grpc-daemon[1754]: time=2026-09-22T15:49:18.205Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 245.609849] nix-grpc-daemon[1754]: time=2026-09-22T15:49:18.205Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)" lb # [ 245.631169] niks3-server[1077]: 2026/09/22 15:49:18 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst lb # [ 245.645808] niks3-server[1077]: 2026/09/22 15:49:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign worker2 # [ 245.653665] nix-grpc-daemon[1754]: time=2026-09-22T15:49:18.252Z level=INFO msg="Uploading 1 narinfos" lb # [ 245.660381] niks3-server[1077]: 2026/09/22 15:49:18 INFO Signed narinfos id=53 count=1 lb # [ 245.663456] niks3-server[1077]: 2026/09/22 15:49:18 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls lb # [ 245.680199] niks3-server[1077]: 2026/09/22 15:49:18 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete lb # [ 245.687946] niks3-server[1077]: 2026/09/22 15:49:18 INFO Completed upload id=53 worker2 # [ 245.688392] nix-grpc-daemon[1754]: time=2026-09-22T15:49:18.287Z level=INFO msg="Upload complete. (361ms)" lb # [ 245.690830] niks3-server[1077]: 2026/09/22 15:49:18 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo worker2 # [ 245.696947] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 245.711546] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 245.719174] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) worker2 # [ 245.724100] nix-daemon[856]: reaped child process 1752, status = succeeded client # [ 245.742172] nix[1839]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051... worker2 # [ 245.737924] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) client: must succeed: systemctl kill -s INT intr client # [ 247.790579] systemd[1]: intr.service: Sent signal SIGINT to main process 1839 (nix) on client request. worker2 # [ 247.787064] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:20Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv client # [ 247.797112] nix[1839]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client: (finished: must succeed: systemctl kill -s INT intr, in 0.06 seconds) client # [ 247.804931] nix[1839]: [mutex.cc : 451] RAW: Unlock 0xed2e75dfc788 @ 0xed327a4359a4 0xed327b50d40c 0xed329a9a57bc 0xed329a9a7414 0xed329a99e1d4 0xed32999e4f7c 0xed3299772d80 client # [ 247.821703] systemd-coredump[1853]: Process 1839 (nix) of user 0 terminated abnormally with signal 11/SEGV, processing... client # [ 247.843314] systemd[1]: Created slice Slice /system/systemd-coredump. client # [ 247.851756] systemd[1]: Started Process Core Dump (PID 1853/UID 0). worker2 # [ 247.963827] nix-daemon[1783]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 247.969454] nix-daemon[856]: reaped child process 1781, status = succeeded worker2 # [ 247.974255] nix-daemon[856]: reaped child process 1783, status = failed with exit code 1 lb # [ 248.549867] envoy[894]: conn peer=[2001:db8:1::1]:35062 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4288 tx=2113 ms=100129 (finished: subtest: interrupting the client stops the build on the worker, in 5.01 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw client # [ 249.119178] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: fbb71412e3ba4a83846065debe327456 client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.06 seconds) client # [ 249.504909] nix[1860]: don't know how to build these paths: worker1 # [ 249.523187] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:42214 client # [ 249.509849] nix[1860]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker1 # [ 249.540160] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=0 worker1 # [ 249.546792] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 249.596398] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42214 worker1 # [ 249.615775] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42214 client # [ 249.613216] nix[1860]: copying 1 paths to 'lb:50051'... worker1 # [ 249.630619] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42214 client # [ 249.617621] nix[1860]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'... worker1 # [ 249.649161] nix-daemon[829]: accepted connection from pid 1573, user nix-grpc-daemon (trusted) worker1 # [ 249.812460] nix-daemon[829]: accepted connection from pid 1739, user nix-grpc-daemon (trusted) worker1 # [ 249.842845] nix-daemon[829]: reaped child process 1742, status = succeeded worker1 # [ 249.935864] nix-daemon[829]: accepted connection from pid 1744, user nix-grpc-daemon (trusted) worker1 # [ 249.955057] nix-daemon[829]: reaped child process 1747, status = succeeded lb # [ 249.932836] niks3-server[1077]: 2026/09/22 15:49:22 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 249.970050] nix-grpc-daemon[1686]: time=2026-09-22T15:49:22.572Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 249.971515] nix-grpc-daemon[1686]: time=2026-09-22T15:49:22.572Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)" lb # [ 249.961291] niks3-server[1077]: 2026/09/22 15:49:22 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst lb # [ 249.976510] niks3-server[1077]: 2026/09/22 15:49:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign worker1 # [ 250.007872] nix-grpc-daemon[1686]: time=2026-09-22T15:49:22.610Z level=INFO msg="Uploading 1 narinfos" lb # [ 249.985700] niks3-server[1077]: 2026/09/22 15:49:22 INFO Signed narinfos id=54 count=1 lb # [ 249.991042] niks3-server[1077]: 2026/09/22 15:49:22 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls lb # [ 250.000328] niks3-server[1077]: 2026/09/22 15:49:22 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete worker1 # [ 250.032220] nix-grpc-daemon[1686]: time=2026-09-22T15:49:22.635Z level=INFO msg="Upload complete. (365ms)" lb # [ 250.008403] niks3-server[1077]: 2026/09/22 15:49:22 INFO Completed upload id=54 lb # [ 250.010460] niks3-server[1077]: 2026/09/22 15:49:22 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo worker1 # [ 250.040349] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42214 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 250.052343] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42214 worker1 # [ 250.061442] nix-daemon[829]: accepted connection from pid 1573, user nix-grpc-daemon (trusted) worker1 # [ 250.065911] nix-daemon[829]: reaped child process 1737, status = succeeded client # [ 250.056375] nix[1860]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051... worker1 # [ 250.078270] nix-daemon[829]: accepted connection from pid 1573, user nix-grpc-daemon (trusted) client # [ 250.386161] systemd-coredump[1854]: Cannot store coredump of 1839 (nix): No space left on device client # [ 250.523450] systemd-coredump[1854]: Process 1839 (nix) of user 0 terminated abnormally without generating a coredump. client # [ 250.540785] systemd[1]: systemd-coredump@0-1-1853_1854-0.service: Deactivated successfully. client # [ 250.542337] systemd[1]: systemd-coredump@0-1-1853_1854-0.service: Consumed 2.018s CPU time over 2.683s wall clock time, 968.7M memory peak, 926.2M written to disk. client # [ 250.550494] systemd[1]: intr.service: Main process exited, code=dumped, status=11/SEGV client # [ 250.551915] systemd[1]: intr.service: Failed with result 'core-dump'. client # [ 250.554195] systemd[1]: intr.service: Consumed 365ms CPU time over 6.458s wall clock time, 41.5M memory peak, 4K written to disk, 6.5K incoming IP traffic, 8.2K outgoing IP traffic. worker1: must succeed: systemctl reload nix-grpc-daemon.service worker1 # [ 251.730007] systemd[1]: Reloading Nix worker-protocol over gRPC... worker1 # [ 251.775965] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds) worker1: must succeed: pgrep -f 'read -t [9]0 x' worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2 worker2 # [ 252.021947] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37710 client # don't know how to build these paths: client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker2 # [ 252.033362] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37710 worker1 # [ 252.082109] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:24Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=0 worker1 # [ 252.084088] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:24Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 252.100514] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 252.115437] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37710 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'... worker2 # [ 252.130852] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 252.145712] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) worker2 # [ 252.303866] nix-daemon[856]: accepted connection from pid 1834, user nix-grpc-daemon (trusted) worker2 # [ 252.322367] nix-daemon[856]: reaped child process 1837, status = succeeded worker2 # [ 252.425741] nix-daemon[856]: accepted connection from pid 1839, user nix-grpc-daemon (trusted) worker2 # [ 252.445020] nix-daemon[856]: reaped child process 1842, status = succeeded lb # [ 252.450431] niks3-server[1077]: 2026/09/22 15:49:25 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 252.460447] nix-grpc-daemon[1754]: time=2026-09-22T15:49:25.059Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 252.462396] nix-grpc-daemon[1754]: time=2026-09-22T15:49:25.061Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)" lb # [ 252.480988] niks3-server[1077]: 2026/09/22 15:49:25 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst worker2 # [ 252.493525] nix-grpc-daemon[1754]: time=2026-09-22T15:49:25.092Z level=INFO msg="Uploading 1 narinfos" lb # [ 252.493481] niks3-server[1077]: 2026/09/22 15:49:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign lb # [ 252.506555] niks3-server[1077]: 2026/09/22 15:49:25 INFO Signed narinfos id=55 count=1 worker2 # [ 252.515573] nix-grpc-daemon[1754]: time=2026-09-22T15:49:25.114Z level=INFO msg="Upload complete. (352ms)" lb # [ 252.514770] niks3-server[1077]: 2026/09/22 15:49:25 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls lb # [ 252.521596] niks3-server[1077]: 2026/09/22 15:49:25 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete worker2 # [ 252.524256] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 paths=1 nar_bytes_in=528 lb # [ 252.526834] niks3-server[1077]: 2026/09/22 15:49:25 INFO Completed upload id=55 worker1 # [ 252.555824] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:25Z level=info event=draining lb # [ 252.529309] niks3-server[1077]: 2026/09/22 15:49:25 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo worker2 # [ 252.535892] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 252.541922] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) worker2 # [ 252.547219] nix-daemon[856]: reaped child process 1832, status = succeeded worker2 # [ 252.557043] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051... client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker2 # [ 253.655585] nix-daemon[856]: reaped child process 1846, status = succeeded worker2 # [ 253.734339] nix-daemon[856]: accepted connection from pid 1862, user nix-grpc-daemon (trusted) worker2 # [ 253.753558] nix-daemon[856]: reaped child process 1865, status = succeeded lb # [ 253.758619] niks3-server[1077]: 2026/09/22 15:49:26 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 253.771628] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.367Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 253.774649] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.367Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)" lb # [ 253.791707] niks3-server[1077]: 2026/09/22 15:49:26 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv lb # [ 253.809458] niks3-server[1077]: 2026/09/22 15:49:26 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst lb # [ 253.826004] niks3-server[1077]: 2026/09/22 15:49:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign worker2 # [ 253.827814] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.426Z level=INFO msg="Uploading 1 narinfos" lb # [ 253.833410] niks3-server[1077]: 2026/09/22 15:49:26 INFO Signed narinfos id=56 count=1 lb # [ 253.838751] niks3-server[1077]: 2026/09/22 15:49:26 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls lb # [ 253.849241] niks3-server[1077]: 2026/09/22 15:49:26 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 253.856302] niks3-server[1077]: 2026/09/22 15:49:26 INFO Completed upload id=56 worker2 # [ 253.856388] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.455Z level=INFO msg="Upload complete. (288ms)" lb # [ 253.859068] niks3-server[1077]: 2026/09/22 15:49:26 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo worker1 # [ 253.896913] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:26Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=0 worker1 # [ 253.898824] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:26Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 253.869140] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=1 worker2 # [ 253.889435] nix-daemon[856]: reaped child process 1844, status = succeeded worker2 # [ 253.921810] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 253.936903] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 253.951323] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.550Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'... worker2 # [ 253.962185] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 253.979594] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) worker2 # [ 254.130258] nix-daemon[856]: accepted connection from pid 1869, user nix-grpc-daemon (trusted) worker2 # [ 254.149128] nix-daemon[856]: reaped child process 1872, status = succeeded worker2 # [ 254.249859] nix-daemon[856]: accepted connection from pid 1874, user nix-grpc-daemon (trusted) worker2 # [ 254.268408] nix-daemon[856]: reaped child process 1877, status = succeeded lb # [ 254.266411] niks3-server[1077]: 2026/09/22 15:49:26 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 254.276155] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.875Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 254.278229] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.877Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)" lb # [ 254.295826] niks3-server[1077]: 2026/09/22 15:49:26 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst lb # [ 254.311065] niks3-server[1077]: 2026/09/22 15:49:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign worker2 # [ 254.318659] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.917Z level=INFO msg="Uploading 1 narinfos" lb # [ 254.323950] niks3-server[1077]: 2026/09/22 15:49:26 INFO Signed narinfos id=57 count=1 lb # [ 254.329723] niks3-server[1077]: 2026/09/22 15:49:26 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls worker2 # [ 254.340104] nix-grpc-daemon[1754]: time=2026-09-22T15:49:26.939Z level=INFO msg="Upload complete. (348ms)" lb # [ 254.340775] niks3-server[1077]: 2026/09/22 15:49:26 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete lb # [ 254.346870] niks3-server[1077]: 2026/09/22 15:49:26 INFO Completed upload id=57 worker2 # [ 254.348122] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 paths=1 nar_bytes_in=640 lb # [ 254.349925] niks3-server[1077]: 2026/09/22 15:49:26 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo worker2 # [ 254.358643] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 254.366666] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) worker2 # [ 254.368900] nix-daemon[856]: reaped child process 1867, status = succeeded client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051... worker2 # [ 254.381218] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker2 # [ 255.848910] nix-daemon[856]: reaped child process 1881, status = succeeded worker2 # [ 255.956819] nix-daemon[856]: accepted connection from pid 1897, user nix-grpc-daemon (trusted) worker2 # [ 255.975220] nix-daemon[856]: reaped child process 1900, status = succeeded lb # [ 255.978974] niks3-server[1077]: 2026/09/22 15:49:28 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 255.989669] nix-grpc-daemon[1754]: time=2026-09-22T15:49:28.588Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 255.991442] nix-grpc-daemon[1754]: time=2026-09-22T15:49:28.588Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)" lb # [ 256.015851] niks3-server[1077]: 2026/09/22 15:49:28 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst worker2 # [ 256.040917] nix-grpc-daemon[1754]: time=2026-09-22T15:49:28.639Z level=INFO msg="Uploading 1 narinfos" lb # [ 256.028920] niks3-server[1077]: 2026/09/22 15:49:28 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv lb # [ 256.044524] niks3-server[1077]: 2026/09/22 15:49:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign lb # [ 256.047774] niks3-server[1077]: 2026/09/22 15:49:28 INFO Signed narinfos id=58 count=1 lb # [ 256.064172] niks3-server[1077]: 2026/09/22 15:49:28 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls lb # [ 256.071240] niks3-server[1077]: 2026/09/22 15:49:28 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete lb # [ 256.075047] niks3-server[1077]: 2026/09/22 15:49:28 INFO Completed upload id=58 lb # [ 256.077074] niks3-server[1077]: 2026/09/22 15:49:28 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo worker2 # [ 256.075920] nix-grpc-daemon[1754]: time=2026-09-22T15:49:28.675Z level=INFO msg="Upload complete. (284ms)" worker1 # [ 256.114021] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:28Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=0 worker1 # [ 256.115951] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:28Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 256.090353] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=1 worker2 # [ 256.108697] nix-daemon[856]: reaped child process 1879, status = succeeded worker2 # [ 256.132878] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 256.148788] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 256.163515] nix-grpc-daemon[1754]: time=2026-09-22T15:49:28.762Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'... worker2 # [ 256.174784] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 256.188885] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) worker2 # [ 256.338166] nix-daemon[856]: accepted connection from pid 1904, user nix-grpc-daemon (trusted) worker2 # [ 256.356132] nix-daemon[856]: reaped child process 1907, status = succeeded worker2 # [ 256.456153] nix-daemon[856]: accepted connection from pid 1909, user nix-grpc-daemon (trusted) worker2 # [ 256.472697] nix-daemon[856]: reaped child process 1912, status = succeeded lb # [ 256.477472] niks3-server[1077]: 2026/09/22 15:49:29 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 256.486825] nix-grpc-daemon[1754]: time=2026-09-22T15:49:29.085Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 256.488504] nix-grpc-daemon[1754]: time=2026-09-22T15:49:29.085Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)" lb # [ 256.506860] niks3-server[1077]: 2026/09/22 15:49:29 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst worker2 # [ 256.520425] nix-grpc-daemon[1754]: time=2026-09-22T15:49:29.119Z level=INFO msg="Uploading 1 narinfos" lb # [ 256.523228] niks3-server[1077]: 2026/09/22 15:49:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign lb # [ 256.533895] niks3-server[1077]: 2026/09/22 15:49:29 INFO Signed narinfos id=59 count=1 lb # [ 256.538664] niks3-server[1077]: 2026/09/22 15:49:29 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls lb # [ 256.547842] niks3-server[1077]: 2026/09/22 15:49:29 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete lb # [ 256.554430] niks3-server[1077]: 2026/09/22 15:49:29 INFO Completed upload id=59 worker2 # [ 256.554791] nix-grpc-daemon[1754]: time=2026-09-22T15:49:29.154Z level=INFO msg="Upload complete. (350ms)" lb # [ 256.557708] niks3-server[1077]: 2026/09/22 15:49:29 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo worker2 # [ 256.564308] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 paths=1 nar_bytes_in=768 worker2 # [ 256.575279] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 256.584574] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) worker2 # [ 256.587475] nix-daemon[856]: reaped child process 1902, status = succeeded client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051... worker2 # [ 256.598693] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker2 # [ 258.068869] nix-daemon[856]: reaped child process 1916, status = succeeded worker2 # [ 258.172788] nix-daemon[856]: accepted connection from pid 1932, user nix-grpc-daemon (trusted) worker2 # [ 258.191050] nix-daemon[856]: reaped child process 1935, status = succeeded lb # [ 258.197785] niks3-server[1077]: 2026/09/22 15:49:30 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 258.209877] nix-grpc-daemon[1754]: time=2026-09-22T15:49:30.807Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 258.211717] nix-grpc-daemon[1754]: time=2026-09-22T15:49:30.807Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)" lb # [ 258.236613] niks3-server[1077]: 2026/09/22 15:49:30 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv lb # [ 258.250173] niks3-server[1077]: 2026/09/22 15:49:30 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst lb # [ 258.262289] niks3-server[1077]: 2026/09/22 15:49:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign worker2 # [ 258.268462] nix-grpc-daemon[1754]: time=2026-09-22T15:49:30.867Z level=INFO msg="Uploading 1 narinfos" lb # [ 258.272852] niks3-server[1077]: 2026/09/22 15:49:30 INFO Signed narinfos id=60 count=1 lb # [ 258.279925] niks3-server[1077]: 2026/09/22 15:49:30 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls worker2 # [ 258.293415] nix-grpc-daemon[1754]: time=2026-09-22T15:49:30.892Z level=INFO msg="Upload complete. (281ms)" lb # [ 258.292979] niks3-server[1077]: 2026/09/22 15:49:30 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete lb # [ 258.297552] niks3-server[1077]: 2026/09/22 15:49:30 INFO Completed upload id=60 lb # [ 258.301009] niks3-server[1077]: 2026/09/22 15:49:30 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo worker2 # [ 258.305196] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=1 worker2 # [ 258.325169] nix-daemon[856]: reaped child process 1914, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.55 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.17 seconds) worker1: must succeed: pkill -f 'read -t [9]0 x' lb # [ 258.552288] envoy[894]: conn peer=[2001:db8:1::1]:43232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554536:system_library:OPENSSL_internal:Connection_reset_by_peer:TLS_error_end" rx=3000 tx=913 ms=6269 lb # [ 258.560644] envoy[894]: conn peer=[2001:db8:1::1]:57646 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9638 tx=4451 ms=6300 worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds) client: waiting for success: ! systemctl is-active sw worker1 # [ 258.724829] nix-daemon[829]: reaped child process 1751, status = succeeded worker1 # [ 258.851432] nix-daemon[829]: accepted connection from pid 1798, user nix-grpc-daemon (trusted) worker1 # [ 258.869013] nix-daemon[829]: reaped child process 1801, status = succeeded lb # [ 258.848256] niks3-server[1077]: 2026/09/22 15:49:31 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 258.883908] nix-grpc-daemon[1686]: time=2026-09-22T15:49:31.485Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 258.885668] nix-grpc-daemon[1686]: time=2026-09-22T15:49:31.485Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)" lb # [ 258.877512] niks3-server[1077]: 2026/09/22 15:49:31 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv lb # [ 258.893725] niks3-server[1077]: 2026/09/22 15:49:31 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst lb # [ 258.910393] niks3-server[1077]: 2026/09/22 15:49:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign worker1 # [ 258.938560] nix-grpc-daemon[1686]: time=2026-09-22T15:49:31.539Z level=INFO msg="Uploading 1 narinfos" lb # [ 258.917921] niks3-server[1077]: 2026/09/22 15:49:31 INFO Signed narinfos id=61 count=1 lb # [ 258.926441] niks3-server[1077]: 2026/09/22 15:49:31 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls worker1 # [ 258.967178] nix-grpc-daemon[1686]: time=2026-09-22T15:49:31.569Z level=INFO msg="Upload complete. (272ms)" lb # [ 258.942405] niks3-server[1077]: 2026/09/22 15:49:31 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete lb # [ 258.946513] niks3-server[1077]: 2026/09/22 15:49:31 INFO Completed upload id=61 lb # [ 258.951448] niks3-server[1077]: 2026/09/22 15:49:31 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo worker1 # [ 258.981155] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42214 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=1 worker1 # [ 259.001278] nix-daemon[829]: reaped child process 1749, status = succeeded client # [ 259.008747] systemd[1]: sw.service: Deactivated successfully. client # [ 259.013662] systemd[1]: sw.service: Consumed 199ms CPU time over 9.890s wall clock time, 41.4M memory peak, 4K written to disk, 7.2K incoming IP traffic, 8.3K outgoing IP traffic. worker1 # [ 259.560851] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:32Z level=info event=signal_exit client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds) worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service lb # [ 260.513910] niks3-server[1077]: 2026/09/22 15:49:33 INFO lead: released remote=192.168.1.3:37838 worker1 # [ 261.533614] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:34Z level=info event=scheduler_restarting worker2 # [ 261.509663] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 261.546081] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:34Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:58180 worker1 # [ 261.550915] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:34Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 261.555602] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:34Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 261.561227] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:34Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:58180 worker1 # [ 261.565901] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:34Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 261.588618] nix-daemon[829]: reaped child process 1647, status = succeeded worker1 # [ 261.592506] nix-grpc-daemon[1573]: E0000 00:00:1790092174.194884 1618 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true worker1 # [ 261.598867] nix-grpc-daemon[1573]: E0000 00:00:1790092174.195284 1618 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker2 # [ 261.710122] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 261.715648] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 261.760561] nix-grpc-daemon[1573]: ts=2026-09-22T15:49:34Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 261.916659] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 261.922071] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 262.123239] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 262.129113] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 262.330328] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 262.336700] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 262.539648] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 262.544337] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 262.609669] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_take_over lb # [ 262.609655] niks3-server[1077]: 2026/09/22 15:49:35 INFO lead: acquired remote=192.168.1.4:45150 worker1 # [ 262.658291] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 262.664225] systemd[1]: nix-grpc-daemon.service: Consumed 1.145s CPU time over 2min 310ms wall clock time, 36.8M memory peak, 27.7K incoming IP traffic, 36.5K outgoing IP traffic. worker1 # [ 262.687189] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 262.741054] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 262.747061] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 262.822550] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 262.824936] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 262.828752] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 262.835388] nix-grpc-daemon[1815]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 262.856215] nix-grpc-daemon[1815]: E0000 00:00:1790092175.435967 1815 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 262.858565] nix-grpc-daemon[1815]: E0000 00:00:1790092175.438135 1815 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.13 seconds) lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 262.884651] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 262.886899] nix-grpc-daemon[1815]: E0000 00:00:1790092175.447841 1854 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 262.889603] nix-grpc-daemon[1815]: E0000 00:00:1790092175.469506 1854 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 262.891819] nix-grpc-daemon[1815]: E0000 00:00:1790092175.469725 1854 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 262.894376] nix-grpc-daemon[1815]: E0000 00:00:1790092175.486555 1854 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 262.899850] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 262.902774] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 262.904139] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker2 # [ 262.948321] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 262.953655] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 263.103910] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 263.109106] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 263.155154] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 263.160627] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 263.309862] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 263.315208] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 263.361767] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 263.367251] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 263.516180] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 263.521620] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 263.570742] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 263.575157] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 263.722257] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 263.713782] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:47470 worker2 # [ 263.719564] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 263.771031] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 263.784137] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 263.790087] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:47470 worker2 # [ 263.795649] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:36Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.06 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker2 # [ 264.461136] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 264.469588] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 264.514323] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:37Z level=debug event=want client=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=0 worker2 # [ 264.520807] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:37Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 264.526939] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:37Z level=debug event=want client=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=0 worker2 # [ 264.532364] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:37Z level=debug event=want client=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=0 worker1 # [ 264.598385] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 264.614116] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 264.639248] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 1/5) worker1 # [ 264.709253] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 2/5) worker1 # [ 264.768089] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 189 ms (attempt 3/5) worker1 # [ 264.966364] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 512 ms (attempt 4/5) worker1 # [ 265.480574] nix-grpc-daemon[1815]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 265.486666] nix-grpc-daemon[1815]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 265.503382] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 265.518255] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 265.532230] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 265.558351] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:38Z level=info event=niks3_push_started pid=1875 worker1 # [ 265.706837] nix-grpc-daemon[1875]: time=2026-09-22T15:49:38.309Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 265.815409] nix-daemon[829]: accepted connection from pid 1888, user nix-grpc-daemon (trusted) worker1 # [ 265.832252] nix-daemon[829]: reaped child process 1891, status = succeeded worker1 # [ 265.935088] nix-daemon[829]: accepted connection from pid 1893, user nix-grpc-daemon (trusted) worker1 # [ 265.950748] nix-daemon[829]: reaped child process 1896, status = succeeded lb # [ 265.931760] niks3-server[1077]: 2026/09/22 15:49:38 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 265.968064] nix-grpc-daemon[1875]: time=2026-09-22T15:49:38.570Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 265.969539] nix-grpc-daemon[1875]: time=2026-09-22T15:49:38.570Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)" lb # [ 265.970006] niks3-server[1077]: 2026/09/22 15:49:38 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst worker1 # [ 266.016758] nix-grpc-daemon[1875]: time=2026-09-22T15:49:38.619Z level=INFO msg="Uploading 1 narinfos" lb # [ 265.989798] niks3-server[1077]: 2026/09/22 15:49:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign lb # [ 266.000776] niks3-server[1077]: 2026/09/22 15:49:38 INFO Signed narinfos id=62 count=1 lb # [ 266.008675] niks3-server[1077]: 2026/09/22 15:49:38 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls lb # [ 266.029699] niks3-server[1077]: 2026/09/22 15:49:38 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete worker1 # [ 266.063108] nix-grpc-daemon[1875]: time=2026-09-22T15:49:38.665Z level=INFO msg="Upload complete. (352ms)" lb # [ 266.040592] niks3-server[1077]: 2026/09/22 15:49:38 INFO Completed upload id=62 worker1 # [ 266.075273] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 266.097495] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 266.104840] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 266.109383] nix-daemon[829]: reaped child process 1873, status = succeeded worker1 # [ 266.122436] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 266.958120] nix-daemon[829]: reaped child process 1900, status = succeeded worker1 # [ 267.075552] nix-daemon[829]: accepted connection from pid 1916, user nix-grpc-daemon (trusted) worker1 # [ 267.093928] nix-daemon[829]: reaped child process 1919, status = succeeded lb # [ 267.075095] niks3-server[1077]: 2026/09/22 15:49:39 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 267.120447] nix-grpc-daemon[1875]: time=2026-09-22T15:49:39.720Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 267.122238] nix-grpc-daemon[1875]: time=2026-09-22T15:49:39.720Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)" worker1 # [ 267.170632] nix-grpc-daemon[1875]: time=2026-09-22T15:49:39.772Z level=INFO msg="Uploading 1 narinfos" lb # [ 267.132911] niks3-server[1077]: 2026/09/22 15:49:39 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst lb # [ 267.149516] niks3-server[1077]: 2026/09/22 15:49:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign lb # [ 267.163485] niks3-server[1077]: 2026/09/22 15:49:39 INFO Signed narinfos id=63 count=1 lb # [ 267.170030] niks3-server[1077]: 2026/09/22 15:49:39 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls lb # [ 267.181087] niks3-server[1077]: 2026/09/22 15:49:39 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete worker1 # [ 267.212710] nix-grpc-daemon[1875]: time=2026-09-22T15:49:39.815Z level=INFO msg="Upload complete. (280ms)" lb # [ 267.187649] niks3-server[1077]: 2026/09/22 15:49:39 INFO Completed upload id=63 lb # [ 267.195670] niks3-server[1077]: 2026/09/22 15:49:39 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo lb # [ 267.203105] niks3-server[1077]: 2026/09/22 15:49:39 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker2 # [ 267.201059] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:39Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 267.225232] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=1 outputs=1 worker2 # [ 267.216280] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:39Z level=debug event=want client=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=0 worker1 # [ 267.246785] nix-daemon[829]: reaped child process 1898, status = succeeded worker1 # [ 267.273690] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 267.289339] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 267.303965] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 267.317832] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 267.474561] nix-daemon[829]: accepted connection from pid 1923, user nix-grpc-daemon (trusted) worker1 # [ 267.502226] nix-daemon[829]: reaped child process 1926, status = succeeded worker1 # [ 267.594501] nix-daemon[829]: accepted connection from pid 1928, user nix-grpc-daemon (trusted) worker1 # [ 267.613544] nix-daemon[829]: reaped child process 1931, status = succeeded lb # [ 267.584993] niks3-server[1077]: 2026/09/22 15:49:40 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 267.624065] nix-grpc-daemon[1875]: time=2026-09-22T15:49:40.226Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 267.626057] nix-grpc-daemon[1875]: time=2026-09-22T15:49:40.226Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)" lb # [ 267.621039] niks3-server[1077]: 2026/09/22 15:49:40 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst worker1 # [ 267.661913] nix-grpc-daemon[1875]: time=2026-09-22T15:49:40.264Z level=INFO msg="Uploading 1 narinfos" lb # [ 267.638812] niks3-server[1077]: 2026/09/22 15:49:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign lb # [ 267.648594] niks3-server[1077]: 2026/09/22 15:49:40 INFO Signed narinfos id=64 count=1 lb # [ 267.662045] niks3-server[1077]: 2026/09/22 15:49:40 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls worker1 # [ 267.695876] nix-grpc-daemon[1875]: time=2026-09-22T15:49:40.298Z level=INFO msg="Upload complete. (361ms)" lb # [ 267.670573] niks3-server[1077]: 2026/09/22 15:49:40 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete lb # [ 267.675691] niks3-server[1077]: 2026/09/22 15:49:40 INFO Completed upload id=64 lb # [ 267.679096] niks3-server[1077]: 2026/09/22 15:49:40 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo worker1 # [ 267.704550] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 267.719513] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 267.728202] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 267.730721] nix-daemon[829]: reaped child process 1921, status = succeeded worker1 # [ 267.742162] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) lb # [ 268.554214] envoy[894]: conn peer=[2001:db8:1::1]:43242 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=1516 ms=9508 lb # [ 268.559206] envoy[894]: conn peer=[2001:db8:1::3]:54556 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2295 tx=877 ms=119182 worker1 # [ 268.604732] nix-daemon[829]: reaped child process 1935, status = succeeded worker1 # [ 268.747729] nix-daemon[829]: accepted connection from pid 1951, user nix-grpc-daemon (trusted) worker1 # [ 268.767007] nix-daemon[829]: reaped child process 1954, status = succeeded lb # [ 268.750236] niks3-server[1077]: 2026/09/22 15:49:41 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 268.784426] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.386Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 268.786057] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.386Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)" lb # [ 268.787700] niks3-server[1077]: 2026/09/22 15:49:41 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker1 # [ 268.830130] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.432Z level=INFO msg="Uploading 1 narinfos" lb # [ 268.805042] niks3-server[1077]: 2026/09/22 15:49:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign lb # [ 268.814840] niks3-server[1077]: 2026/09/22 15:49:41 INFO Signed narinfos id=65 count=1 lb # [ 268.820298] niks3-server[1077]: 2026/09/22 15:49:41 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst lb # [ 268.827283] niks3-server[1077]: 2026/09/22 15:49:41 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls lb # [ 268.837693] niks3-server[1077]: 2026/09/22 15:49:41 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete lb # [ 268.844533] niks3-server[1077]: 2026/09/22 15:49:41 INFO Completed upload id=65 worker1 # [ 268.869333] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.472Z level=INFO msg="Upload complete. (314ms)" lb # [ 268.847132] niks3-server[1077]: 2026/09/22 15:49:41 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo worker2 # [ 268.854921] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:41Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 268.869861] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:41Z level=debug event=want client=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=0 worker1 # [ 268.881250] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=2 outputs=1 worker1 # [ 268.903017] nix-daemon[829]: reaped child process 1933, status = succeeded worker1 # [ 268.934630] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 268.950705] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 268.965829] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 268.979924] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 269.135478] nix-daemon[829]: accepted connection from pid 1958, user nix-grpc-daemon (trusted) worker1 # [ 269.153800] nix-daemon[829]: reaped child process 1961, status = succeeded worker1 # [ 269.259548] nix-daemon[829]: accepted connection from pid 1963, user nix-grpc-daemon (trusted) worker1 # [ 269.274722] nix-daemon[829]: reaped child process 1966, status = succeeded lb # [ 269.255762] niks3-server[1077]: 2026/09/22 15:49:41 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 269.291037] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.893Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 269.292670] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.893Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)" lb # [ 269.281656] niks3-server[1077]: 2026/09/22 15:49:41 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst lb # [ 269.297883] niks3-server[1077]: 2026/09/22 15:49:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign worker1 # [ 269.329914] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.932Z level=INFO msg="Uploading 1 narinfos" lb # [ 269.310089] niks3-server[1077]: 2026/09/22 15:49:41 INFO Signed narinfos id=66 count=1 lb # [ 269.318585] niks3-server[1077]: 2026/09/22 15:49:41 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls lb # [ 269.332474] niks3-server[1077]: 2026/09/22 15:49:41 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete worker1 # [ 269.357829] nix-grpc-daemon[1875]: time=2026-09-22T15:49:41.960Z level=INFO msg="Upload complete. (362ms)" lb # [ 269.342793] niks3-server[1077]: 2026/09/22 15:49:41 INFO Completed upload id=66 worker1 # [ 269.368452] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 269.346392] niks3-server[1077]: 2026/09/22 15:49:41 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo worker1 # [ 269.379231] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 269.388875] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 269.391859] nix-daemon[829]: reaped child process 1956, status = succeeded worker1 # [ 269.403442] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 270.857231] nix-daemon[829]: reaped child process 1970, status = succeeded worker1 # [ 270.973464] nix-daemon[829]: accepted connection from pid 1986, user nix-grpc-daemon (trusted) worker1 # [ 270.991899] nix-daemon[829]: reaped child process 1989, status = succeeded lb # [ 270.970650] niks3-server[1077]: 2026/09/22 15:49:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 271.008444] nix-grpc-daemon[1875]: time=2026-09-22T15:49:43.610Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 271.010138] nix-grpc-daemon[1875]: time=2026-09-22T15:49:43.610Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)" lb # [ 271.010403] niks3-server[1077]: 2026/09/22 15:49:43 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv lb # [ 271.021990] niks3-server[1077]: 2026/09/22 15:49:43 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 271.035486] niks3-server[1077]: 2026/09/22 15:49:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign worker1 # [ 271.066949] nix-grpc-daemon[1875]: time=2026-09-22T15:49:43.669Z level=INFO msg="Uploading 1 narinfos" lb # [ 271.042376] niks3-server[1077]: 2026/09/22 15:49:43 INFO Signed narinfos id=67 count=1 lb # [ 271.049615] niks3-server[1077]: 2026/09/22 15:49:43 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls lb # [ 271.057532] niks3-server[1077]: 2026/09/22 15:49:43 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete lb # [ 271.067842] niks3-server[1077]: 2026/09/22 15:49:43 INFO Completed upload id=67 worker1 # [ 271.093662] nix-grpc-daemon[1875]: time=2026-09-22T15:49:43.696Z level=INFO msg="Upload complete. (277ms)" lb # [ 271.071466] niks3-server[1077]: 2026/09/22 15:49:43 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo worker2 # [ 271.079388] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:43Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 271.108861] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=3 outputs=1 worker2 # [ 271.092146] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:43Z level=debug event=want client=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=0 worker1 # [ 271.126769] nix-daemon[829]: reaped child process 1968, status = succeeded worker1 # [ 271.151684] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 271.167091] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 271.181993] nix-grpc-daemon[1875]: time=2026-09-22T15:49:43.784Z level=INFO msg="All 1 paths already cached" worker1 # [ 271.191665] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 271.205335] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 271.355065] nix-daemon[829]: accepted connection from pid 1993, user nix-grpc-daemon (trusted) worker1 # [ 271.381966] nix-daemon[829]: reaped child process 1996, status = succeeded worker1 # [ 271.476567] nix-daemon[829]: accepted connection from pid 1998, user nix-grpc-daemon (trusted) worker1 # [ 271.495079] nix-daemon[829]: reaped child process 2001, status = succeeded lb # [ 271.473162] niks3-server[1077]: 2026/09/22 15:49:44 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 271.509217] nix-grpc-daemon[1875]: time=2026-09-22T15:49:44.111Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 271.510697] nix-grpc-daemon[1875]: time=2026-09-22T15:49:44.111Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)" lb # [ 271.500662] niks3-server[1077]: 2026/09/22 15:49:44 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst lb # [ 271.517054] niks3-server[1077]: 2026/09/22 15:49:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign worker1 # [ 271.550556] nix-grpc-daemon[1875]: time=2026-09-22T15:49:44.153Z level=INFO msg="Uploading 1 narinfos" lb # [ 271.529344] niks3-server[1077]: 2026/09/22 15:49:44 INFO Signed narinfos id=68 count=1 lb # [ 271.536423] niks3-server[1077]: 2026/09/22 15:49:44 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls lb # [ 271.548911] niks3-server[1077]: 2026/09/22 15:49:44 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete worker1 # [ 271.586103] nix-grpc-daemon[1875]: time=2026-09-22T15:49:44.188Z level=INFO msg="Upload complete. (365ms)" lb # [ 271.559675] niks3-server[1077]: 2026/09/22 15:49:44 INFO Completed upload id=68 lb # [ 271.563910] niks3-server[1077]: 2026/09/22 15:49:44 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo worker1 # [ 271.596384] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 271.612253] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 271.619074] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 271.624827] nix-daemon[829]: reaped child process 1991, status = succeeded worker1 # [ 271.638262] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 272.924468] nix-daemon[829]: reaped child process 2005, status = succeeded worker1 # [ 273.009619] nix-daemon[829]: accepted connection from pid 2021, user nix-grpc-daemon (trusted) worker1 # [ 273.028630] nix-daemon[829]: reaped child process 2024, status = succeeded lb # [ 273.008827] niks3-server[1077]: 2026/09/22 15:49:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 273.048422] nix-grpc-daemon[1875]: time=2026-09-22T15:49:45.650Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 273.049945] nix-grpc-daemon[1875]: time=2026-09-22T15:49:45.650Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)" lb # [ 273.054488] niks3-server[1077]: 2026/09/22 15:49:45 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst lb # [ 273.066525] niks3-server[1077]: 2026/09/22 15:49:45 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker1 # [ 273.104272] nix-grpc-daemon[1875]: time=2026-09-22T15:49:45.706Z level=INFO msg="Uploading 1 narinfos" lb # [ 273.081451] niks3-server[1077]: 2026/09/22 15:49:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign lb # [ 273.089621] niks3-server[1077]: 2026/09/22 15:49:45 INFO Signed narinfos id=69 count=1 lb # [ 273.095770] niks3-server[1077]: 2026/09/22 15:49:45 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls lb # [ 273.107123] niks3-server[1077]: 2026/09/22 15:49:45 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 273.113042] niks3-server[1077]: 2026/09/22 15:49:45 INFO Completed upload id=69 worker1 # [ 273.139407] nix-grpc-daemon[1875]: time=2026-09-22T15:49:45.742Z level=INFO msg="Upload complete. (289ms)" lb # [ 273.115850] niks3-server[1077]: 2026/09/22 15:49:45 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo worker2 # [ 273.123421] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:45Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 273.153263] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=4 outputs=1 worker2 # [ 273.135851] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:45Z level=debug event=want client=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=0 worker1 # [ 273.174882] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 273.182095] nix-daemon[829]: reaped child process 2003, status = succeeded worker1 # [ 273.190693] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 273.205185] nix-grpc-daemon[1875]: time=2026-09-22T15:49:45.807Z level=INFO msg="All 1 paths already cached" worker1 # [ 273.212303] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 273.225819] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 273.387853] nix-daemon[829]: accepted connection from pid 2028, user nix-grpc-daemon (trusted) worker1 # [ 273.411306] nix-daemon[829]: reaped child process 2031, status = succeeded worker1 # [ 273.517270] nix-daemon[829]: accepted connection from pid 2033, user nix-grpc-daemon (trusted) worker1 # [ 273.538377] nix-daemon[829]: reaped child process 2036, status = succeeded lb # [ 273.510027] niks3-server[1077]: 2026/09/22 15:49:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 273.547815] nix-grpc-daemon[1875]: time=2026-09-22T15:49:46.150Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 273.549572] nix-grpc-daemon[1875]: time=2026-09-22T15:49:46.150Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)" lb # [ 273.540561] niks3-server[1077]: 2026/09/22 15:49:46 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst worker1 # [ 273.580253] nix-grpc-daemon[1875]: time=2026-09-22T15:49:46.182Z level=INFO msg="Uploading 1 narinfos" lb # [ 273.555354] niks3-server[1077]: 2026/09/22 15:49:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign lb # [ 273.567448] niks3-server[1077]: 2026/09/22 15:49:46 INFO Signed narinfos id=70 count=1 lb # [ 273.572797] niks3-server[1077]: 2026/09/22 15:49:46 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls worker1 # [ 273.614286] nix-grpc-daemon[1875]: time=2026-09-22T15:49:46.217Z level=INFO msg="Upload complete. (374ms)" lb # [ 273.589414] niks3-server[1077]: 2026/09/22 15:49:46 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete lb # [ 273.598781] niks3-server[1077]: 2026/09/22 15:49:46 INFO Completed upload id=70 worker1 # [ 273.625456] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 273.601728] niks3-server[1077]: 2026/09/22 15:49:46 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo worker1 # [ 273.632343] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 273.643488] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 273.648451] nix-daemon[829]: reaped child process 2026, status = succeeded worker1 # [ 273.658996] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 275.094417] nix-daemon[829]: reaped child process 2040, status = succeeded worker1 # [ 275.194842] nix-daemon[829]: accepted connection from pid 2056, user nix-grpc-daemon (trusted) worker1 # [ 275.216603] nix-daemon[829]: reaped child process 2059, status = succeeded lb # [ 275.193154] niks3-server[1077]: 2026/09/22 15:49:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 275.232614] nix-grpc-daemon[1875]: time=2026-09-22T15:49:47.835Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 275.235727] nix-grpc-daemon[1875]: time=2026-09-22T15:49:47.835Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)" lb # [ 275.227999] niks3-server[1077]: 2026/09/22 15:49:47 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv lb # [ 275.247847] niks3-server[1077]: 2026/09/22 15:49:47 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst lb # [ 275.262011] niks3-server[1077]: 2026/09/22 15:49:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign worker1 # [ 275.293252] nix-grpc-daemon[1875]: time=2026-09-22T15:49:47.895Z level=INFO msg="Uploading 1 narinfos" lb # [ 275.269083] niks3-server[1077]: 2026/09/22 15:49:47 INFO Signed narinfos id=71 count=1 lb # [ 275.275633] niks3-server[1077]: 2026/09/22 15:49:47 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls lb # [ 275.284252] niks3-server[1077]: 2026/09/22 15:49:47 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete worker1 # [ 275.314910] nix-grpc-daemon[1875]: time=2026-09-22T15:49:47.917Z level=INFO msg="Upload complete. (243ms)" lb # [ 275.292112] niks3-server[1077]: 2026/09/22 15:49:47 INFO Completed upload id=71 lb # [ 275.295263] niks3-server[1077]: 2026/09/22 15:49:47 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo worker2 # [ 275.298155] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:47Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 275.324948] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=5 outputs=1 worker1 # [ 275.336439] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker2 # [ 275.311353] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:47Z level=debug event=want client=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=0 worker1 # [ 275.345861] nix-daemon[829]: reaped child process 2038, status = succeeded worker1 # [ 275.353279] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 275.364404] nix-grpc-daemon[1875]: time=2026-09-22T15:49:47.967Z level=INFO msg="All 1 paths already cached" worker1 # [ 275.372628] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 275.384669] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 275.524728] nix-daemon[829]: accepted connection from pid 2063, user nix-grpc-daemon (trusted) worker1 # [ 275.547662] nix-daemon[829]: reaped child process 2066, status = succeeded worker1 # [ 275.665870] nix-daemon[829]: accepted connection from pid 2068, user nix-grpc-daemon (trusted) worker1 # [ 275.686669] nix-daemon[829]: reaped child process 2071, status = succeeded lb # [ 275.663596] niks3-server[1077]: 2026/09/22 15:49:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 275.699750] nix-grpc-daemon[1875]: time=2026-09-22T15:49:48.302Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 275.702213] nix-grpc-daemon[1875]: time=2026-09-22T15:49:48.302Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)" lb # [ 275.694168] niks3-server[1077]: 2026/09/22 15:49:48 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst worker1 # [ 275.736214] nix-grpc-daemon[1875]: time=2026-09-22T15:49:48.338Z level=INFO msg="Uploading 1 narinfos" lb # [ 275.710422] niks3-server[1077]: 2026/09/22 15:49:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign lb # [ 275.718407] niks3-server[1077]: 2026/09/22 15:49:48 INFO Signed narinfos id=72 count=1 lb # [ 275.725431] niks3-server[1077]: 2026/09/22 15:49:48 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls worker1 # [ 275.763806] nix-grpc-daemon[1875]: time=2026-09-22T15:49:48.366Z level=INFO msg="Upload complete. (367ms)" lb # [ 275.738455] niks3-server[1077]: 2026/09/22 15:49:48 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete worker1 # [ 275.768264] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 275.774038] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 lb # [ 275.747993] niks3-server[1077]: 2026/09/22 15:49:48 INFO Completed upload id=72 lb # [ 275.750679] niks3-server[1077]: 2026/09/22 15:49:48 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo worker1 # [ 275.782677] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 275.783925] nix-daemon[829]: reaped child process 2061, status = succeeded worker1 # [ 275.791094] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 276.425086] nix-daemon[829]: reaped child process 2075, status = succeeded worker1 # [ 276.541243] nix-daemon[829]: accepted connection from pid 2091, user nix-grpc-daemon (trusted) worker1 # [ 276.559293] nix-daemon[829]: reaped child process 2094, status = succeeded lb # [ 276.539043] niks3-server[1077]: 2026/09/22 15:49:49 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 276.577690] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.179Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 276.579152] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.179Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)" worker1 # [ 276.622187] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.224Z level=INFO msg="Uploading 1 narinfos" lb # [ 276.584507] niks3-server[1077]: 2026/09/22 15:49:49 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv lb # [ 276.600167] niks3-server[1077]: 2026/09/22 15:49:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign lb # [ 276.612333] niks3-server[1077]: 2026/09/22 15:49:49 INFO Signed narinfos id=73 count=1 lb # [ 276.619036] niks3-server[1077]: 2026/09/22 15:49:49 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst lb # [ 276.627599] niks3-server[1077]: 2026/09/22 15:49:49 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls worker1 # [ 276.664750] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.267Z level=INFO msg="Upload complete. (264ms)" lb # [ 276.639091] niks3-server[1077]: 2026/09/22 15:49:49 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 276.642988] niks3-server[1077]: 2026/09/22 15:49:49 INFO Completed upload id=73 lb # [ 276.647283] niks3-server[1077]: 2026/09/22 15:49:49 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo worker2 # [ 276.648662] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:49Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 276.677322] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=6 outputs=1 worker1 # [ 276.698583] nix-daemon[829]: reaped child process 2073, status = succeeded worker1 # [ 276.720808] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker2 # [ 276.698505] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:49Z level=debug event=want client=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=0 worker1 # [ 276.736639] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 276.750679] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.353Z level=INFO msg="All 2 paths already cached" worker1 # [ 276.760476] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 276.773105] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 276.922953] nix-daemon[829]: accepted connection from pid 2098, user nix-grpc-daemon (trusted) worker1 # [ 276.943725] nix-daemon[829]: reaped child process 2101, status = succeeded worker1 # [ 277.042412] nix-daemon[829]: accepted connection from pid 2103, user nix-grpc-daemon (trusted) worker1 # [ 277.058908] nix-daemon[829]: reaped child process 2106, status = succeeded lb # [ 277.037817] niks3-server[1077]: 2026/09/22 15:49:49 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 277.074321] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.676Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 277.075814] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.677Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)" lb # [ 277.065219] niks3-server[1077]: 2026/09/22 15:49:49 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst worker1 # [ 277.105640] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.708Z level=INFO msg="Uploading 1 narinfos" lb # [ 277.079891] niks3-server[1077]: 2026/09/22 15:49:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign lb # [ 277.091459] niks3-server[1077]: 2026/09/22 15:49:49 INFO Signed narinfos id=74 count=1 lb # [ 277.098059] niks3-server[1077]: 2026/09/22 15:49:49 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls worker1 # [ 277.137465] nix-grpc-daemon[1875]: time=2026-09-22T15:49:49.740Z level=INFO msg="Upload complete. (348ms)" lb # [ 277.111512] niks3-server[1077]: 2026/09/22 15:49:49 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete lb # [ 277.120220] niks3-server[1077]: 2026/09/22 15:49:49 INFO Completed upload id=74 worker1 # [ 277.145565] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 277.122927] niks3-server[1077]: 2026/09/22 15:49:49 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo worker1 # [ 277.158609] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 277.166213] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 277.172292] nix-daemon[829]: reaped child process 2096, status = succeeded worker1 # [ 277.181030] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 278.456193] nix-daemon[829]: reaped child process 2110, status = succeeded worker1 # [ 278.561196] nix-daemon[829]: accepted connection from pid 2126, user nix-grpc-daemon (trusted) worker1 # [ 278.581580] nix-daemon[829]: reaped child process 2129, status = succeeded lb # [ 278.562171] niks3-server[1077]: 2026/09/22 15:49:51 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 278.600366] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.200Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 278.601858] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.200Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)" lb # [ 278.592807] niks3-server[1077]: 2026/09/22 15:49:51 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv lb # [ 278.608957] niks3-server[1077]: 2026/09/22 15:49:51 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst lb # [ 278.624695] niks3-server[1077]: 2026/09/22 15:49:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign worker1 # [ 278.657854] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.260Z level=INFO msg="Uploading 1 narinfos" lb # [ 278.635753] niks3-server[1077]: 2026/09/22 15:49:51 INFO Signed narinfos id=75 count=1 lb # [ 278.641485] niks3-server[1077]: 2026/09/22 15:49:51 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls lb # [ 278.652893] niks3-server[1077]: 2026/09/22 15:49:51 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete lb # [ 278.659962] niks3-server[1077]: 2026/09/22 15:49:51 INFO Completed upload id=75 worker1 # [ 278.686812] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.289Z level=INFO msg="Upload complete. (295ms)" lb # [ 278.665353] niks3-server[1077]: 2026/09/22 15:49:51 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo worker2 # [ 278.673513] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:51Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 278.700735] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=7 outputs=1 worker1 # [ 278.722075] nix-daemon[829]: reaped child process 2108, status = succeeded worker1 # [ 278.746718] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 278.762086] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 278.777798] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.380Z level=INFO msg="All 2 paths already cached" worker1 # [ 278.787909] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 278.802165] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 278.956096] nix-daemon[829]: accepted connection from pid 2133, user nix-grpc-daemon (trusted) worker1 # [ 278.976148] nix-daemon[829]: reaped child process 2136, status = succeeded worker1 # [ 279.078588] nix-daemon[829]: accepted connection from pid 2138, user nix-grpc-daemon (trusted) worker1 # [ 279.093997] nix-daemon[829]: reaped child process 2141, status = succeeded lb # [ 279.074561] niks3-server[1077]: 2026/09/22 15:49:51 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 279.110902] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.713Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 279.112446] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.713Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)" lb # [ 279.101672] niks3-server[1077]: 2026/09/22 15:49:51 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst lb # [ 279.118527] niks3-server[1077]: 2026/09/22 15:49:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign worker1 # [ 279.151912] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.754Z level=INFO msg="Uploading 1 narinfos" lb # [ 279.131618] niks3-server[1077]: 2026/09/22 15:49:51 INFO Signed narinfos id=76 count=1 lb # [ 279.136632] niks3-server[1077]: 2026/09/22 15:49:51 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls lb # [ 279.147964] niks3-server[1077]: 2026/09/22 15:49:51 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete lb # [ 279.153424] niks3-server[1077]: 2026/09/22 15:49:51 INFO Completed upload id=76 worker1 # [ 279.179011] nix-grpc-daemon[1875]: time=2026-09-22T15:49:51.781Z level=INFO msg="Upload complete. (360ms)" lb # [ 279.155585] niks3-server[1077]: 2026/09/22 15:49:51 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo worker1 # [ 279.189068] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 279.199651] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 279.206607] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 279.211350] nix-daemon[829]: reaped child process 2131, status = succeeded worker1 # [ 279.224317] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 280.704161] nix-daemon[829]: reaped child process 2145, status = succeeded worker1 # [ 280.808114] nix-daemon[829]: accepted connection from pid 2161, user nix-grpc-daemon (trusted) worker1 # [ 280.824106] nix-daemon[829]: reaped child process 2164, status = succeeded lb # [ 280.806830] niks3-server[1077]: 2026/09/22 15:49:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 280.844423] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.446Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 280.846593] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.446Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)" lb # [ 280.843302] niks3-server[1077]: 2026/09/22 15:49:53 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv lb # [ 280.857729] niks3-server[1077]: 2026/09/22 15:49:53 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst worker1 # [ 280.897915] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.500Z level=INFO msg="Uploading 1 narinfos" lb # [ 280.871784] niks3-server[1077]: 2026/09/22 15:49:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign lb # [ 280.879541] niks3-server[1077]: 2026/09/22 15:49:53 INFO Signed narinfos id=77 count=1 lb # [ 280.884822] niks3-server[1077]: 2026/09/22 15:49:53 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls lb # [ 280.893707] niks3-server[1077]: 2026/09/22 15:49:53 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete lb # [ 280.899721] niks3-server[1077]: 2026/09/22 15:49:53 INFO Completed upload id=77 worker1 # [ 280.926064] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.528Z level=INFO msg="Upload complete. (290ms)" lb # [ 280.902090] niks3-server[1077]: 2026/09/22 15:49:53 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo worker2 # [ 280.909818] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:53Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 280.941261] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=8 outputs=1 worker1 # [ 280.960366] nix-daemon[829]: reaped child process 2143, status = succeeded worker1 # [ 280.986598] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 281.001998] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 281.016636] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.619Z level=INFO msg="All 2 paths already cached" worker1 # [ 281.026814] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 281.040580] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 281.197472] nix-daemon[829]: accepted connection from pid 2168, user nix-grpc-daemon (trusted) worker1 # [ 281.218626] nix-daemon[829]: reaped child process 2171, status = succeeded worker1 # [ 281.332977] nix-daemon[829]: accepted connection from pid 2173, user nix-grpc-daemon (trusted) worker1 # [ 281.350513] nix-daemon[829]: reaped child process 2176, status = succeeded lb # [ 281.328281] niks3-server[1077]: 2026/09/22 15:49:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 281.364505] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.967Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 281.365995] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.967Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)" lb # [ 281.355184] niks3-server[1077]: 2026/09/22 15:49:53 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst worker1 # [ 281.394255] nix-grpc-daemon[1875]: time=2026-09-22T15:49:53.997Z level=INFO msg="Uploading 1 narinfos" lb # [ 281.371104] niks3-server[1077]: 2026/09/22 15:49:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign lb # [ 281.380688] niks3-server[1077]: 2026/09/22 15:49:53 INFO Signed narinfos id=78 count=1 lb # [ 281.391840] niks3-server[1077]: 2026/09/22 15:49:53 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete worker1 # [ 281.427291] nix-grpc-daemon[1875]: time=2026-09-22T15:49:54.030Z level=INFO msg="Upload complete. (370ms)" lb # [ 281.402191] niks3-server[1077]: 2026/09/22 15:49:53 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls lb # [ 281.407676] niks3-server[1077]: 2026/09/22 15:49:53 INFO Completed upload id=78 worker1 # [ 281.433432] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 281.411211] niks3-server[1077]: 2026/09/22 15:49:53 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo worker1 # [ 281.444280] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 worker1 # [ 281.454193] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 281.458840] nix-daemon[829]: reaped child process 2166, status = succeeded worker1 # [ 281.469206] nix-daemon[829]: accepted connection from pid 1815, user nix-grpc-daemon (trusted) worker1 # [ 282.575950] nix-daemon[829]: reaped child process 2180, status = succeeded worker1 # [ 282.643970] nix-daemon[829]: accepted connection from pid 2196, user nix-grpc-daemon (trusted) worker1 # [ 282.662204] nix-daemon[829]: reaped child process 2199, status = succeeded lb # [ 282.643630] niks3-server[1077]: 2026/09/22 15:49:55 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 282.683066] nix-grpc-daemon[1875]: time=2026-09-22T15:49:55.284Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 282.684960] nix-grpc-daemon[1875]: time=2026-09-22T15:49:55.284Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)" lb # [ 282.683654] niks3-server[1077]: 2026/09/22 15:49:55 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv lb # [ 282.696787] niks3-server[1077]: 2026/09/22 15:49:55 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst worker1 # [ 282.734498] nix-grpc-daemon[1875]: time=2026-09-22T15:49:55.337Z level=INFO msg="Uploading 1 narinfos" lb # [ 282.710973] niks3-server[1077]: 2026/09/22 15:49:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign lb # [ 282.719771] niks3-server[1077]: 2026/09/22 15:49:55 INFO Signed narinfos id=79 count=1 lb # [ 282.726163] niks3-server[1077]: 2026/09/22 15:49:55 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls lb # [ 282.732907] niks3-server[1077]: 2026/09/22 15:49:55 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete lb # [ 282.742110] niks3-server[1077]: 2026/09/22 15:49:55 INFO Completed upload id=79 worker1 # [ 282.767894] nix-grpc-daemon[1875]: time=2026-09-22T15:49:55.370Z level=INFO msg="Upload complete. (286ms)" lb # [ 282.745567] niks3-server[1077]: 2026/09/22 15:49:55 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo worker1 # [ 282.781236] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50068 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=9 outputs=1 worker1 # [ 282.799905] nix-daemon[829]: reaped child process 2178, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.74 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 283.003077] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 515d8eac035444d59343ce9b425311b1 client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2 # [ 283.206764] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37710 client # [ 283.221260] nix[1949]: don't know how to build these paths: client # [ 283.222419] nix[1949]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1 # [ 283.245685] nix-grpc-daemon[1815]: ts=2026-09-22T15:49:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:50068 worker2 # [ 283.244257] nix-grpc-daemon[1682]: ts=2026-09-22T15:49:55Z level=debug event=want client=2 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=0 client # [ 283.257618] nix[1949]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code lb # [ 288.560538] envoy[894]: conn peer=[2001:db8:1::1]:41934 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27118 tx=12410 ms=18362 client # [ 313.264708] nix[1949]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip} client # [ 313.279773] nix[1949]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed client # [ 313.314169] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 313.320964] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 313.324727] systemd[1]: gpu.service: Consumed 195ms CPU time over 30.310s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.09 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.59 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1 # [ 314.869216] nix-grpc-daemon[1815]: ts=2026-09-22T15:50:27Z level=info event=unhealthy reason=min_free worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.20 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.03 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2 worker1 # [ 315.189540] nix-grpc-daemon[1815]: ts=2026-09-22T15:50:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:50068 client # don't know how to build these paths: client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker2 # [ 315.177334] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 315.202872] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:27Z level=debug event=want client=3 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=0 worker2 # [ 315.210362] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:27Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 315.248987] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 315.265165] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37710 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'... worker2 # [ 315.281239] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 315.298015] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) worker2 # [ 315.456195] nix-daemon[856]: accepted connection from pid 1975, user nix-grpc-daemon (trusted) worker2 # [ 315.476172] nix-daemon[856]: reaped child process 1978, status = succeeded worker2 # [ 315.577724] nix-daemon[856]: accepted connection from pid 1980, user nix-grpc-daemon (trusted) worker2 # [ 315.595701] nix-daemon[856]: reaped child process 1983, status = succeeded lb # [ 315.602436] niks3-server[1077]: 2026/09/22 15:50:28 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 315.616275] nix-grpc-daemon[1754]: time=2026-09-22T15:50:28.215Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 315.617801] nix-grpc-daemon[1754]: time=2026-09-22T15:50:28.215Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)" lb # [ 315.636856] niks3-server[1077]: 2026/09/22 15:50:28 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst worker2 # [ 315.650364] nix-grpc-daemon[1754]: time=2026-09-22T15:50:28.249Z level=INFO msg="Uploading 1 narinfos" lb # [ 315.650417] niks3-server[1077]: 2026/09/22 15:50:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign lb # [ 315.660176] niks3-server[1077]: 2026/09/22 15:50:28 INFO Signed narinfos id=80 count=1 lb # [ 315.668034] niks3-server[1077]: 2026/09/22 15:50:28 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls lb # [ 315.685798] niks3-server[1077]: 2026/09/22 15:50:28 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete lb # [ 315.692241] niks3-server[1077]: 2026/09/22 15:50:28 INFO Completed upload id=80 worker2 # [ 315.691367] nix-grpc-daemon[1754]: time=2026-09-22T15:50:28.290Z level=INFO msg="Upload complete. (378ms)" lb # [ 315.700329] niks3-server[1077]: 2026/09/22 15:50:28 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo worker2 # [ 315.700843] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 315.716551] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 315.726593] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) worker2 # [ 315.735650] nix-daemon[856]: reaped child process 1973, status = succeeded client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051... worker2 # [ 315.743266] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv lb # [ 316.011666] postgres[798]: [798] LOG: checkpoint starting: time client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 316.436258] nix-daemon[856]: reaped child process 1987, status = succeeded worker2 # [ 316.504289] nix-daemon[856]: accepted connection from pid 2003, user nix-grpc-daemon (trusted) worker2 # [ 316.521155] nix-daemon[856]: reaped child process 2006, status = succeeded lb # [ 316.525710] niks3-server[1077]: 2026/09/22 15:50:29 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 316.536090] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.134Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 316.537721] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.134Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)" lb # [ 316.568627] niks3-server[1077]: 2026/09/22 15:50:29 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 316.582119] niks3-server[1077]: 2026/09/22 15:50:29 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker2 # [ 316.590807] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.189Z level=INFO msg="Uploading 1 narinfos" lb # [ 316.593119] niks3-server[1077]: 2026/09/22 15:50:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign lb # [ 316.602240] niks3-server[1077]: 2026/09/22 15:50:29 INFO Signed narinfos id=81 count=1 lb # [ 316.608977] niks3-server[1077]: 2026/09/22 15:50:29 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls lb # [ 316.615785] niks3-server[1077]: 2026/09/22 15:50:29 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete lb # [ 316.624528] niks3-server[1077]: 2026/09/22 15:50:29 INFO Completed upload id=81 worker2 # [ 316.629401] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.222Z level=INFO msg="Upload complete. (268ms)" lb # [ 316.629314] niks3-server[1077]: 2026/09/22 15:50:29 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo worker2 # [ 316.641844] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=10 outputs=1 worker2 # [ 316.662512] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=debug event=want client=3 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=0 worker2 # [ 316.668539] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 316.674712] nix-daemon[856]: reaped child process 1985, status = succeeded worker2 # [ 316.709101] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 316.724473] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 316.739931] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.339Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'... worker2 # [ 316.751063] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 316.765838] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) worker2 # [ 316.918716] nix-daemon[856]: accepted connection from pid 2012, user nix-grpc-daemon (trusted) worker2 # [ 316.938294] nix-daemon[856]: reaped child process 2015, status = succeeded worker2 # [ 317.038710] nix-daemon[856]: accepted connection from pid 2017, user nix-grpc-daemon (trusted) worker2 # [ 317.057453] nix-daemon[856]: reaped child process 2020, status = succeeded lb # [ 317.054283] niks3-server[1077]: 2026/09/22 15:50:29 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 317.065252] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.664Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 317.066792] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.664Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)" lb # [ 317.085352] niks3-server[1077]: 2026/09/22 15:50:29 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst worker2 # [ 317.095815] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.695Z level=INFO msg="Uploading 1 narinfos" lb # [ 317.098416] niks3-server[1077]: 2026/09/22 15:50:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign lb # [ 317.108667] niks3-server[1077]: 2026/09/22 15:50:29 INFO Signed narinfos id=82 count=1 lb # [ 317.114163] niks3-server[1077]: 2026/09/22 15:50:29 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls lb # [ 317.123554] niks3-server[1077]: 2026/09/22 15:50:29 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete worker2 # [ 317.128631] nix-grpc-daemon[1754]: time=2026-09-22T15:50:29.727Z level=INFO msg="Upload complete. (346ms)" lb # [ 317.129312] niks3-server[1077]: 2026/09/22 15:50:29 INFO Completed upload id=82 lb # [ 317.131691] niks3-server[1077]: 2026/09/22 15:50:29 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo worker2 # [ 317.136233] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 paths=1 nar_bytes_in=632 worker2 # [ 317.150019] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 317.155627] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) worker2 # [ 317.160669] nix-daemon[856]: reaped child process 2010, status = succeeded client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051... worker2 # [ 317.172616] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds lb # [ 318.573042] envoy[894]: conn peer=[2001:db8:1::1]:51474 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=580 ms=30076 client # farm-b> LOG-farm-b-drain worker2 # [ 318.882565] nix-daemon[856]: reaped child process 2024, status = succeeded worker2 # [ 318.956300] nix-daemon[856]: accepted connection from pid 2040, user nix-grpc-daemon (trusted) worker2 # [ 318.975886] nix-daemon[856]: reaped child process 2043, status = succeeded lb # [ 318.979139] niks3-server[1077]: 2026/09/22 15:50:31 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 318.992555] nix-grpc-daemon[1754]: time=2026-09-22T15:50:31.590Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 318.994775] nix-grpc-daemon[1754]: time=2026-09-22T15:50:31.590Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)" lb # [ 319.021891] niks3-server[1077]: 2026/09/22 15:50:31 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv lb # [ 319.035429] niks3-server[1077]: 2026/09/22 15:50:31 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst worker2 # [ 319.049955] nix-grpc-daemon[1754]: time=2026-09-22T15:50:31.649Z level=INFO msg="Uploading 1 narinfos" lb # [ 319.050653] niks3-server[1077]: 2026/09/22 15:50:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign lb # [ 319.059282] niks3-server[1077]: 2026/09/22 15:50:31 INFO Signed narinfos id=83 count=1 lb # [ 319.066928] niks3-server[1077]: 2026/09/22 15:50:31 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls worker2 # [ 319.075280] nix-grpc-daemon[1754]: time=2026-09-22T15:50:31.674Z level=INFO msg="Upload complete. (291ms)" lb # [ 319.076809] niks3-server[1077]: 2026/09/22 15:50:31 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete lb # [ 319.084475] niks3-server[1077]: 2026/09/22 15:50:31 INFO Completed upload id=83 lb # [ 319.087006] niks3-server[1077]: 2026/09/22 15:50:31 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo worker2 # [ 319.089138] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=11 outputs=1 worker2 # [ 319.112353] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:31Z level=debug event=want client=3 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=0 worker2 # [ 319.118282] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:31Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 319.123272] nix-daemon[856]: reaped child process 2022, status = succeeded worker2 # [ 319.153378] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 319.168743] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 319.182507] nix-grpc-daemon[1754]: time=2026-09-22T15:50:31.781Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'... worker2 # [ 319.193363] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 319.208834] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) worker2 # [ 319.362343] nix-daemon[856]: accepted connection from pid 2049, user nix-grpc-daemon (trusted) worker2 # [ 319.381741] nix-daemon[856]: reaped child process 2052, status = succeeded worker2 # [ 319.486983] nix-daemon[856]: accepted connection from pid 2054, user nix-grpc-daemon (trusted) worker2 # [ 319.503728] nix-daemon[856]: reaped child process 2057, status = succeeded lb # [ 319.508952] niks3-server[1077]: 2026/09/22 15:50:32 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 319.518370] nix-grpc-daemon[1754]: time=2026-09-22T15:50:32.117Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 319.519920] nix-grpc-daemon[1754]: time=2026-09-22T15:50:32.117Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)" lb # [ 319.535057] niks3-server[1077]: 2026/09/22 15:50:32 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst worker2 # [ 319.551704] nix-grpc-daemon[1754]: time=2026-09-22T15:50:32.150Z level=INFO msg="Uploading 1 narinfos" lb # [ 319.551594] niks3-server[1077]: 2026/09/22 15:50:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign lb # [ 319.560956] niks3-server[1077]: 2026/09/22 15:50:32 INFO Signed narinfos id=84 count=1 lb # [ 319.563657] niks3-server[1077]: 2026/09/22 15:50:32 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls worker2 # [ 319.580232] nix-grpc-daemon[1754]: time=2026-09-22T15:50:32.179Z level=INFO msg="Upload complete. (356ms)" lb # [ 319.580784] niks3-server[1077]: 2026/09/22 15:50:32 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete lb # [ 319.588633] niks3-server[1077]: 2026/09/22 15:50:32 INFO Completed upload id=84 lb # [ 319.592539] niks3-server[1077]: 2026/09/22 15:50:32 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo worker2 # [ 319.588243] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=0 paths=1 nar_bytes_in=760 worker2 # [ 319.601453] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 worker2 # [ 319.608249] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) worker2 # [ 319.613022] nix-daemon[856]: reaped child process 2047, status = succeeded client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051... worker2 # [ 319.626646] nix-daemon[856]: accepted connection from pid 1682, user nix-grpc-daemon (trusted) client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 320.885647] nix-daemon[856]: reaped child process 2061, status = succeeded worker2 # [ 320.983324] nix-daemon[856]: accepted connection from pid 2077, user nix-grpc-daemon (trusted) worker2 # [ 321.002191] nix-daemon[856]: reaped child process 2080, status = succeeded lb # [ 321.009549] niks3-server[1077]: 2026/09/22 15:50:33 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 321.020418] nix-grpc-daemon[1754]: time=2026-09-22T15:50:33.618Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 321.022090] nix-grpc-daemon[1754]: time=2026-09-22T15:50:33.618Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)" lb # [ 321.051107] niks3-server[1077]: 2026/09/22 15:50:33 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst lb # [ 321.065148] niks3-server[1077]: 2026/09/22 15:50:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign worker2 # [ 321.071492] nix-grpc-daemon[1754]: time=2026-09-22T15:50:33.670Z level=INFO msg="Uploading 1 narinfos" lb # [ 321.078053] niks3-server[1077]: 2026/09/22 15:50:33 INFO Signed narinfos id=85 count=1 lb # [ 321.083818] niks3-server[1077]: 2026/09/22 15:50:33 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls lb # [ 321.095099] niks3-server[1077]: 2026/09/22 15:50:33 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv lb # [ 321.106784] niks3-server[1077]: 2026/09/22 15:50:33 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete worker2 # [ 321.114732] nix-grpc-daemon[1754]: time=2026-09-22T15:50:33.714Z level=INFO msg="Upload complete. (278ms)" lb # [ 321.114852] niks3-server[1077]: 2026/09/22 15:50:33 INFO Completed upload id=85 lb # [ 321.120074] niks3-server[1077]: 2026/09/22 15:50:33 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo worker2 # [ 321.129227] nix-grpc-daemon[1682]: ts=2026-09-22T15:50:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37710 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=12 outputs=1 worker2 # [ 321.149770] nix-daemon[856]: reaped child process 2059, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 6.23 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 321.873309] nix-grpc-daemon[1815]: ts=2026-09-22T15:50:34Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 8.99 seconds) (finished: run the VM test script, in 323.63 seconds) test script finished in 323.90s cleanup kill QemuMachine (pid 49) client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) client # [2026-09-22T15:50:35Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-22T15:50:35Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-22T15:50:35Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 50) lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) lb # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker1 # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker2 # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-22T15:50:36Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.46 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh