tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm worker2 # Disk image does not exist, creating the virtualisation disk image... worker2 # Formatting '/build/vm-state-worker2/tmp.A64hm4GJOD', fmt=raw size=1073741824 worker2 # mke2fs 1.47.4 (6-Mar-2025) worker2 # Discarding device blocks: 0/262144 done worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes lb # Disk image does not exist, creating the virtualisation disk image... worker2 # Filesystem UUID: f9c82e2c-8814-4592-9a51-fe3bb167e285 lb # Formatting '/build/vm-state-lb/tmp.21k70Ixftg', fmt=raw size=1073741824 worker2 # Superblock backups stored on blocks: worker2: QEMU running (pid 52) lb # mke2fs 1.47.4 (6-Mar-2025) lb # Discarding device blocks: 0/262144 done lb: QEMU running (pid 48) lb # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # 32768, 98304, 163840, 229376 lb # Filesystem UUID: e419f460-32a8-4ac9-96c7-3ad177ebeeda client # Disk image does not exist, creating the virtualisation disk image... client # Formatting '/build/vm-state-client/tmp.mvIEBejypW', fmt=raw size=1073741824 worker1 # Disk image does not exist, creating the virtualisation disk image... worker1: QEMU running (pid 51) client # mke2fs 1.47.4 (6-Mar-2025) client: QEMU running (pid 46) worker1 # Formatting '/build/vm-state-worker1/tmp.J7aeQ8aePv', fmt=raw size=1073741824 worker2 # worker2 # Allocating group tables: 0/8 done lb # Superblock backups stored on blocks: worker1 # mke2fs 1.47.4 (6-Mar-2025) worker2 # Writing inode tables: 0/8 done worker2 # Creating journal (8192 blocks): done client # Discarding device blocks: 0/262144 done worker2 # Writing superblocks and filesystem accounting information: 0/8 done lb # 32768, 98304, 163840, 229376 client # Creating filesystem with 262144 4k blocks and 65536 inodes lb # worker1 # Discarding device blocks: 0/262144 done (finished: start all VMs, in 0.64 seconds) worker2 # client # Filesystem UUID: 377aa17b-16db-4838-8736-3487cf584fcc lb # Allocating group tables: 0/8 done client: waiting for unit network-addresses-eth1.service client: waiting for the VM to finish booting worker2 # Virtualisation disk image created. worker2 # Starting virtiofs daemons... worker2 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests lb # Writing inode tables: 0/8 done lb # Creating journal (8192 blocks): done lb # Writing superblocks and filesystem accounting information: 0/8 done lb # lb # Virtualisation disk image created. lb # Starting virtiofs daemons... lb # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests client # Superblock backups stored on blocks: client # 32768, 98304, 163840, 229376 worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # Filesystem UUID: a7ecca0c-350f-44fc-aa57-2fc72629c496 worker1 # Superblock backups stored on blocks: worker1 # 32768, 98304, 163840, 229376 worker1 # worker1 # Allocating group tables: 0/8 done worker1 # Writing inode tables: 0/8 done worker1 # Creating journal (8192 blocks): done worker1 # Writing superblocks and filesystem accounting information: 0/8 done worker1 # worker1 # Virtualisation disk image created. worker1 # Starting virtiofs daemons... worker1 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests client # client # Allocating group tables: 0/8 done client # Writing inode tables: 0/8 done client # Creating journal (8192 blocks): done client # Writing superblocks and filesystem accounting information: 0/8 done client # client # Virtualisation disk image created. client # Starting virtiofs daemons... client # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T16:12:25Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T16:12:25Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-23T16:12:25Z INFO virtiofsd] Client connected, servicing requests worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker2 # [ 0.000000] KASLR enabled worker2 # [ 0.000000] random: crng init done worker2 # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker2 # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] KASLR enabled client # [ 0.000000] random: crng init done worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] Zone ranges: client # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] DMA32 empty worker2 # [ 0.000000] Normal empty worker2 # [ 0.000000] Device empty client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker2 # [ 0.000000] Movable zone start for each node client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] Early memory node ranges client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] Zone ranges: worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] DMA32 empty worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker2 # [ 0.000000] psci: probing for conduit method from DT. worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] Normal empty worker1 # [ 0.000000] KASLR enabled lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] Device empty client # [ 0.000000] Movable zone start for each node client # [ 0.000000] Early memory node ranges worker2 # [ 0.000000] psci: Trusted OS migration not required client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] random: crng init done lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker2 # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) lb # [ 0.000000] KASLR enabled worker1 # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] efi: UEFI not found. client # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT lb # [ 0.000000] random: crng init done client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 lb # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] Detected PIPT I-cache on CPU0 worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] Zone ranges: lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] DMA32 empty worker2 # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker1 # [ 0.000000] Normal empty worker2 # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker1 # [ 0.000000] Device empty client # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.000000] Zone ranges: worker1 # [ 0.000000] Movable zone start for each node worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] Early memory node ranges worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 lb # [ 0.000000] DMA32 empty client # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Normal empty worker2 # [ 0.000000] alternatives: applying boot alternatives lb # [ 0.000000] Device empty client # [ 0.000000] CPU features: detected: Spectre-v4 worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Movable zone start for each node client # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] Early memory node ranges worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker1 # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] alternatives: applying boot alternatives lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6fvhs2ngm0npgxjv84bzdcnf0823nkly-nixos-system-worker2-test/init regInfo=/nix/store/i80rg3z8f160nwf6i6wkg9y7ng75s232-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker1 # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.000000] psci: probing for conduit method from DT. worker1 # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/i80rg3z8f160nwf6i6wkg9y7ng75s232-closure-info/registration", will be passed to user space. lb # [ 0.000000] psci: Trusted OS migration not required worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes lb # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zbvxb4cbwfrqzrgr4i5486i3lak215fd-nixos-system-client-test/init regInfo=/nix/store/bhkqppcdz1hbk04v8bdn54wi7d6k8amk-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/bhkqppcdz1hbk04v8bdn54wi7d6k8amk-closure-info/registration", will be passed to user space. lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 lb # [ 0.000000] Detected PIPT I-cache on CPU0 worker1 # [ 0.000000] CPU features: detected: Spectre-v4 worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker1 # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 lb # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB client # [ 0.000000] software IO TLB: area num 1. worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker1 # [ 0.000000] alternatives: applying boot alternatives worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] Policy zone: DMA worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.000000] CPU features: detected: Spectre-v4 lb # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 lb # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] allocated 3145728 bytes of page_ext client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/70lv2j0cfdc8kw2f0kxgczfdih2jlj3k-nixos-system-worker1-test/init regInfo=/nix/store/ynqd41fnq28nbq3cfqld13z1wi4vm2cy-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.000000] Policy zone: DMA client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/ynqd41fnq28nbq3cfqld13z1wi4vm2cy-closure-info/registration", will be passed to user space. client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5hmvppg1fs8xkiccvl0m50n4dxjz92fr-nixos-system-lb-test/init regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.000000] allocated 3145728 bytes of page_ext worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker2 # [ 0.000000] rcu: Hierarchical RCU implementation. worker2 # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.000000] Rude variant of Tasks RCU enabled. worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker1 # [ 0.000000] software IO TLB: area num 1. lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration", will be passed to user space. lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages client # [ 0.000000] ftrace: allocated 294 pages with 4 groups client # [ 0.000000] rcu: Hierarchical RCU implementation. client # [ 0.000000] rcu: RCU event tracing is enabled. client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.000000] Tracing variant of Tasks RCU enabled. worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker2 # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] software IO TLB: area num 1. lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker1 # [ 0.000000] Fallback order for Node 0: 0 worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] Policy zone: DMA worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) lb # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 lb # [ 0.000000] Policy zone: DMA worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.000000] rcu: Hierarchical RCU implementation. client # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages client # [ 0.000000] GICv3: 0 Extended SPIs implemented client # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI lb # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker1 # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled. client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.000000] Trampoline variant of Tasks RCU enabled. lb # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.000000] Tracing variant of Tasks RCU enabled. worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker2 # [ 0.000043] arm-pv: using stolen time PV client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 lb # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] Root IRQ handler: gic_handle_irq client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker2 # [ 0.000616] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.000842] Console: colour dummy device 80x25 client # [ 0.000035] arm-pv: using stolen time PV lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker2 # [ 0.000852] printk: legacy console [tty0] enabled lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) client # [ 0.000446] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker2 # [ 0.001053] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker2 # [ 0.001061] pid_max: default: 32768 minimum: 301 worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker2 # [ 0.001144] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.001347] landlock: Up and running. worker2 # [ 0.001350] Yama: becoming mindful. lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker2 # [ 0.001940] LSM support for eBPF active client # [ 0.000657] Console: colour dummy device 80x25 client # [ 0.000665] printk: legacy console [tty0] enabled client # [ 0.000864] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) client # [ 0.000871] pid_max: default: 32768 minimum: 301 lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker2 # [ 0.002166] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 client # [ 0.000951] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). client # [ 0.001082] landlock: Up and running. lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.002203] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.004346] rcu: Hierarchical SRCU implementation. client # [ 0.001086] Yama: becoming mindful. client # [ 0.001597] LSM support for eBPF active worker2 # [ 0.004355] rcu: Max phase no-delay instances is 1000. worker2 # [ 0.005759] fsl-mc MSI: its@8080000 domain created lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns client # [ 0.001776] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.005864] EFI services will not be available. worker2 # [ 0.005973] smp: Bringing up secondary CPUs ... client # [ 0.001809] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.003952] rcu: Hierarchical SRCU implementation. worker2 # [ 0.005984] smp: Brought up 1 node, 1 CPU worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.000046] arm-pv: using stolen time PV worker1 # [ 0.000503] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.000761] Console: colour dummy device 80x25 lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns client # [ 0.003960] rcu: Max phase no-delay instances is 1000. client # [ 0.005418] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.005987] SMP: Total of 1 processors activated. worker2 # [ 0.005994] CPU: All CPU(s) started at EL1 worker2 # [ 0.006007] CPU features: detected: Branch Target Identification worker1 # [ 0.000770] printk: legacy console [tty0] enabled worker1 # [ 0.000980] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.000987] pid_max: default: 32768 minimum: 301 lb # [ 0.000039] arm-pv: using stolen time PV lb # [ 0.000498] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.000780] Console: colour dummy device 80x25 lb # [ 0.000789] printk: legacy console [tty0] enabled worker1 # [ 0.001071] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker1 # [ 0.001253] landlock: Up and running. worker1 # [ 0.001256] Yama: becoming mindful. lb # [ 0.001002] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.001831] LSM support for eBPF active lb # [ 0.001010] pid_max: default: 32768 minimum: 301 worker2 # [ 0.006014] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.006017] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.006021] CPU features: detected: Data cache clean to the PoU not required for I/D coherence client # [ 0.005520] EFI services will not be available. client # [ 0.005616] smp: Bringing up secondary CPUs ... client # [ 0.005627] smp: Brought up 1 node, 1 CPU client # [ 0.005631] SMP: Total of 1 processors activated. client # [ 0.005637] CPU: All CPU(s) started at EL1 client # [ 0.005650] CPU features: detected: Branch Target Identification client # [ 0.005655] CPU features: detected: ARMv8.4 Translation Table Level client # [ 0.005659] CPU features: detected: Instruction cache invalidation not required for I/D coherence lb # [ 0.001097] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.001267] landlock: Up and running. worker1 # [ 0.001993] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.006025] CPU features: detected: Common not Private translations worker2 # [ 0.006029] CPU features: detected: CRC32 instructions lb # [ 0.001270] Yama: becoming mindful. worker1 # [ 0.002027] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.006032] CPU features: detected: Data cache clean to Point of Deep Persistence client # [ 0.005663] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.001791] LSM support for eBPF active worker2 # [ 0.006036] CPU features: detected: Data cache clean to Point of Persistence worker1 # [ 0.004168] rcu: Hierarchical SRCU implementation. client # [ 0.005667] CPU features: detected: Common not Private translations worker2 # [ 0.006039] CPU features: detected: Data independent timing control (DIT) worker1 # [ 0.004175] rcu: Max phase no-delay instances is 1000. lb # [ 0.001959] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.006042] CPU features: detected: E0PD client # [ 0.005670] CPU features: detected: CRC32 instructions worker2 # [ 0.006045] CPU features: detected: Enhanced Counter Virtualization lb # [ 0.001994] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.005673] CPU features: detected: Data cache clean to Point of Deep Persistence worker1 # [ 0.005541] fsl-mc MSI: its@8080000 domain created lb # [ 0.004102] rcu: Hierarchical SRCU implementation. worker2 # [ 0.006049] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker1 # [ 0.005644] EFI services will not be available. client # [ 0.005677] CPU features: detected: Data cache clean to Point of Persistence lb # [ 0.004113] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.005741] smp: Bringing up secondary CPUs ... worker2 # [ 0.006052] CPU features: detected: Enhanced Virtualization Traps lb # [ 0.005583] fsl-mc MSI: its@8080000 domain created client # [ 0.005680] CPU features: detected: Data independent timing control (DIT) worker1 # [ 0.005753] smp: Brought up 1 node, 1 CPU worker2 # [ 0.006056] CPU features: detected: Fine Grained Traps lb # [ 0.005691] EFI services will not be available. client # [ 0.005683] CPU features: detected: E0PD worker1 # [ 0.005757] SMP: Total of 1 processors activated. lb # [ 0.005845] smp: Bringing up secondary CPUs ... worker1 # [ 0.005762] CPU: All CPU(s) started at EL1 worker2 # [ 0.006060] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.005686] CPU features: detected: Enhanced Counter Virtualization lb # [ 0.005857] smp: Brought up 1 node, 1 CPU worker1 # [ 0.005777] CPU features: detected: Branch Target Identification worker2 # [ 0.006065] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.005689] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker2 # [ 0.006069] CPU features: detected: LSE atomic instructions client # [ 0.005693] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.005782] CPU features: detected: ARMv8.4 Translation Table Level lb # [ 0.005860] SMP: Total of 1 processors activated. lb # [ 0.005868] CPU: All CPU(s) started at EL1 worker2 # [ 0.006072] CPU features: detected: Privileged Access Never lb # [ 0.005882] CPU features: detected: Branch Target Identification client # [ 0.005696] CPU features: detected: Fine Grained Traps worker1 # [ 0.005785] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.006075] CPU features: detected: PMUv3 lb # [ 0.005888] CPU features: detected: ARMv8.4 Translation Table Level client # [ 0.005701] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.006078] CPU features: detected: RAS Extension Support worker1 # [ 0.005789] CPU features: detected: Data cache clean to the PoU not required for I/D coherence client # [ 0.005706] CPU features: detected: RCpc load-acquire (LDAPR) lb # [ 0.005892] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.006081] CPU features: detected: RASv1p1 Extension Support worker1 # [ 0.005793] CPU features: detected: Common not Private translations client # [ 0.005709] CPU features: detected: LSE atomic instructions worker2 # [ 0.006084] CPU features: detected: Random Number Generator worker1 # [ 0.005796] CPU features: detected: CRC32 instructions lb # [ 0.005896] CPU features: detected: Data cache clean to the PoU not required for I/D coherence client # [ 0.005712] CPU features: detected: Privileged Access Never worker2 # [ 0.006087] CPU features: detected: Speculation barrier (SB) lb # [ 0.005899] CPU features: detected: Common not Private translations client # [ 0.005715] CPU features: detected: PMUv3 worker1 # [ 0.005799] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.006089] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.005903] CPU features: detected: CRC32 instructions client # [ 0.005721] CPU features: detected: RAS Extension Support worker1 # [ 0.005803] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.006092] CPU features: detected: TLB range maintenance instructions client # [ 0.005724] CPU features: detected: RASv1p1 Extension Support lb # [ 0.005906] CPU features: detected: Data cache clean to Point of Deep Persistence worker1 # [ 0.005807] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.006097] CPU features: detected: Speculative Store Bypassing Safe (SSBS) client # [ 0.005727] CPU features: detected: Random Number Generator worker1 # [ 0.005810] CPU features: detected: E0PD lb # [ 0.005909] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.006169] alternatives: applying system-wide alternatives client # [ 0.005730] CPU features: detected: Speculation barrier (SB) worker1 # [ 0.005813] CPU features: detected: Enhanced Counter Virtualization lb # [ 0.005913] CPU features: detected: Data independent timing control (DIT) client # [ 0.005732] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.009892] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.005916] CPU features: detected: E0PD worker1 # [ 0.005816] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.005735] CPU features: detected: TLB range maintenance instructions lb # [ 0.005919] CPU features: detected: Enhanced Counter Virtualization worker1 # [ 0.005820] CPU features: detected: Enhanced Virtualization Traps client # [ 0.005741] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker2 # [ 0.010145] Memory: 1405692K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved) worker1 # [ 0.005823] CPU features: detected: Fine Grained Traps lb # [ 0.005922] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker2 # [ 0.010633] devtmpfs: initialized client # [ 0.005784] alternatives: applying system-wide alternatives lb # [ 0.005926] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.005827] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.012728] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.009520] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.005929] CPU features: detected: Fine Grained Traps worker1 # [ 0.005833] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.012757] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.005836] CPU features: detected: LSE atomic instructions lb # [ 0.005933] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.012989] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.009773] Memory: 1405644K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved) worker1 # [ 0.005839] CPU features: detected: Privileged Access Never lb # [ 0.005938] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.010286] devtmpfs: initialized worker2 # [ 0.012995] 0 pages in range for non-PLT usage worker1 # [ 0.005842] CPU features: detected: PMUv3 lb # [ 0.005940] CPU features: detected: LSE atomic instructions worker2 # [ 0.012997] 508272 pages in range for PLT usage worker1 # [ 0.005845] CPU features: detected: RAS Extension Support client # [ 0.012308] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.005943] CPU features: detected: Privileged Access Never worker2 # [ 0.013156] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.014193] DMI not present or invalid. lb # [ 0.005946] CPU features: detected: PMUv3 lb # [ 0.005949] CPU features: detected: RAS Extension Support lb # [ 0.005953] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.017607] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.020298] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.012334] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.005848] CPU features: detected: RASv1p1 Extension Support client # [ 0.012572] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker1 # [ 0.005851] CPU features: detected: Random Number Generator client # [ 0.012577] 0 pages in range for non-PLT usage lb # [ 0.005956] CPU features: detected: Random Number Generator worker1 # [ 0.005853] CPU features: detected: Speculation barrier (SB) client # [ 0.012579] 508272 pages in range for PLT usage worker1 # [ 0.005856] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.020528] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.012723] pinctrl core: initialized pinctrl subsystem lb # [ 0.005958] CPU features: detected: Speculation barrier (SB) client # [ 0.013635] DMI not present or invalid. client # [ 0.017123] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.005859] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.005865] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.005911] alternatives: applying system-wide alternatives worker2 # [ 0.020770] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.020813] audit: initializing netlink subsys (disabled) lb # [ 0.005961] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.005965] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.009834] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.021498] thermal_sys: Registered thermal governor 'fair_share' client # [ 0.020097] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations lb # [ 0.005970] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker2 # [ 0.021501] thermal_sys: Registered thermal governor 'bang_bang' client # [ 0.020315] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.006016] alternatives: applying system-wide alternatives worker2 # [ 0.021506] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.009785] CPU features: detected: BBM Level 2 without TLB conflict abort client # [ 0.020540] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker1 # [ 0.010117] Memory: 1405600K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved) worker1 # [ 0.010647] devtmpfs: initialized worker2 # [ 0.021509] thermal_sys: Registered thermal governor 'user_space' client # [ 0.020574] audit: initializing netlink subsys (disabled) worker1 # [ 0.012657] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.021244] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.021515] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.021247] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.021541] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.012685] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker2 # [ 0.021554] cpuidle: using governor ladder client # [ 0.021251] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.012916] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker2 # [ 0.021560] cpuidle: using governor menu client # [ 0.021254] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.012920] 0 pages in range for non-PLT usage worker2 # [ 0.021776] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.012922] 508272 pages in range for PLT usage client # [ 0.021260] thermal_sys: Registered thermal governor 'power_allocator' worker2 # [ 0.021798] ASID allocator initialised with 65536 entries worker1 # [ 0.013061] pinctrl core: initialized pinctrl subsystem lb # [ 0.010017] Memory: 1405636K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved) worker2 # [ 0.023162] Serial: AMBA PL011 UART driver worker1 # [ 0.013962] DMI not present or invalid. lb # [ 0.010515] devtmpfs: initialized client # [ 0.021289] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.012444] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.012471] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.012767] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.021301] cpuidle: using governor ladder worker1 # [ 0.018178] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.012772] 0 pages in range for non-PLT usage worker2 # [ 0.028633] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 client # [ 0.021306] cpuidle: using governor menu lb # [ 0.012773] 508272 pages in range for PLT usage worker2 # [ 0.028810] printk: console [ttyAMA0] enabled worker1 # [ 0.020827] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.021515] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.012915] pinctrl core: initialized pinctrl subsystem client # [ 0.021533] ASID allocator initialised with 65536 entries lb # [ 0.013742] DMI not present or invalid. client # [ 0.022871] Serial: AMBA PL011 UART driver worker1 # [ 0.021063] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.017361] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.021294] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.028448] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.020145] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.021334] audit: initializing netlink subsys (disabled) client # [ 0.028678] printk: console [ttyAMA0] enabled lb # [ 0.020382] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker1 # [ 0.021889] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.022082] thermal_sys: Registered thermal governor 'fair_share' lb # [ 0.020644] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.020687] audit: initializing netlink subsys (disabled) worker1 # [ 0.022085] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.022089] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.021388] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.022095] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.021390] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.022097] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.021394] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.154875] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.022125] cpuidle: using governor ladder lb # [ 0.021397] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.022133] cpuidle: using governor menu worker2 # [ 0.154902] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.021403] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.022349] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.154908] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.022368] ASID allocator initialised with 65536 entries lb # [ 0.021435] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.154913] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker1 # [ 0.023682] Serial: AMBA PL011 UART driver lb # [ 0.021444] cpuidle: using governor ladder worker2 # [ 0.154918] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.021450] cpuidle: using governor menu worker1 # [ 0.029317] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker2 # [ 0.154922] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.021666] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.029528] printk: console [ttyAMA0] enabled client # [ 0.156651] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker2 # [ 0.154927] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.021683] ASID allocator initialised with 65536 entries lb # [ 0.023083] Serial: AMBA PL011 UART driver worker2 # [ 0.154932] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.028836] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.029008] printk: console [ttyAMA0] enabled client # [ 0.156680] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.156686] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.156690] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.156695] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.163412] fbcon: Taking over console client # [ 0.156700] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.163435] ACPI: Interpreter disabled. client # [ 0.156705] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages client # [ 0.156709] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page client # [ 0.164711] fbcon: Taking over console client # [ 0.164733] ACPI: Interpreter disabled. worker1 # [ 0.158384] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages client # [ 0.166716] iommu: Default domain type: Translated worker1 # [ 0.158412] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.170761] iommu: Default domain type: Translated lb # [ 0.156825] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages client # [ 0.166730] iommu: DMA domain TLB invalidation policy: strict mode worker1 # [ 0.158418] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker2 # [ 0.170777] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.156853] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.172788] SCSI subsystem initialized worker1 # [ 0.158423] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.156859] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.158428] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.156863] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker1 # [ 0.158432] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.173550] usbcore: registered new interface driver usbfs lb # [ 0.156868] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker1 # [ 0.158436] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker2 # [ 0.173590] usbcore: registered new interface driver hub lb # [ 0.156872] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.173614] usbcore: registered new device driver usb worker1 # [ 0.158440] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.156876] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages client # [ 0.172695] SCSI subsystem initialized worker2 # [ 0.173933] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.156881] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker2 # [ 0.173941] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti client # [ 0.173476] usbcore: registered new interface driver usbfs worker2 # [ 0.173951] PTP clock support registered worker2 # [ 0.174006] EDAC MC: Ver: 3.0.0 client # [ 0.173513] usbcore: registered new interface driver hub worker1 # [ 0.166676] fbcon: Taking over console client # [ 0.173530] usbcore: registered new device driver usb worker1 # [ 0.166697] ACPI: Interpreter disabled. worker2 # [ 0.179043] scmi_core: SCMI protocol bus registered client # [ 0.173854] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.165226] fbcon: Taking over console lb # [ 0.165249] ACPI: Interpreter disabled. worker2 # [ 0.180140] FPGA manager framework client # [ 0.173862] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti client # [ 0.173873] PTP clock support registered worker2 # [ 0.181272] vgaarb: loaded client # [ 0.173938] EDAC MC: Ver: 3.0.0 worker2 # [ 0.181983] clocksource: Switched to clocksource arch_sys_counter client # [ 0.178851] scmi_core: SCMI protocol bus registered client # [ 0.179897] FPGA manager framework client # [ 0.181011] vgaarb: loaded worker1 # [ 0.174108] iommu: Default domain type: Translated worker1 # [ 0.174122] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.172538] iommu: Default domain type: Translated worker1 # [ 0.176039] SCSI subsystem initialized lb # [ 0.172554] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.173043] SCSI subsystem initialized worker2 # [ 0.188260] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.176825] usbcore: registered new interface driver usbfs worker1 # [ 0.176864] usbcore: registered new interface driver hub worker2 # [ 0.188317] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) client # [ 0.185225] clocksource: Switched to clocksource arch_sys_counter lb # [ 0.175298] usbcore: registered new interface driver usbfs worker1 # [ 0.176880] usbcore: registered new device driver usb client # [ 0.185969] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.177230] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.175336] usbcore: registered new interface driver hub lb # [ 0.175352] usbcore: registered new device driver usb worker1 # [ 0.177237] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti client # [ 0.186013] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.175778] pps_core: LinuxPPS API ver. 1 registered client # [ 0.188541] netfs: FS-Cache loaded client # [ 0.188696] pnp: PnP ACPI: disabled worker1 # [ 0.177248] PTP clock support registered worker1 # [ 0.177299] EDAC MC: Ver: 3.0.0 worker2 # [ 0.192682] netfs: FS-Cache loaded lb # [ 0.175791] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.192870] pnp: PnP ACPI: disabled worker1 # [ 0.182442] scmi_core: SCMI protocol bus registered lb # [ 0.175803] PTP clock support registered lb # [ 0.175864] EDAC MC: Ver: 3.0.0 worker1 # [ 0.183591] FPGA manager framework lb # [ 0.181449] scmi_core: SCMI protocol bus registered worker1 # [ 0.184752] vgaarb: loaded worker1 # [ 0.185474] clocksource: Switched to clocksource arch_sys_counter lb # [ 0.182660] FPGA manager framework lb # [ 0.183868] vgaarb: loaded worker2 # [ 0.197308] NET: Registered PF_INET protocol family lb # [ 0.184668] clocksource: Switched to clocksource arch_sys_counter worker2 # [ 0.197639] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.195555] NET: Registered PF_INET protocol family client # [ 0.195912] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.192129] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.192216] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.191225] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.194215] netfs: FS-Cache loaded worker1 # [ 0.194375] pnp: PnP ACPI: disabled lb # [ 0.191296] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.195667] netfs: FS-Cache loaded lb # [ 0.195852] pnp: PnP ACPI: disabled worker1 # [ 0.198721] NET: Registered PF_INET protocol family worker1 # [ 0.199040] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.200699] NET: Registered PF_INET protocol family lb # [ 0.201066] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.231611] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.231687] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.231728] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.231838] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.231985] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.232106] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.232153] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.232843] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.232194] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.232918] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.232313] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.232959] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.232344] NET: Registered PF_XDP protocol family client # [ 0.232367] PCI: CLS 0 bytes, default 64 worker2 # [ 0.233060] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.233206] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.233325] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.233383] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.233425] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.233534] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.233558] NET: Registered PF_XDP protocol family worker2 # [ 0.233579] PCI: CLS 0 bytes, default 64 worker2 # [ 0.233882] Trying to unpack rootfs image as initramfs... client # [ 0.242800] Trying to unpack rootfs image as initramfs... worker1 # [ 0.234690] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.234764] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.234802] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.234913] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.235059] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.235223] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.252667] kvm [1]: HYP mode not available worker1 # [ 0.235276] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.235318] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.235435] NET: Registered PF_UNIX/PF_LOCAL protocol family worker1 # [ 0.235501] NET: Registered PF_XDP protocol family client # [ 0.250157] kvm [1]: HYP mode not available worker1 # [ 0.235522] PCI: CLS 0 bytes, default 64 worker1 # [ 0.235813] Trying to unpack rootfs image as initramfs... lb # [ 0.239829] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.239902] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.251414] kvm [1]: HYP mode not available lb # [ 0.239943] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.240053] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.240202] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.240326] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.240377] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.240418] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.240550] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.240625] NET: Registered PF_XDP protocol family lb # [ 0.240649] PCI: CLS 0 bytes, default 64 lb # [ 0.240785] Trying to unpack rootfs image as initramfs... lb # [ 0.259222] kvm [1]: HYP mode not available worker2 # [ 0.463428] Initialise system trusted keyrings worker2 # [ 0.464318] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker2 # [ 0.465758] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker2 # [ 0.466846] 9p: Installing v9fs 9p2000 file system support client # [ 0.469939] Initialise system trusted keyrings client # [ 0.470896] workingset: timestamp_bits=42 max_order=19 bucket_order=0 client # [ 0.472361] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.473320] 9p: Installing v9fs 9p2000 file system support worker1 # [ 0.471777] Initialise system trusted keyrings worker1 # [ 0.472707] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.474366] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker1 # [ 0.475295] 9p: Installing v9fs 9p2000 file system support lb # [ 0.481496] Initialise system trusted keyrings worker2 # [ 0.495099] Key type asymmetric registered worker2 # [ 0.495137] Asymmetric key parser 'x509' registered lb # [ 0.482364] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker2 # [ 0.495235] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) lb # [ 0.483817] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker2 # [ 0.497658] io scheduler mq-deadline registered worker2 # [ 0.497672] io scheduler kyber registered lb # [ 0.484810] 9p: Installing v9fs 9p2000 file system support client # [ 0.501694] Key type asymmetric registered client # [ 0.501729] Asymmetric key parser 'x509' registered worker1 # [ 0.495510] Key type asymmetric registered client # [ 0.501824] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker1 # [ 0.495544] Asymmetric key parser 'x509' registered client # [ 0.504370] io scheduler mq-deadline registered client # [ 0.504386] io scheduler kyber registered worker1 # [ 0.495642] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker2 # [ 0.511632] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker2 # [ 0.513112] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.513562] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.513585] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker1 # [ 0.509594] io scheduler mq-deadline registered worker1 # [ 0.509628] io scheduler kyber registered worker2 # [ 0.513597] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.513606] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.513638] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.513665] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker2 # [ 0.513764] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker2 # [ 0.513774] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.513065] Key type asymmetric registered worker1 # [ 0.515352] pl061_gpio 9030000.pl061: PL061 GPIO chip registered lb # [ 0.513102] Asymmetric key parser 'x509' registered worker2 # [ 0.513781] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker2 # [ 0.513787] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] lb # [ 0.513193] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker1 # [ 0.516748] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.518396] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker2 # [ 0.513792] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker1 # [ 0.517195] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.519184] ledtrig-cpu: registered to indicate activity on CPUs lb # [ 0.515583] io scheduler mq-deadline registered lb # [ 0.515598] io scheduler kyber registered worker2 # [ 0.513856] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.519643] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker1 # [ 0.517214] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker2 # [ 0.514416] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.519665] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker2 # [ 0.514614] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.517227] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.519678] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker1 # [ 0.517235] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.514633] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.514662] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.519687] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.514678] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.517270] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.519712] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.515157] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.517294] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker2 # [ 0.515357] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.517403] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.519736] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker2 # [ 0.515373] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.519821] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker2 # [ 0.515404] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.519832] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.517413] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.517420] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.517425] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.515917] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.519838] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker2 # [ 0.516121] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker1 # [ 0.517430] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.519844] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.516139] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.519849] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.516171] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.529023] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.519934] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.516675] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.516894] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.520442] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.516913] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.520654] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.520675] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.516947] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.516968] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.520707] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.520726] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.517469] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.521259] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.517681] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.521470] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.517716] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.521487] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.521520] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.529575] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.522040] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.530211] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.522242] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.522260] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.530633] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.522302] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.530654] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker1 # [ 0.537662] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.537911] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.522846] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.537934] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.530668] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.523053] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.537968] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.523070] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.530677] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker1 # [ 0.537990] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.523101] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.530702] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.523123] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.538603] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.538818] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.523638] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker1 # [ 0.538835] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.523844] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.530727] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] lb # [ 0.530817] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker1 # [ 0.538866] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.523878] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.530827] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.530834] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.539426] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.524380] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.530839] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker1 # [ 0.539626] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.524586] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.530844] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker1 # [ 0.539643] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.524619] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.554481] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker1 # [ 0.539677] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.530921] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.554725] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.525042] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker1 # [ 0.540181] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.531415] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.554763] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.531619] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.540384] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.555215] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker1 # [ 0.540403] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.531639] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.555415] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker1 # [ 0.540438] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.531671] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.531689] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.540459] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.555711] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.555916] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.532178] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.540981] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint lb # [ 0.532380] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.541187] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.555950] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.532398] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.541222] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.556437] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.532429] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.556651] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.532988] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker2 # [ 0.556681] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.565354] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.533174] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker2 # [ 0.557272] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.533190] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.565781] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.557478] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.533221] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.566015] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.557510] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.566050] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.558070] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.533694] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.560692] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.533896] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.566584] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.533912] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.558409] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] client # [ 0.566844] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.560909] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.533941] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.558430] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.560942] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.566880] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.533960] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.558464] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.561366] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.567400] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.534447] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.558979] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.561578] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] client # [ 0.567611] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.534641] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.559194] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.567646] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.561884] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.534672] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.559213] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.562088] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.568183] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.559247] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.535152] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker1 # [ 0.562119] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.568529] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.535352] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.559890] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned client # [ 0.568552] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.562596] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.535387] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.559903] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.562796] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.568584] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.535841] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.559908] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.562829] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.569104] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.536038] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker2 # [ 0.559959] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.563339] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.536299] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.563551] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.560011] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.536496] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.563585] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.560071] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned lb # [ 0.536528] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.564080] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.560120] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker1 # [ 0.564425] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.560176] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker1 # [ 0.564447] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.560228] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker1 # [ 0.564483] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.560278] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker1 # [ 0.565013] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.565220] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.560332] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker1 # [ 0.565239] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.560382] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker1 # [ 0.565275] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.560464] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.560553] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.589422] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.589470] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.560577] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned client # [ 0.589505] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.560603] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned lb # [ 0.581736] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.590295] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.560626] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned lb # [ 0.581981] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.590310] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned lb # [ 0.582015] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.560652] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.560674] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned client # [ 0.590316] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.582540] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.560698] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.582741] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.590378] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.560729] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned lb # [ 0.582772] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.590435] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker2 # [ 0.560755] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.583267] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint client # [ 0.590490] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker2 # [ 0.560781] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.583572] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.590127] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned client # [ 0.590552] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned lb # [ 0.583592] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.560805] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.590163] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker2 # [ 0.560827] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.583623] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.590607] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.560851] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.584120] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.590664] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker1 # [ 0.590169] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker2 # [ 0.560877] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker1 # [ 0.590231] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.584321] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.590721] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.560902] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.584339] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.590287] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker2 # [ 0.560925] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.584372] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.590344] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker2 # [ 0.560949] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned client # [ 0.590821] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned client # [ 0.590880] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.560974] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned client # [ 0.590995] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.561003] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] client # [ 0.591129] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.561013] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker1 # [ 0.590395] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.561018] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker1 # [ 0.590446] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned client # [ 0.591158] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker1 # [ 0.590496] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker1 # [ 0.590548] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.561930] pci 0000:00:07.0: enabling device (0000 -> 0002) client # [ 0.591182] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned client # [ 0.591207] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned client # [ 0.591234] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.590605] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker1 # [ 0.590657] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned client # [ 0.591259] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.590771] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned client # [ 0.591282] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.590890] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.591316] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned lb # [ 0.601388] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker1 # [ 0.590918] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned client # [ 0.591342] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.590943] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned lb # [ 0.601428] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.591367] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker1 # [ 0.590967] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned lb # [ 0.601435] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned client # [ 0.591395] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.590992] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.601494] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned client # [ 0.591425] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.591016] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned lb # [ 0.601545] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned client # [ 0.591452] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker1 # [ 0.591043] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned client # [ 0.591475] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.601599] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned client # [ 0.591501] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.601663] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned client # [ 0.591526] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.601715] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned client # [ 0.591548] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.591076] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned lb # [ 0.601805] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker1 # [ 0.591099] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned client # [ 0.591573] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.591125] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.601859] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker1 # [ 0.591151] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned client # [ 0.591605] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.591174] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned client # [ 0.591618] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker1 # [ 0.591222] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.601939] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker1 # [ 0.591243] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.626224] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 62789 usecs client # [ 0.591623] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker1 # [ 0.591265] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned client # [ 0.592530] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.601989] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker1 # [ 0.591294] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.602174] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.591315] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.602229] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker1 # [ 0.591339] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.602253] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker1 # [ 0.591368] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.602316] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.591378] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker1 # [ 0.591383] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.602341] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.592258] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.602365] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.602390] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned lb # [ 0.602412] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.602441] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned lb # [ 0.602469] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.602492] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.602515] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned lb # [ 0.602564] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.602585] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.602608] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.602629] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.602654] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.602675] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.602700] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.602732] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.602742] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.602747] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.603782] pci 0000:00:07.0: enabling device (0000 -> 0002) worker2 # [ 0.647993] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.658424] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.649625] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker1 # [ 0.652273] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.654132] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) client # [ 0.656731] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker2 # [ 0.663970] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) client # [ 0.668142] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.670382] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker2 # [ 0.673702] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker1 # [ 0.663291] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.661952] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) client # [ 0.674922] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker2 # [ 0.684190] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) lb # [ 0.672774] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.675808] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) client # [ 0.686085] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) client # [ 0.688804] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker1 # [ 0.674027] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.677072] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker1 # [ 0.679808] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker2 # [ 0.695614] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) lb # [ 0.686543] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.690305] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.700241] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.692815] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker2 # [ 0.702299] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker2 # [ 0.704903] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) client # [ 0.703824] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.703883] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker2 # [ 0.715950] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) lb # [ 0.697010] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) lb # [ 0.701014] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) lb # [ 0.703294] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.714061] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) client # [ 0.716578] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker2 # [ 0.723453] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker1 # [ 0.714289] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) lb # [ 0.713515] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) lb # [ 0.716394] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) client # [ 0.727844] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker1 # [ 0.718657] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.737176] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.726807] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.732629] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.740178] msm_serial: driver initialized worker2 # [ 0.740350] SuperH (H)SCI(F) driver initialized worker2 # [ 0.740406] STM32 USART driver initialized client # [ 0.742918] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker1 # [ 0.735734] msm_serial: driver initialized worker1 # [ 0.735893] SuperH (H)SCI(F) driver initialized worker1 # [ 0.735947] STM32 USART driver initialized client # [ 0.746389] msm_serial: driver initialized lb # [ 0.733499] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.746548] SuperH (H)SCI(F) driver initialized client # [ 0.746604] STM32 USART driver initialized lb # [ 0.747701] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.750848] msm_serial: driver initialized lb # [ 0.750994] SuperH (H)SCI(F) driver initialized lb # [ 0.751053] STM32 USART driver initialized worker1 # [ 0.776664] loop: module loaded client # [ 0.784664] loop: module loaded worker1 # [ 0.776958] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 0.784925] virtio_blk virtio2: 1/0/0 default/read/poll queues worker1 # [ 0.779455] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.790486] loop: module loaded worker2 # [ 0.790746] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 0.791897] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 0.794188] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.798900] megasas: 07.734.00.00-rc1 worker2 # [ 0.799770] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] client # [ 0.798016] megasas: 07.734.00.00-rc1 client # [ 0.798854] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker1 # [ 0.790415] megasas: 07.734.00.00-rc1 worker1 # [ 0.791527] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.795837] loop: module loaded lb # [ 0.796133] virtio_blk virtio2: 1/0/0 default/read/poll queues worker1 # [ 0.794141] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.794188] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.796396] Using buffer write method worker1 # [ 0.796523] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.810067] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.810654] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.810188] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.810710] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.813724] Using buffer write method client # [ 0.812949] Using buffer write method worker2 # [ 0.813853] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] client # [ 0.813071] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.805673] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.815958] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.816000] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.810936] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.811048] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.813015] Using buffer write method worker1 # [ 0.813065] Concatenating MTD devices: worker1 # [ 0.813070] (0): "0.flash" worker1 # [ 0.813075] (1): "0.flash" worker1 # [ 0.813089] into device "0.flash" lb # [ 0.809610] megasas: 07.734.00.00-rc1 lb # [ 0.810456] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.814613] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.814662] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.834036] Using buffer write method worker2 # [ 0.834088] Concatenating MTD devices: worker2 # [ 0.834093] (0): "0.flash" worker2 # [ 0.834123] (1): "0.flash" worker2 # [ 0.834128] into device "0.flash" client # [ 0.831489] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.831561] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.836389] Using buffer write method client # [ 0.836443] Concatenating MTD devices: client # [ 0.836448] (0): "0.flash" client # [ 0.836452] (1): "0.flash" client # [ 0.836468] into device "0.flash" lb # [ 0.832727] Using buffer write method lb # [ 0.832832] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.835519] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.835569] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.837982] Using buffer write method lb # [ 0.838021] Concatenating MTD devices: lb # [ 0.838026] (0): "0.flash" lb # [ 0.838031] (1): "0.flash" lb # [ 0.838047] into device "0.flash" client # [ 1.155680] Freeing initrd memory: 26972K worker2 # [ 1.159127] Freeing initrd memory: 26944K worker1 # [ 1.152023] Freeing initrd memory: 26976K worker2 # [ 1.165914] tun: Universal TUN/TAP device driver, 1.6 client # [ 1.162605] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 1.158919] tun: Universal TUN/TAP device driver, 1.6 client # [ 1.167268] thunder_xcv, ver 1.0 client # [ 1.167337] thunder_bgx, ver 1.0 client # [ 1.167359] nicpf, ver 1.0 client # [ 1.167968] e1000: Intel(R) PRO/1000 Network Driver client # [ 1.167977] e1000: Copyright (c) 1999-2006 Intel Corporation. worker2 # [ 1.170782] thunder_xcv, ver 1.0 client # [ 1.168001] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 1.170836] thunder_bgx, ver 1.0 worker2 # [ 1.170859] nicpf, ver 1.0 worker1 # [ 1.163597] thunder_xcv, ver 1.0 client # [ 1.168012] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 1.171470] e1000: Intel(R) PRO/1000 Network Driver lb # [ 1.160911] Freeing initrd memory: 26948K worker1 # [ 1.163659] thunder_bgx, ver 1.0 worker1 # [ 1.163683] nicpf, ver 1.0 client # [ 1.168042] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 1.171478] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 1.164278] e1000: Intel(R) PRO/1000 Network Driver client # [ 1.168047] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 1.171504] e1000e: Intel(R) PRO/1000 Network Driver worker1 # [ 1.164287] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 1.168074] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 1.171514] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker1 # [ 1.164313] e1000e: Intel(R) PRO/1000 Network Driver client # [ 1.168081] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 1.171545] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 1.164324] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 1.168220] sky2: driver version 1.30 worker2 # [ 1.171551] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 1.164357] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 1.171576] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 1.164363] igb: Copyright (c) 2007-2014 Intel Corporation. lb # [ 1.167902] tun: Universal TUN/TAP device driver, 1.6 worker2 # [ 1.171583] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 1.164414] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 1.171723] sky2: driver version 1.30 client # [ 1.177825] usbcore: registered new interface driver usb-storage worker1 # [ 1.164420] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 1.173398] usbcore: registered new interface driver usb-storage worker1 # [ 1.164565] sky2: driver version 1.30 client # [ 1.177899] usbcore: registered new interface driver usbserial_generic worker2 # [ 1.173548] usbcore: registered new interface driver usbserial_generic client # [ 1.177914] usbserial: USB Serial support registered for generic worker2 # [ 1.173563] usbserial: USB Serial support registered for generic client # [ 1.178521] hv_vmbus: registering driver hyperv_keyboard worker2 # [ 1.174938] ehci-pci 0000:00:07.0: EHCI Host Controller lb # [ 1.172623] thunder_xcv, ver 1.0 worker2 # [ 1.174973] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker2 # [ 1.175221] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 client # [ 1.183665] ehci-pci 0000:00:07.0: EHCI Host Controller worker1 # [ 1.174045] usbcore: registered new interface driver usb-storage client # [ 1.183711] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 1.174221] usbcore: registered new interface driver usbserial_generic worker2 # [ 1.188056] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker2 # [ 1.188451] hub 1-0:1.0: USB hub found worker2 # [ 1.188475] hub 1-0:1.0: 6 ports detected worker1 # [ 1.174238] usbserial: USB Serial support registered for generic worker1 # [ 1.174875] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 1.176580] ehci-pci 0000:00:07.0: EHCI Host Controller lb # [ 1.173302] thunder_bgx, ver 1.0 worker2 # [ 1.191005] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 1.176618] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 lb # [ 1.173336] nicpf, ver 1.0 client # [ 1.183910] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 client # [ 1.186950] rtc-pl031 9010000.pl031: registered as rtc0 worker1 # [ 1.176867] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 lb # [ 1.173970] e1000: Intel(R) PRO/1000 Network Driver client # [ 1.186988] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T16:12:27 UTC (1790179947) lb # [ 1.173980] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 1.187446] i2c_dev: i2c /dev entries driver worker2 # [ 1.192734] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 1.174005] e1000e: Intel(R) PRO/1000 Network Driver worker1 # [ 1.182686] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 1.174015] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 1.192770] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T16:12:27 UTC (1790179947) lb # [ 1.174045] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 1.193104] i2c_dev: i2c /dev entries driver worker1 # [ 1.182724] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T16:12:27 UTC (1790179947) lb # [ 1.174051] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 1.183221] i2c_dev: i2c /dev entries driver lb # [ 1.174075] igbvf: Intel(R) Gigabit Virtual Function Network Driver lb # [ 1.174081] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 1.186418] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 lb # [ 1.174221] sky2: driver version 1.30 worker1 # [ 1.186820] hub 1-0:1.0: USB hub found lb # [ 1.175861] usbcore: registered new interface driver usb-storage worker1 # [ 1.186843] hub 1-0:1.0: 6 ports detected lb # [ 1.176016] usbcore: registered new interface driver usbserial_generic client # [ 1.193337] sdhci: Secure Digital Host Controller Interface driver lb # [ 1.176032] usbserial: USB Serial support registered for generic client # [ 1.193349] sdhci: Copyright(c) Pierre Ossman lb # [ 1.177479] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 1.193614] Synopsys Designware Multimedia Card Interface Driver worker2 # [ 1.199101] sdhci: Secure Digital Host Controller Interface driver lb # [ 1.177512] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 client # [ 1.193992] sdhci-pltfm: SDHCI platform and OF driver helper worker2 # [ 1.199118] sdhci: Copyright(c) Pierre Ossman lb # [ 1.178233] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker2 # [ 1.199387] Synopsys Designware Multimedia Card Interface Driver client # [ 1.195612] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 1.199756] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 1.191706] sdhci: Secure Digital Host Controller Interface driver lb # [ 1.190284] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 1.195875] usbcore: registered new interface driver usbhid client # [ 1.195883] usbhid: USB HID core driver worker2 # [ 1.201358] hid: raw HID events driver (C) Jiri Kosina lb # [ 1.190705] hub 1-0:1.0: USB hub found worker1 # [ 1.191723] sdhci: Copyright(c) Pierre Ossman client # [ 1.201746] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker2 # [ 1.201608] usbcore: registered new interface driver usbhid lb # [ 1.190740] hub 1-0:1.0: 6 ports detected worker1 # [ 1.192001] Synopsys Designware Multimedia Card Interface Driver client # [ 1.202863] hub 1-0:1.0: USB hub found worker2 # [ 1.201616] usbhid: USB HID core driver worker1 # [ 1.192389] sdhci-pltfm: SDHCI platform and OF driver helper client # [ 1.203348] hub 1-0:1.0: 6 ports detected lb # [ 1.193235] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 1.196825] hid: raw HID events driver (C) Jiri Kosina client # [ 1.204336] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 1.197109] usbcore: registered new interface driver usbhid worker1 # [ 1.197116] usbhid: USB HID core driver lb # [ 1.194942] rtc-pl031 9010000.pl031: registered as rtc0 worker2 # [ 1.207513] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available lb # [ 1.194982] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T16:12:27 UTC (1790179947) worker2 # [ 1.209126] drop_monitor: Initializing network drop monitor service client # [ 1.207207] drop_monitor: Initializing network drop monitor service lb # [ 1.195331] i2c_dev: i2c /dev entries driver worker2 # [ 1.209327] NET: Registered PF_INET6 protocol family client # [ 1.207386] NET: Registered PF_INET6 protocol family client # [ 1.209519] Segment Routing with IPv6 worker2 # [ 1.212968] Segment Routing with IPv6 worker1 # [ 1.200183] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available client # [ 1.209546] In-situ OAM (IOAM) with IPv6 worker2 # [ 1.213006] In-situ OAM (IOAM) with IPv6 client # [ 1.209582] NET: Registered PF_PACKET protocol family worker2 # [ 1.213038] NET: Registered PF_PACKET protocol family worker1 # [ 1.203155] drop_monitor: Initializing network drop monitor service worker1 # [ 1.203384] NET: Registered PF_INET6 protocol family worker1 # [ 1.205579] Segment Routing with IPv6 client # [ 1.211366] 9pnet: Installing 9P2000 support worker1 # [ 1.205615] In-situ OAM (IOAM) with IPv6 client # [ 1.211426] Key type dns_resolver registered worker1 # [ 1.205678] NET: Registered PF_PACKET protocol family worker2 # [ 1.214883] 9pnet: Installing 9P2000 support lb # [ 1.201147] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 1.214940] Key type dns_resolver registered lb # [ 1.201163] sdhci: Copyright(c) Pierre Ossman lb # [ 1.201438] Synopsys Designware Multimedia Card Interface Driver lb # [ 1.201827] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 1.207562] 9pnet: Installing 9P2000 support worker1 # [ 1.207624] Key type dns_resolver registered lb # [ 1.203435] hid: raw HID events driver (C) Jiri Kosina lb # [ 1.203687] usbcore: registered new interface driver usbhid lb # [ 1.203696] usbhid: USB HID core driver client # [ 1.218538] registered taskstats version 1 client # [ 1.218722] Loading compiled-in X.509 certificates worker2 # [ 1.222048] registered taskstats version 1 lb # [ 1.209942] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 1.222235] Loading compiled-in X.509 certificates lb # [ 1.211653] drop_monitor: Initializing network drop monitor service lb # [ 1.211833] NET: Registered PF_INET6 protocol family worker1 # [ 1.214957] registered taskstats version 1 lb # [ 1.215646] Segment Routing with IPv6 worker1 # [ 1.215115] Loading compiled-in X.509 certificates lb # [ 1.215684] In-situ OAM (IOAM) with IPv6 lb # [ 1.215714] NET: Registered PF_PACKET protocol family worker2 # [ 1.231599] Demotion targets for Node 0: null worker2 # [ 1.231744] Key type .fscrypt registered worker2 # [ 1.231751] Key type fscrypt-provisioning registered worker2 # [ 1.231868] ima: No TPM chip found, activating TPM-bypass! lb # [ 1.217598] 9pnet: Installing 9P2000 support lb # [ 1.217654] Key type dns_resolver registered worker2 # [ 1.231892] ima: Allocated hash algorithm: sha1 worker2 # [ 1.231917] ima: No architecture policies found client # [ 1.228261] Demotion targets for Node 0: null client # [ 1.228421] Key type .fscrypt registered client # [ 1.228429] Key type fscrypt-provisioning registered client # [ 1.228544] ima: No TPM chip found, activating TPM-bypass! client # [ 1.228566] ima: Allocated hash algorithm: sha1 client # [ 1.228592] ima: No architecture policies found worker1 # [ 1.224532] Demotion targets for Node 0: null worker1 # [ 1.224693] Key type .fscrypt registered client # [ 1.233196] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker1 # [ 1.224700] Key type fscrypt-provisioning registered worker2 # [ 1.236871] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker1 # [ 1.224823] ima: No TPM chip found, activating TPM-bypass! lb # [ 1.224497] registered taskstats version 1 worker1 # [ 1.224846] ima: Allocated hash algorithm: sha1 worker1 # [ 1.224871] ima: No architecture policies found worker1 # [ 1.229633] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 1.225281] Loading compiled-in X.509 certificates lb # [ 1.233984] Demotion targets for Node 0: null lb # [ 1.234134] Key type .fscrypt registered lb # [ 1.234142] Key type fscrypt-provisioning registered lb # [ 1.234265] ima: No TPM chip found, activating TPM-bypass! lb # [ 1.234287] ima: Allocated hash algorithm: sha1 lb # [ 1.234313] ima: No architecture policies found lb # [ 1.239151] input: gpio-keys as /devices/platform/gpio-keys/input/input0 client # [ 1.256728] clk: Disabling unused clocks client # [ 1.256778] PM: genpd: Disabling unused power domains worker2 # [ 1.260330] clk: Disabling unused clocks worker2 # [ 1.260378] PM: genpd: Disabling unused power domains worker1 # [ 1.253355] clk: Disabling unused clocks worker1 # [ 1.253407] PM: genpd: Disabling unused power domains client # [ 1.261330] Freeing unused kernel memory: 4736K worker2 # [ 1.265036] Freeing unused kernel memory: 4736K client # [ 1.261623] Run /init as init process worker2 # [ 1.265250] Run /init as init process worker1 # [ 1.258029] Freeing unused kernel memory: 4736K worker1 # [ 1.258274] Run /init as init process lb # [ 1.261971] clk: Disabling unused clocks lb # [ 1.262018] PM: genpd: Disabling unused power domains lb # [ 1.266584] Freeing unused kernel memory: 4736K lb # [ 1.266877] Run /init as init process client # [ 1.280048] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 1.284670] systemd[1]: Successfully made /usr/ read-only. worker1 # [ 1.276313] systemd[1]: Successfully made /usr/ read-only. lb # [ 1.283488] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 1.434130] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.433569] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 1.436902] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.449350] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 1.590976] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 lb # [ 1.589318] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker1 # [ 1.588584] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 client # [ 1.608783] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.619831] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.616442] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.633722] systemd[1]: Detected virtualization qemu. worker1 # [ 1.611536] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.636352] systemd[1]: Detected architecture arm64. worker1 # [ 1.625661] systemd[1]: Detected virtualization qemu. client # [ 1.631020] systemd[1]: Detected virtualization qemu. lb # [ 1.618928] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.638595] systemd[1]: Running in initrd. lb # [ 1.626062] systemd[1]: Detected virtualization qemu. worker1 # [ 1.628057] systemd[1]: Detected architecture arm64. client # [ 1.631620] systemd[1]: Detected architecture arm64. lb # [ 1.627201] systemd[1]: Detected architecture arm64. lb # [ 1.628319] systemd[1]: Running in initrd. worker1 # [ 1.630243] systemd[1]: Running in initrd. client # [ 1.631677] systemd[1]: Running in initrd. worker2 # [ 1.641538] systemd[1]: Initializing machine ID from random generator. lb # [ 1.629853] systemd[1]: Initializing machine ID from random generator. lb # [ 1.631418] systemd[1]: Hostname set to . client # [ 1.632833] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.633161] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.645076] systemd[1]: Hostname set to . client # [ 1.633171] systemd[1]: Hostname set to . worker1 # [ 1.636517] systemd[1]: Hostname set to . worker2 # [ 1.686412] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker1 # [ 1.681932] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 lb # [ 1.684981] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 client # [ 1.701654] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker2 # [ 1.810043] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.805561] usb 1-2: new high-speed USB device number 3 using ehci-pci lb # [ 1.804771] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.825289] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.947446] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.939055] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 1.961092] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 1.970500] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 1.974720] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.965409] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.971490] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.983118] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 lb # [ 1.971591] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.986336] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.977414] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 lb # [ 1.976978] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 client # [ 2.023983] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 2.036460] systemd[1]: Created slice Slice /system/modprobe. client # [ 2.037735] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 2.039032] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 2.039993] systemd[1]: Reached target Path Units. client # [ 2.040861] systemd[1]: Reached target Slice Units. client # [ 2.041625] systemd[1]: Reached target Swaps. client # [ 2.042263] systemd[1]: Reached target Timer Units. client # [ 2.043220] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 2.044357] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 2.045448] systemd[1]: Listening on Journal Sockets. client # [ 2.046386] systemd[1]: Listening on udev Control Socket. client # [ 2.047376] systemd[1]: Listening on udev Kernel Socket. client # [ 2.048181] systemd[1]: Reached target Socket Units. lb # [ 2.040439] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 2.051244] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 2.055221] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 2.052397] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 2.047071] systemd[1]: Created slice Slice /system/modprobe. lb # [ 2.048303] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 2.064201] systemd[1]: Created slice Slice /system/modprobe. client # [ 2.061576] systemd[1]: Mounting Kernel Configuration File System... lb # [ 2.049686] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 2.065307] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 2.049727] systemd[1]: Reached target Path Units. lb # [ 2.049748] systemd[1]: Reached target Slice Units. lb # [ 2.049768] systemd[1]: Reached target Swaps. lb # [ 2.049787] systemd[1]: Reached target Timer Units. worker2 # [ 2.066610] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 2.066647] systemd[1]: Reached target Path Units. lb # [ 2.049995] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 2.066664] systemd[1]: Reached target Slice Units. lb # [ 2.050143] systemd[1]: Listening on Journal Socket (/dev/log). worker2 # [ 2.066684] systemd[1]: Reached target Swaps. lb # [ 2.050277] systemd[1]: Listening on Journal Sockets. worker2 # [ 2.066702] systemd[1]: Reached target Timer Units. lb # [ 2.050400] systemd[1]: Listening on udev Control Socket. worker2 # [ 2.066911] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 2.050512] systemd[1]: Listening on udev Kernel Socket. client # [ 2.069609] systemd[1]: Starting Journal Service... worker2 # [ 2.067046] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 2.050535] systemd[1]: Reached target Socket Units. worker2 # [ 2.067165] systemd[1]: Listening on Journal Sockets. worker2 # [ 2.067273] systemd[1]: Listening on udev Control Socket. worker2 # [ 2.067373] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 2.067390] systemd[1]: Reached target Socket Units. lb # [ 2.061246] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 2.077538] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 2.061344] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 2.078682] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 2.074013] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 2.072943] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 2.089843] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 2.084025] systemd[1]: Created slice Slice /system/modprobe. lb # [ 2.081830] systemd[1]: Starting Journal Service... worker1 # [ 2.085297] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 2.095264] systemd[1]: Starting Load Kernel Modules... client # [ 2.096300] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 2.086791] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 2.086836] systemd[1]: Reached target Path Units. worker1 # [ 2.086857] systemd[1]: Reached target Slice Units. worker1 # [ 2.086879] systemd[1]: Reached target Swaps. worker1 # [ 2.086898] systemd[1]: Reached target Timer Units. worker1 # [ 2.087126] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 2.087312] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 2.087451] systemd[1]: Listening on Journal Sockets. worker1 # [ 2.087584] systemd[1]: Listening on udev Control Socket. worker1 # [ 2.087708] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 2.087731] systemd[1]: Reached target Socket Units. worker1 # [ 2.099691] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 2.100902] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 2.114370] systemd[1]: Starting Journal Service... client # [ 2.117577] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 2.113100] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 2.125848] systemd[1]: Starting Load Kernel Modules... lb # [ 2.112994] systemd[1]: Starting Load Kernel Modules... client # [ 2.123562] systemd-journald[72]: Collecting audit messages is disabled. worker2 # [ 2.127054] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 2.114017] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 2.126194] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 2.121239] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 2.139676] systemd[1]: Starting Coldplug All udev Devices... lb # [ 2.129153] systemd[1]: Finished Create List of Static Device Nodes. client # [ 2.147258] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 2.142153] systemd[1]: Starting Journal Service... worker2 # [ 2.151806] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 2.152772] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 2.145304] systemd[1]: Starting Load Kernel Modules... worker1 # [ 2.145443] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 2.141300] systemd-journald[72]: Collecting audit messages is disabled. worker2 # [ 2.158649] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 2.161708] systemd-journald[72]: Collecting audit messages is disabled. client # [ 2.157879] systemd[1]: Mounted Kernel Configuration File System. lb # [ 2.158304] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 2.161788] systemd[1]: Starting Coldplug All udev Devices... lb # [ 2.160132] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 2.181704] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 2.182837] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 2.194061] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 2.190802] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 2.210108] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 2.213442] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 2.213050] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 2.217636] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 2.228611] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 2.231173] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 2.228407] systemd-journald[72]: Collecting audit messages is disabled. worker2 # [ 2.246749] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 2.245342] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 2.234668] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker2 # [ 2.249448] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 2.234970] [drm] features: -virgl +edid -resource_blob -host_visible lb # [ 2.234984] [drm] features: -context_init lb # [ 2.235830] [drm] number of scanouts: 1 lb # [ 2.235853] [drm] number of cap sets: 0 lb # [ 2.240138] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 2.254561] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 2.248465] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 2.265372] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker2 # [ 2.265669] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 2.265680] [drm] features: -context_init lb # [ 2.254321] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic lb # [ 2.254353] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 client # [ 2.267422] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 client # [ 2.267735] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 2.267749] [drm] features: -context_init client # [ 2.268614] [drm] number of scanouts: 1 client # [ 2.268638] [drm] number of cap sets: 0 worker1 # [ 2.265603] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 2.270322] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 2.273382] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 2.286852] [drm] number of scanouts: 1 worker2 # [ 2.286892] [drm] number of cap sets: 0 worker2 # [ 2.288853] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker2 # [ 2.288870] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 client # [ 2.289692] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic client # [ 2.289721] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker1 # [ 2.287517] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker1 # [ 2.287795] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 2.302088] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 2.287809] [drm] features: -context_init worker1 # [ 2.288633] [drm] number of scanouts: 1 worker1 # [ 2.288652] [drm] number of cap sets: 0 lb # [ 2.285245] Console: switching to colour frame buffer device 160x50 client # [ 2.305583] systemd[1]: Reached target Preparation for Local File Systems. client # [ 2.305628] systemd[1]: Reached target Local File Systems. client # [ 2.313006] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 2.315809] systemd[1]: Started Journal Service. worker1 # [ 2.306049] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker1 # [ 2.306078] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 client # [ 2.304694] systemd-modules-load[74]: Inserted module 'dm_mod' client # [ 2.306239] systemd-modules-load[74]: Module 'virtio_balloon' is built in client # [ 2.307552] systemd-modules-load[74]: Module 'virtio_console' is built in lb # [ 2.313465] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 2.313717] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 2.313744] systemd[1]: Reached target Local File Systems. lb # [ 2.299118] systemd-modules-load[73]: Inserted module 'dm_mod' lb # [ 2.321400] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 2.326449] Console: switching to colour frame buffer device 160x50 lb # [ 2.322673] systemd[1]: Started Journal Service. lb # [ 2.323965] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device client # [ 2.329735] Console: switching to colour frame buffer device 160x50 client # [ 2.335400] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device lb # [ 2.307548] systemd-modules-load[73]: Module 'virtio_balloon' is built in worker2 # [ 2.346302] systemd[1]: Started Journal Service. lb # [ 2.316587] systemd-modules-load[73]: Module 'virtio_console' is built in lb # [ 2.318202] systemd-modules-load[73]: Inserted module 'virtio_gpu' lb # [ 2.319319] systemd-modules-load[73]: Module 'virtio_rng' is built in client # [ 2.334525] systemd[1]: Starting Create System Files and Directories... worker2 # [ 2.354210] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker2 # [ 2.331798] systemd-modules-load[73]: Inserted module 'dm_mod' worker2 # [ 2.339850] systemd-modules-load[73]: Module 'virtio_balloon' is built in lb # [ 2.325403] systemd[1]: Finished Load Kernel Modules. client # [ 2.339537] systemd-modules-load[74]: Inserted module 'virtio_gpu' worker1 # [ 2.341868] Console: switching to colour frame buffer device 160x50 worker2 # [ 2.343947] systemd-modules-load[73]: Module 'virtio_console' is built in client # [ 2.344363] systemd-modules-load[74]: Module 'virtio_rng' is built in worker1 # [ 2.352028] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 2.352271] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 2.352298] systemd[1]: Reached target Local File Systems. worker2 # [ 2.352677] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 2.353904] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 2.360780] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 2.355087] systemd[1]: Reached target Local File Systems. client # [ 2.358193] systemd[1]: Finished Load Kernel Modules. worker2 # [ 2.364321] systemd[1]: Starting Create System Files and Directories... lb # [ 2.349795] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.366735] systemd-modules-load[73]: Inserted module 'virtio_gpu' lb # [ 2.352660] systemd[1]: Starting Create System Files and Directories... worker2 # [ 2.370683] systemd-modules-load[73]: Module 'virtio_rng' is built in worker1 # [ 2.378256] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker2 # [ 2.376322] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 2.378387] systemd[1]: Finished Load Kernel Modules. client # [ 2.385798] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 2.393906] systemd[1]: Finished Load Kernel Modules. worker1 # [ 2.397993] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.400274] systemd[1]: Starting Apply Kernel Variables... client # [ 2.402042] systemd[1]: Finished Create System Files and Directories. client # [ 2.426500] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.419847] systemd[1]: Finished Create System Files and Directories. worker1 # [ 2.424434] systemd-modules-load[73]: Inserted module 'dm_mod' lb # [ 2.424564] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker2 # [ 2.441392] systemd[1]: Finished Create System Files and Directories. worker1 # [ 2.447386] systemd[1]: Started Journal Service. worker1 # [ 2.429673] systemd-modules-load[73]: Module 'virtio_balloon' is built in lb # [ 2.429034] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 2.430913] systemd-modules-load[73]: Module 'virtio_console' is built in client # [ 2.449364] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker1 # [ 2.444304] systemd-modules-load[73]: Inserted module 'virtio_gpu' worker1 # [ 2.448479] systemd-modules-load[73]: Module 'virtio_rng' is built in worker2 # [ 2.466465] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 2.452381] systemd[1]: Starting Create System Files and Directories... worker2 # [ 2.469628] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker1 # [ 2.458305] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.457924] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 2.468647] systemd-udevd[78]: Using default interface naming scheme 'v261'. client # [ 2.484221] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.501300] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 2.512969] systemd[1]: Finished Create System Files and Directories. worker1 # [ 2.517108] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.520952] systemd[1]: Starting Virtual Console Setup... client # [ 2.560172] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.576211] systemd[1]: Starting Virtual Console Setup... lb # [ 2.580582] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.584776] systemd[1]: Starting Virtual Console Setup... lb # [ 2.584759] systemd[1]: Finished Virtual Console Setup. client # [ 2.616588] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.621087] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.624551] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.628732] systemd[1]: Finished Virtual Console Setup. worker1 # [ 2.652612] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.656728] systemd[1]: Finished Virtual Console Setup. lb # [ 3.204498] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 3.221142] systemd[1]: Finished Coldplug All udev Devices. lb # [ 3.205911] systemd[1]: Reached target System Initialization. lb # [ 3.208452] systemd[1]: Reached target Basic System. worker2 # [ 3.225498] systemd[1]: Reached target System Initialization. worker2 # [ 3.226862] systemd[1]: Reached target Basic System. client # [ 3.251062] systemd[1]: Finished Coldplug All udev Devices. client # [ 3.252113] systemd[1]: Reached target System Initialization. client # [ 3.252964] systemd[1]: Reached target Basic System. worker1 # [ 3.250557] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 3.251567] systemd[1]: Reached target System Initialization. worker1 # [ 3.252701] systemd[1]: Reached target Basic System. worker2 # [ 3.386271] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.400766] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.404432] (udev-worker)[100]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 3.420395] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 3.404347] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 3.423802] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.408923] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.439597] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.444387] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.457996] (udev-worker)[99]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 3.469546] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.472588] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.480160] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.500530] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.507566] systemd[1]: Reached target Initrd Root Device. lb # [ 3.491795] systemd[1]: Reached target Initrd Root Device. lb # [ 3.498036] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 3.515158] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 3.544371] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 3.533176] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 3.540112] systemd[1]: Reached target Initrd Root Device. client # [ 3.552625] systemd[1]: Reached target Initrd Root Device. worker1 # [ 3.545931] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 3.558283] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 3.567770] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 3.554872] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.578797] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 3.563114] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.584262] systemd[1]: Mounting /sysroot... lb # [ 3.568220] systemd[1]: Mounting /sysroot... worker1 # [ 3.596200] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 3.603576] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. client # [ 3.620874] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 3.628438] EXT4-fs (vda): mounted filesystem e419f460-32a8-4ac9-96c7-3ad177ebeeda r/w with ordered data mode. Quota mode: none. worker1 # [ 3.612366] systemd[1]: Mounting /sysroot... worker2 # [ 3.645268] EXT4-fs (vda): mounted filesystem f9c82e2c-8814-4592-9a51-fe3bb167e285 r/w with ordered data mode. Quota mode: none. lb # [ 3.614746] systemd[1]: Mounted /sysroot. client # [ 3.628246] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 3.617087] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.634378] systemd[1]: Mounted /sysroot. client # [ 3.632968] systemd[1]: Mounting /sysroot... worker2 # [ 3.636137] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.638753] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.623798] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 3.673727] EXT4-fs (vda): mounted filesystem a7ecca0c-350f-44fc-aa57-2fc72629c496 r/w with ordered data mode. Quota mode: none. lb # [ 3.656062] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. worker1 # [ 3.659760] systemd[1]: Mounted /sysroot. worker2 # [ 3.673796] systemd-sysroot-fstab-check[117]: /sysroot should be mounted in the initrd, will request daemon-reload. worker1 # [ 3.662079] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.663485] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 3.691971] EXT4-fs (vda): mounted filesystem 377aa17b-16db-4838-8736-3487cf584fcc r/w with ordered data mode. Quota mode: none. lb # [ 3.663390] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.666629] systemd[1]: Reloading... worker2 # [ 3.682199] systemd[1]: Reload requested from client PID 117 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.681454] systemd[1]: Mounted /sysroot. client # [ 3.683822] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.686920] systemd[1]: Reloading... client # [ 3.689774] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 3.695914] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload. worker1 # [ 3.702241] systemd[1]: Reload requested from client PID 119 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 3.708187] systemd[1]: Reloading... client # [ 3.724790] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 3.729797] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.733477] systemd[1]: Reloading... lb # [ 3.892761] systemd[1]: Reloading finished in 232 ms. worker2 # [ 3.913633] systemd[1]: Reloading finished in 231 ms. lb # [ 3.926398] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... lb # [ 3.929480] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... worker2 # [ 3.947469] systemd-sysroot-fstab-check[117]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.950816] systemd-sysroot-fstab-check[117]: Requesting swap.target/start/replace... worker1 # [ 3.936762] systemd[1]: Reloading finished in 230 ms. lb # [ 3.936553] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.939704] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.942090] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.958848] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.961673] systemd[1]: Finished Mountpoints Configured in the Real Root. worker2 # [ 3.963357] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.963547] systemd[1]: Reloading finished in 230 ms. worker1 # [ 3.967692] systemd-sysroot-fstab-check[119]: Requesting initrd-fs.target/start/replace... worker1 # [ 3.973022] systemd-sysroot-fstab-check[119]: Requesting swap.target/start/replace... worker1 # [ 3.979614] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.982945] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.984739] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.998828] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... client # [ 4.002819] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... client # [ 4.010893] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 4.013583] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 4.015749] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 4.106948] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 4.119986] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 4.110141] systemd[1]: Mounting /sysroot/nix/.ro-store... worker2 # [ 4.128861] systemd[1]: Mounting /sysroot/nix/.ro-store... client # [ 4.133733] systemd[1]: Mounting /sysroot/run... worker2 # [ 4.137117] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 4.130141] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 4.134737] systemd[1]: Mounting /sysroot/run... client # [ 4.156183] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 4.159846] systemd[1]: Mounting /sysroot/run... worker1 # [ 4.154246] systemd[1]: Mounting /sysroot/nix/.ro-store... worker2 # [ 4.176752] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 4.161639] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 4.164256] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 4.184323] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 4.179486] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 4.216504] fuse: init (API version 7.45) worker1 # [ 4.189097] systemd[1]: Mounting /sysroot/run... worker1 # [ 4.194708] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 4.217172] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 4.236788] virtiofs virtio6: discovered new tag: nix-store worker2 # [ 4.245424] fuse: init (API version 7.45) client # [ 4.245629] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 4.221714] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 4.263164] virtiofs virtio7: discovered new tag: shared worker2 # [ 4.266314] virtiofs virtio6: discovered new tag: nix-store client # [ 4.263979] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 4.267131] virtiofs virtio6: virtio_fs_setup_dax: No cache capability client # [ 4.271571] virtiofs virtio8: discovered new tag: xchg client # [ 4.272380] virtiofs virtio8: virtio_fs_setup_dax: No cache capability lb # [ 4.245563] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 4.283914] virtiofs virtio7: discovered new tag: shared worker2 # [ 4.284773] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 4.273560] fuse: init (API version 7.45) client # [ 4.272349] systemd[1]: Mounted /sysroot/run. worker2 # [ 4.293184] virtiofs virtio8: discovered new tag: xchg worker2 # [ 4.282748] systemd[1]: Mounted /sysroot/run. client # [ 4.282303] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 4.287550] virtiofs virtio6: discovered new tag: nix-store lb # [ 4.288407] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 4.302881] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker2 # [ 4.294436] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 4.278572] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 4.295958] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 4.302402] virtiofs virtio7: discovered new tag: shared lb # [ 4.303221] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 4.288199] systemd[1]: Mounted /sysroot/run. client # [ 4.302982] systemd[1]: Mounted /sysroot/tmp/shared. client # [ 4.307268] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 4.313183] virtiofs virtio8: discovered new tag: xchg lb # [ 4.313986] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker1 # [ 4.299990] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 4.310936] systemd[1]: Mounted /sysroot/run. client # [ 4.323449] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 4.326679] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 4.333802] fuse: init (API version 7.45) lb # [ 4.314688] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 4.331251] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 4.332283] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 4.340388] virtiofs virtio6: discovered new tag: nix-store worker1 # [ 4.341229] virtiofs virtio6: virtio_fs_setup_dax: No cache capability lb # [ 4.322966] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 4.342342] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 4.326013] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 4.346436] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 4.355314] virtiofs virtio7: discovered new tag: shared client # [ 4.347939] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 4.356132] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 4.337383] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.339965] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 4.364189] virtiofs virtio8: discovered new tag: xchg worker1 # [ 4.364996] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker1 # [ 4.351145] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 4.365926] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.367734] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 4.374097] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 4.377044] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 4.379442] systemd[1]: Mounted /sysroot/tmp/xchg. worker1 # [ 4.382192] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 4.385171] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.881330] (udev-worker)[93]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory lb # [ 4.888169] (udev-worker)[93]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker2 # [ 4.909676] (udev-worker)[99]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker2 # [ 4.916710] (udev-worker)[99]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory lb # [ 4.918093] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.921396] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.924156] systemd[1]: Stopping Virtual Console Setup... lb # [ 4.926158] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.932699] (udev-worker)[97]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.939853] (udev-worker)[97]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory lb # [ 4.940083] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.941589] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.957700] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.959654] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.945452] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.962061] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.964267] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.981578] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.983168] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.968843] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.970520] systemd[1]: Stopped Virtual Console Setup. client # [ 4.983044] (udev-worker)[91]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.973390] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.988297] systemd[1]: Starting Virtual Console Setup... client # [ 4.985972] (udev-worker)[91]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.976854] systemd[1]: Starting Virtual Console Setup... lb # [ 4.974294] systemd-vconsole-setup[153]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 4.977962] systemd[1]: Finished Virtual Console Setup. worker1 # [ 4.992837] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.994450] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.996940] systemd[1]: Starting Virtual Console Setup... worker2 # [ 5.013937] systemd-vconsole-setup[152]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 5.016745] systemd[1]: Finished Virtual Console Setup. client # [ 5.026076] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 5.028551] systemd[1]: Stopped Virtual Console Setup. client # [ 5.032265] systemd[1]: Stopping Virtual Console Setup... client # [ 5.033171] systemd[1]: Starting Virtual Console Setup... worker1 # [ 5.020572] systemd-vconsole-setup[156]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 5.024199] systemd[1]: Finished Virtual Console Setup. client # [ 5.057851] systemd-vconsole-setup[151]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 5.061076] systemd[1]: Finished Virtual Console Setup. client # [ 5.104602] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 5.125883] systemd[1]: Mounting /sysroot/nix/store... lb # [ 5.111217] systemd[1]: Mounting /sysroot/nix/store... client # [ 5.143696] systemd[1]: Mounted /sysroot/nix/store. client # [ 5.147244] systemd[1]: Reached target Initrd File Systems. client # [ 5.152633] systemd[1]: Starting Find NixOS closure... worker2 # [ 5.158548] systemd[1]: Mounted /sysroot/nix/store. client # [ 5.155427] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker2 # [ 5.164482] systemd[1]: Reached target Initrd File Systems. worker2 # [ 5.165547] systemd[1]: Starting Find NixOS closure... worker2 # [ 5.167865] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 5.156372] systemd[1]: Mounting /sysroot/nix/store... client # [ 5.190000] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 5.194626] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker2 # [ 5.198509] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 5.183320] systemd[1]: Mounted /sysroot/nix/store. lb # [ 5.186377] systemd[1]: Reached target Initrd File Systems. worker2 # [ 5.202984] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 5.205111] systemd[1]: Finished Find NixOS closure. lb # [ 5.192180] systemd[1]: Starting Find NixOS closure... client # [ 5.207985] systemd[1]: Reached target Initrd Default Target. worker2 # [ 5.213365] systemd[1]: Finished Find NixOS closure. worker2 # [ 5.214944] systemd[1]: Reached target Initrd Default Target. client # [ 5.212601] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 5.198033] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker2 # [ 5.217622] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 5.220403] systemd[1]: Mounted /sysroot/nix/store. client # [ 5.236140] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.237714] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 5.223333] systemd[1]: Reached target Initrd File Systems. worker1 # [ 5.228344] systemd[1]: Starting Find NixOS closure... worker2 # [ 5.243538] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.241207] systemd[1]: Stopped target Initrd Default Target. client # [ 5.244215] systemd[1]: Stopped target Basic System. client # [ 5.245014] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 5.247540] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 5.245916] systemd[1]: Stopped target Path Units. worker2 # [ 5.248686] systemd[1]: Stopped target Initrd Default Target. worker2 # [ 5.249832] systemd[1]: Stopped target Basic System. worker2 # [ 5.252204] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 5.253706] systemd[1]: Stopped target Path Units. client # [ 5.251387] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker2 # [ 5.254737] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 5.240447] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 5.257665] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.260237] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 5.258791] systemd[1]: Stopped target Slice Units. worker2 # [ 5.261430] systemd[1]: Stopped target Slice Units. worker2 # [ 5.262148] systemd[1]: Stopped target Socket Units. client # [ 5.259492] systemd[1]: Stopped target Socket Units. worker2 # [ 5.262879] systemd[1]: Stopped target System Initialization. worker2 # [ 5.265612] systemd[1]: Stopped target Swaps. client # [ 5.264583] systemd[1]: Stopped target System Initialization. worker2 # [ 5.267862] systemd[1]: Stopped target Timer Units. client # [ 5.266379] systemd[1]: Stopped target Swaps. client # [ 5.267187] systemd[1]: Stopped target Timer Units. worker2 # [ 5.270387] systemd[1]: dbus.socket: Deactivated successfully. client # [ 5.267927] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 5.272194] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 5.256846] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 5.274059] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 5.261208] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 5.278260] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 5.282422] systemd[1]: Stopped Find NixOS closure. worker2 # [ 5.283563] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 5.285919] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 5.284267] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker2 # [ 5.288391] systemd[1]: Stopped Apply Kernel Variables. lb # [ 5.271956] systemd[1]: Finished Find NixOS closure. lb # [ 5.273930] systemd[1]: Reached target Initrd Default Target. worker2 # [ 5.290922] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 5.288672] systemd[1]: Stopped Find NixOS closure. client # [ 5.289561] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 5.290412] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 5.277518] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 5.292123] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 5.294833] systemd[1]: Stopped Load Kernel Modules. client # [ 5.292956] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 5.293986] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 5.296381] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. client # [ 5.294725] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. client # [ 5.295822] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 5.299843] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 5.301494] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker1 # [ 5.287924] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 5.304395] systemd[1]: Stopped Create System Files and Directories. client # [ 5.302822] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 5.303920] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 5.308506] systemd[1]: Stopped target Local File Systems. worker2 # [ 5.309360] systemd[1]: Stopped target Preparation for Local File Systems. worker2 # [ 5.310328] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 5.308440] systemd[1]: Stopped target Local File Systems. worker2 # [ 5.311334] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 5.295752] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 5.309425] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 5.310422] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 5.313121] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 5.314214] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 5.312533] systemd[1]: Stopped Coldplug All udev Devices. worker2 # [ 5.315242] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 5.316044] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.317092] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 5.317928] systemd[1]: systemd-udevd.service: Deactivated successfully. client # [ 5.316137] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 5.318863] systemd[1]: Stopped Rule-based Manager for Device Events and Files. client # [ 5.317211] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 5.319890] systemd[1]: systemd-udevd.service: Consumed 1.521s CPU time over 2.932s wall clock time, 25.3M memory peak. worker1 # [ 5.306802] systemd[1]: Finished Find NixOS closure. client # [ 5.320153] systemd[1]: Stopped Virtual Console Setup. client # [ 5.320924] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 5.309655] systemd[1]: Reached target Initrd Default Target. worker2 # [ 5.324361] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 5.308622] systemd[1]: Stopped target Initrd Default Target. worker2 # [ 5.325437] systemd[1]: Closed udev Control Socket. worker1 # [ 5.311283] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 5.309888] systemd[1]: Stopped target Basic System. client # [ 5.324327] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 5.310874] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 5.328275] systemd[1]: Starting Cleanup udev Database... client # [ 5.325375] systemd[1]: systemd-udevd.service: Consumed 1.567s CPU time over 2.976s wall clock time, 25.3M memory peak. worker2 # [ 5.329283] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 5.313466] systemd[1]: Stopped target Path Units. client # [ 5.328336] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 5.314794] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker2 # [ 5.332268] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 5.332399] systemd[1]: Closed udev Control Socket. client # [ 5.333154] systemd[1]: Starting Cleanup udev Database... lb # [ 5.319560] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.336190] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 5.333940] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 5.337380] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 5.334994] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 5.338377] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 5.340161] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 5.325544] systemd[1]: Stopped target Slice Units. lb # [ 5.326427] systemd[1]: Stopped target Socket Units. client # [ 5.340333] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 5.327275] systemd[1]: Stopped target System Initialization. client # [ 5.341534] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 5.342513] systemd[1]: kmod-static-nodes.service: Deactivated successfully. client # [ 5.343473] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 5.333898] systemd[1]: Stopped target Swaps. client # [ 5.348546] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 5.349583] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 5.338810] systemd[1]: Stopped target Timer Units. worker1 # [ 5.340442] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 5.341982] systemd[1]: Stopped target Basic System. lb # [ 5.341908] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 5.344293] systemd[1]: Stopped target Initrd Root Device. lb # [ 5.343524] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 5.345875] systemd[1]: Stopped target Path Units. worker1 # [ 5.348348] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 5.347628] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 5.361540] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 5.351068] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 5.364953] systemd[1]: Finished Cleanup udev Database. client # [ 5.365794] systemd[1]: Reached target Switch Root. worker2 # [ 5.369853] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 5.367942] systemd[1]: Starting NixOS Activation... lb # [ 5.354505] systemd[1]: Stopped Find NixOS closure. worker1 # [ 5.356288] systemd[1]: Stopped target Slice Units. worker1 # [ 5.357223] systemd[1]: Stopped target Socket Units. worker2 # [ 5.371670] systemd[1]: Finished Cleanup udev Database. lb # [ 5.355447] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 5.358155] systemd[1]: Stopped target System Initialization. worker1 # [ 5.359136] systemd[1]: Stopped target Swaps. worker1 # [ 5.359958] systemd[1]: Stopped target Timer Units. worker2 # [ 5.375215] systemd[1]: Reached target Switch Root. lb # [ 5.361663] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 5.362742] systemd[1]: Stopped Apply Kernel Variables. lb # [ 5.363507] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 5.380301] systemd[1]: Starting NixOS Activation... worker1 # [ 5.369463] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 5.368741] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 5.372342] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 5.371223] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 5.374167] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 5.376255] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 5.376164] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker1 # [ 5.380428] systemd[1]: Stopped Find NixOS closure. lb # [ 5.380211] systemd[1]: Stopped Create System Files and Directories. lb # [ 5.381176] systemd[1]: Stopped target Local File Systems. lb # [ 5.382400] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 5.383959] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker1 # [ 5.386437] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 5.385350] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 5.387484] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 5.386142] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 5.388609] systemd[1]: Stopped Apply Kernel Variables. lb # [ 5.387172] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 5.389444] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 5.388396] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 5.390555] systemd[1]: Stopped Load Kernel Modules. lb # [ 5.389148] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 5.391335] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 5.390081] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. lb # [ 5.390989] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 5.391912] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 5.396231] systemd[1]: systemd-udevd.service: Consumed 1.511s CPU time over 3.060s wall clock time, 22.3M memory peak. lb # [ 5.397700] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 5.400354] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 5.400499] systemd[1]: Closed udev Control Socket. worker1 # [ 5.405209] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 5.404198] systemd[1]: Starting Cleanup udev Database... lb # [ 5.405072] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 5.406113] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 5.408491] systemd[1]: Stopped Create System Files and Directories. lb # [ 5.408330] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 5.411023] systemd[1]: Stopped target Local File Systems. worker1 # [ 5.412992] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 5.412183] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 5.413215] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 5.415583] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 5.414158] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 5.416112] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 5.420323] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 5.421166] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 5.420330] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 5.422201] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 5.423192] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 5.423905] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 5.427654] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 5.428731] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 5.429649] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 5.430673] systemd[1]: systemd-udevd.service: Consumed 1.514s CPU time over 3.065s wall clock time, 25.4M memory peak. worker1 # [ 5.436880] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 5.440187] systemd[1]: Closed udev Control Socket. worker1 # [ 5.441307] systemd[1]: Starting Cleanup udev Database... worker1 # [ 5.444411] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker1 # [ 5.445555] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 5.457145] initrd-nixos-activation-start[175]: booting system configuration /nix/store/zbvxb4cbwfrqzrgr4i5486i3lak215fd-nixos-system-client-test worker1 # [ 5.448131] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 5.447246] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 5.463426] initrd-nixos-activation-start[178]: booting system configuration /nix/store/6fvhs2ngm0npgxjv84bzdcnf0823nkly-nixos-system-worker2-test lb # [ 5.448965] systemd[1]: Finished Cleanup udev Database. lb # [ 5.449708] systemd[1]: Reached target Switch Root. worker1 # [ 5.452161] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 5.453191] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 5.454174] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 5.452734] systemd[1]: Starting NixOS Activation... worker1 # [ 5.456255] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 5.460363] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 5.492165] initrd-nixos-activation-start[175]: running activation script... worker2 # [ 5.496932] initrd-nixos-activation-start[178]: running activation script... worker1 # [ 5.484947] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 5.487652] systemd[1]: Finished Cleanup udev Database. worker1 # [ 5.492309] systemd[1]: Reached target Switch Root. worker1 # [ 5.493104] systemd[1]: Starting NixOS Activation... lb # [ 5.533356] initrd-nixos-activation-start[176]: booting system configuration /nix/store/5hmvppg1fs8xkiccvl0m50n4dxjz92fr-nixos-system-lb-test lb # [ 5.566152] initrd-nixos-activation-start[176]: running activation script... worker1 # [ 5.574021] initrd-nixos-activation-start[179]: booting system configuration /nix/store/70lv2j0cfdc8kw2f0kxgczfdih2jlj3k-nixos-system-worker1-test worker1 # [ 5.608503] initrd-nixos-activation-start[179]: running activation script... worker2 # [ 5.754885] initrd-nixos-activation-start[201]: setting up /etc... client # [ 5.760885] initrd-nixos-activation-start[198]: setting up /etc... lb # [ 5.835144] initrd-nixos-activation-start[199]: setting up /etc... worker2 # [ 5.879766] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 5.882817] systemd[1]: Finished NixOS Activation. worker2 # [ 5.883961] systemd[1]: Starting Switch Root... worker1 # [ 5.871527] initrd-nixos-activation-start[202]: setting up /etc... client # [ 5.887371] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 5.890517] systemd[1]: Finished NixOS Activation. client # [ 5.891792] systemd[1]: Starting Switch Root... worker2 # [ 5.903754] systemd[1]: Switching root. client # [ 5.916583] systemd[1]: Switching root. lb # [ 5.956553] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 5.959355] systemd[1]: Finished NixOS Activation. lb # [ 5.960629] systemd[1]: Starting Switch Root... lb # [ 5.983692] systemd[1]: Switching root. worker1 # [ 5.997131] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 5.999985] systemd[1]: Finished NixOS Activation. worker1 # [ 6.001320] systemd[1]: Starting Switch Root... worker1 # [ 6.023916] systemd[1]: Switching root. worker2 # [ 6.091154] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). client # [ 6.107682] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). lb # [ 6.170853] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker1 # [ 6.209876] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker2 # [ 6.639760] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 6.652720] systemd[1]: Detected virtualization qemu. client # [ 6.639685] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 6.652835] systemd[1]: Detected virtualization qemu. worker2 # [ 6.656065] systemd[1]: Detected architecture arm64. worker2 # [ 6.660015] systemd[1]: Detected first boot. client # [ 6.656030] systemd[1]: Detected architecture arm64. client # [ 6.659701] systemd[1]: Detected first boot. worker2 # [ 6.663884] systemd[1]: Initializing machine ID from random generator. client # [ 6.664482] systemd[1]: Initializing machine ID from random generator. lb # [ 6.709142] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 6.721906] systemd[1]: Detected virtualization qemu. lb # [ 6.725085] systemd[1]: Detected architecture arm64. lb # [ 6.728895] systemd[1]: Detected first boot. lb # [ 6.734811] systemd[1]: Initializing machine ID from random generator. worker1 # [ 6.745257] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 6.758211] systemd[1]: Detected virtualization qemu. worker1 # [ 6.760429] systemd[1]: Detected architecture arm64. worker1 # [ 6.764386] systemd[1]: Detected first boot. worker1 # [ 6.769592] systemd[1]: Initializing machine ID from random generator. worker2 # [ 6.970217] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.994015] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 7.062079] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 7.076492] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 7.134829] systemd[1]: Applying preset policy. client # [ 7.187828] systemd[1]: Applying preset policy. worker1 # [ 7.229909] systemd[1]: Applying preset policy. lb # [ 7.262466] systemd[1]: Applying preset policy. worker2 # [ 7.389415] systemd[1]: Populated /etc with preset unit settings. client # [ 7.461756] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 7.485302] systemd[1]: Populated /etc with preset unit settings. lb # [ 7.522180] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 7.621236] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 7.622744] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 7.626310] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 7.629089] systemd[1]: Created slice Slice /system/getty. worker2 # [ 7.631408] systemd[1]: Created slice User and Session Slice. worker2 # [ 7.632687] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 7.634840] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 7.636423] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 7.637865] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.639423] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 7.641905] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.643053] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 7.644574] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 7.646215] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 7.648602] systemd[1]: Reached target Path Units. worker2 # [ 7.649591] systemd[1]: Reached target Remote File Systems. worker2 # [ 7.651292] systemd[1]: Reached target Slice Units. worker2 # [ 7.653545] systemd[1]: Reached target Swaps. worker2 # [ 7.656127] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 7.659257] systemd[1]: Listening on Process Core Dump Socket. worker2 # [ 7.661534] systemd[1]: Listening on Credential Encryption/Decryption. worker2 # [ 7.664050] systemd[1]: Listening on Factory Reset Management. worker2 # [ 7.665301] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 7.669797] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 7.672055] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 7.674927] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 7.676524] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 7.678623] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.680695] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 7.686028] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 7.687423] systemd[1]: Listening on udev Control Socket. worker2 # [ 7.689044] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 7.692829] systemd[1]: Mounting Huge Pages File System... worker2 # [ 7.698372] systemd[1]: Mounting POSIX Message Queue File System... client # [ 7.696687] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 7.698306] systemd[1]: Stopped initrd-switch-root.service. client # [ 7.702279] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 7.706290] systemd[1]: Mounting Kernel Debug File System... client # [ 7.705078] systemd[1]: Created slice Slice /system/getty. client # [ 7.708020] systemd[1]: Created slice User and Session Slice. client # [ 7.709624] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 7.711327] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 7.713105] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 7.719077] systemd[1]: Mounting Kernel Trace File System... client # [ 7.714624] systemd[1]: Expecting device /dev/ttyAMA0... client # [ 7.717017] systemd[1]: Reached target Local Encrypted Volumes. client # [ 7.718238] systemd[1]: Stopped target initrd-fs.target. client # [ 7.719825] systemd[1]: Stopped target initrd-root-fs.target. client # [ 7.721504] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.723169] systemd[1]: Reached target Virtual Machines and Containers. client # [ 7.724953] systemd[1]: Reached target Path Units. client # [ 7.726458] systemd[1]: Reached target Remote File Systems. client # [ 7.728086] systemd[1]: Reached target Slice Units. worker2 # [ 7.731792] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 7.722019] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 7.729777] systemd[1]: Reached target Swaps. worker1 # [ 7.723828] systemd[1]: Stopped initrd-switch-root.service. client # [ 7.732961] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 7.738243] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 7.727717] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 7.736289] systemd[1]: Listening on Process Core Dump Socket. client # [ 7.738814] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 7.731515] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.733863] systemd[1]: Created slice User and Session Slice. client # [ 7.741488] systemd[1]: Listening on Factory Reset Management. client # [ 7.742824] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 7.735173] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 7.748051] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.738134] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 7.749711] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 7.747525] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 7.740377] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 7.752129] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.742336] systemd[1]: Expecting device /dev/ttyAMA0... client # [ 7.750173] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.744243] systemd[1]: Reached target Local Encrypted Volumes. client # [ 7.754249] systemd[1]: Listening on Console Output Muting Service Socket. worker1 # [ 7.746225] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.758260] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 7.756007] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 7.748050] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 7.749964] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.757711] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.751837] systemd[1]: Reached target Virtual Machines and Containers. client # [ 7.761028] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 7.753783] systemd[1]: Reached target Path Units. worker1 # [ 7.755494] systemd[1]: Reached target Remote File Systems. worker1 # [ 7.757302] systemd[1]: Reached target Slice Units. worker1 # [ 7.759057] systemd[1]: Reached target Swaps. client # [ 7.767395] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 7.771896] systemd[1]: Mounting FUSE Control File System... client # [ 7.768862] systemd[1]: Listening on udev Control Socket. worker1 # [ 7.762333] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 7.770670] systemd[1]: Listening on udev Varlink Socket. worker2 # [ 7.773216] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.765553] systemd[1]: Listening on Process Core Dump Socket. lb # [ 7.763714] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 7.775359] systemd[1]: Mounting Huge Pages File System... worker1 # [ 7.767865] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 7.765546] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 7.770377] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.770816] systemd[1]: Listening on Hostname Service Socket. lb # [ 7.768960] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 7.781732] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 7.771972] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.775393] systemd[1]: Starting Journal Log Access Socket... lb # [ 7.774031] systemd[1]: Created slice User and Session Slice. worker1 # [ 7.777686] systemd[1]: Listening on Journal Audit Socket. lb # [ 7.775277] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 7.778310] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 7.781404] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 7.779655] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.783060] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 7.781334] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 7.785112] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.783796] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 7.786920] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 7.784986] systemd[1]: Stopped target initrd-fs.target. client # [ 7.797369] systemd[1]: Mounting Kernel Debug File System... lb # [ 7.787349] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 7.788435] systemd[1]: Stopped target initrd-switch-root.target. lb # [ 7.790145] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 7.793172] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 7.801815] systemd[1]: Starting Journal Service... worker1 # [ 7.794651] systemd[1]: Listening on udev Control Socket. lb # [ 7.791864] systemd[1]: Reached target Path Units. worker1 # [ 7.796258] systemd[1]: Listening on udev Varlink Socket. lb # [ 7.793485] systemd[1]: Reached target Remote File Systems. lb # [ 7.795990] systemd[1]: Reached target Slice Units. lb # [ 7.797071] systemd[1]: Reached target Swaps. client # [ 7.807849] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 7.800217] systemd[1]: Mounting Huge Pages File System... lb # [ 7.801022] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 7.804694] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 7.804079] systemd[1]: Listening on Process Core Dump Socket. lb # [ 7.806437] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 7.808879] systemd[1]: Listening on Factory Reset Management. lb # [ 7.810126] systemd[1]: Listening on Hostname Service Socket. lb # [ 7.814504] systemd[1]: Starting Journal Log Access Socket... client # [ 7.824349] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 7.817180] systemd[1]: Mounting Kernel Debug File System... lb # [ 7.816789] systemd[1]: Listening on Journal Audit Socket. client # [ 7.827317] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 7.820574] systemd[1]: Listening on Console Output Muting Service Socket. worker2 # [ 7.834405] systemd[1]: Starting Load Kernel Modules... lb # [ 7.822192] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 7.824565] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.826195] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 7.830531] systemd[1]: Mounting Kernel Trace File System... lb # [ 7.833187] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 7.847619] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... client # [ 7.843903] systemd[1]: Mounting Kernel Configuration File System... lb # [ 7.834587] systemd[1]: Listening on udev Control Socket. lb # [ 7.836293] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 7.841202] systemd[1]: Starting Create List of Static Device Nodes... client # [ 7.848723] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 7.840104] systemd[1]: Mounting Huge Pages File System... worker2 # [ 7.854313] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 7.844347] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 7.843411] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 7.856773] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.857776] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 7.860639] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.866033] systemd[1]: Starting Coldplug All udev Devices... lb # [ 7.853344] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 7.871941] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 7.867778] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 7.878346] systemd[1]: Mounted Huge Pages File System. lb # [ 7.866581] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 7.880401] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 7.872066] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 7.880873] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 7.881256] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 7.881631] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 7.886171] systemd[1]: Mounted FUSE Control File System. client # [ 7.884577] systemd[1]: Mounting FUSE Control File System... lb # [ 7.874401] systemd[1]: Starting Create List of Static Device Nodes... client # [ 7.886183] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.880866] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 7.881840] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.882277] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 7.892649] systemd[1]: Mounting Kernel Configuration File System... lb # [ 7.896813] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.901756] systemd[1]: Mounting FUSE Control File System... worker1 # [ 7.902191] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.904275] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 7.918621] systemd[1]: Starting Journal Service... lb # [ 7.907112] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.925192] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 7.931893] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.929517] systemd[1]: Starting Load Kernel Modules... worker1 # [ 7.925695] systemd[1]: Starting Journal Service... worker1 # [ 7.942847] systemd[1]: Starting Load Kernel Modules... lb # [ 7.943033] systemd[1]: Mounting FUSE Control File System... client # [ 7.958700] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 7.948096] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 7.969006] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 7.963277] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... client # [ 7.971698] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.970074] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 7.971956] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.987715] systemd[1]: Finished Load Kernel Modules. worker2 # [ 7.992187] systemd-journald[271]: Collecting audit messages is enabled. worker1 # [ 7.983892] systemd[1]: Starting Coldplug All udev Devices... client # [ 7.991679] systemd-journald[268]: Collecting audit messages is enabled. lb # [ 7.980968] systemd[1]: Starting Journal Service... worker2 # [ 7.996043] systemd[1]: Starting Firewall... worker1 # [ 7.987970] systemd[1]: Listening on Journal Log Access Socket. client # [ 8.001772] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 7.994611] systemd[1]: Mounted Huge Pages File System. worker1 # [ 7.996116] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 7.993815] systemd[1]: Starting Load Kernel Modules... worker1 # [ 7.998645] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 8.010577] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 8.000843] systemd[1]: Mounted Kernel Trace File System. client # [ 7.992663] systemd[1]: Queued start job for default target Multi-User System. client # [ 7.994272] systemd[1]: systemd-journald.service: Deactivated successfully. worker1 # [ 8.005855] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 8.008223] systemd[1]: Mounted FUSE Control File System. client # [ 8.003549] systemd-modules-load[269]: Module 'atkbd' is built in worker2 # [ 8.023069] EXT4-fs (vda): re-mounted f9c82e2c-8814-4592-9a51-fe3bb167e285. client # [ 8.022501] systemd[1]: Started Journal Service. client # [ 8.006546] systemd-modules-load[269]: Module 'loop' is built in lb # [ 8.014675] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker1 # [ 8.023899] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 8.034908] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 8.025692] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 8.019822] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 8.037675] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 8.030773] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 8.038233] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 8.034919] systemd[1]: Queued start job for default target Multi-User System. client # [ 8.033871] systemd[1]: Mounted Huge Pages File System. lb # [ 8.040072] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 8.054667] systemd[1]: Starting Load/Save OS Random Seed... client # [ 8.037361] systemd[1]: Mounted POSIX Message Queue File System. client # [ 8.038300] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 8.057053] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 8.039087] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 8.060560] systemd[1]: Started Journal Service. lb # [ 8.049585] systemd[1]: Starting Coldplug All udev Devices... lb # [ 8.052323] systemd[1]: Listening on Journal Log Access Socket. client # [ 8.049591] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 8.052426] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 8.050633] systemd[1]: Mounted Kernel Configuration File System. client # [ 8.051495] systemd[1]: Finished Load Kernel Modules. lb # [ 8.063041] systemd[1]: Mounted Huge Pages File System. worker2 # [ 8.064162] systemd-modules-load[272]: Module 'atkbd' is built in lb # [ 8.068445] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 8.069806] systemd[1]: Mounted Kernel Debug File System. lb # [ 8.071233] systemd[1]: Mounted Kernel Trace File System. lb # [ 8.077007] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 8.078407] systemd[1]: Mounted Kernel Configuration File System. client # [ 8.071688] systemd[1]: Starting Apply Kernel Variables... client # [ 8.093378] EXT4-fs (vda): re-mounted 377aa17b-16db-4838-8736-3487cf584fcc. worker2 # [ 8.080877] systemd-modules-load[272]: Module 'loop' is built in client # [ 8.080280] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 8.091554] systemd[1]: Starting Flush Journal to Persistent Storage... worker1 # [ 8.101769] systemd[1]: Finished Load Kernel Modules. lb # [ 8.104070] systemd-journald[269]: Collecting audit messages is enabled. worker1 # [ 8.110378] systemd[1]: Starting Firewall... client # [ 8.102692] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 8.109475] systemd[1]: Listening on Disk Image Download Service Socket. lb # [ 8.116797] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 8.121644] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 8.130192] EXT4-fs (vda): re-mounted a7ecca0c-350f-44fc-aa57-2fc72629c496. client # [ 8.126912] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 8.127973] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 8.137598] systemd[1]: Finished Load Kernel Modules. lb # [ 8.122654] systemd[1]: Queued start job for default target Multi-User System. worker1 # [ 8.142441] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 8.143130] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 8.143450] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 8.140352] systemd[1]: Starting Load/Save OS Random Seed... worker1 # [ 8.149787] systemd-journald[272]: Collecting audit messages is enabled. client # [ 8.141259] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 8.127718] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 8.150917] systemd[1]: Starting Firewall... lb # [ 8.133750] systemd-modules-load[270]: Module 'atkbd' is built in lb # [ 8.134791] systemd-modules-load[270]: Module 'loop' is built in worker1 # [ 8.158761] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 8.159962] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 8.161697] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 8.155591] systemd-oomd[271]: No swap; memory pressure usage will be degraded lb # [ 8.163538] systemd[1]: Started Journal Service. client # [ 8.166417] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 8.173205] systemd-oomd[274]: No swap; memory pressure usage will be degraded lb # [ 8.183388] EXT4-fs (vda): re-mounted e419f460-32a8-4ac9-96c7-3ad177ebeeda. lb # [ 8.165596] systemd[1]: Mounted FUSE Control File System. lb # [ 8.166793] systemd-oomd[271]: No swap; memory pressure usage will be degraded worker2 # [ 8.201963] systemd-journald[271]: Received client request to flush runtime journal. client # [ 8.190769] systemd[1]: Mounted FUSE Control File System. worker1 # [ 8.187331] systemd[1]: Queued start job for default target Multi-User System. lb # [ 8.188487] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 8.210015] systemd[1]: Started Journal Service. lb # [ 8.203498] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 8.207307] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 8.242647] systemd-journald[268]: Received client request to flush runtime journal. worker1 # [ 8.217465] systemd-modules-load[273]: Module 'atkbd' is built in lb # [ 8.228571] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 8.230917] systemd-modules-load[273]: Module 'loop' is built in worker1 # [ 8.239645] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 8.254530] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 8.238286] systemd[1]: Starting Flush Journal to Persistent Storage... worker2 # [ 8.255612] systemd[1]: Reached target First Boot Complete. lb # [ 8.239289] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker2 # [ 8.262368] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 8.264337] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 8.265407] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 8.266319] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 8.267084] systemd[1]: Finished Flush Journal to Persistent Storage. lb # [ 8.252367] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 8.253213] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 8.284528] systemd[1]: Finished Load/Save OS Random Seed. client # [ 8.285484] systemd[1]: Reached target First Boot Complete. worker1 # [ 8.279792] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 8.293050] systemd[1]: Finished Apply Kernel Variables. client # [ 8.294466] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 8.292734] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 8.294059] systemd-oomd[274]: No swap; memory pressure usage will be degraded worker1 # [ 8.308863] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 8.309810] systemd[1]: Reached target First Boot Complete. lb # [ 8.329186] systemd-journald[269]: Received client request to flush runtime journal. worker1 # [ 8.314297] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 8.321154] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 8.358628] systemd-journald[272]: Received client request to flush runtime journal. lb # [ 8.373982] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 8.381634] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 8.382728] systemd[1]: Finished Apply Kernel Variables. lb # [ 8.388269] systemd[1]: Finished Flush Journal to Persistent Storage. lb # [ 8.390242] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 8.391031] systemd[1]: Reached target First Boot Complete. worker1 # [ 8.412640] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 8.451237] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 8.452502] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 8.456404] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 8.470837] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 8.471899] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 8.474386] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 8.538355] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 8.540278] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 8.576463] systemd-udevd[311]: Using default interface naming scheme 'v261'. worker1 # [ 8.594759] systemd-udevd[311]: Using default interface naming scheme 'v261'. worker2 # [ 8.610793] systemd[1]: Mounting /run/wrappers... lb # [ 8.630026] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 8.631045] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 8.633355] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 8.673286] systemd[1]: Mounted /run/wrappers. worker2 # [ 8.674118] systemd[1]: Reached target Local File Systems. worker2 # [ 8.684127] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 8.685214] systemd[1]: Starting register-nix-paths.service... worker2 # [ 8.689044] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 8.698531] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 8.713456] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 8.705176] systemd[1]: Mounting /run/wrappers... worker2 # [ 8.721034] systemd[1]: Starting Create System Files and Directories... worker1 # [ 8.736663] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 8.752851] systemd[1]: Mounting /run/wrappers... worker1 # [ 8.769203] systemd[1]: Mounted /run/wrappers. worker1 # [ 8.770059] systemd[1]: Reached target Local File Systems. worker1 # [ 8.782897] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 8.792459] systemd[1]: Starting register-nix-paths.service... worker1 # [ 8.804539] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 8.827596] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 8.813421] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 8.820754] systemd[1]: Mounted /run/wrappers. worker1 # [ 8.822612] systemd[1]: Starting Save Transient machine-id to Disk... lb # [ 8.821577] systemd[1]: Reached target Local File Systems. worker1 # [ 8.827334] systemd[1]: Starting Create System Files and Directories... lb # [ 8.826428] systemd[1]: Listening on Boot Loader Control Service Socket. lb # [ 8.834660] systemd[1]: Starting register-nix-paths.service... lb # [ 8.838019] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 8.838917] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 8.861440] systemd[1]: Finished Coldplug All udev Devices. lb # [ 8.849220] systemd-udevd[306]: Using default interface naming scheme 'v261'. lb # [ 8.850231] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 8.870397] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 8.873084] systemd[1]: Reached target Preparation for Local File Systems. client # [ 8.876461] systemd[1]: Mounting /run/wrappers... lb # [ 8.862718] systemd[1]: Starting Create System Files and Directories... client # [ 8.884008] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 8.918799] systemd[1]: Mounted /run/wrappers. client # [ 8.919607] systemd[1]: Reached target Local File Systems. worker2 # [ 8.928801] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 8.926343] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 8.916534] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 8.929961] systemd[1]: Starting register-nix-paths.service... worker1 # [ 8.923152] systemd[1]: Finished Save Transient machine-id to Disk. worker2 # [ 8.938140] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 8.937643] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 8.938889] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 8.952442] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 8.962217] systemd[1]: Starting Create System Files and Directories... worker2 # [ 8.964890] systemd[1]: Finished Create System Files and Directories. worker2 # [ 8.970395] systemd[1]: Starting Rebuild Journal Catalog... client # [ 8.969434] systemd-udevd[299]: Using default interface naming scheme 'v261'. worker2 # [ 8.985122] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 9.014635] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 9.032484] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 9.022190] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 9.032953] systemd[1]: Finished Create System Files and Directories. client # [ 9.047851] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 9.042269] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 9.052708] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 9.069084] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 9.099049] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 9.115832] systemd[1]: Finished Create System Files and Directories. client # [ 9.123637] systemd[1]: Starting Rebuild Journal Catalog... client # [ 9.129883] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 9.130936] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 9.124675] systemd[1]: Finished Create System Files and Directories. worker2 # [ 9.158112] systemd[1]: Finished Rebuild Journal Catalog. worker2 # [ 9.161293] systemd[1]: Starting Update is Completed... lb # [ 9.145443] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 9.151126] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 9.163141] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker1 # [ 9.230003] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 9.233772] systemd[1]: Starting Update is Completed... client # [ 9.246465] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker2 # [ 9.266141] systemd[1]: Finished Update is Completed. client # [ 9.286874] systemd[1]: Finished Rebuild Journal Catalog. client # [ 9.294506] systemd[1]: Starting Update is Completed... lb # [ 9.282108] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker1 # [ 9.328424] systemd[1]: Finished Update is Completed. client # [ 9.373366] systemd[1]: Finished Update is Completed. lb # [ 9.363434] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 9.373827] systemd[1]: Starting Update is Completed... lb # [ 9.493349] systemd[1]: Finished Update is Completed. client # [ 9.550531] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 9.592957] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 9.706488] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 9.710202] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.729672] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 9.734075] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 9.736973] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.821634] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 9.830554] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 9.834095] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.838678] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 9.858187] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. lb # [ 9.918269] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 9.967889] systemd[1]: Finished Firewall. worker2 # [ 9.993970] systemd[1]: Finished Firewall. client # [ 10.065480] (udev-worker)[352]: Network interface NamePolicy= disabled on kernel command line. client # [ 10.069424] (udev-worker)[347]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 10.071557] (udev-worker)[347]: Network interface NamePolicy= disabled on kernel command line. lb # [ 10.106329] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 10.125852] systemd[1]: Finished Coldplug All udev Devices. lb # [ 10.146178] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 10.150920] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 10.188096] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 10.189846] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 10.210312] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 10.201812] systemd[1]: Finished register-nix-paths.service. lb # [ 10.204436] systemd[1]: Reached target System Initialization. lb # [ 10.208465] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 10.209530] systemd[1]: Started niks3 garbage collection timer. lb # [ 10.210377] systemd[1]: Started Daily Cleanup of Temporary Directories. lb # [ 10.211307] systemd[1]: Reached target Timer Units. lb # [ 10.216965] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 10.222630] systemd[1]: Listening on niks3 server socket. lb # [ 10.223503] systemd[1]: Listening on Nix Daemon Socket. lb # [ 10.228942] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 10.230140] systemd[1]: Reached target Socket Units. lb # [ 10.230846] systemd[1]: Reached target Basic System. lb # [ 10.231574] systemd[1]: Started backdoor.service. lb # [ 10.242254] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 10.243302] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.253878] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.260388] systemd[1]: Finished Coldplug All udev Devices. lb # [ 10.292680] systemd[1]: Started Reset console on configuration changes. lb # [ 10.321815] systemd[1]: Starting resolvconf update... client # [ 10.335168] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 10.336788] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 10.342074] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 10.348282] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 10.349589] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 10.338437] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 10.351745] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 10.360569] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 10.361920] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 10.365841] systemd[1]: Finished register-nix-paths.service. client # [ 10.399422] mousedev: PS/2 mouse device common for all mice worker2 # [ 10.386797] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. lb # [ 10.371194] systemd[1]: Started rustfs.service. worker1 # [ 10.378566] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 10.380068] systemd[1]: Starting rustfs-bucket.service... lb # [ 10.390480] nsncd[448]: Sep 23 16:12:36.710 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.413605] systemd[1]: Finished register-nix-paths.service. worker2 # [ 10.415622] systemd[1]: Reached target System Initialization. lb # [ 10.404632] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.427165] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 10.421121] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 10.422146] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.439602] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 10.448695] systemd[1]: Reached target Timer Units. worker2 # [ 10.455021] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 10.444315] systemd[1]: Finished register-nix-paths.service. worker2 # [ 10.460360] systemd[1]: Listening on Nix Daemon Socket. worker2 # [ 10.465555] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 10.452179] systemd[1]: Reached target System Initialization. worker2 # [ 10.475804] systemd[1]: Reached target Socket Units. worker1 # [ 10.464638] systemd[1]: Started Discard unused filesystem blocks once a week. lb # connecting to host... worker2 # [ 10.486593] systemd[1]: Reached target Basic System. worker1 # [ 10.482503] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 10.500964] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 10.488163] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 10.514239] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.502578] systemd[1]: Reached target Timer Units. worker2 # [ 10.522281] systemd[1]: Starting Post-Boot Actions... lb # [ 10.510640] systemd[1]: Starting User Login Management... worker2 # [ 10.533183] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.521095] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 10.519554] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.546303] systemd[1]: Starting resolvconf update... worker1 # [ 10.533253] systemd[1]: Listening on Nix Daemon Socket. worker2 # [ 10.554697] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 10.544273] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 10.563569] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 10.577232] systemd[1]: Started backdoor.service. worker1 # [ 10.564355] systemd[1]: Reached target Socket Units. worker2 # [ 10.584719] systemd[1]: Finished Post-Boot Actions. worker1 # [ 10.582048] systemd[1]: Reached target Basic System. lb # [ 10.579949] (udev-worker)[366]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 10.590854] systemd[1]: Finished Import lastlog data into lastlog2 database. lb # [ 10.597744] (udev-worker)[344]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.600103] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 10.613736] nsncd[489]: Sep 23 16:12:36.936 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 10.611577] (udev-worker)[366]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 10.637667] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.624580] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 10.644316] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.650785] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.656406] systemd[1]: Starting User Login Management... worker1 # [ 10.645661] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 10.674575] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 10.669335] systemd[1]: Starting Post-Boot Actions... worker2 # connecting to host... worker1 # [ 10.678304] systemd[1]: Started Reset console on configuration changes. worker2 # [ 10.695178] dbus-broker-launch[493]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.703003] dbus-broker-launch[493]: NSS returned no entry for 'systemd-timesync' worker1 # [ 10.697643] systemd[1]: Starting resolvconf update... worker2 # [ 10.711083] dbus-broker-launch[493]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 10.710285] nix-grpc-daemon[487]: ts=2026-09-23T16:12:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 10.729081] systemd[1]: Started D-Bus System Message Bus. worker1 # [ 10.763825] nix-grpc-daemon[487]: ts=2026-09-23T16:12:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 10.784577] dbus-broker-launch[493]: Ready worker1 # [ 10.796918] nix-grpc-daemon[487]: ts=2026-09-23T16:12:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 10.839085] systemd-logind[502]: New seat seat0. client # [ 10.836335] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 10.840739] systemd[1]: Finished Create SUID/SGID Wrappers. worker1 # [ 10.832655] systemd[1]: Starting D-Bus System Message Bus... client # [ 10.845231] systemd[1]: Reached target System Initialization. worker2 # [ 10.848506] systemd[1]: Started User Login Management. client # [ 10.846733] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 10.851635] systemd[1]: Starting linger-users.service... client # [ 10.849036] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 10.850056] systemd[1]: Reached target Timer Units. client # [ 10.850785] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 10.851665] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 10.842628] nsncd[489]: Sep 23 16:12:37.146 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 10.859124] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 10.863512] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 10.863204] systemd[1]: Reached target Socket Units. lb # [ 10.848693] systemd-logind[478]: New seat seat0. worker2 # [ 10.868240] systemd[1]: Stopping Host and Network Name Lookups... client # [ 10.865934] systemd[1]: Reached target Basic System. client # [ 10.866796] systemd[1]: Started backdoor.service. worker2 # [ 10.869248] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 10.854546] nix-grpc-daemon[487]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 10.870104] systemd[1]: Stopping User and Group Name Lookups... client # [ 10.867485] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 10.870914] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... lb # [ 10.859732] systemd[1]: Started User Login Management. worker1 # [ 10.863577] nix-grpc-daemon[487]: E0000 00:00:1790179957.073070 487 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. client # [ 10.878500] systemd[1]: Starting nftables firewall... lb # [ 10.867067] systemd[1]: Starting linger-users.service... worker2 # [ 10.887190] systemd[1]: nscd.service: Deactivated successfully. lb # [ 10.873278] dbus-broker-launch[476]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 10.875670] nix-grpc-daemon[487]: E0000 00:00:1790179957.081881 487 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 10.896514] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 10.897446] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.881912] dbus-broker-launch[476]: NSS returned no entry for 'systemd-timesync' lb # [ 10.882948] dbus-broker-launch[476]: Invalid user-name in /nix/store/4yiprls1sll47jid3xzq0m9nmi1qz2ww-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 10.888626] nix-grpc-daemon[487]: ts=2026-09-23T16:12:37Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket client # [ 10.903907] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 10.911743] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.900400] nix-grpc-daemon[487]: ts=2026-09-23T16:12:37Z level=info event=scheduler_lock_lost error="error: http://lb:5751/api/farm/lead: Could not connect to server" worker2 # [ 10.922135] (udev-worker)[485]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.911669] nix-grpc-daemon[487]: E0000 00:00:1790179957.098979 542 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 10.918446] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 10.926238] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 10.927190] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 10.928375] nix-grpc-daemon[487]: E0000 00:00:1790179957.128171 542 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 10.947396] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 10.953036] systemd[1]: Finished linger-users.service. lb # [ 10.937141] systemd[1]: Stopping User and Group Name Lookups... lb # [ 10.938003] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 10.938322] nix-grpc-daemon[487]: E0000 00:00:1790179957.128908 542 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. client # [ 10.951621] systemd[1]: Started Reset console on configuration changes. lb # [ 10.938859] systemd[1]: nscd.service: Deactivated successfully. client # [ 10.966655] systemd[1]: Starting resolvconf update... worker2 # [ 10.970121] (udev-worker)[361]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 10.955494] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker1 # [ 10.956408] nix-grpc-daemon[487]: E0000 00:00:1790179957.160492 542 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 10.961729] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 10.980684] (udev-worker)[361]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.965663] nix-grpc-daemon[487]: ts=2026-09-23T16:12:37Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: FAILED_PRECONDITION: ipv4:192.168.1.2:50051: connect failed: addr: ipv4:192.168.1.2:50051 error: Network is unreachable" worker2 # [ 10.982235] nsncd[561]: Sep 23 16:12:37.301 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.991099] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.994584] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.995506] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.985997] systemd[1]: Finished Post-Boot Actions. worker1 # [ 10.989052] dbus-broker-launch[493]: Looking up NSS user entry for 'systemd-timesync'... lb # [ 10.991080] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.995478] dbus-broker-launch[493]: NSS returned no entry for 'systemd-timesync' worker2 # [ 11.012419] systemd[1]: Finished resolvconf update. worker2 # [ 11.013178] systemd[1]: Reached target Preparation for Network. worker1 # [ 10.998854] dbus-broker-launch[493]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 11.017765] systemd[1]: Starting DHCP Client... worker2 # [ 11.024215] systemd[1]: Starting Extra networking commands.... worker1 # [ 11.010628] systemd[1]: Started Nix worker-protocol over gRPC. client # [ 11.027808] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 11.016665] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 11.016871] dbus-broker-launch[476]: Ready worker1 # [ 11.023459] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 11.026789] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 11.031060] systemd[1]: Starting User Login Management... client # connecting to host... worker1 # [ 11.035075] nix-grpc-daemon[487]: ts=2026-09-23T16:12:37Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: FAILED_PRECONDITION: ipv4:192.168.1.2:50051: connect failed: addr: ipv4:192.168.1.2:50051 error: Network is unreachable" client # [ 11.055480] systemd[1]: Finished Post-Boot Actions. lb # [ 11.042498] systemd[1]: linger-users.service: Deactivated successfully. client # [ 11.058172] nsncd[451]: Sep 23 16:12:37.387 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 11.050342] systemd[1]: Finished Import lastlog data into lastlog2 database. lb # [ 11.049174] systemd[1]: Finished linger-users.service. worker1 # [ 11.054085] systemd[1]: Started D-Bus System Message Bus. worker1 # [ 11.058742] dbus-broker-launch[493]: Ready client # [ 11.073830] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 11.062669] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 11.070770] systemd-logind[551]: New seat seat0. worker1 # [ 11.079881] systemd[1]: Started User Login Management. worker1 # [ 11.082754] systemd[1]: Starting linger-users.service... worker1 # [ 11.092708] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 11.101091] systemd[1]: Stopping Host and Network Name Lookups... client: Guest shell says: b'Spawning backdoor root shell...\n' worker1 # [ 11.107124] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 11.099337] nsncd[558]: Sep 23 16:12:37.422 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 11.122918] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 11.115951] systemd[1]: Stopping User and Group Name Lookups... client # [ 11.128813] systemd[1]: Reached target User and Group Name Lookups. lb # [ 11.107150] systemd[1]: Started Name Service Cache Daemon (nsncd). client: connected to guest root shell client # [ 11.155772] systemd[1]: Starting User Login Management... worker1 # [ 11.126123] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... lb # [ 11.111625] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 11.130039] systemd[1]: nscd.service: Deactivated successfully. lb # [ 11.114284] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 11.133003] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 11.142978] systemd[1]: Finished resolvconf update. worker1 # [ 11.137954] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 11.155854] systemd[1]: Starting DHCP Client... client: (connecting took 11.63 seconds) client: (finished: waiting for the VM to finish booting, in 11.63 seconds) worker1 # [ 11.174504] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 11.178428] systemd[1]: Finished linger-users.service. client # [ 11.217483] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 11.205865] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 11.211521] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 11.214534] nsncd[610]: Sep 23 16:12:37.545 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 11.220925] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 11.226474] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker1 # [ 11.233415] systemd[1]: Started backdoor.service. worker2 # [ 11.249005] dhcpcd[597]: dhcpcd-10.3.2 starting worker1 # [ 11.241801] systemd[1]: Finished resolvconf update. worker2 # [ 11.259568] dhcpcd[637]: dev: loaded udev lb # [ 11.243471] systemd-logind[478]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.257482] dbus-broker-launch[459]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 11.247875] systemd[1]: Reached target Preparation for Network. worker1 # [ 11.258054] systemd[1]: Starting DHCP Client... worker1 # [ 11.260915] systemd[1]: Starting Extra networking commands.... client # [ 11.272981] dbus-broker-launch[459]: NSS returned no entry for 'systemd-timesync' client # [ 11.274086] dbus-broker-launch[459]: Invalid user-name in /nix/store/6wvgzhp3xj1v1xybf1ixpbqi5gif5fbv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 11.309716] 8021q: 802.1Q VLAN Support v1.8 lb # [ 11.307388] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 11.337772] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 11.356082] systemd[1]: Starting Address configuration of eth1... worker1 # connecting to host... client # [ 11.364122] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 11.375059] systemd[1]: Finished Extra networking commands.. worker2 # [ 11.375983] systemd[1]: Reached target Network. worker2 # [ 11.379608] systemd[1]: Starting Permit User Sessions... client # [ 11.402226] systemd-logind[480]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker2 # [ 11.422396] cfg80211: Loading compiled-in X.509 certificates for regulatory database client # [ 11.420313] systemd-logind[480]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.421823] systemd-logind[480]: New seat seat0. client # [ 11.429796] systemd[1]: Started User Login Management. worker2 # [ 11.450356] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 11.430953] dbus-broker-launch[459]: Ready worker2 # [ 11.450888] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 11.453542] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 11.453888] cfg80211: failed to load regulatory.db client # [ 11.440274] systemd[1]: Starting linger-users.service... worker1 # [ 11.427689] nix-grpc-daemon[487]: ts=2026-09-23T16:12:37Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: FAILED_PRECONDITION: ipv4:192.168.1.2:50051: connect failed: addr: ipv4:192.168.1.2:50051 error: Network is unreachable" lb # [ 11.438006] dhcpcd[605]: dhcpcd-10.3.2 starting worker2 # [ 11.468125] systemd[1]: Finished Permit User Sessions. lb # [ 11.453141] dhcpcd[620]: dev: loaded udev worker2 # [ 11.474264] systemd[1]: Started Getty on tty1. worker2 # [ 11.475037] systemd[1]: Reached target Login Prompts. worker2 # [ 11.489616] systemd-logind[502]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.489539] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 11.490584] systemd[1]: Stopping Host and Network Name Lookups... client # [ 11.491404] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 11.482762] (udev-worker)[342]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 11.504434] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 11.492479] (udev-worker)[342]: Network interface NamePolicy= disabled on kernel command line. client # [ 11.505376] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 11.506269] systemd[1]: nscd.service: Deactivated successfully. client # [ 11.507055] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 11.532680] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 11.506560] (udev-worker)[351]: Network interface NamePolicy= disabled on kernel command line. lb # [ 11.530849] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 11.535658] network-addresses-eth1-start[650]: adding address 192.168.1.4/24... done worker1 # [ 11.529759] dhcpcd[647]: dhcpcd-10.3.2 starting worker2 # [ 11.567738] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 11.553106] network-addresses-eth1-start[650]: adding address 2001:db8:1::4/64... done worker1 # [ 11.539866] dhcpcd[689]: dev: loaded udev client # [ 11.553581] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 11.559452] dhcpcd[637]: eth0: waiting for carrier worker2 # [ 11.562380] dhcpcd[637]: eth0: waiting for carrier worker2 # [ 11.563119] dhcpcd[637]: eth0: carrier acquired client # [ 11.565772] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 11.577826] systemd[1]: Finished Address configuration of eth1. client # [ 11.577676] systemd[1]: Finished linger-users.service. worker2 # [ 11.580458] dhcpcd[637]: DUID 00:01:00:01:32:46:b6:f5:52:54:00:12:34:56 worker2 # [ 11.581442] dhcpcd[637]: eth0: IAID 00:12:34:56 worker2 # [ 11.582087] dhcpcd[637]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 11.598711] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 11.615475] systemd[1]: Finished Extra networking commands.. worker1 # [ 11.625313] systemd[1]: Reached target Network. worker1 # [ 11.629238] systemd[1]: Starting Permit User Sessions... client # [ 11.642096] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 11.645928] nsncd[545]: Sep 23 16:12:37.973 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 11.653915] systemd[1]: Finished resolvconf update. client # [ 11.654695] systemd[1]: Reached target Host and Network Name Lookups. client # [ 11.655571] systemd[1]: Reached target User and Group Name Lookups. client # [ 11.662383] systemd[1]: Starting DHCP Client... lb # [ 11.686559] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 11.699803] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 11.678568] systemd[1]: Finished Firewall. lb # [ 11.681718] systemd[1]: Reached target Preparation for Network. worker2 # [ 11.718129] mousedev: PS/2 mouse device common for all mice lb # [ 11.706148] mousedev: PS/2 mouse device common for all mice lb # [ 11.692230] systemd[1]: Starting Address configuration of eth1... lb # [ 11.700988] systemd[1]: Starting Extra networking commands.... worker1 # [ 11.707450] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 11.715499] systemd[1]: Starting Address configuration of eth1... worker1 # [ 11.742094] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 11.742616] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker1 # [ 11.745246] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 11.745632] cfg80211: failed to load regulatory.db worker1 # [ 11.729707] systemd[1]: Finished Permit User Sessions. worker1 # [ 11.738686] systemd[1]: Started Getty on tty1. worker1 # [ 11.739420] systemd[1]: Reached target Login Prompts. lb # [ 11.776222] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 11.756732] nix-grpc-daemon[487]: ts=2026-09-23T16:12:38Z level=info event=scheduler_lock_lost error="error: http://lb:5751/api/farm/lead: Could not connect to server" lb # [ 11.777876] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 11.783969] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 11.784288] cfg80211: failed to load regulatory.db worker2 # [ 11.822692] systemd-logind[502]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker1 # [ 11.842769] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 11.853693] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 11.831918] systemd-logind[478]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker1 # [ 11.836118] dhcpcd[689]: eth0: waiting for carrier worker1 # [ 11.839009] dhcpcd[689]: eth0: waiting for carrier worker1 # [ 11.841854] dhcpcd[689]: eth0: carrier acquired worker1 # [ 11.851487] network-addresses-eth1-start[705]: adding address 192.168.1.3/24... done worker1 # [ 11.857935] dhcpcd[689]: DUID 00:01:00:01:32:46:b6:f6:52:54:00:12:34:56 worker1 # [ 11.860269] dhcpcd[689]: eth0: IAID 00:12:34:56 worker1 # [ 11.864107] dhcpcd[689]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 11.883685] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 11.878662] dhcpcd[577]: dhcpcd-10.3.2 starting worker1 # [ 11.867850] network-addresses-eth1-start[705]: adding address 2001:db8:1::3/64... done client # [ 11.887442] dhcpcd[585]: dev: loaded udev worker1 # [ 11.889306] systemd[1]: Finished Address configuration of eth1. lb # [ 11.892545] network-addresses-eth1-start[636]: adding address 192.168.1.2/24... done lb # [ 11.911227] network-addresses-eth1-start[636]: adding address 2001:db8:1::2/64... done lb # [ 11.934912] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 11.946424] 8021q: 802.1Q VLAN Support v1.8 lb # [ 11.919140] dhcpcd[620]: libudev: received NULL device lb # [ 11.919971] dhcpcd[620]: eth0: waiting for carrier lb # [ 11.928466] dhcpcd[620]: libudev: received NULL device lb # [ 11.929361] dhcpcd[620]: eth0: carrier acquired lb # [ 11.941074] dhcpcd[620]: DUID 00:01:00:01:32:46:b6:f6:52:54:00:12:34:56 lb # [ 11.945239] dhcpcd[620]: eth0: IAID 00:12:34:56 lb # [ 11.945937] dhcpcd[620]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 11.950449] systemd[1]: Finished Address configuration of eth1. client # [ 12.020267] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 12.022492] systemd-logind[551]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 12.054796] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 12.055301] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 12.058038] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 12.058350] cfg80211: failed to load regulatory.db client # [ 12.161495] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 12.146644] dhcpcd[585]: eth0: waiting for carrier client # [ 12.149334] dhcpcd[585]: libudev: received NULL device client # [ 12.151259] dhcpcd[585]: libudev: received NULL device client # [ 12.154031] dhcpcd[585]: eth0: carrier acquired client # [ 12.162137] systemd[1]: Finished nftables firewall. client # [ 12.165317] systemd[1]: Reached target Preparation for Network. worker1 # [ 12.177939] mousedev: PS/2 mouse device common for all mice client # [ 12.172955] systemd[1]: Starting Address configuration of eth1... client # [ 12.177294] dhcpcd[585]: DUID 00:01:00:01:32:46:b6:f6:52:54:00:12:34:56 client # [ 12.178493] dhcpcd[585]: eth0: IAID 00:12:34:56 client # [ 12.179129] dhcpcd[585]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 12.188259] systemd[1]: Starting Extra networking commands.... worker1 # [ 12.220411] systemd-logind[551]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker1 # [ 12.228166] nix-grpc-daemon[487]: ts=2026-09-23T16:12:38Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: FAILED_PRECONDITION: ipv4:192.168.1.2:50051: connect failed: addr: ipv4:192.168.1.2:50051 error: Network is unreachable" client # [ 12.288349] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 12.285570] dhcpcd[585]: eth0: soliciting a DHCP lease client # [ 12.292544] dhcpcd[585]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 12.300333] dhcpcd[585]: eth0: probing address 10.0.2.15/24 client # [ 12.303543] network-addresses-eth1-start[620]: adding address 192.168.1.1/24... done client # [ 12.324688] network-addresses-eth1-start[620]: adding address 2001:db8:1::1/64... done client # [ 12.353402] systemd[1]: Finished Address configuration of eth1. worker2 # [ 12.436170] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client # [ 12.425067] systemd[1]: Finished Extra networking commands.. client # [ 12.429489] systemd[1]: Reached target Network. client # [ 12.435032] systemd[1]: Starting Permit User Sessions... client # [ 12.512934] systemd[1]: Finished Permit User Sessions. client # [ 12.521215] systemd[1]: Started Getty on tty1. client # [ 12.525195] systemd[1]: Reached target Login Prompts. client # [ 12.585956] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client: (finished: waiting for unit network-addresses-eth1.service, in 13.16 seconds) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } lb # [ 12.719290] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker2 # [ 12.735898] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 12.748084] systemd[1]: Starting Virtual Console Setup... worker2 # [ 12.769459] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 12.775840] systemd[1]: Stopped Virtual Console Setup. client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.07 seconds) worker2 # [ 12.780926] systemd[1]: Starting Virtual Console Setup... lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker2 # [ 12.831726] systemd-logind[502]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 12.858839] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client # [ 12.894112] systemd[1]: Starting Virtual Console Setup... worker2 # [ 12.918531] systemd-vconsole-setup[693]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 12.917140] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 12.922650] systemd[1]: Finished Virtual Console Setup. client # [ 12.920236] systemd[1]: Stopped Virtual Console Setup. client # [ 12.928203] systemd[1]: Starting Virtual Console Setup... client # [ 12.954806] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb: (finished: waiting for unit network-addresses-eth1.service, in 0.19 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } client # [ 12.998800] systemd-logind[480]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.06 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) client # [ 13.084741] systemd-vconsole-setup[696]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 13.088093] systemd[1]: Finished Virtual Console Setup. lb # [ 13.080503] systemd[1]: Finished Extra networking commands.. lb # [ 13.083263] systemd[1]: Reached target Network. lb # [ 13.090339] systemd[1]: Starting PostgreSQL Server... worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } lb # [ 13.110939] systemd[1]: Starting Permit User Sessions... worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.04 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb # [ 13.160964] systemd[1]: Finished Permit User Sessions. lb # [ 13.169932] systemd[1]: Started Getty on tty1. lb # [ 13.170654] systemd[1]: Reached target Login Prompts. worker1 # [ 13.188983] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1 # [ 13.193498] systemd[1]: Starting Virtual Console Setup... lb # [ 13.208592] systemd[1]: Starting Virtual Console Setup... worker1 # [ 13.215828] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } worker1 # [ 13.221926] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 13.228641] systemd[1]: Starting Virtual Console Setup... lb # [ 13.229976] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 13.231071] systemd[1]: Stopped Virtual Console Setup. lb # [ 13.258309] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.04 seconds) lb: waiting for unit niks3.service lb # [ 13.267090] systemd[1]: Starting Virtual Console Setup... worker2 # [ 13.289070] dhcpcd[637]: eth0: soliciting an IPv6 router worker2 # [ 13.290140] dhcpcd[637]: eth0: Router Advertisement from fe80::2 worker2 # [ 13.291173] dhcpcd[637]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 13.292394] dhcpcd[637]: eth0: adding route to fec0::/64 worker2 # [ 13.293318] dhcpcd[637]: eth0: adding default route via fe80::2 worker1 # [ 13.298446] systemd-logind[551]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 13.385177] dhcpcd[637]: eth0: soliciting a DHCP lease worker1 # [ 13.372377] systemd-vconsole-setup[757]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 13.388988] dhcpcd[637]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 13.374922] systemd-logind[478]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 13.377430] systemd[1]: Finished Virtual Console Setup. worker2 # [ 13.392534] dhcpcd[637]: eth0: probing address 10.0.2.15/24 lb # [ 13.429987] postgresql-pre-start[745]: The files belonging to this database system will be owned by user "postgres". lb # [ 13.431404] postgresql-pre-start[745]: This user must also own the server process. lb # [ 13.436128] postgresql-pre-start[745]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 13.437386] postgresql-pre-start[745]: The default database encoding has accordingly been set to "UTF8". lb # [ 13.438559] postgresql-pre-start[745]: The default text search configuration will be set to "english". lb # [ 13.439723] postgresql-pre-start[745]: Data page checksums are enabled. lb # [ 13.441699] postgresql-pre-start[745]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 13.442964] postgresql-pre-start[745]: creating subdirectories ... ok lb # [ 13.443821] postgresql-pre-start[745]: selecting dynamic shared memory implementation ... posix lb # [ 13.507643] postgresql-pre-start[745]: selecting default "max_connections" ... 100 lb # [ 13.568485] postgresql-pre-start[745]: selecting default "shared_buffers" ... 128MB lb # [ 13.631900] systemd-vconsole-setup[744]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 13.636323] systemd[1]: Finished Virtual Console Setup. lb # [ 13.714967] dhcpcd[620]: eth0: soliciting a DHCP lease lb # [ 13.716549] dhcpcd[620]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 13.724167] dhcpcd[620]: eth0: probing address 10.0.2.15/24 worker1 # [ 13.743949] dhcpcd[689]: eth0: soliciting a DHCP lease worker1 # [ 13.749491] dhcpcd[689]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 13.752539] dhcpcd[689]: eth0: probing address 10.0.2.15/24 worker1 # [ 13.803248] dhcpcd[689]: eth0: soliciting an IPv6 router worker1 # [ 13.807039] dhcpcd[689]: eth0: Router Advertisement from fe80::2 worker1 # [ 13.809729] dhcpcd[689]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 13.812515] dhcpcd[689]: eth0: adding route to fec0::/64 worker1 # [ 13.814770] dhcpcd[689]: eth0: adding default route via fe80::2 worker1 # [ 13.829992] nix-grpc-daemon[487]: ts=2026-09-23T16:12:40Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" lb # [ 14.198910] dhcpcd[620]: eth0: soliciting an IPv6 router lb # [ 14.201209] dhcpcd[620]: eth0: Router Advertisement from fe80::2 lb # [ 14.202064] dhcpcd[620]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 14.202945] dhcpcd[620]: eth0: adding route to fec0::/64 lb # [ 14.203663] dhcpcd[620]: eth0: adding default route via fe80::2 lb # [ 14.235102] postgresql-pre-start[745]: selecting default time zone ... UTC lb # [ 14.237777] postgresql-pre-start[745]: creating configuration files ... ok lb # [ 14.466274] postgresql-pre-start[745]: running bootstrap script ... ok client # [ 14.758860] dhcpcd[585]: eth0: soliciting an IPv6 router client # [ 14.762563] dhcpcd[585]: eth0: Router Advertisement from fe80::2 client # [ 14.765562] dhcpcd[585]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 14.768404] dhcpcd[585]: eth0: adding route to fec0::/64 client # [ 14.770634] dhcpcd[585]: eth0: adding default route via fe80::2 lb # [ 14.987482] postgresql-pre-start[745]: performing post-bootstrap initialization ... ok lb # [ 15.183321] postgresql-pre-start[745]: syncing data to disk ... ok lb # [ 15.185613] postgresql-pre-start[745]: initdb: warning: enabling "trust" authentication for local connections lb # [ 15.186931] postgresql-pre-start[745]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 15.189125] postgresql-pre-start[745]: Success. You can now start the database server using: lb # [ 15.190308] postgresql-pre-start[745]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 15.284524] postgres[777]: [777] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 15.287275] postgres[777]: [777] LOG: listening on IPv6 address "::1", port 5432 lb # [ 15.288677] postgres[777]: [777] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 15.290403] postgres[777]: [777] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 15.303554] postgres[786]: [786] LOG: database system was shut down at 2026-09-23 16:12:41 GMT lb # [ 15.308453] postgres[777]: [777] LOG: database system is ready to accept connections lb # [ 15.313319] systemd[1]: Started PostgreSQL Server. lb # [ 15.319720] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 15.487869] postgresql-setup-start[797]: CREATE DATABASE lb # [ 15.527701] postgresql-setup-start[802]: CREATE ROLE lb # [ 15.544544] postgresql-setup-start[804]: ALTER DATABASE lb # [ 15.550009] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 15.551963] systemd[1]: Reached target PostgreSQL. worker1 # [ 17.030777] nix-grpc-daemon[487]: ts=2026-09-23T16:12:43Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv6:%5B2001:db8:1::2%5D:50051: Failed to connect to remote host: Connection refused" client # [ 18.065951] dhcpcd[585]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 18.066324] dhcpcd[585]: eth0: adding route to 10.0.2.0/24 client # [ 18.066470] dhcpcd[585]: eth0: adding default route via 10.0.2.2 worker2 # [ 18.121460] dhcpcd[637]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 18.125644] dhcpcd[637]: eth0: adding route to 10.0.2.0/24 worker2 # [ 18.132446] dhcpcd[637]: eth0: adding default route via 10.0.2.2 client # [ 18.202543] systemd[1]: Started DHCP Client. client # [ 18.205179] systemd[1]: Reached target Multi-User System. client # [ 18.206641] systemd[1]: Startup finished in 1.249s (kernel) + 4.952s (initrd) + 12.004s (userspace) = 18.206s. worker2 # [ 18.296372] systemd[1]: Started DHCP Client. worker2 # [ 18.299451] systemd[1]: Reached target Network is Online. worker2 # [ 18.302625] systemd[1]: Starting scheduler-token.service... worker2 # [ 18.346850] systemd[1]: scheduler-token.service: Main process exited, code=exited, status=7/NOTRUNNING worker2 # [ 18.350019] systemd[1]: scheduler-token.service: Failed with result 'exit-code'. worker2 # [ 18.351330] systemd[1]: Failed to start scheduler-token.service. worker2 # [ 18.352672] systemd[1]: Dependency failed for Nix worker-protocol over gRPC. worker2 # [ 18.353893] systemd[1]: nix-grpc-daemon.service: Job nix-grpc-daemon.service/start failed with result 'dependency'. worker2 # [ 18.355336] systemd[1]: scheduler-token.service: Consumed 26ms CPU time over 47ms wall clock time, 3.1M memory peak, 40B incoming IP traffic, 60B outgoing IP traffic. worker2 # [ 18.357523] systemd[1]: Reached target Multi-User System. worker2 # [ 18.364334] systemd[1]: Startup finished in 1.252s (kernel) + 4.952s (initrd) + 12.159s (userspace) = 18.364s. lb # [ 18.451365] dhcpcd[620]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 18.455572] dhcpcd[620]: eth0: adding route to 10.0.2.0/24 lb # [ 18.461200] dhcpcd[620]: eth0: adding default route via 10.0.2.2 lb # [ 18.576688] systemd[1]: Started DHCP Client. lb # [ 18.580502] systemd[1]: Reached target Network is Online. lb # [ 18.582968] systemd[1]: Started Envoy reverse proxy. lb # [ 18.589090] systemd[1]: Started mock-oidc.service. worker1 # [ 18.589775] dhcpcd[689]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 18.592984] dhcpcd[689]: eth0: adding route to 10.0.2.0/24 worker1 # [ 18.595353] dhcpcd[689]: eth0: adding default route via 10.0.2.2 worker1 # [ 18.733717] systemd[1]: Started DHCP Client. worker1 # [ 18.735452] systemd[1]: Reached target Multi-User System. worker1 # [ 18.736771] systemd[1]: Startup finished in 1.242s (kernel) + 5.055s (initrd) + 12.437s (userspace) = 18.735s. lb # [ 18.849230] mock-oidc-server[896]: Mock OIDC Server running lb # [ 18.851958] mock-oidc-server[896]: OIDC Address: 192.168.1.2:8080 lb # [ 18.856454] mock-oidc-server[896]: Issue Address: 0.0.0.0:8081 lb # [ 18.857660] mock-oidc-server[896]: Issuer: http://192.168.1.2:8080/oidc lb # [ 18.860074] mock-oidc-server[896]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 18.863761] mock-oidc-server[896]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 18.865602] mock-oidc-server[896]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 19.002873] envoy[895]: [2026-09-23 16:12:45.328][895][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120) lb # [ 19.005710] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 19.007170] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 19.009400] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd lb # [ 19.014822] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 19.018823] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 19.024170] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 19.025825] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 19.032618] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 19.035676] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 19.085824] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 19.089427] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 19.092766] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 19.094964] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 19.097596] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 19.104872] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 19.108703] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 19.111156] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 19.112969] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 19.115037] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 19.116801] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 19.119423] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 19.121649] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 19.129748] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 19.131563] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 19.135339] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 19.138200] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 19.141665] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 19.143484] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 19.146291] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 19.148368] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 19.150119] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 19.154738] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 19.157844] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 19.163493] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 19.166391] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 19.168121] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 19.169800] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 19.171903] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 19.175217] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 19.178531] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 19.180465] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 19.182326] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 19.184216] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 19.186306] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 19.188000] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 19.191376] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 19.193777] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 19.198622] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 19.202189] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 19.206223] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 19.208342] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 19.210350] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 19.224913] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 19.232583] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 19.235263] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 19.237604] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 19.240211] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 19.242227] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 19.244882] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 19.253227] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 19.254747] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 19.261907] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 19.267685] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 19.272790] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 19.274931] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 19.277628] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 19.286446] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder lb # [ 19.289063] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 19.294865] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 19.297336] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 19.301522] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 19.304153] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 19.305743] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 19.316590] envoy[895]: [2026-09-23 16:12:45.330][895][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 19.318420] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 19.320054] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 19.321855] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 19.324774] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 19.326466] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 19.328514] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 19.331129] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 19.333581] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 19.335455] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 19.337427] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 19.339856] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 19.341744] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 19.344301] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 19.346359] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 19.349641] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 19.351310] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 19.353077] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher lb # [ 19.355436] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 19.358050] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 19.360297] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 19.362066] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 19.364791] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 19.367129] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 19.369026] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 19.371063] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 19.376026] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 19.378545] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default lb # [ 19.380481] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 19.382042] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 19.384619] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 19.392729] envoy[895]: [2026-09-23 16:12:45.331][895][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 19.394562] envoy[895]: [2026-09-23 16:12:45.357][895][info][main] [source/server/server.cc:531] HTTP header map info: lb # [ 19.396019] envoy[895]: [2026-09-23 16:12:45.358][895][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 19.412647] envoy[895]: [2026-09-23 16:12:45.358][895][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: lb # [ 19.415471] envoy[895]: [2026-09-23 16:12:45.358][895][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 19.424546] envoy[895]: [2026-09-23 16:12:45.358][895][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 19.426299] envoy[895]: [2026-09-23 16:12:45.409][895][info][main] [source/server/server.cc:968] runtime: {} lb # [ 19.427596] envoy[895]: [2026-09-23 16:12:45.414][895][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. lb # [ 19.429404] envoy[895]: [2026-09-23 16:12:45.414][895][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 19.431119] envoy[895]: [2026-09-23 16:12:45.414][895][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 lb # [ 19.432682] envoy[895]: [2026-09-23 16:12:45.415][895][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 19.434254] envoy[895]: [2026-09-23 16:12:45.415][895][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 19.435802] envoy[895]: [2026-09-23 16:12:45.415][895][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) lb # [ 19.437432] envoy[895]: [2026-09-23 16:12:45.577][895][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) lb # [ 19.438933] envoy[895]: [2026-09-23 16:12:45.596][895][info][config] [source/server/configuration_impl.cc:164] loading stats configuration lb # [ 19.440524] envoy[895]: [2026-09-23 16:12:45.597][895][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 19.443543] envoy[895]: [2026-09-23 16:12:45.597][895][info][main] [source/server/server.cc:1077] starting main dispatch loop lb # [ 19.445036] envoy[895]: [2026-09-23 16:12:45.597][895][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization lb # [ 19.446656] envoy[895]: [2026-09-23 16:12:45.597][895][info][upstream] [source/common/upstream/cluster_manager_impl.cc:227] cm init: initializing secondary clusters worker2 # [ 19.553719] systemd[1]: scheduler-token.service: Scheduled restart job, restart counter is at 1. worker2 # [ 19.566095] systemd[1]: Starting scheduler-token.service... worker2 # [ 19.657992] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 19.661849] systemd[1]: Finished scheduler-token.service. worker2 # [ 19.665848] systemd[1]: scheduler-token.service: Consumed 57ms CPU time over 1.311s wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker1 # [ 22.031507] nix-grpc-daemon[487]: ts=2026-09-23T16:12:48Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" lb # [ 26.233508] rustfs-bucket-start[975]: mb s3://farm lb # [ 26.251750] systemd[1]: Finished rustfs-bucket.service. lb # [ 26.267161] systemd[1]: Starting niks3 server... lb # [ 26.439296] postgres[991]: [991] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 26.440724] postgres[991]: [991] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC lb # [ 26.475054] niks3-server[985]: 2026/09/23 16:12:52 OK 20241026095416_initial_model.sql (21.92ms) lb # [ 26.478919] niks3-server[985]: 2026/09/23 16:12:52 OK 20251210153512_drop_unused_gin_index.sql (4.23ms) lb # [ 26.485811] niks3-server[985]: 2026/09/23 16:12:52 OK 20251218171726_add_pins.sql (6.76ms) lb # [ 26.491709] niks3-server[985]: 2026/09/23 16:12:52 OK 20260628120000_add_object_size_and_stats.sql (5.91ms) lb # [ 26.497823] niks3-server[985]: 2026/09/23 16:12:52 OK 20260905000000_add_claims.sql (6.07ms) lb # [ 26.502261] niks3-server[985]: 2026/09/23 16:12:52 OK 20260920000000_drop_claims.sql (4.41ms) lb # [ 26.505618] niks3-server[985]: 2026/09/23 16:12:52 OK 20260923120000_add_pushes.sql (3.2ms) lb # [ 26.508280] niks3-server[985]: 2026/09/23 16:12:52 goose: successfully migrated database to version: 20260923120000 lb # [ 26.511724] niks3-server[985]: 2026/09/23 16:12:52 OK 1_commit_pending_closure.sql (5.99ms) lb # [ 26.515089] niks3-server[985]: 2026/09/23 16:12:52 OK 2_object_stats_trigger.sql (3.37ms) lb # [ 26.518386] niks3-server[985]: 2026/09/23 16:12:52 OK 3_commit_push.sql (2.84ms) lb # [ 26.519453] niks3-server[985]: 2026/09/23 16:12:52 goose: up to current file version: 3 lb # [ 26.522103] niks3-server[985]: 2026/09/23 16:12:52 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 26.549114] niks3-server[985]: 2026/09/23 16:12:52 INFO Read proxy enabled — serving cache objects from S3 lb # [ 26.556034] niks3-server[985]: 2026/09/23 16:12:52 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 26.562678] niks3-server[985]: 2026/09/23 16:12:52 INFO systemd watchdog enabled interval=15s lb # [ 26.563856] niks3-server[985]: 2026/09/23 16:12:52 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 26.565663] systemd[1]: Started niks3 server. lb # [ 26.566566] systemd[1]: Reached target Multi-User System. lb # [ 26.567408] systemd[1]: Startup finished in 1.251s (kernel) + 5.009s (initrd) + 20.299s (userspace) = 26.559s. lb: (finished: waiting for unit niks3.service, in 13.43 seconds) lb: waiting for TCP port 5751 on localhost lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.04 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.03 seconds) lb: waiting for TCP port 50051 on localhost worker1 # [ 27.033300] nix-grpc-daemon[487]: ts=2026-09-23T16:12:53Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker1 # [ 32.035409] nix-grpc-daemon[487]: ts=2026-09-23T16:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" lb # [ 34.281036] envoy[895]: [2026-09-23 16:13:00.599][895][warning][config] [source/extensions/config_subscription/grpc/grpc_subscription_impl.cc:130] gRPC config: initial fetch timed out for type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment lb # [ 34.290294] envoy[895]: [2026-09-23 16:13:00.600][895][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized lb # [ 34.295153] envoy[895]: [2026-09-23 16:13:00.600][895][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager lb # [ 34.306607] envoy[895]: [2026-09-23 16:13:00.631][895][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 35.347445] envoy[895]: conn peer=[::1]:33328 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=4 lb: (finished: waiting for TCP port 50051 on localhost, in 8.59 seconds) worker2 # [ 35.539946] systemd[1]: Starting scheduler-token.service... worker2 # [ 35.616469] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 35.621502] systemd[1]: Finished scheduler-token.service. worker2 # [ 35.623120] systemd[1]: scheduler-token.service: Consumed 46ms CPU time over 81ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 35.643248] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 35.777300] nix-grpc-daemon[786]: ts=2026-09-23T16:13:02Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 35.786678] nix-grpc-daemon[786]: ts=2026-09-23T16:13:02Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 35.789222] nix-grpc-daemon[786]: ts=2026-09-23T16:13:02Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 35.804177] nix-grpc-daemon[786]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 35.807376] nix-grpc-daemon[786]: E0000 00:00:1790179982.128838 786 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 35.812658] nix-grpc-daemon[786]: E0000 00:00:1790179982.137611 786 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 35.816047] nix-grpc-daemon[786]: ts=2026-09-23T16:13:02Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 35.828058] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 35.832078] nix-grpc-daemon[786]: E0000 00:00:1790179982.156551 828 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 35.851120] nix-grpc-daemon[786]: E0000 00:00:1790179982.176153 828 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 35.856080] nix-grpc-daemon[786]: E0000 00:00:1790179982.179027 828 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 35.871423] nix-grpc-daemon[786]: E0000 00:00:1790179982.196163 828 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 35.891781] nix-grpc-daemon[786]: ts=2026-09-23T16:13:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.894394] nix-grpc-daemon[786]: ts=2026-09-23T16:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) worker2 # [ 36.095649] nix-grpc-daemon[786]: ts=2026-09-23T16:13:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.098606] nix-grpc-daemon[786]: ts=2026-09-23T16:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.299659] nix-grpc-daemon[786]: ts=2026-09-23T16:13:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.302555] nix-grpc-daemon[786]: ts=2026-09-23T16:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.503575] nix-grpc-daemon[786]: ts=2026-09-23T16:13:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.506777] nix-grpc-daemon[786]: ts=2026-09-23T16:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 36.567029] niks3-server[985]: 2026/09/23 16:13:02 INFO lead: acquired remote=192.168.1.3:54078 worker1 # [ 36.569387] nix-grpc-daemon[487]: ts=2026-09-23T16:13:02Z level=info event=scheduler_take_over worker2 # [ 36.708360] nix-grpc-daemon[786]: ts=2026-09-23T16:13:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.713866] nix-grpc-daemon[786]: ts=2026-09-23T16:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.917269] nix-grpc-daemon[786]: ts=2026-09-23T16:13:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.921490] nix-grpc-daemon[786]: ts=2026-09-23T16:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 37.034491] nix-grpc-daemon[487]: ts=2026-09-23T16:13:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.039969] nix-grpc-daemon[487]: ts=2026-09-23T16:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.118555] nix-grpc-daemon[786]: ts=2026-09-23T16:13:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.124569] nix-grpc-daemon[786]: ts=2026-09-23T16:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.240793] nix-grpc-daemon[487]: ts=2026-09-23T16:13:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.246156] nix-grpc-daemon[487]: ts=2026-09-23T16:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.325897] nix-grpc-daemon[786]: ts=2026-09-23T16:13:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.331434] nix-grpc-daemon[786]: ts=2026-09-23T16:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.446972] nix-grpc-daemon[487]: ts=2026-09-23T16:13:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.467133] nix-grpc-daemon[487]: ts=2026-09-23T16:13:03Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:59094 worker1 # [ 37.469766] nix-grpc-daemon[487]: ts=2026-09-23T16:13:03Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 37.532774] nix-grpc-daemon[786]: ts=2026-09-23T16:13:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.527630] nix-grpc-daemon[487]: ts=2026-09-23T16:13:03Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 37.529922] nix-grpc-daemon[487]: ts=2026-09-23T16:13:03Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:59094 worker1 # [ 37.532397] nix-grpc-daemon[487]: ts=2026-09-23T16:13:03Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 40.766859] nix-grpc-daemon[487]: E0000 00:00:1790179987.103231 539 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 40.820095] nix-grpc-daemon[487]: E0000 00:00:1790179987.158400 539 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 40.852347] nix-grpc-daemon[487]: E0000 00:00:1790179987.165534 539 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 40.855866] nix-grpc-daemon[487]: E0000 00:00:1790179987.189503 539 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 51.141394] nix-grpc-daemon[487]: ts=2026-09-23T16:13:17Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:39548 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 16.10 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.74 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker1 # [ 52.964768] nix-grpc-daemon[487]: ts=2026-09-23T16:13:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 52.983749] systemd[1]: Started Nix Daemon. worker1 # [ 53.178239] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 53.200968] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 1/5) worker1 # [ 53.232068] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 2/5) worker1 # [ 53.319309] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 378 ms (attempt 3/5) worker1 # [ 53.699521] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 4/5) worker1 # [ 53.711310] nix-grpc-daemon[487]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 53.715945] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 53.763544] nix-grpc-daemon[786]: ts=2026-09-23T16:13:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 53.779366] systemd[1]: Started Nix Daemon. worker2 # [ 53.959634] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) worker2 # [ 53.981050] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 1/5) worker2 # [ 54.039386] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 46 ms (attempt 2/5) worker2 # [ 54.087355] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 3/5) worker2 # [ 54.126558] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 296 ms (attempt 4/5) worker2 # [ 54.426228] nix-grpc-daemon[786]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 54.431529] nix-grpc-daemon[786]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 54.442599] nix-grpc-daemon[786]: ts=2026-09-23T16:13:20Z level=debug event=path_info_miss path=zbvxb4cbwfrqzrgr4i5486i3lak215fd-nixos-system-client-test error="error: path '/nix/store/zbvxb4cbwfrqzrgr4i5486i3lak215fd-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 54.459814] nix-grpc-daemon[786]: ts=2026-09-23T16:13:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 54.472413] nix-grpc-daemon[786]: ts=2026-09-23T16:13:20Z level=debug event=path_info_miss path=zbvxb4cbwfrqzrgr4i5486i3lak215fd-nixos-system-client-test error="error: path '/nix/store/zbvxb4cbwfrqzrgr4i5486i3lak215fd-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds) client: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds) worker2 # [ 54.811777] nix-grpc-daemon[786]: ts=2026-09-23T16:13:21Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 54.816929] nix-grpc-daemon[786]: ts=2026-09-23T16:13:21Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:37594 worker2 # [ 54.830309] nix-grpc-daemon[786]: ts=2026-09-23T16:13:21Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:37594 worker2 # [ 54.842132] nix-grpc-daemon[786]: ts=2026-09-23T16:13:21Z level=debug event=path_info_miss path=zbvxb4cbwfrqzrgr4i5486i3lak215fd-nixos-system-client-test error="error: path '/nix/store/zbvxb4cbwfrqzrgr4i5486i3lak215fd-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 54.856650] nix-grpc-daemon[786]: ts=2026-09-23T16:13:21Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:37594 worker2 # [ 54.864819] nix-grpc-daemon[786]: ts=2026-09-23T16:13:21Z level=debug event=path_info_miss path=zbvxb4cbwfrqzrgr4i5486i3lak215fd-nixos-system-client-test error="error: path '/nix/store/zbvxb4cbwfrqzrgr4i5486i3lak215fd-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds) worker1 # [ 55.094254] nix-grpc-daemon[487]: ts=2026-09-23T16:13:21Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 55.283319] nix-grpc-daemon[786]: ts=2026-09-23T16:13:21Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 2.63 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 55.442222] nix-grpc-daemon[487]: ts=2026-09-23T16:13:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 55.451456] nix-grpc-daemon[487]: ts=2026-09-23T16:13:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 55.482131] nix-grpc-daemon[487]: ts=2026-09-23T16:13:21Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=0 worker1 # [ 55.489641] nix-grpc-daemon[487]: ts=2026-09-23T16:13:21Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 55.497258] nix-grpc-daemon[487]: ts=2026-09-23T16:13:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 55.514603] nix-grpc-daemon[487]: ts=2026-09-23T16:13:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 client # copying 1 paths to 'lb:50051'... worker1 # [ 55.530966] nix-grpc-daemon[487]: ts=2026-09-23T16:13:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'... worker1 # [ 55.551484] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 55.592153] nix-grpc-daemon[487]: ts=2026-09-23T16:13:21Z level=info event=niks3_push_started pid=870 worker1 # [ 55.799263] nix-grpc-daemon[870]: time=2026-09-23T16:13:22.137Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 55.915451] nix-daemon[856]: accepted connection from pid 883, user nix-grpc-daemon (trusted) worker1 # [ 55.937344] nix-daemon[856]: reaped child process 886, status = succeeded lb # [ 55.938976] niks3-server[985]: 2026/09/23 16:13:22 INFO Received push request method=POST path=/api/pushes worker1 # [ 55.961745] nix-grpc-daemon[870]: time=2026-09-23T16:13:22.299Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 55.963216] nix-grpc-daemon[870]: time=2026-09-23T16:13:22.299Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)" lb # [ 56.003526] niks3-server[985]: 2026/09/23 16:13:22 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign lb # [ 56.028883] niks3-server[985]: 2026/09/23 16:13:22 INFO Signed narinfos id=1 count=1 worker1 # [ 56.035761] nix-grpc-daemon[870]: time=2026-09-23T16:13:22.372Z level=INFO msg="Uploading 1 narinfos" lb # [ 56.056116] niks3-server[985]: 2026/09/23 16:13:22 INFO Received complete push request method=POST path=/api/pushes/1/complete lb # [ 56.086607] niks3-server[985]: 2026/09/23 16:13:22 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst lb # [ 56.096353] niks3-server[985]: 2026/09/23 16:13:22 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls lb # [ 56.100509] niks3-server[985]: 2026/09/23 16:13:22 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo worker1 # [ 56.110094] nix-grpc-daemon[870]: time=2026-09-23T16:13:22.448Z level=INFO msg="Upload complete. (306ms)" worker1 # [ 56.121186] nix-grpc-daemon[487]: ts=2026-09-23T16:13:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 56.140407] nix-grpc-daemon[487]: ts=2026-09-23T16:13:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 56.155216] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 56.159504] nix-daemon[856]: reaped child process 868, status = succeeded client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051... worker1 # [ 56.174285] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker1 # [ 57.248981] nix-daemon[856]: reaped child process 890, status = succeeded worker1 # [ 57.356958] nix-daemon[856]: accepted connection from pid 906, user nix-grpc-daemon (trusted) worker1 # [ 57.374668] nix-daemon[856]: reaped child process 909, status = succeeded lb # [ 57.380923] niks3-server[985]: 2026/09/23 16:13:23 INFO Received push request method=POST path=/api/pushes worker1 # [ 57.395119] nix-grpc-daemon[870]: time=2026-09-23T16:13:23.731Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 57.398124] nix-grpc-daemon[870]: time=2026-09-23T16:13:23.731Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)" lb # [ 57.430808] niks3-server[985]: 2026/09/23 16:13:23 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst worker1 # [ 57.443108] nix-grpc-daemon[870]: time=2026-09-23T16:13:23.781Z level=INFO msg="Uploading 1 narinfos" lb # [ 57.446609] niks3-server[985]: 2026/09/23 16:13:23 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign lb # [ 57.453747] niks3-server[985]: 2026/09/23 16:13:23 INFO Signed narinfos id=2 count=1 lb # [ 57.465176] niks3-server[985]: 2026/09/23 16:13:23 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls lb # [ 57.471882] niks3-server[985]: 2026/09/23 16:13:23 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv lb # [ 57.481020] niks3-server[985]: 2026/09/23 16:13:23 INFO Received complete push request method=POST path=/api/pushes/2/complete lb # [ 57.484530] niks3-server[985]: 2026/09/23 16:13:23 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo worker1 # [ 57.490598] nix-grpc-daemon[870]: time=2026-09-23T16:13:23.829Z level=INFO msg="Upload complete. (302ms)" worker1 # [ 57.514607] nix-grpc-daemon[487]: ts=2026-09-23T16:13:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=1 worker1 # [ 57.536425] nix-grpc-daemon[487]: ts=2026-09-23T16:13:23Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=0 worker1 # [ 57.540932] nix-grpc-daemon[487]: ts=2026-09-23T16:13:23Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 57.546454] nix-grpc-daemon[487]: ts=2026-09-23T16:13:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 57.551407] nix-grpc-daemon[487]: ts=2026-09-23T16:13:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 57.556639] nix-daemon[856]: reaped child process 888, status = succeeded client # copying 1 paths to 'lb:50051'... worker1 # [ 57.558586] nix-grpc-daemon[870]: time=2026-09-23T16:13:23.894Z level=INFO msg="All 1 paths already cached" worker1 # [ 57.562194] nix-grpc-daemon[487]: ts=2026-09-23T16:13:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'... worker1 # [ 57.582679] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 57.726364] nix-daemon[856]: accepted connection from pid 915, user nix-grpc-daemon (trusted) worker1 # [ 57.744441] nix-daemon[856]: reaped child process 918, status = succeeded lb # [ 57.746935] niks3-server[985]: 2026/09/23 16:13:24 INFO Received push request method=POST path=/api/pushes worker1 # [ 57.759810] nix-grpc-daemon[870]: time=2026-09-23T16:13:24.098Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 57.761560] nix-grpc-daemon[870]: time=2026-09-23T16:13:24.098Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)" lb # [ 57.778409] niks3-server[985]: 2026/09/23 16:13:24 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls lb # [ 57.791571] niks3-server[985]: 2026/09/23 16:13:24 INFO Received sign narinfos request method=POST path=/api/pushes/3/sign worker1 # [ 57.798976] nix-grpc-daemon[870]: time=2026-09-23T16:13:24.137Z level=INFO msg="Uploading 1 narinfos" lb # [ 57.803768] niks3-server[985]: 2026/09/23 16:13:24 INFO Signed narinfos id=3 count=1 lb # [ 57.809333] niks3-server[985]: 2026/09/23 16:13:24 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst lb # [ 57.827054] niks3-server[985]: 2026/09/23 16:13:24 INFO Received complete push request method=POST path=/api/pushes/3/complete lb # [ 57.833826] niks3-server[985]: 2026/09/23 16:13:24 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo worker1 # [ 57.842798] nix-grpc-daemon[870]: time=2026-09-23T16:13:24.181Z level=INFO msg="Upload complete. (246ms)" worker1 # [ 57.852857] nix-grpc-daemon[487]: ts=2026-09-23T16:13:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 57.868542] nix-grpc-daemon[487]: ts=2026-09-23T16:13:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 57.878919] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 57.886015] nix-daemon[856]: reaped child process 913, status = succeeded client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051... client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker1 # [ 57.901358] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker1 # [ 58.738861] nix-daemon[856]: reaped child process 922, status = succeeded worker1 # [ 58.854353] nix-daemon[856]: accepted connection from pid 938, user nix-grpc-daemon (trusted) worker1 # [ 58.875357] nix-daemon[856]: reaped child process 941, status = succeeded lb # [ 58.878184] niks3-server[985]: 2026/09/23 16:13:25 INFO Received push request method=POST path=/api/pushes worker1 # [ 58.893009] nix-grpc-daemon[870]: time=2026-09-23T16:13:25.230Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 58.894602] nix-grpc-daemon[870]: time=2026-09-23T16:13:25.230Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)" lb # [ 58.921147] niks3-server[985]: 2026/09/23 16:13:25 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst worker1 # [ 58.937152] nix-grpc-daemon[870]: time=2026-09-23T16:13:25.275Z level=INFO msg="Uploading 1 narinfos" lb # [ 58.936839] niks3-server[985]: 2026/09/23 16:13:25 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv lb # [ 58.947145] niks3-server[985]: 2026/09/23 16:13:25 INFO Received sign narinfos request method=POST path=/api/pushes/4/sign lb # [ 58.959952] niks3-server[985]: 2026/09/23 16:13:25 INFO Signed narinfos id=4 count=1 lb # [ 58.962517] niks3-server[985]: 2026/09/23 16:13:25 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls lb # [ 58.969402] niks3-server[985]: 2026/09/23 16:13:25 INFO Received complete push request method=POST path=/api/pushes/4/complete lb # [ 58.972927] niks3-server[985]: 2026/09/23 16:13:25 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo worker1 # [ 58.980654] nix-grpc-daemon[870]: time=2026-09-23T16:13:25.319Z level=INFO msg="Upload complete. (267ms)" worker1 # [ 59.000413] nix-grpc-daemon[487]: ts=2026-09-23T16:13:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=1 worker1 # [ 59.016305] nix-grpc-daemon[487]: ts=2026-09-23T16:13:25Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=0 worker1 # [ 59.023817] nix-grpc-daemon[487]: ts=2026-09-23T16:13:25Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 59.029420] nix-grpc-daemon[487]: ts=2026-09-23T16:13:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 59.034084] nix-grpc-daemon[487]: ts=2026-09-23T16:13:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 59.038773] nix-daemon[856]: reaped child process 920, status = succeeded worker1 # [ 59.042677] nix-grpc-daemon[870]: time=2026-09-23T16:13:25.381Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'... worker1 # [ 59.052910] nix-grpc-daemon[487]: ts=2026-09-23T16:13:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 59.067218] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 59.203495] nix-daemon[856]: accepted connection from pid 947, user nix-grpc-daemon (trusted) worker1 # [ 59.224116] nix-daemon[856]: reaped child process 950, status = succeeded lb # [ 59.228367] niks3-server[985]: 2026/09/23 16:13:25 INFO Received push request method=POST path=/api/pushes worker1 # [ 59.240860] nix-grpc-daemon[870]: time=2026-09-23T16:13:25.579Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 59.243394] nix-grpc-daemon[870]: time=2026-09-23T16:13:25.579Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)" lb # [ 59.255102] envoy[895]: conn peer=[2001:db8:1::1]:48964 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=1535 lb # [ 59.268511] envoy[895]: conn peer=[2001:db8:1::1]:48970 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=388 ms=76 worker1 # [ 59.281768] nix-grpc-daemon[870]: time=2026-09-23T16:13:25.620Z level=INFO msg="Uploading 1 narinfos" lb # [ 59.282222] envoy[895]: conn peer=[2001:db8:1::1]:48986 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=17 lb # [ 59.291637] envoy[895]: conn peer=[2001:db8:1::1]:48990 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=12 lb # [ 59.308887] niks3-server[985]: 2026/09/23 16:13:25 INFO Received sign narinfos request method=POST path=/api/pushes/5/sign lb # [ 59.315339] niks3-server[985]: 2026/09/23 16:13:25 INFO Signed narinfos id=5 count=1 worker1 # [ 59.316569] nix-grpc-daemon[870]: time=2026-09-23T16:13:25.655Z level=INFO msg="Upload complete. (238ms)" lb # [ 59.322052] niks3-server[985]: 2026/09/23 16:13:25 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst lb # [ 59.327770] niks3-server[985]: 2026/09/23 16:13:25 INFO Received complete push request method=POST path=/api/pushes/5/complete worker1 # [ 59.328216] nix-grpc-daemon[487]: ts=2026-09-23T16:13:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 59.331080] niks3-server[985]: 2026/09/23 16:13:25 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo lb # [ 59.335548] niks3-server[985]: 2026/09/23 16:13:25 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls worker1 # [ 59.344892] nix-grpc-daemon[487]: ts=2026-09-23T16:13:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 59.352780] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 59.358944] nix-daemon[856]: reaped child process 945, status = succeeded client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051... worker1 # [ 59.372187] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker1 # [ 60.416713] nix-daemon[856]: reaped child process 954, status = succeeded worker1 # [ 60.501302] nix-daemon[856]: accepted connection from pid 970, user nix-grpc-daemon (trusted) worker1 # [ 60.522016] nix-daemon[856]: reaped child process 973, status = succeeded lb # [ 60.529436] niks3-server[985]: 2026/09/23 16:13:26 INFO Received push request method=POST path=/api/pushes worker1 # [ 60.550505] nix-grpc-daemon[870]: time=2026-09-23T16:13:26.887Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 60.552070] nix-grpc-daemon[870]: time=2026-09-23T16:13:26.887Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)" worker1 # [ 60.591129] nix-grpc-daemon[870]: time=2026-09-23T16:13:26.929Z level=INFO msg="Uploading 1 narinfos" lb # [ 60.582044] niks3-server[985]: 2026/09/23 16:13:26 INFO Received sign narinfos request method=POST path=/api/pushes/6/sign lb # [ 60.595901] niks3-server[985]: 2026/09/23 16:13:26 INFO Signed narinfos id=6 count=1 lb # [ 60.604366] niks3-server[985]: 2026/09/23 16:13:26 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls lb # [ 60.618072] niks3-server[985]: 2026/09/23 16:13:26 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst lb # [ 60.627683] niks3-server[985]: 2026/09/23 16:13:26 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv lb # [ 60.632239] niks3-server[985]: 2026/09/23 16:13:26 INFO Received complete push request method=POST path=/api/pushes/6/complete worker1 # [ 60.638415] nix-grpc-daemon[870]: time=2026-09-23T16:13:26.977Z level=INFO msg="Upload complete. (255ms)" lb # [ 60.639606] niks3-server[985]: 2026/09/23 16:13:26 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo worker1 # [ 60.653224] nix-grpc-daemon[487]: ts=2026-09-23T16:13:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=1 worker1 # [ 60.673029] nix-daemon[856]: reaped child process 952, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 5.41 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.19 seconds) client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client # copying 3 paths... client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.28 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 5.99 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds) worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 61.995986] nix-grpc-daemon[786]: ts=2026-09-23T16:13:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 62.004105] nix-grpc-daemon[786]: ts=2026-09-23T16:13:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37594 worker1 # [ 62.041426] nix-grpc-daemon[487]: ts=2026-09-23T16:13:28Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.34 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.16 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 1.24 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2 client # don't know how to build these paths: client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker1 # [ 62.915734] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 62.924656] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 62.933850] nix-grpc-daemon[870]: time=2026-09-23T16:13:29.269Z level=INFO msg="All 2 paths already cached" worker1 # [ 62.946480] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=0 worker1 # [ 62.953819] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 62.963083] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 62.975092] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 62.990209] nix-grpc-daemon[870]: time=2026-09-23T16:13:29.328Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'... worker1 # [ 63.002304] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 63.017659] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 63.167444] nix-daemon[856]: accepted connection from pid 1002, user nix-grpc-daemon (trusted) worker1 # [ 63.188963] nix-daemon[856]: reaped child process 1005, status = succeeded lb # [ 63.191172] niks3-server[985]: 2026/09/23 16:13:29 INFO Received push request method=POST path=/api/pushes worker1 # [ 63.203520] nix-grpc-daemon[870]: time=2026-09-23T16:13:29.541Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 63.205274] nix-grpc-daemon[870]: time=2026-09-23T16:13:29.541Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)" lb # [ 63.221419] niks3-server[985]: 2026/09/23 16:13:29 INFO Received sign narinfos request method=POST path=/api/pushes/7/sign worker1 # [ 63.231743] nix-grpc-daemon[870]: time=2026-09-23T16:13:29.570Z level=INFO msg="Uploading 1 narinfos" lb # [ 63.232069] niks3-server[985]: 2026/09/23 16:13:29 INFO Signed narinfos id=7 count=1 lb # [ 63.244566] niks3-server[985]: 2026/09/23 16:13:29 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls lb # [ 63.253757] niks3-server[985]: 2026/09/23 16:13:29 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst lb # [ 63.264612] niks3-server[985]: 2026/09/23 16:13:29 INFO Received complete push request method=POST path=/api/pushes/7/complete lb # [ 63.271904] niks3-server[985]: 2026/09/23 16:13:29 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo worker1 # [ 63.274068] nix-grpc-daemon[870]: time=2026-09-23T16:13:29.612Z level=INFO msg="Upload complete. (240ms)" worker1 # [ 63.285451] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 63.300705] nix-grpc-daemon[487]: ts=2026-09-23T16:13:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 63.309223] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 63.320337] nix-daemon[856]: reaped child process 1000, status = succeeded client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051... client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker1 # [ 63.331937] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker1 # [ 64.641484] nix-daemon[856]: reaped child process 1009, status = succeeded worker1 # [ 64.712742] nix-daemon[856]: accepted connection from pid 1025, user nix-grpc-daemon (trusted) worker1 # [ 64.734640] nix-daemon[856]: reaped child process 1028, status = succeeded lb # [ 64.739631] niks3-server[985]: 2026/09/23 16:13:31 INFO Received push request method=POST path=/api/pushes worker1 # [ 64.754794] nix-grpc-daemon[870]: time=2026-09-23T16:13:31.090Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 64.757889] nix-grpc-daemon[870]: time=2026-09-23T16:13:31.090Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)" lb # [ 64.785776] niks3-server[985]: 2026/09/23 16:13:31 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker1 # [ 64.800111] nix-grpc-daemon[870]: time=2026-09-23T16:13:31.138Z level=INFO msg="Uploading 1 narinfos" lb # [ 64.801053] niks3-server[985]: 2026/09/23 16:13:31 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls lb # [ 64.809962] niks3-server[985]: 2026/09/23 16:13:31 INFO Received sign narinfos request method=POST path=/api/pushes/8/sign lb # [ 64.817429] niks3-server[985]: 2026/09/23 16:13:31 INFO Signed narinfos id=8 count=1 lb # [ 64.827631] niks3-server[985]: 2026/09/23 16:13:31 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst lb # [ 64.835761] niks3-server[985]: 2026/09/23 16:13:31 INFO Received complete push request method=POST path=/api/pushes/8/complete worker1 # [ 64.843629] nix-grpc-daemon[870]: time=2026-09-23T16:13:31.182Z level=INFO msg="Upload complete. (300ms)" lb # [ 64.844128] niks3-server[985]: 2026/09/23 16:13:31 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo worker1 # [ 64.857178] nix-grpc-daemon[487]: ts=2026-09-23T16:13:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 worker1 # [ 64.878532] nix-daemon[856]: reaped child process 1007, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.21 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.19 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.74 seconds) subtest: outputs read back through any worker carry the cache signature client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2 client # don't know how to build these paths: client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker2 # [ 65.487008] nix-grpc-daemon[786]: ts=2026-09-23T16:13:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 65.495187] nix-grpc-daemon[786]: ts=2026-09-23T16:13:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37594 worker1 # [ 65.499094] nix-grpc-daemon[487]: ts=2026-09-23T16:13:31Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=0 worker1 # [ 65.503979] nix-grpc-daemon[487]: ts=2026-09-23T16:13:31Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 65.521340] nix-grpc-daemon[786]: ts=2026-09-23T16:13:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 65.537166] nix-grpc-daemon[786]: ts=2026-09-23T16:13:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37594 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'... worker2 # [ 65.551407] nix-grpc-daemon[786]: ts=2026-09-23T16:13:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 65.566915] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) worker2 # [ 65.600154] nix-grpc-daemon[786]: ts=2026-09-23T16:13:31Z level=info event=niks3_push_started pid=869 worker2 # [ 65.767617] nix-grpc-daemon[869]: time=2026-09-23T16:13:32.092Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 65.881104] nix-daemon[844]: accepted connection from pid 882, user nix-grpc-daemon (trusted) worker2 # [ 65.901615] nix-daemon[844]: reaped child process 885, status = succeeded lb # [ 65.891034] niks3-server[985]: 2026/09/23 16:13:32 INFO Received push request method=POST path=/api/pushes worker2 # [ 65.919001] nix-grpc-daemon[869]: time=2026-09-23T16:13:32.242Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 65.923078] nix-grpc-daemon[869]: time=2026-09-23T16:13:32.242Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)" lb # [ 65.933270] niks3-server[985]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls worker2 # [ 65.965857] nix-grpc-daemon[869]: time=2026-09-23T16:13:32.289Z level=INFO msg="Uploading 1 narinfos" lb # [ 65.942484] niks3-server[985]: 2026/09/23 16:13:32 INFO Received sign narinfos request method=POST path=/api/pushes/9/sign lb # [ 65.955746] niks3-server[985]: 2026/09/23 16:13:32 INFO Signed narinfos id=9 count=1 lb # [ 65.963093] niks3-server[985]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst lb # [ 65.977780] niks3-server[985]: 2026/09/23 16:13:32 INFO Received complete push request method=POST path=/api/pushes/9/complete worker2 # [ 66.002583] nix-grpc-daemon[869]: time=2026-09-23T16:13:32.327Z level=INFO msg="Upload complete. (230ms)" lb # [ 65.988103] niks3-server[985]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo worker2 # [ 66.013431] nix-grpc-daemon[786]: ts=2026-09-23T16:13:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 66.026731] nix-grpc-daemon[786]: ts=2026-09-23T16:13:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 66.033132] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) worker2 # [ 66.038051] nix-daemon[844]: reaped child process 867, status = succeeded client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051... worker2 # [ 66.050612] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-sig worker2 # [ 66.478388] nix-daemon[844]: reaped child process 889, status = succeeded worker2 # [ 66.576683] nix-daemon[844]: accepted connection from pid 905, user nix-grpc-daemon (trusted) worker2 # [ 66.598658] nix-daemon[844]: reaped child process 908, status = succeeded lb # [ 66.580225] niks3-server[985]: 2026/09/23 16:13:32 INFO Received push request method=POST path=/api/pushes worker2 # [ 66.607633] nix-grpc-daemon[869]: time=2026-09-23T16:13:32.930Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 66.610257] nix-grpc-daemon[869]: time=2026-09-23T16:13:32.930Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)" lb # [ 66.621934] niks3-server[985]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst lb # [ 66.637073] niks3-server[985]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv lb # [ 66.650646] niks3-server[985]: 2026/09/23 16:13:32 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign worker2 # [ 66.675601] nix-grpc-daemon[869]: time=2026-09-23T16:13:33.000Z level=INFO msg="Uploading 1 narinfos" lb # [ 66.662741] niks3-server[985]: 2026/09/23 16:13:32 INFO Signed narinfos id=10 count=1 lb # [ 66.669722] niks3-server[985]: 2026/09/23 16:13:32 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls lb # [ 66.681142] niks3-server[985]: 2026/09/23 16:13:33 INFO Received complete push request method=POST path=/api/pushes/10/complete worker2 # [ 66.705580] nix-grpc-daemon[869]: time=2026-09-23T16:13:33.030Z level=INFO msg="Upload complete. (246ms)" lb # [ 66.691755] niks3-server[985]: 2026/09/23 16:13:33 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo worker1 # [ 66.710853] nix-grpc-daemon[487]: ts=2026-09-23T16:13:33Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=0 worker1 # [ 66.717293] nix-grpc-daemon[487]: ts=2026-09-23T16:13:33Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 66.721245] nix-grpc-daemon[786]: ts=2026-09-23T16:13:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=1 worker2 # [ 66.740656] nix-grpc-daemon[786]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 66.747752] nix-daemon[844]: reaped child process 887, status = succeeded worker2 # [ 66.756769] nix-grpc-daemon[786]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 66.771368] nix-grpc-daemon[869]: time=2026-09-23T16:13:33.095Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'... worker2 # [ 66.785617] nix-grpc-daemon[786]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 66.799661] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) worker2 # [ 66.952347] nix-daemon[844]: accepted connection from pid 912, user nix-grpc-daemon (trusted) worker2 # [ 66.975623] nix-daemon[844]: reaped child process 915, status = succeeded lb # [ 66.966098] niks3-server[985]: 2026/09/23 16:13:33 INFO Received push request method=POST path=/api/pushes worker2 # [ 67.085472] nix-grpc-daemon[869]: time=2026-09-23T16:13:33.409Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 67.090387] nix-grpc-daemon[869]: time=2026-09-23T16:13:33.409Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)" lb # [ 67.098420] niks3-server[985]: 2026/09/23 16:13:33 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign worker2 # [ 67.123057] nix-grpc-daemon[869]: time=2026-09-23T16:13:33.447Z level=INFO msg="Uploading 1 narinfos" lb # [ 67.116578] niks3-server[985]: 2026/09/23 16:13:33 INFO Signed narinfos id=11 count=1 lb # [ 67.119950] niks3-server[985]: 2026/09/23 16:13:33 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst lb # [ 67.142164] niks3-server[985]: 2026/09/23 16:13:33 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls worker2 # [ 67.166659] nix-grpc-daemon[869]: time=2026-09-23T16:13:33.491Z level=INFO msg="Upload complete. (352ms)" lb # [ 67.155030] niks3-server[985]: 2026/09/23 16:13:33 INFO Received complete push request method=POST path=/api/pushes/11/complete lb # [ 67.160097] niks3-server[985]: 2026/09/23 16:13:33 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo worker2 # [ 67.180771] nix-grpc-daemon[786]: ts=2026-09-23T16:13:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 67.201094] nix-grpc-daemon[786]: ts=2026-09-23T16:13:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 67.211937] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) worker2 # [ 67.219983] nix-daemon[844]: reaped child process 910, status = succeeded client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051... client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker2 # [ 67.239900] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-sig worker2 # [ 68.924402] nix-daemon[844]: reaped child process 919, status = succeeded worker2 # [ 69.043504] nix-daemon[844]: accepted connection from pid 935, user nix-grpc-daemon (trusted) worker2 # [ 69.065976] nix-daemon[844]: reaped child process 938, status = succeeded lb # [ 69.060151] niks3-server[985]: 2026/09/23 16:13:35 INFO Received push request method=POST path=/api/pushes worker2 # [ 69.086317] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.409Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 69.088259] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.409Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)" lb # [ 69.105768] niks3-server[985]: 2026/09/23 16:13:35 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst worker2 # [ 69.134490] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.459Z level=INFO msg="Uploading 1 narinfos" lb # [ 69.118907] niks3-server[985]: 2026/09/23 16:13:35 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv lb # [ 69.127973] niks3-server[985]: 2026/09/23 16:13:35 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign lb # [ 69.130770] niks3-server[985]: 2026/09/23 16:13:35 INFO Signed narinfos id=12 count=1 lb # [ 69.139914] niks3-server[985]: 2026/09/23 16:13:35 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls lb # [ 69.149108] niks3-server[985]: 2026/09/23 16:13:35 INFO Received complete push request method=POST path=/api/pushes/12/complete worker2 # [ 69.165771] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.490Z level=INFO msg="Upload complete. (309ms)" lb # [ 69.153466] niks3-server[985]: 2026/09/23 16:13:35 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo worker2 # [ 69.181486] nix-grpc-daemon[786]: ts=2026-09-23T16:13:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=1 worker1 # [ 69.173643] nix-grpc-daemon[487]: ts=2026-09-23T16:13:35Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=0 worker1 # [ 69.174008] nix-grpc-daemon[487]: ts=2026-09-23T16:13:35Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 69.204529] nix-grpc-daemon[786]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 69.213555] nix-daemon[844]: reaped child process 917, status = succeeded worker2 # [ 69.222853] nix-grpc-daemon[786]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 69.237262] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.562Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 69.248504] nix-grpc-daemon[786]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'... worker2 # [ 69.271797] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) lb # [ 69.257226] envoy[895]: conn peer=[2001:db8:1::1]:48994 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9502 tx=4373 ms=5232 lb # [ 69.260895] envoy[895]: conn peer=[2001:db8:1::1]:38990 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=84 lb # [ 69.263820] envoy[895]: conn peer=[2001:db8:1::1]:38992 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4185 tx=1895 ms=1962 worker2 # [ 69.444138] nix-daemon[844]: accepted connection from pid 942, user nix-grpc-daemon (trusted) worker2 # [ 69.466582] nix-daemon[844]: reaped child process 945, status = succeeded lb # [ 69.457181] niks3-server[985]: 2026/09/23 16:13:35 INFO Received push request method=POST path=/api/pushes worker2 # [ 69.483658] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.808Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 69.485336] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.808Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)" lb # [ 69.489323] niks3-server[985]: 2026/09/23 16:13:35 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst worker2 # [ 69.515443] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.840Z level=INFO msg="Uploading 1 narinfos" lb # [ 69.502760] niks3-server[985]: 2026/09/23 16:13:35 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign lb # [ 69.515584] niks3-server[985]: 2026/09/23 16:13:35 INFO Signed narinfos id=13 count=1 lb # [ 69.521138] niks3-server[985]: 2026/09/23 16:13:35 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls lb # [ 69.532504] niks3-server[985]: 2026/09/23 16:13:35 INFO Received complete push request method=POST path=/api/pushes/13/complete lb # [ 69.540279] niks3-server[985]: 2026/09/23 16:13:35 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo worker2 # [ 69.556658] nix-grpc-daemon[869]: time=2026-09-23T16:13:35.881Z level=INFO msg="Upload complete. (265ms)" worker2 # [ 69.567454] nix-grpc-daemon[786]: ts=2026-09-23T16:13:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 69.580734] nix-grpc-daemon[786]: ts=2026-09-23T16:13:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 69.591707] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) worker2 # [ 69.597778] nix-daemon[844]: reaped child process 940, status = succeeded worker2 # [ 69.608609] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051... client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-sig worker2 # [ 70.845004] nix-daemon[844]: reaped child process 949, status = succeeded worker2 # [ 70.970678] nix-daemon[844]: accepted connection from pid 965, user nix-grpc-daemon (trusted) worker2 # [ 70.991636] nix-daemon[844]: reaped child process 968, status = succeeded lb # [ 70.975535] niks3-server[985]: 2026/09/23 16:13:37 INFO Received push request method=POST path=/api/pushes worker2 # [ 71.004429] nix-grpc-daemon[869]: time=2026-09-23T16:13:37.327Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 71.005949] nix-grpc-daemon[869]: time=2026-09-23T16:13:37.327Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)" lb # [ 71.023817] niks3-server[985]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls worker2 # [ 71.054812] nix-grpc-daemon[869]: time=2026-09-23T16:13:37.379Z level=INFO msg="Uploading 1 narinfos" lb # [ 71.039792] niks3-server[985]: 2026/09/23 16:13:37 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign lb # [ 71.048799] niks3-server[985]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv lb # [ 71.059091] niks3-server[985]: 2026/09/23 16:13:37 INFO Signed narinfos id=14 count=1 lb # [ 71.070825] niks3-server[985]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst lb # [ 71.078921] niks3-server[985]: 2026/09/23 16:13:37 INFO Received complete push request method=POST path=/api/pushes/14/complete worker2 # [ 71.100501] nix-grpc-daemon[869]: time=2026-09-23T16:13:37.425Z level=INFO msg="Upload complete. (278ms)" lb # [ 71.083001] niks3-server[985]: 2026/09/23 16:13:37 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo worker2 # [ 71.113361] nix-grpc-daemon[786]: ts=2026-09-23T16:13:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=1 worker2 # [ 71.132884] nix-daemon[844]: reaped child process 947, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.89 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 71.466617] nix-grpc-daemon[487]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:58204 client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked): client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 71.501071] nix-grpc-daemon[487]: ts=2026-09-23T16:13:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:58204 worker1 # [ 71.510277] nix-grpc-daemon[487]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'... worker1 # [ 71.533693] nix-grpc-daemon[487]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'... worker1 # [ 71.548453] nix-grpc-daemon[487]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.26 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker2 # [ 71.785457] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:47362 worker2 # [ 71.793854] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:47362 client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.21 seconds) client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top client # finding garbage collector roots... client # removing stale temporary roots file "/nix/var/nix/temproots/897" client # removing stale temporary roots file "/nix/var/nix/temproots/917" client # removing stale temporary roots file "/nix/var/nix/temproots/849" client # removing stale temporary roots file "/nix/var/nix/temproots/905" client # removing stale temporary roots file "/nix/var/nix/temproots/940" client # removing stale temporary roots file "/nix/var/nix/temproots/885" client # removing stale temporary roots file "/nix/var/nix/temproots/925" client # removing stale temporary roots file "/nix/var/nix/temproots/864" client # deleting specified paths... client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' client # deleting unused links... client # note: hard linking is currently saving 0.0 KiB worker2 # [ 72.126057] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37594 worker1 # [ 72.122104] nix-grpc-daemon[487]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 72.137872] nix-grpc-daemon[870]: time=2026-09-23T16:13:38.474Z level=INFO msg="All 1 paths already cached" worker2 # [ 72.155839] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 72.166318] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 72.176198] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37594 client # copying 3 paths... worker2 # [ 72.191429] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37594 client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'... worker2 # [ 72.218371] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 72.225648] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_out=232 worker2 # [ 72.242224] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 72.249252] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 72.258355] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37594 client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'... worker2 # [ 72.263468] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_out=128 client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'... client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.49 seconds) (finished: subtest: outputs read back through any worker carry the cache signature, in 7.04 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 72.346712] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: c09a6918455c425fb56f9eee49bf339d client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.06 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup worker2 # [ 72.533417] nix-grpc-daemon[786]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37594 client # [ 72.535909] nix[993]: don't know how to build these paths: client # [ 72.537301] nix[993]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 72.536831] nix-grpc-daemon[487]: ts=2026-09-23T16:13:38Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker1 # [ 72.544941] nix-grpc-daemon[487]: ts=2026-09-23T16:13:38Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 72.552346] nix-grpc-daemon[487]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 72.563550] nix-grpc-daemon[487]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 client # [ 72.590417] nix[993]: copying 1 paths to 'lb:50051'... client # [ 72.592142] nix[993]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'... worker1 # [ 72.579655] nix-grpc-daemon[487]: ts=2026-09-23T16:13:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 72.595628] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 72.739122] nix-daemon[856]: accepted connection from pid 1048, user nix-grpc-daemon (trusted) worker1 # [ 72.759903] nix-daemon[856]: reaped child process 1051, status = succeeded lb # [ 72.763183] niks3-server[985]: 2026/09/23 16:13:39 INFO Received push request method=POST path=/api/pushes worker1 # [ 72.776084] nix-grpc-daemon[870]: time=2026-09-23T16:13:39.114Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 72.777571] nix-grpc-daemon[870]: time=2026-09-23T16:13:39.114Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)" lb # [ 72.796761] niks3-server[985]: 2026/09/23 16:13:39 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst worker1 # [ 72.812155] nix-grpc-daemon[870]: time=2026-09-23T16:13:39.150Z level=INFO msg="Uploading 1 narinfos" lb # [ 72.811623] niks3-server[985]: 2026/09/23 16:13:39 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign lb # [ 72.822799] niks3-server[985]: 2026/09/23 16:13:39 INFO Signed narinfos id=15 count=1 lb # [ 72.829030] niks3-server[985]: 2026/09/23 16:13:39 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls worker1 # [ 72.847119] nix-grpc-daemon[870]: time=2026-09-23T16:13:39.185Z level=INFO msg="Upload complete. (232ms)" lb # [ 72.844986] niks3-server[985]: 2026/09/23 16:13:39 INFO Received complete push request method=POST path=/api/pushes/15/complete lb # [ 72.853827] niks3-server[985]: 2026/09/23 16:13:39 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo worker1 # [ 72.854702] nix-grpc-daemon[487]: ts=2026-09-23T16:13:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=432 worker1 # [ 72.871345] nix-grpc-daemon[487]: ts=2026-09-23T16:13:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 72.878991] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 72.885059] nix-daemon[856]: reaped child process 1046, status = succeeded client # [ 72.907944] nix[993]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker1 # [ 72.900196] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) client # [ 73.435773] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: bb6de19e44dd4f1f8001a38a6f9d122a client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds) worker2 # [ 73.673181] nix-grpc-daemon[786]: ts=2026-09-23T16:13:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37594 client # [ 73.678449] nix[1009]: don't know how to build these paths: client # [ 73.679538] nix[1009]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 73.676604] nix-grpc-daemon[487]: ts=2026-09-23T16:13:40Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 client # [ 73.697372] nix[1009]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker1 # [ 73.684848] nix-grpc-daemon[487]: ts=2026-09-23T16:13:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.05 seconds) worker1 # [ 76.927232] nix-daemon[856]: reaped child process 1055, status = succeeded client: waiting for success: ! systemctl is-active dedup1 dedup2 worker1 # [ 77.028099] nix-daemon[856]: accepted connection from pid 1087, user nix-grpc-daemon (trusted) worker1 # [ 77.045308] nix-daemon[856]: reaped child process 1090, status = succeeded lb # [ 77.049167] niks3-server[985]: 2026/09/23 16:13:43 INFO Received push request method=POST path=/api/pushes worker1 # [ 77.059187] nix-grpc-daemon[870]: time=2026-09-23T16:13:43.395Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 77.061657] nix-grpc-daemon[870]: time=2026-09-23T16:13:43.395Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)" lb # [ 77.088107] niks3-server[985]: 2026/09/23 16:13:43 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 77.104610] nix-grpc-daemon[870]: time=2026-09-23T16:13:43.443Z level=INFO msg="Uploading 1 narinfos" lb # [ 77.103071] niks3-server[985]: 2026/09/23 16:13:43 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign lb # [ 77.110593] niks3-server[985]: 2026/09/23 16:13:43 INFO Signed narinfos id=16 count=1 lb # [ 77.115361] niks3-server[985]: 2026/09/23 16:13:43 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls lb # [ 77.127969] niks3-server[985]: 2026/09/23 16:13:43 INFO Received complete push request method=POST path=/api/pushes/16/complete lb # [ 77.131349] niks3-server[985]: 2026/09/23 16:13:43 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst worker1 # [ 77.141942] nix-grpc-daemon[870]: time=2026-09-23T16:13:43.480Z level=INFO msg="Upload complete. (225ms)" lb # [ 77.142575] niks3-server[985]: 2026/09/23 16:13:43 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo worker1 # [ 77.157100] nix-grpc-daemon[487]: ts=2026-09-23T16:13:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 77.175114] nix-grpc-daemon[487]: ts=2026-09-23T16:13:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 77.181343] nix-daemon[856]: reaped child process 1053, status = succeeded client # [ 77.224144] systemd[1]: dedup2.service: Deactivated successfully. client # [ 77.228185] systemd[1]: dedup2.service: Consumed 208ms CPU time over 3.792s wall clock time, 35.2M memory peak, 6K incoming IP traffic, 5.6K outgoing IP traffic. client # [ 77.235985] systemd[1]: dedup1.service: Deactivated successfully. client # [ 77.240759] systemd[1]: dedup1.service: Consumed 169ms CPU time over 4.892s wall clock time, 41.5M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds) (finished: subtest: two clients wanting the same drv build it once, in 6.07 seconds) subtest: --timeout and --max-silent-time reach the builder worker2 # [ 78.602461] nix-grpc-daemon[786]: ts=2026-09-23T16:13:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37594 worker1 # [ 78.606309] nix-grpc-daemon[487]: ts=2026-09-23T16:13:44Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=0 worker1 # [ 78.612706] nix-grpc-daemon[487]: ts=2026-09-23T16:13:44Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 78.630605] nix-grpc-daemon[786]: ts=2026-09-23T16:13:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 78.645066] nix-grpc-daemon[786]: ts=2026-09-23T16:13:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 78.659088] nix-grpc-daemon[786]: ts=2026-09-23T16:13:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 78.667211] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) worker2 # [ 78.795358] nix-daemon[844]: accepted connection from pid 993, user nix-grpc-daemon (trusted) worker2 # [ 78.817212] nix-daemon[844]: reaped child process 996, status = succeeded lb # [ 78.805392] niks3-server[985]: 2026/09/23 16:13:45 INFO Received push request method=POST path=/api/pushes worker2 # [ 78.832427] nix-grpc-daemon[869]: time=2026-09-23T16:13:45.156Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 78.834629] nix-grpc-daemon[869]: time=2026-09-23T16:13:45.157Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)" lb # [ 78.841762] niks3-server[985]: 2026/09/23 16:13:45 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign worker2 # [ 78.866790] nix-grpc-daemon[869]: time=2026-09-23T16:13:45.191Z level=INFO msg="Uploading 1 narinfos" lb # [ 78.856006] niks3-server[985]: 2026/09/23 16:13:45 INFO Signed narinfos id=17 count=1 lb # [ 78.863490] niks3-server[985]: 2026/09/23 16:13:45 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst lb # [ 78.879123] niks3-server[985]: 2026/09/23 16:13:45 INFO Received complete push request method=POST path=/api/pushes/17/complete lb # [ 78.887745] niks3-server[985]: 2026/09/23 16:13:45 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls worker2 # [ 78.910748] nix-grpc-daemon[869]: time=2026-09-23T16:13:45.235Z level=INFO msg="Upload complete. (233ms)" lb # [ 78.894824] niks3-server[985]: 2026/09/23 16:13:45 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo worker2 # [ 78.921338] nix-grpc-daemon[786]: ts=2026-09-23T16:13:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 78.936490] nix-grpc-daemon[786]: ts=2026-09-23T16:13:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 78.944835] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) worker2 # [ 78.955210] nix-daemon[844]: reaped child process 991, status = succeeded worker2 # [ 78.962933] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) lb # [ 79.258489] envoy[895]: conn peer=[2001:db8:1::1]:39004 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9541 tx=4382 ms=5641 lb # [ 79.264445] envoy[895]: conn peer=[2001:db8:1::1]:49408 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=72 lb # [ 79.269590] envoy[895]: conn peer=[2001:db8:1::1]:49422 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=375 ms=56 lb # [ 79.275242] envoy[895]: conn peer=[2001:db8:1::1]:49394 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2928 ms=167 lb # [ 79.281297] envoy[895]: conn peer=[2001:db8:1::1]:49432 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=37 lb # [ 79.286368] envoy[895]: conn peer=[2001:db8:1::1]:49452 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=3515 lb # [ 79.292194] envoy[895]: conn peer=[2001:db8:1::1]:49444 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3044 tx=1563 ms=4651 worker2 # [ 84.777264] nix-grpc-daemon[786]: ts=2026-09-23T16:13:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=5 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=0 worker2 # [ 84.802006] nix-daemon[844]: reaped child process 998, status = succeeded worker2 # [ 84.803823] nix-daemon[844]: reaped child process 1000, status = succeeded worker2 # [ 85.023126] nix-grpc-daemon[786]: ts=2026-09-23T16:13:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37594 worker1 # [ 85.028425] nix-grpc-daemon[487]: ts=2026-09-23T16:13:51Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=0 worker1 # [ 85.028750] nix-grpc-daemon[487]: ts=2026-09-23T16:13:51Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 85.041885] nix-grpc-daemon[487]: ts=2026-09-23T16:13:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 85.058370] nix-grpc-daemon[487]: ts=2026-09-23T16:13:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 85.070246] nix-grpc-daemon[487]: ts=2026-09-23T16:13:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 85.085388] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 85.240597] nix-daemon[856]: accepted connection from pid 1107, user nix-grpc-daemon (trusted) worker1 # [ 85.257466] nix-daemon[856]: reaped child process 1110, status = succeeded lb # [ 85.266481] niks3-server[985]: 2026/09/23 16:13:51 INFO Received push request method=POST path=/api/pushes worker1 # [ 85.276667] nix-grpc-daemon[870]: time=2026-09-23T16:13:51.614Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 85.278160] nix-grpc-daemon[870]: time=2026-09-23T16:13:51.614Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)" lb # [ 85.296818] niks3-server[985]: 2026/09/23 16:13:51 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign worker1 # [ 85.305305] nix-grpc-daemon[870]: time=2026-09-23T16:13:51.643Z level=INFO msg="Uploading 1 narinfos" lb # [ 85.307873] niks3-server[985]: 2026/09/23 16:13:51 INFO Signed narinfos id=18 count=1 lb # [ 85.325723] niks3-server[985]: 2026/09/23 16:13:51 INFO Received complete push request method=POST path=/api/pushes/18/complete lb # [ 85.335375] niks3-server[985]: 2026/09/23 16:13:51 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls worker1 # [ 85.342105] nix-grpc-daemon[870]: time=2026-09-23T16:13:51.680Z level=INFO msg="Upload complete. (241ms)" lb # [ 85.342142] niks3-server[985]: 2026/09/23 16:13:51 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst lb # [ 85.348560] niks3-server[985]: 2026/09/23 16:13:51 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo worker1 # [ 85.352300] nix-grpc-daemon[487]: ts=2026-09-23T16:13:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 85.368737] nix-grpc-daemon[487]: ts=2026-09-23T16:13:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 85.376358] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 85.384397] nix-daemon[856]: reaped child process 1105, status = succeeded worker1 # [ 85.394685] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) lb # [ 89.260828] envoy[895]: conn peer=[2001:db8:1::1]:49456 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3014 tx=978 ms=6203 worker1 # [ 91.409310] nix-grpc-daemon[487]: ts=2026-09-23T16:13:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=0 worker1 # [ 91.431555] nix-daemon[856]: reaped child process 1114, status = succeeded worker1 # [ 91.433206] nix-daemon[856]: reaped child process 1112, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 13.19 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 91.627381] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: 971a6aba2c7e4663976ca2f65ce3b22f client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.08 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share worker1 # [ 91.822942] nix-grpc-daemon[487]: ts=2026-09-23T16:13:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:39554 client # [ 91.842292] nix[1081]: don't know how to build these paths: client # [ 91.843376] nix[1081]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 91.843224] nix-grpc-daemon[487]: ts=2026-09-23T16:13:58Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker1 # [ 91.850461] nix-grpc-daemon[487]: ts=2026-09-23T16:13:58Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 91.868774] nix-grpc-daemon[786]: ts=2026-09-23T16:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 91.881308] nix-grpc-daemon[786]: ts=2026-09-23T16:13:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37594 client # [ 91.891901] nix[1081]: copying 1 paths to 'lb:50051'... client # [ 91.894043] nix[1081]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'... worker2 # [ 91.894478] nix-grpc-daemon[786]: ts=2026-09-23T16:13:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 91.909821] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) worker2 # [ 92.067685] nix-daemon[844]: accepted connection from pid 1023, user nix-grpc-daemon (trusted) worker2 # [ 92.087307] nix-daemon[844]: reaped child process 1026, status = succeeded lb # [ 92.075987] niks3-server[985]: 2026/09/23 16:13:58 INFO Received push request method=POST path=/api/pushes worker2 # [ 92.103411] nix-grpc-daemon[869]: time=2026-09-23T16:13:58.427Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 92.105224] nix-grpc-daemon[869]: time=2026-09-23T16:13:58.427Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)" lb # [ 92.120797] niks3-server[985]: 2026/09/23 16:13:58 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign worker2 # [ 92.146529] nix-grpc-daemon[869]: time=2026-09-23T16:13:58.471Z level=INFO msg="Uploading 1 narinfos" lb # [ 92.134479] niks3-server[985]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst lb # [ 92.143006] niks3-server[985]: 2026/09/23 16:13:58 INFO Signed narinfos id=19 count=1 lb # [ 92.151845] niks3-server[985]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls lb # [ 92.156861] niks3-server[985]: 2026/09/23 16:13:58 INFO Received complete push request method=POST path=/api/pushes/19/complete lb # [ 92.161501] niks3-server[985]: 2026/09/23 16:13:58 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo worker2 # [ 92.177226] nix-grpc-daemon[869]: time=2026-09-23T16:13:58.502Z level=INFO msg="Upload complete. (249ms)" worker2 # [ 92.188510] nix-grpc-daemon[786]: ts=2026-09-23T16:13:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 92.204785] nix-grpc-daemon[786]: ts=2026-09-23T16:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 92.213510] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) worker2 # [ 92.219594] nix-daemon[844]: reaped child process 1021, status = succeeded worker2 # [ 92.230148] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) client # [ 92.227070] nix[1081]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client # [ 92.730436] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 20c68947242647e480b0157788aee4b7 client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds) client # [ 92.946470] nix[1097]: don't know how to build these paths: client # [ 92.948526] nix[1097]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 92.933635] nix-grpc-daemon[487]: ts=2026-09-23T16:13:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 92.947405] nix-grpc-daemon[487]: ts=2026-09-23T16:13:59Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker2 # [ 92.965703] nix-grpc-daemon[786]: ts=2026-09-23T16:13:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 client # [ 92.966619] nix[1097]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client: must succeed: sleep 2; systemctl kill -s INT share1 client # [ 96.136824] systemd[1]: share1.service: Sent signal SIGINT to main process 1081 (nix) on client request. client # [ 96.147291] nix[1081]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 96.149525] nix[1081]: error: interrupted by the user client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 96.217890] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 96.220085] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 96.223231] systemd[1]: share1.service: Consumed 177ms CPU time over 4.595s wall clock time, 39.2M memory peak, 6.2K incoming IP traffic, 8K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 1.11 seconds) client: waiting for success: ! systemctl is-active share2 worker2 # [ 97.737660] nix-daemon[844]: reaped child process 1030, status = succeeded worker2 # [ 97.806615] nix-daemon[844]: accepted connection from pid 1064, user nix-grpc-daemon (trusted) worker2 # [ 97.826554] nix-daemon[844]: reaped child process 1067, status = succeeded lb # [ 97.815984] niks3-server[985]: 2026/09/23 16:14:04 INFO Received push request method=POST path=/api/pushes worker2 # [ 97.847871] nix-grpc-daemon[869]: time=2026-09-23T16:14:04.169Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 97.849729] nix-grpc-daemon[869]: time=2026-09-23T16:14:04.169Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)" lb # [ 97.869031] niks3-server[985]: 2026/09/23 16:14:04 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 97.882633] niks3-server[985]: 2026/09/23 16:14:04 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign worker2 # [ 97.903853] nix-grpc-daemon[869]: time=2026-09-23T16:14:04.228Z level=INFO msg="Uploading 1 narinfos" lb # [ 97.887204] niks3-server[985]: 2026/09/23 16:14:04 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv lb # [ 97.894545] niks3-server[985]: 2026/09/23 16:14:04 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls lb # [ 97.902309] niks3-server[985]: 2026/09/23 16:14:04 INFO Signed narinfos id=20 count=1 lb # [ 97.913149] niks3-server[985]: 2026/09/23 16:14:04 INFO Received complete push request method=POST path=/api/pushes/20/complete worker2 # [ 97.934847] nix-grpc-daemon[869]: time=2026-09-23T16:14:04.259Z level=INFO msg="Upload complete. (285ms)" lb # [ 97.921145] niks3-server[985]: 2026/09/23 16:14:04 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo worker2 # [ 97.948494] nix-grpc-daemon[786]: ts=2026-09-23T16:14:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker2 # [ 97.966001] nix-grpc-daemon[786]: ts=2026-09-23T16:14:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker2 # [ 97.972527] nix-daemon[844]: reaped child process 1028, status = succeeded client # [ 97.978718] systemd[1]: share2.service: Deactivated successfully. client # [ 97.984244] systemd[1]: share2.service: Consumed 188ms CPU time over 5.249s wall clock time, 33.3M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds) (finished: subtest: the first client leaving does not fail the second, in 7.14 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 98.837917] systemd[1]: Started Nix Daemon. client # [ 98.953416] nix-daemon[1149]: accepted connection from pid 1146, user root (trusted) worker2 # [ 99.103695] nix-grpc-daemon[786]: ts=2026-09-23T16:14:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:37594 worker1 # [ 99.094613] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 99.109995] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=0 worker1 # [ 99.118024] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 99.127797] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 99.139649] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 99.154792] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 99.170169] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) lb # [ 99.259541] envoy[895]: conn peer=[2001:db8:1::1]:45576 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2976 tx=951 ms=6419 lb # [ 99.265029] envoy[895]: conn peer=[2001:db8:1::1]:35438 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2986 tx=797 ms=4336 lb # [ 99.269923] envoy[895]: conn peer=[2001:db8:1::1]:35452 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=5014 worker1 # [ 99.328779] nix-daemon[856]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker1 # [ 99.348450] nix-daemon[856]: reaped child process 1162, status = succeeded lb # [ 99.353412] niks3-server[985]: 2026/09/23 16:14:05 INFO Received push request method=POST path=/api/pushes worker1 # [ 99.365784] nix-grpc-daemon[870]: time=2026-09-23T16:14:05.703Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 99.367227] nix-grpc-daemon[870]: time=2026-09-23T16:14:05.703Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)" lb # [ 99.388550] niks3-server[985]: 2026/09/23 16:14:05 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls worker1 # [ 99.407565] nix-grpc-daemon[870]: time=2026-09-23T16:14:05.745Z level=INFO msg="Uploading 1 narinfos" lb # [ 99.405611] niks3-server[985]: 2026/09/23 16:14:05 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign lb # [ 99.413423] niks3-server[985]: 2026/09/23 16:14:05 INFO Signed narinfos id=21 count=1 lb # [ 99.421425] niks3-server[985]: 2026/09/23 16:14:05 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst lb # [ 99.436224] niks3-server[985]: 2026/09/23 16:14:05 INFO Received complete push request method=POST path=/api/pushes/21/complete worker1 # [ 99.446935] nix-grpc-daemon[870]: time=2026-09-23T16:14:05.785Z level=INFO msg="Upload complete. (260ms)" lb # [ 99.446852] niks3-server[985]: 2026/09/23 16:14:05 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo worker1 # [ 99.459128] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 99.476468] nix-grpc-daemon[487]: ts=2026-09-23T16:14:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 99.483965] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 99.490892] nix-daemon[856]: reaped child process 1157, status = succeeded worker1 # [ 99.503724] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 100.591587] nix-daemon[856]: reaped child process 1166, status = succeeded worker1 # [ 100.655946] nix-daemon[856]: accepted connection from pid 1182, user nix-grpc-daemon (trusted) worker1 # [ 100.674294] nix-daemon[856]: reaped child process 1185, status = succeeded lb # [ 100.681723] niks3-server[985]: 2026/09/23 16:14:07 INFO Received push request method=POST path=/api/pushes worker1 # [ 100.696466] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.032Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 100.698080] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.032Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)" lb # [ 100.729158] niks3-server[985]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst worker1 # [ 100.742121] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.080Z level=INFO msg="Uploading 1 narinfos" lb # [ 100.742629] niks3-server[985]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv lb # [ 100.753021] niks3-server[985]: 2026/09/23 16:14:07 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign lb # [ 100.764442] niks3-server[985]: 2026/09/23 16:14:07 INFO Signed narinfos id=22 count=1 lb # [ 100.766804] niks3-server[985]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls lb # [ 100.775107] niks3-server[985]: 2026/09/23 16:14:07 INFO Received complete push request method=POST path=/api/pushes/22/complete worker1 # [ 100.777342] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.115Z level=INFO msg="Upload complete. (259ms)" lb # [ 100.779630] niks3-server[985]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo worker1 # [ 100.792871] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=1 worker2 # [ 100.811881] nix-grpc-daemon[786]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37594 worker1 # [ 100.809791] nix-daemon[856]: reaped child process 1164, status = succeeded worker2 # [ 100.825475] nix-grpc-daemon[786]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'... worker1 # [ 100.854429] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 100.862291] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_out=128 worker1 # [ 101.101788] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:39554 worker2 # [ 101.120927] nix-grpc-daemon[786]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 101.133264] nix-grpc-daemon[869]: time=2026-09-23T16:14:07.456Z level=INFO msg="All 1 paths already cached" worker1 # [ 101.131491] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=0 worker1 # [ 101.138503] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 101.147401] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 101.158437] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 101.172780] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.511Z level=INFO msg="All 1 paths already cached" worker1 # [ 101.182690] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 101.196949] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 101.355716] nix-daemon[856]: accepted connection from pid 1191, user nix-grpc-daemon (trusted) worker1 # [ 101.372003] nix-daemon[856]: reaped child process 1194, status = succeeded lb # [ 101.377739] niks3-server[985]: 2026/09/23 16:14:07 INFO Received push request method=POST path=/api/pushes worker1 # [ 101.388967] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.727Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 101.390411] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.727Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)" lb # [ 101.408144] niks3-server[985]: 2026/09/23 16:14:07 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign worker1 # [ 101.418008] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.756Z level=INFO msg="Uploading 1 narinfos" lb # [ 101.421468] niks3-server[985]: 2026/09/23 16:14:07 INFO Signed narinfos id=23 count=1 lb # [ 101.430861] niks3-server[985]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst worker1 # [ 101.446283] nix-grpc-daemon[870]: time=2026-09-23T16:14:07.784Z level=INFO msg="Upload complete. (233ms)" lb # [ 101.445280] niks3-server[985]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls lb # [ 101.450421] niks3-server[985]: 2026/09/23 16:14:07 INFO Received complete push request method=POST path=/api/pushes/23/complete worker1 # [ 101.456286] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 101.455395] niks3-server[985]: 2026/09/23 16:14:07 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo worker1 # [ 101.471051] nix-grpc-daemon[487]: ts=2026-09-23T16:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 101.477561] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 101.482873] nix-daemon[856]: reaped child process 1189, status = succeeded worker1 # [ 101.494998] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 102.587214] nix-daemon[856]: reaped child process 1198, status = succeeded worker1 # [ 102.670934] nix-daemon[856]: accepted connection from pid 1214, user nix-grpc-daemon (trusted) worker1 # [ 102.691166] nix-daemon[856]: reaped child process 1217, status = succeeded lb # [ 102.693084] niks3-server[985]: 2026/09/23 16:14:09 INFO Received push request method=POST path=/api/pushes worker1 # [ 102.708438] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.044Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 102.710173] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.044Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)" lb # [ 102.748484] niks3-server[985]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst worker1 # [ 102.762788] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.101Z level=INFO msg="Uploading 1 narinfos" lb # [ 102.762496] niks3-server[985]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls lb # [ 102.772863] niks3-server[985]: 2026/09/23 16:14:09 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign lb # [ 102.787910] niks3-server[985]: 2026/09/23 16:14:09 INFO Signed narinfos id=24 count=1 worker1 # [ 102.796742] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.135Z level=INFO msg="Upload complete. (289ms)" lb # [ 102.796380] niks3-server[985]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv lb # [ 102.804426] niks3-server[985]: 2026/09/23 16:14:09 INFO Received complete push request method=POST path=/api/pushes/24/complete worker1 # [ 102.804582] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=1 lb # [ 102.810179] niks3-server[985]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo worker1 # [ 102.815011] nix-daemon[856]: reaped child process 1196, status = succeeded worker2 # [ 102.836119] nix-grpc-daemon[786]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 102.844283] nix-grpc-daemon[786]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'... worker1 # [ 102.868587] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 102.875733] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_out=176 worker1 # [ 103.114876] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 103.122598] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 103.130329] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.466Z level=INFO msg="All 2 paths already cached" worker1 # [ 103.142281] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=0 worker1 # [ 103.151171] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 103.159851] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 103.167417] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 103.178469] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.517Z level=INFO msg="All 2 paths already cached" worker1 # [ 103.188566] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 103.202618] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 103.349070] nix-daemon[856]: accepted connection from pid 1223, user nix-grpc-daemon (trusted) worker1 # [ 103.371293] nix-daemon[856]: reaped child process 1226, status = succeeded lb # [ 103.373235] niks3-server[985]: 2026/09/23 16:14:09 INFO Received push request method=POST path=/api/pushes worker1 # [ 103.385165] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.723Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 103.386679] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.723Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)" lb # [ 103.404673] niks3-server[985]: 2026/09/23 16:14:09 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign worker1 # [ 103.414952] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.753Z level=INFO msg="Uploading 1 narinfos" lb # [ 103.419640] niks3-server[985]: 2026/09/23 16:14:09 INFO Signed narinfos id=25 count=1 lb # [ 103.427491] niks3-server[985]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst lb # [ 103.440842] niks3-server[985]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls lb # [ 103.444559] niks3-server[985]: 2026/09/23 16:14:09 INFO Received complete push request method=POST path=/api/pushes/25/complete worker1 # [ 103.450346] nix-grpc-daemon[870]: time=2026-09-23T16:14:09.788Z level=INFO msg="Upload complete. (232ms)" lb # [ 103.451520] niks3-server[985]: 2026/09/23 16:14:09 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo worker1 # [ 103.460370] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 103.472603] nix-grpc-daemon[487]: ts=2026-09-23T16:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 103.484229] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 103.492647] nix-daemon[856]: reaped child process 1221, status = succeeded worker1 # [ 103.503132] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 104.792776] nix-daemon[856]: reaped child process 1230, status = succeeded worker1 # [ 104.876950] nix-daemon[856]: accepted connection from pid 1246, user nix-grpc-daemon (trusted) worker1 # [ 104.898121] nix-daemon[856]: reaped child process 1249, status = succeeded lb # [ 104.902804] niks3-server[985]: 2026/09/23 16:14:11 INFO Received push request method=POST path=/api/pushes worker1 # [ 104.916444] nix-grpc-daemon[870]: time=2026-09-23T16:14:11.253Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 104.918049] nix-grpc-daemon[870]: time=2026-09-23T16:14:11.253Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)" lb # [ 104.947375] niks3-server[985]: 2026/09/23 16:14:11 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst worker1 # [ 104.962231] nix-grpc-daemon[870]: time=2026-09-23T16:14:11.300Z level=INFO msg="Uploading 1 narinfos" lb # [ 104.963686] niks3-server[985]: 2026/09/23 16:14:11 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign lb # [ 104.973918] niks3-server[985]: 2026/09/23 16:14:11 INFO Signed narinfos id=26 count=1 lb # [ 104.979513] niks3-server[985]: 2026/09/23 16:14:11 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv lb # [ 104.991311] niks3-server[985]: 2026/09/23 16:14:11 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls worker1 # [ 105.000490] nix-grpc-daemon[870]: time=2026-09-23T16:14:11.339Z level=INFO msg="Upload complete. (287ms)" lb # [ 104.999200] niks3-server[985]: 2026/09/23 16:14:11 INFO Received complete push request method=POST path=/api/pushes/26/complete lb # [ 105.002963] niks3-server[985]: 2026/09/23 16:14:11 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo worker1 # [ 105.017454] nix-grpc-daemon[487]: ts=2026-09-23T16:14:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=1 worker1 # [ 105.035683] nix-grpc-daemon[487]: ts=2026-09-23T16:14:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 105.040312] nix-daemon[856]: reaped child process 1228, status = succeeded worker2 # [ 105.075387] nix-grpc-daemon[786]: ts=2026-09-23T16:14:11Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 105.095937] nix-grpc-daemon[786]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'... worker2 # [ 105.119825] nix-grpc-daemon[786]: ts=2026-09-23T16:14:11Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_out=232 client # [ 105.230371] nix-daemon[1149]: reaped child process 1151, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.55 seconds) client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.68 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.44 seconds) client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.19 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.23 seconds) worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 106.480455] nix-daemon[1149]: accepted connection from pid 1231, user root (trusted) client # this derivation will be built: client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 106.643635] nix-grpc-daemon[786]: ts=2026-09-23T16:14:12Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 106.652268] nix-grpc-daemon[786]: ts=2026-09-23T16:14:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37594 client # copying 1 paths... client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'... worker1 # [ 106.650837] nix-grpc-daemon[487]: ts=2026-09-23T16:14:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 106.668236] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 106.839486] nix-daemon[856]: accepted connection from pid 1258, user nix-grpc-daemon (trusted) worker1 # [ 106.859269] nix-daemon[856]: reaped child process 1261, status = succeeded lb # [ 106.862933] niks3-server[985]: 2026/09/23 16:14:13 INFO Received push request method=POST path=/api/pushes worker1 # [ 106.877079] nix-grpc-daemon[870]: time=2026-09-23T16:14:13.214Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 106.878784] nix-grpc-daemon[870]: time=2026-09-23T16:14:13.214Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)" lb # [ 106.896472] niks3-server[985]: 2026/09/23 16:14:13 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign worker1 # [ 106.908676] nix-grpc-daemon[870]: time=2026-09-23T16:14:13.246Z level=INFO msg="Uploading 1 narinfos" lb # [ 106.910117] niks3-server[985]: 2026/09/23 16:14:13 INFO Signed narinfos id=27 count=1 lb # [ 106.918443] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst lb # [ 106.935838] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls lb # [ 106.943364] niks3-server[985]: 2026/09/23 16:14:13 INFO Received complete push request method=POST path=/api/pushes/27/complete worker1 # [ 106.950411] nix-grpc-daemon[870]: time=2026-09-23T16:14:13.289Z level=INFO msg="Upload complete. (265ms)" lb # [ 106.951604] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo worker1 # [ 106.960281] nix-grpc-daemon[487]: ts=2026-09-23T16:14:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_in=120 worker1 # [ 106.968944] nix-daemon[856]: reaped child process 1256, status = succeeded worker1 # [ 106.977849] nix-grpc-daemon[487]: ts=2026-09-23T16:14:13Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=0 worker1 # [ 106.982249] nix-grpc-daemon[487]: ts=2026-09-23T16:14:13Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 107.000900] nix-grpc-daemon[786]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 107.016927] nix-grpc-daemon[786]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37594 client # copying 2 paths to 'lb:50051'... client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'... worker2 # [ 107.036240] nix-grpc-daemon[786]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'... worker2 # [ 107.059501] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) worker2 # [ 107.232126] nix-daemon[844]: accepted connection from pid 1077, user nix-grpc-daemon (trusted) worker2 # [ 107.253726] nix-daemon[844]: reaped child process 1080, status = succeeded lb # [ 107.241633] niks3-server[985]: 2026/09/23 16:14:13 INFO Received push request method=POST path=/api/pushes worker2 # [ 107.271876] nix-grpc-daemon[869]: time=2026-09-23T16:14:13.594Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 107.275489] nix-grpc-daemon[869]: time=2026-09-23T16:14:13.594Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)" worker2 # [ 107.277892] nix-grpc-daemon[869]: time=2026-09-23T16:14:13.594Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)" lb # [ 107.285947] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst lb # [ 107.303226] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls lb # [ 107.312519] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst worker2 # [ 107.337426] nix-grpc-daemon[869]: time=2026-09-23T16:14:13.661Z level=INFO msg="Uploading 2 narinfos" lb # [ 107.321545] niks3-server[985]: 2026/09/23 16:14:13 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign lb # [ 107.328958] niks3-server[985]: 2026/09/23 16:14:13 INFO Signed narinfos id=28 count=2 lb # [ 107.334737] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls lb # [ 107.347859] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo lb # [ 107.360720] niks3-server[985]: 2026/09/23 16:14:13 INFO Received complete push request method=POST path=/api/pushes/28/complete worker2 # [ 107.386505] nix-grpc-daemon[869]: time=2026-09-23T16:14:13.711Z level=INFO msg="Upload complete. (303ms)" lb # [ 107.371303] niks3-server[985]: 2026/09/23 16:14:13 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo worker2 # [ 107.396388] nix-grpc-daemon[786]: ts=2026-09-23T16:14:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=2 nar_bytes_in=984 worker2 # [ 107.412952] nix-grpc-daemon[786]: ts=2026-09-23T16:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 107.425949] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) worker2 # [ 107.433208] nix-daemon[844]: reaped child process 1075, status = succeeded worker2 # [ 107.451243] nix-grpc-daemon[786]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'... client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051... client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker2 # [ 107.473923] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 108.740177] nix-daemon[844]: reaped child process 1085, status = succeeded worker2 # [ 108.849408] nix-daemon[844]: accepted connection from pid 1101, user nix-grpc-daemon (trusted) worker2 # [ 108.869548] nix-daemon[844]: reaped child process 1104, status = succeeded lb # [ 108.852974] niks3-server[985]: 2026/09/23 16:14:15 INFO Received push request method=POST path=/api/pushes worker2 # [ 108.888427] nix-grpc-daemon[869]: time=2026-09-23T16:14:15.210Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 108.896587] niks3-server[985]: 2026/09/23 16:14:15 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls worker2 # [ 108.925582] nix-grpc-daemon[869]: time=2026-09-23T16:14:15.250Z level=INFO msg="Uploading 1 narinfos" lb # [ 108.908974] niks3-server[985]: 2026/09/23 16:14:15 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign lb # [ 108.920082] niks3-server[985]: 2026/09/23 16:14:15 INFO Signed narinfos id=29 count=1 lb # [ 108.925507] niks3-server[985]: 2026/09/23 16:14:15 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv lb # [ 108.944704] niks3-server[985]: 2026/09/23 16:14:15 INFO Received complete push request method=POST path=/api/pushes/29/complete lb # [ 108.951627] niks3-server[985]: 2026/09/23 16:14:15 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo worker2 # [ 108.967157] nix-grpc-daemon[869]: time=2026-09-23T16:14:15.292Z level=INFO msg="Upload complete. (279ms)" client # copying 1 paths... worker2 # [ 108.981149] nix-grpc-daemon[786]: ts=2026-09-23T16:14:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=1 worker1 # [ 108.970977] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:39554 worker2 # [ 108.999292] nix-daemon[844]: reaped child process 1082, status = succeeded worker1 # [ 108.986933] nix-grpc-daemon[487]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'... client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'... worker1 # [ 109.029045] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 109.036949] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_out=120 client # [ 109.140244] nix-daemon[1149]: reaped child process 1234, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.87 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2 lb # [ 109.261968] envoy[895]: conn peer=[2001:db8:1::1]:42738 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=30 lb # [ 109.266985] envoy[895]: conn peer=[2001:db8:1::1]:35458 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3433 tx=1942 ms=1792 lb # [ 109.271837] envoy[895]: conn peer=[2001:db8:1::1]:42754 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3888 tx=2156 ms=1791 lb # [ 109.276782] envoy[895]: conn peer=[2001:db8:1::1]:42768 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=29 lb # [ 109.282287] envoy[895]: conn peer=[2001:db8:1::1]:42780 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=407 ms=75 lb # [ 109.287852] envoy[895]: conn peer=[2001:db8:1::1]:42778 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4340 tx=2390 ms=2012 client # [ 109.309294] nix-daemon[1149]: accepted connection from pid 1253, user root (trusted) lb # [ 109.292851] envoy[895]: conn peer=[2001:db8:1::1]:42796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4240 tx=2073 ms=2434 lb # [ 109.297605] envoy[895]: conn peer=[2001:db8:1::1]:42810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=33 client # this derivation will be built: client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 109.442680] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:39554 client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 109.450643] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 109.459249] nix-grpc-daemon[870]: time=2026-09-23T16:14:15.795Z level=INFO msg="All 1 paths already cached" worker1 # [ 109.471249] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=0 worker1 # [ 109.479221] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 109.488345] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 109.499390] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39554 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'... worker1 # [ 109.520746] nix-grpc-daemon[487]: ts=2026-09-23T16:14:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 109.535810] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 109.556214] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 1/5) worker1 # [ 109.605277] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 125 ms (attempt 2/5) worker1 # [ 109.732342] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 361 ms (attempt 3/5) worker1 # [ 110.097065] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 376 ms (attempt 4/5) worker1 # [ 110.474578] nix-grpc-daemon[487]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 110.481526] nix-grpc-daemon[487]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 110.490256] nix-grpc-daemon[487]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'... worker1 # [ 110.661636] nix-daemon[856]: accepted connection from pid 1275, user nix-grpc-daemon (trusted) worker1 # [ 110.684495] nix-daemon[856]: reaped child process 1278, status = succeeded lb # [ 110.686791] niks3-server[985]: 2026/09/23 16:14:17 INFO Received push request method=POST path=/api/pushes worker1 # [ 110.699825] nix-grpc-daemon[870]: time=2026-09-23T16:14:17.037Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 110.701571] nix-grpc-daemon[870]: time=2026-09-23T16:14:17.037Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)" lb # [ 110.716907] niks3-server[985]: 2026/09/23 16:14:17 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst worker1 # [ 110.734685] nix-grpc-daemon[870]: time=2026-09-23T16:14:17.072Z level=INFO msg="Uploading 1 narinfos" lb # [ 110.732840] niks3-server[985]: 2026/09/23 16:14:17 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign lb # [ 110.740271] niks3-server[985]: 2026/09/23 16:14:17 INFO Signed narinfos id=30 count=1 lb # [ 110.749376] niks3-server[985]: 2026/09/23 16:14:17 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls lb # [ 110.761440] niks3-server[985]: 2026/09/23 16:14:17 INFO Received complete push request method=POST path=/api/pushes/30/complete lb # [ 110.768776] niks3-server[985]: 2026/09/23 16:14:17 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo worker1 # [ 110.771492] nix-grpc-daemon[870]: time=2026-09-23T16:14:17.110Z level=INFO msg="Upload complete. (260ms)" worker1 # [ 110.782784] nix-grpc-daemon[487]: ts=2026-09-23T16:14:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 paths=1 nar_bytes_in=568 worker1 # [ 110.794565] nix-grpc-daemon[487]: ts=2026-09-23T16:14:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 110.807215] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) worker1 # [ 110.813399] nix-daemon[856]: reaped child process 1268, status = succeeded client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051... worker1 # [ 110.828604] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted) client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 111.878412] nix-daemon[856]: reaped child process 1282, status = succeeded worker1 # [ 111.973816] nix-daemon[856]: accepted connection from pid 1298, user nix-grpc-daemon (trusted) worker1 # [ 111.996568] nix-daemon[856]: reaped child process 1301, status = succeeded lb # [ 111.999329] niks3-server[985]: 2026/09/23 16:14:18 INFO Received push request method=POST path=/api/pushes worker1 # [ 112.012405] nix-grpc-daemon[870]: time=2026-09-23T16:14:18.349Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 112.033995] niks3-server[985]: 2026/09/23 16:14:18 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls worker1 # [ 112.047905] nix-grpc-daemon[870]: time=2026-09-23T16:14:18.386Z level=INFO msg="Uploading 1 narinfos" lb # [ 112.046672] niks3-server[985]: 2026/09/23 16:14:18 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign lb # [ 112.056328] niks3-server[985]: 2026/09/23 16:14:18 INFO Signed narinfos id=31 count=1 lb # [ 112.067057] niks3-server[985]: 2026/09/23 16:14:18 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 112.077548] nix-grpc-daemon[870]: time=2026-09-23T16:14:18.416Z level=INFO msg="Upload complete. (237ms)" lb # [ 112.076798] niks3-server[985]: 2026/09/23 16:14:18 INFO Received complete push request method=POST path=/api/pushes/31/complete lb # [ 112.083007] niks3-server[985]: 2026/09/23 16:14:18 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo worker1 # [ 112.089025] nix-grpc-daemon[487]: ts=2026-09-23T16:14:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=1 client # copying 1 paths... worker2 # [ 112.119801] nix-grpc-daemon[786]: ts=2026-09-23T16:14:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37594 worker1 # [ 112.107077] nix-daemon[856]: reaped child process 1280, status = succeeded worker2 # [ 112.128529] nix-grpc-daemon[786]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'... client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'... worker1 # [ 112.159368] nix-grpc-daemon[487]: ts=2026-09-23T16:14:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 112.167291] nix-grpc-daemon[487]: ts=2026-09-23T16:14:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_out=120 client # [ 112.278695] nix-daemon[1149]: reaped child process 1256, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.14 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 112.451344] nix-daemon[1149]: accepted connection from pid 1275, user root (trusted) client # this derivation will be built: client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker1 # [ 112.601624] nix-grpc-daemon[487]: ts=2026-09-23T16:14:18Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:39554 client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 112.624864] nix-grpc-daemon[786]: ts=2026-09-23T16:14:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 112.635739] nix-grpc-daemon[869]: time=2026-09-23T16:14:18.960Z level=INFO msg="All 1 paths already cached" worker1 # [ 112.638517] nix-grpc-daemon[487]: ts=2026-09-23T16:14:18Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=0 worker1 # [ 112.645409] nix-grpc-daemon[487]: ts=2026-09-23T16:14:18Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 112.663824] nix-grpc-daemon[786]: ts=2026-09-23T16:14:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 112.680585] nix-grpc-daemon[786]: ts=2026-09-23T16:14:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 112.696738] nix-grpc-daemon[869]: time=2026-09-23T16:14:19.021Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'... worker2 # [ 112.708437] nix-grpc-daemon[786]: ts=2026-09-23T16:14:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 112.724804] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) worker2 # [ 112.884759] nix-daemon[844]: accepted connection from pid 1109, user nix-grpc-daemon (trusted) worker2 # [ 112.905288] nix-daemon[844]: reaped child process 1112, status = succeeded lb # [ 112.896654] niks3-server[985]: 2026/09/23 16:14:19 INFO Received push request method=POST path=/api/pushes worker2 # [ 112.924659] nix-grpc-daemon[869]: time=2026-09-23T16:14:19.249Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 112.926606] nix-grpc-daemon[869]: time=2026-09-23T16:14:19.249Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)" lb # [ 112.939379] niks3-server[985]: 2026/09/23 16:14:19 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst worker2 # [ 112.970657] nix-grpc-daemon[869]: time=2026-09-23T16:14:19.295Z level=INFO msg="Uploading 1 narinfos" lb # [ 112.957751] niks3-server[985]: 2026/09/23 16:14:19 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign lb # [ 112.967066] niks3-server[985]: 2026/09/23 16:14:19 INFO Signed narinfos id=32 count=1 lb # [ 112.973196] niks3-server[985]: 2026/09/23 16:14:19 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls lb # [ 112.984396] niks3-server[985]: 2026/09/23 16:14:19 INFO Received complete push request method=POST path=/api/pushes/32/complete worker2 # [ 113.012717] nix-grpc-daemon[869]: time=2026-09-23T16:14:19.337Z level=INFO msg="Upload complete. (270ms)" lb # [ 112.997151] niks3-server[985]: 2026/09/23 16:14:19 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo worker2 # [ 113.025491] nix-grpc-daemon[786]: ts=2026-09-23T16:14:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=0 paths=1 nar_bytes_in=568 worker2 # [ 113.040693] nix-grpc-daemon[786]: ts=2026-09-23T16:14:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 worker2 # [ 113.051799] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) worker2 # [ 113.058638] nix-daemon[844]: reaped child process 1107, status = succeeded worker2 # [ 113.070661] nix-daemon[844]: accepted connection from pid 786, user nix-grpc-daemon (trusted) client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051... client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 113.941912] nix-daemon[844]: reaped child process 1116, status = succeeded worker2 # [ 114.061357] nix-daemon[844]: accepted connection from pid 1132, user nix-grpc-daemon (trusted) worker2 # [ 114.082257] nix-daemon[844]: reaped child process 1135, status = succeeded lb # [ 114.070494] niks3-server[985]: 2026/09/23 16:14:20 INFO Received push request method=POST path=/api/pushes worker2 # [ 114.096527] nix-grpc-daemon[869]: time=2026-09-23T16:14:20.421Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 114.106971] niks3-server[985]: 2026/09/23 16:14:20 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker2 # [ 114.135225] nix-grpc-daemon[869]: time=2026-09-23T16:14:20.460Z level=INFO msg="Uploading 1 narinfos" lb # [ 114.122983] niks3-server[985]: 2026/09/23 16:14:20 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign lb # [ 114.132213] niks3-server[985]: 2026/09/23 16:14:20 INFO Signed narinfos id=33 count=1 lb # [ 114.141196] niks3-server[985]: 2026/09/23 16:14:20 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls worker2 # [ 114.167812] nix-grpc-daemon[869]: time=2026-09-23T16:14:20.492Z level=INFO msg="Upload complete. (281ms)" lb # [ 114.152351] niks3-server[985]: 2026/09/23 16:14:20 INFO Received complete push request method=POST path=/api/pushes/33/complete lb # [ 114.157404] niks3-server[985]: 2026/09/23 16:14:20 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo client # copying 1 paths... worker2 # [ 114.181218] nix-grpc-daemon[786]: ts=2026-09-23T16:14:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37594 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=1 worker1 # [ 114.178120] nix-grpc-daemon[487]: ts=2026-09-23T16:14:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:39554 worker2 # [ 114.200558] nix-daemon[844]: reaped child process 1114, status = succeeded worker1 # [ 114.186373] nix-grpc-daemon[487]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'... client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'... worker1 # [ 114.230156] nix-grpc-daemon[487]: ts=2026-09-23T16:14:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 114.237153] nix-grpc-daemon[487]: ts=2026-09-23T16:14:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39554 duration_s=0 paths=1 nar_bytes_out=120 client # [ 114.339796] nix-daemon[1149]: reaped child process 1278, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.06 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 8.97 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.34 seconds) client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.30 seconds) client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds) worker2 # [ 115.367908] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 115.859273] nix-grpc-daemon[786]: ts=2026-09-23T16:14:22Z level=info event=signal_exit worker2 # [ 117.839224] nix-grpc-daemon[786]: ts=2026-09-23T16:14:24Z level=info event=scheduler_restarting worker1 # [ 117.828670] nix-grpc-daemon[487]: ts=2026-09-23T16:14:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:59094 worker1 # [ 117.829112] nix-grpc-daemon[487]: ts=2026-09-23T16:14:24Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 117.869478] nix-grpc-daemon[786]: ts=2026-09-23T16:14:24Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 117.904167] nix-daemon[844]: reaped child process 846, status = succeeded worker2 # [ 118.442077] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 118.449374] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 118.455404] systemd[1]: nix-grpc-daemon.service: Consumed 2.599s CPU time over 1min 22.813s wall clock time, 63.4M memory peak, 154.9K incoming IP traffic, 202.8K outgoing IP traffic. lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.07 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2 worker1 # [ 118.739207] nix-grpc-daemon[487]: ts=2026-09-23T16:14:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:39554 worker1 # [ 118.749155] nix-grpc-daemon[487]: ts=2026-09-23T16:14:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:39554 client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.26 seconds) worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds) worker2 # [ 118.893715] systemd[1]: Starting scheduler-token.service... worker2 # [ 118.953859] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 118.960281] systemd[1]: Finished scheduler-token.service. worker2 # [ 118.961735] systemd[1]: scheduler-token.service: Consumed 33ms CPU time over 62ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 118.978167] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 119.082116] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 119.084470] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 119.088337] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 119.094844] nix-grpc-daemon[1159]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 119.104636] nix-grpc-daemon[1159]: E0000 00:00:1790180065.417813 1159 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 119.120137] nix-grpc-daemon[1159]: E0000 00:00:1790180065.419839 1159 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 119.122422] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:25Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 119.143857] nix-grpc-daemon[1159]: E0000 00:00:1790180065.434244 1199 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 119.147018] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 119.148040] nix-grpc-daemon[1159]: E0000 00:00:1790180065.453502 1199 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 119.150287] nix-grpc-daemon[1159]: E0000 00:00:1790180065.453689 1199 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 119.155927] nix-grpc-daemon[1159]: E0000 00:00:1790180065.468055 1199 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 119.160470] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 119.151312] nix-grpc-daemon[487]: ts=2026-09-23T16:14:25Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:59094 worker1 # [ 119.176609] nix-grpc-daemon[487]: ts=2026-09-23T16:14:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 119.182415] systemd[1]: Stopping Nix worker-protocol over gRPC... lb # [ 119.261572] envoy[895]: conn peer=[2001:db8:1::1]:42826 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3757 tx=2125 ms=2743 lb # [ 119.266632] envoy[895]: conn peer=[2001:db8:1::1]:43390 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=33 lb # [ 119.271471] envoy[895]: conn peer=[2001:db8:1::1]:43396 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3745 tx=2126 ms=1654 lb # [ 119.276738] envoy[895]: conn peer=[2001:db8:1::1]:43404 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=29 lb # [ 119.281849] envoy[895]: conn peer=[2001:db8:1::4]:37578 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=8622 tx=1175 ms=81973 lb # [ 119.287334] envoy[895]: conn peer=[2001:db8:1::1]:43410 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=33 worker1 # [ 119.826150] nix-grpc-daemon[487]: ts=2026-09-23T16:14:26Z level=info event=signal_exit lb # [ 120.568867] niks3-server[985]: 2026/09/23 16:14:26 INFO lead: released remote=192.168.1.3:54078 worker2 # [ 121.588496] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:27Z level=info event=scheduler_restarting addr=lb:50051 lb # [ 121.573397] envoy[895]: [2026-09-23 16:14:27.895][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 121.582375] nix-grpc-daemon[487]: ts=2026-09-23T16:14:27Z level=info event=scheduler_restarting worker1 # [ 121.582767] nix-grpc-daemon[487]: ts=2026-09-23T16:14:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:59094 worker1 # [ 121.582891] nix-grpc-daemon[487]: ts=2026-09-23T16:14:27Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 121.594994] nix-grpc-daemon[487]: ts=2026-09-23T16:14:27Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 121.595242] nix-grpc-daemon[487]: ts=2026-09-23T16:14:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:59094 worker1 # [ 121.595358] nix-grpc-daemon[487]: ts=2026-09-23T16:14:27Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 121.622479] nix-daemon[856]: reaped child process 858, status = succeeded worker2 # [ 121.788533] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 121.794242] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 121.804588] nix-grpc-daemon[487]: ts=2026-09-23T16:14:28Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 121.995596] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 122.001252] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 122.201986] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 122.207452] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 122.408772] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 122.397889] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 122.414274] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 122.408804] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 122.411440] systemd[1]: nix-grpc-daemon.service: Consumed 3.823s CPU time over 1min 51.958s wall clock time, 67.8M memory peak, 463.1K incoming IP traffic, 322.1K outgoing IP traffic. lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.08 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2 worker2 # [ 122.616990] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 122.621870] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 122.749538] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43642 worker2 # [ 122.765111] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker2 # [ 122.792587] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 1/5) worker2 # [ 122.822901] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 122.828207] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 122.898270] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 2/5) worker2 # [ 122.993528] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 267 ms (attempt 3/5) worker2 # [ 123.028901] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 123.031647] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 123.232919] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 123.238235] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 123.262640] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 545 ms (attempt 4/5) worker2 # [ 123.439564] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 123.445235] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 123.648396] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 123.652584] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 123.810355] nix-grpc-daemon[1159]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 123.817405] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 123.841462] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43642 client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 123.848885] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 123.862752] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 123.877291] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker2 # [ 123.902952] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=niks3_push_started pid=1219 worker2 # [ 124.050881] nix-grpc-daemon[1219]: time=2026-09-23T16:14:30.375Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 124.057906] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.059997] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 124.105527] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_take_over lb # [ 124.090109] niks3-server[985]: 2026/09/23 16:14:30 INFO lead: acquired remote=192.168.1.4:58012 worker2 # [ 124.169747] nix-daemon[844]: accepted connection from pid 1232, user nix-grpc-daemon (trusted) worker2 # [ 124.192181] nix-daemon[844]: reaped child process 1235, status = succeeded lb # [ 124.175841] niks3-server[985]: 2026/09/23 16:14:30 INFO Received push request method=POST path=/api/pushes worker2 # [ 124.223415] nix-grpc-daemon[1219]: time=2026-09-23T16:14:30.548Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 124.229336] nix-grpc-daemon[1219]: time=2026-09-23T16:14:30.550Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)" worker2 # [ 124.231482] nix-grpc-daemon[1219]: time=2026-09-23T16:14:30.550Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)" worker2 # [ 124.258791] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.261213] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 124.266532] niks3-server[985]: 2026/09/23 16:14:30 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign lb # [ 124.282084] niks3-server[985]: 2026/09/23 16:14:30 INFO Signed narinfos id=34 count=2 worker2 # [ 124.300107] nix-grpc-daemon[1219]: time=2026-09-23T16:14:30.623Z level=INFO msg="Uploading 2 narinfos" lb # [ 124.293862] niks3-server[985]: 2026/09/23 16:14:30 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst lb # [ 124.306578] niks3-server[985]: 2026/09/23 16:14:30 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 124.321364] niks3-server[985]: 2026/09/23 16:14:30 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls lb # [ 124.334036] niks3-server[985]: 2026/09/23 16:14:30 INFO Received complete push request method=POST path=/api/pushes/34/complete lb # [ 124.346171] niks3-server[985]: 2026/09/23 16:14:30 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 124.358795] niks3-server[985]: 2026/09/23 16:14:30 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo lb # [ 124.378628] niks3-server[985]: 2026/09/23 16:14:30 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls worker2 # [ 124.407973] nix-grpc-daemon[1219]: time=2026-09-23T16:14:30.732Z level=INFO msg="Upload complete. (352ms)" worker2 # [ 124.420159] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43642 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 124.437891] nix-daemon[844]: reaped child process 1217, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.94 seconds) worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache worker2 # [ 124.463965] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.472333] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds) worker1 # [ 124.564719] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 124.671891] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.688127] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:31Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 124.693762] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:31Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:43648 worker2 # [ 124.699331] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 124.692473] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:31Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 124.694453] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 124.698400] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 124.704071] nix-grpc-daemon[1327]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 124.705532] nix-grpc-daemon[1327]: E0000 00:00:1790180071.041692 1327 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 124.709571] nix-grpc-daemon[1327]: E0000 00:00:1790180071.048105 1327 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 124.713107] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:31Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 124.714850] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 124.748724] nix-grpc-daemon[1327]: E0000 00:00:1790180071.060406 1370 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 124.751135] nix-grpc-daemon[1327]: E0000 00:00:1790180071.077726 1370 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 124.761783] nix-grpc-daemon[1327]: E0000 00:00:1790180071.078750 1370 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 124.764822] nix-grpc-daemon[1327]: E0000 00:00:1790180071.099547 1370 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 124.779867] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:31Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.795799] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:31Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:43648 worker2 # [ 124.801155] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:31Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 1.14 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.56 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb # [ 126.015924] niks3-server[985]: 2026/09/23 16:14:32 INFO Shutdown signal received, draining in-flight requests timeout=10s lb # [ 126.023983] systemd[1]: Stopping niks3 server... lb # [ 126.025920] niks3-server[985]: 2026/09/23 16:14:32 INFO lead: released remote=192.168.1.4:58012 worker2 # [ 126.041537] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:32Z level=info event=scheduler_lock_lost status=200 worker1 # [ 126.027382] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:32Z level=info event=scheduler_lock_lost status=200 lb # [ 126.047920] systemd[1]: niks3.service: Deactivated successfully. lb # [ 126.049976] systemd[1]: Stopped niks3 server. lb # [ 126.051732] systemd[1]: niks3.service: Consumed 705ms CPU time over 1min 39.782s wall clock time, 27.7M memory peak, 98.2K incoming IP traffic, 140.7K outgoing IP traffic. lb # [ 126.062213] systemd[1]: Starting niks3 server... lb # [ 126.126983] niks3-server[1178]: 2026/09/23 16:14:32 goose: no migrations to run. current version: 20260923120000 lb # [ 126.135867] niks3-server[1178]: 2026/09/23 16:14:32 OK 1_commit_pending_closure.sql (7.09ms) lb # [ 126.139514] niks3-server[1178]: 2026/09/23 16:14:32 OK 2_object_stats_trigger.sql (2.3ms) lb # [ 126.142825] niks3-server[1178]: 2026/09/23 16:14:32 OK 3_commit_push.sql (2.91ms) lb # [ 126.143880] niks3-server[1178]: 2026/09/23 16:14:32 goose: up to current file version: 3 lb # [ 126.145787] niks3-server[1178]: 2026/09/23 16:14:32 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 126.157257] niks3-server[1178]: 2026/09/23 16:14:32 INFO Read proxy enabled — serving cache objects from S3 lb # [ 126.158582] niks3-server[1178]: 2026/09/23 16:14:32 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 126.159910] systemd[1]: Started niks3 server. lb # [ 126.163925] niks3-server[1178]: 2026/09/23 16:14:32 INFO systemd watchdog enabled interval=15s lb # [ 126.165728] niks3-server[1178]: 2026/09/23 16:14:32 INFO Starting HTTP server address=0.0.0.0:5751 lb: waiting for unit niks3.service lb: (finished: waiting for unit niks3.service, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2 worker1 # [ 126.648170] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 126.663990] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted) client # don't know how to build these paths: client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker1 # [ 126.690416] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 126.700770] nix-grpc-daemon[1327]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5) worker1 # [ 126.763324] nix-grpc-daemon[1327]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 2/5) worker1 # [ 126.801517] nix-grpc-daemon[1327]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 3/5) worker1 # [ 126.927771] nix-grpc-daemon[1327]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 538 ms (attempt 4/5) lb # [ 127.031117] niks3-server[1178]: 2026/09/23 16:14:33 INFO lead: acquired remote=192.168.1.4:46930 worker1 # [ 127.469681] nix-grpc-daemon[1327]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 127.477270] nix-grpc-daemon[1327]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 127.546105] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:33Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=0 worker2 # [ 127.546492] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:33Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 127.560465] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43642 worker2 # [ 127.580658] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43642 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'... worker2 # [ 127.597438] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43642 worker2 # [ 127.612855] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker2 # [ 127.768142] nix-daemon[844]: accepted connection from pid 1254, user nix-grpc-daemon (trusted) worker2 # [ 127.786357] nix-daemon[844]: reaped child process 1257, status = succeeded lb # [ 127.779207] niks3-server[1178]: 2026/09/23 16:14:34 INFO Received push request method=POST path=/api/pushes worker2 # [ 127.811580] nix-grpc-daemon[1219]: time=2026-09-23T16:14:34.135Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 127.813566] nix-grpc-daemon[1219]: time=2026-09-23T16:14:34.135Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)" lb # [ 127.832548] niks3-server[1178]: 2026/09/23 16:14:34 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign lb # [ 127.845285] niks3-server[1178]: 2026/09/23 16:14:34 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls lb # [ 127.872218] niks3-server[1178]: 2026/09/23 16:14:34 INFO Signed narinfos id=35 count=1 worker2 # [ 127.892117] nix-grpc-daemon[1219]: time=2026-09-23T16:14:34.216Z level=INFO msg="Uploading 1 narinfos" lb # [ 127.880157] niks3-server[1178]: 2026/09/23 16:14:34 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst lb # [ 127.892276] niks3-server[1178]: 2026/09/23 16:14:34 INFO Received complete push request method=POST path=/api/pushes/35/complete worker2 # [ 127.917525] nix-grpc-daemon[1219]: time=2026-09-23T16:14:34.242Z level=INFO msg="Upload complete. (289ms)" lb # [ 127.904070] niks3-server[1178]: 2026/09/23 16:14:34 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo worker2 # [ 127.929515] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43642 duration_s=0 paths=1 nar_bytes_in=544 worker2 # [ 127.944356] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43642 worker2 # [ 127.956260] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker2 # [ 127.962675] nix-daemon[844]: reaped child process 1252, status = succeeded client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051... worker2 # [ 127.975697] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker2 # [ 129.032491] nix-daemon[844]: reaped child process 1261, status = succeeded worker2 # [ 129.151717] nix-daemon[844]: accepted connection from pid 1277, user nix-grpc-daemon (trusted) worker2 # [ 129.171269] nix-daemon[844]: reaped child process 1280, status = succeeded lb # [ 129.162495] niks3-server[1178]: 2026/09/23 16:14:35 INFO Received push request method=POST path=/api/pushes worker2 # [ 129.199911] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.521Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 129.202192] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.522Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)" lb # [ 129.220951] niks3-server[1178]: 2026/09/23 16:14:35 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign worker2 # [ 129.249952] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.574Z level=INFO msg="Uploading 1 narinfos" lb # [ 129.234123] niks3-server[1178]: 2026/09/23 16:14:35 INFO Signed narinfos id=36 count=1 lb # [ 129.245904] niks3-server[1178]: 2026/09/23 16:14:35 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst lb # [ 129.256518] niks3-server[1178]: 2026/09/23 16:14:35 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv lb # [ 129.265897] envoy[895]: conn peer=[2001:db8:1::3]:40604 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2864 tx=1357 ms=85266 lb # [ 129.279386] envoy[895]: conn peer=[2001:db8:1::1]:52662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1010 tx=305 ms=1688 lb # [ 129.286235] niks3-server[1178]: 2026/09/23 16:14:35 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls lb # [ 129.292333] niks3-server[1178]: 2026/09/23 16:14:35 INFO Received complete push request method=POST path=/api/pushes/36/complete lb # [ 129.296165] niks3-server[1178]: 2026/09/23 16:14:35 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo worker2 # [ 129.325546] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.650Z level=INFO msg="Upload complete. (336ms)" worker2 # [ 129.351164] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43642 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 worker2 # [ 129.372236] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:35Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=0 worker2 # [ 129.376286] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:35Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 129.382459] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43642 worker2 # [ 129.387736] nix-daemon[844]: reaped child process 1259, status = succeeded worker2 # [ 129.391282] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43642 worker2 # [ 129.395869] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.720Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'... worker2 # [ 129.405457] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43642 worker2 # [ 129.417755] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker2 # [ 129.561323] nix-daemon[844]: accepted connection from pid 1286, user nix-grpc-daemon (trusted) worker2 # [ 129.582529] nix-daemon[844]: reaped child process 1289, status = succeeded lb # [ 129.572142] niks3-server[1178]: 2026/09/23 16:14:35 INFO Received push request method=POST path=/api/pushes worker2 # [ 129.598839] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.923Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 129.601012] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.925Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)" worker2 # [ 129.640653] nix-grpc-daemon[1219]: time=2026-09-23T16:14:35.965Z level=INFO msg="Uploading 1 narinfos" lb # [ 129.614706] niks3-server[1178]: 2026/09/23 16:14:35 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign lb # [ 129.629068] niks3-server[1178]: 2026/09/23 16:14:35 INFO Signed narinfos id=37 count=1 lb # [ 129.636461] niks3-server[1178]: 2026/09/23 16:14:35 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls lb # [ 129.643867] niks3-server[1178]: 2026/09/23 16:14:35 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst lb # [ 129.656046] niks3-server[1178]: 2026/09/23 16:14:35 INFO Received complete push request method=POST path=/api/pushes/37/complete lb # [ 129.666538] niks3-server[1178]: 2026/09/23 16:14:35 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo worker2 # [ 129.682885] nix-grpc-daemon[1219]: time=2026-09-23T16:14:36.007Z level=INFO msg="Upload complete. (251ms)" worker2 # [ 129.693313] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43642 duration_s=0 paths=1 nar_bytes_in=656 worker2 # [ 129.708600] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43642 worker2 # [ 129.719502] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker2 # [ 129.728316] nix-daemon[844]: reaped child process 1284, status = succeeded client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051... worker2 # [ 129.737970] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker2 # [ 129.983259] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:36Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:43658 client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker2 # [ 130.646231] nix-daemon[844]: reaped child process 1293, status = succeeded worker2 # [ 130.718602] nix-daemon[844]: accepted connection from pid 1309, user nix-grpc-daemon (trusted) worker2 # [ 130.737794] nix-daemon[844]: reaped child process 1312, status = succeeded lb # [ 130.725985] niks3-server[1178]: 2026/09/23 16:14:37 INFO Received push request method=POST path=/api/pushes worker2 # [ 130.753966] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.077Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 130.755886] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.077Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)" lb # [ 130.773761] niks3-server[1178]: 2026/09/23 16:14:37 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign worker2 # [ 130.798866] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.123Z level=INFO msg="Uploading 1 narinfos" lb # [ 130.787668] niks3-server[1178]: 2026/09/23 16:14:37 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst lb # [ 130.796274] niks3-server[1178]: 2026/09/23 16:14:37 INFO Signed narinfos id=38 count=1 lb # [ 130.798378] niks3-server[1178]: 2026/09/23 16:14:37 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv lb # [ 130.812436] niks3-server[1178]: 2026/09/23 16:14:37 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls lb # [ 130.816786] niks3-server[1178]: 2026/09/23 16:14:37 INFO Received complete push request method=POST path=/api/pushes/38/complete lb # [ 130.826566] niks3-server[1178]: 2026/09/23 16:14:37 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo worker2 # [ 130.849531] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.174Z level=INFO msg="Upload complete. (297ms)" worker2 # [ 130.870534] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43642 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 worker2 # [ 130.891269] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=0 worker2 # [ 130.895356] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 130.901393] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43642 worker2 # [ 130.906683] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43642 worker2 # [ 130.911873] nix-daemon[844]: reaped child process 1291, status = succeeded worker2 # [ 130.916225] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.241Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'... worker2 # [ 130.925971] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43642 worker2 # [ 130.940378] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker2 # [ 131.078971] nix-daemon[844]: accepted connection from pid 1318, user nix-grpc-daemon (trusted) worker2 # [ 131.098178] nix-daemon[844]: reaped child process 1321, status = succeeded lb # [ 131.085334] niks3-server[1178]: 2026/09/23 16:14:37 INFO Received push request method=POST path=/api/pushes worker2 # [ 131.111877] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.436Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 131.113941] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.436Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)" lb # [ 131.111812] niks3-server[1178]: 2026/09/23 16:14:37 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls lb # [ 131.123643] niks3-server[1178]: 2026/09/23 16:14:37 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign worker2 # [ 131.148562] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.473Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.134649] niks3-server[1178]: 2026/09/23 16:14:37 INFO Signed narinfos id=39 count=1 lb # [ 131.139163] niks3-server[1178]: 2026/09/23 16:14:37 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst lb # [ 131.149194] niks3-server[1178]: 2026/09/23 16:14:37 INFO Received complete push request method=POST path=/api/pushes/39/complete worker2 # [ 131.175140] nix-grpc-daemon[1219]: time=2026-09-23T16:14:37.500Z level=INFO msg="Upload complete. (224ms)" lb # [ 131.160470] niks3-server[1178]: 2026/09/23 16:14:37 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo worker2 # [ 131.184264] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43642 duration_s=0 paths=1 nar_bytes_in=784 worker2 # [ 131.199299] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43642 worker2 # [ 131.205896] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker2 # [ 131.210789] nix-daemon[844]: reaped child process 1316, status = succeeded worker2 # [ 131.222628] nix-daemon[844]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051... client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker2 # [ 132.487298] nix-daemon[844]: reaped child process 1325, status = succeeded worker2 # [ 132.620986] nix-daemon[844]: accepted connection from pid 1341, user nix-grpc-daemon (trusted) worker2 # [ 132.642425] nix-daemon[844]: reaped child process 1344, status = succeeded lb # [ 132.630606] niks3-server[1178]: 2026/09/23 16:14:38 INFO Received push request method=POST path=/api/pushes worker2 # [ 132.659006] nix-grpc-daemon[1219]: time=2026-09-23T16:14:38.982Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 132.661067] nix-grpc-daemon[1219]: time=2026-09-23T16:14:38.982Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)" lb # [ 132.675552] niks3-server[1178]: 2026/09/23 16:14:39 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst worker2 # [ 132.704802] nix-grpc-daemon[1219]: time=2026-09-23T16:14:39.029Z level=INFO msg="Uploading 1 narinfos" lb # [ 132.690478] niks3-server[1178]: 2026/09/23 16:14:39 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv lb # [ 132.698540] niks3-server[1178]: 2026/09/23 16:14:39 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign lb # [ 132.702656] niks3-server[1178]: 2026/09/23 16:14:39 INFO Signed narinfos id=40 count=1 lb # [ 132.711742] niks3-server[1178]: 2026/09/23 16:14:39 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls lb # [ 132.719385] niks3-server[1178]: 2026/09/23 16:14:39 INFO Received complete push request method=POST path=/api/pushes/40/complete lb # [ 132.723707] niks3-server[1178]: 2026/09/23 16:14:39 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo worker2 # [ 132.736656] nix-grpc-daemon[1219]: time=2026-09-23T16:14:39.061Z level=INFO msg="Upload complete. (302ms)" worker2 # [ 132.749361] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43642 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 worker2 # [ 132.769835] nix-daemon[844]: reaped child process 1323, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.36 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.19 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 7.07 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2 # [ 133.231196] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 134.109874] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:40Z level=info event=signal_exit lb # [ 135.043914] niks3-server[1178]: 2026/09/23 16:14:41 INFO lead: released remote=192.168.1.4:46930 worker1 # [ 136.042265] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:42Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 136.060521] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:42Z level=info event=scheduler_restarting lb # [ 136.041114] envoy[895]: [2026-09-23 16:14:42.363][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker2 # [ 136.067676] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:42Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 136.072426] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:43648 worker2 # [ 136.077297] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:42Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 136.083087] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:43648 worker2 # [ 136.088149] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:42Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 136.110797] nix-daemon[844]: reaped child process 1211, status = succeeded worker1 # [ 136.241983] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 136.248084] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 136.278441] nix-grpc-daemon[1159]: ts=2026-09-23T16:14:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 136.343872] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 136.350999] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 136.358554] systemd[1]: nix-grpc-daemon.service: Consumed 1.449s CPU time over 17.380s wall clock time, 39.6M memory peak, 201.9K incoming IP traffic, 125.9K outgoing IP traffic. worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) worker1 # [ 136.448539] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 136.451207] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 136.652398] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 136.658028] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 136.859156] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 136.865007] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 137.031217] niks3-server[1178]: 2026/09/23 16:14:43 INFO lead: acquired remote=192.168.1.3:47298 worker1 # [ 137.033088] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_take_over worker1 # [ 137.067884] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 137.072301] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 137.268833] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 137.274256] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 137.475139] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 137.485920] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) worker1 # [ 137.683382] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 137.703105] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:44Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:40894 worker1 # [ 137.706015] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:44Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 137.721581] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:44Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:40910 worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2 client # don't know how to build these paths: client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 138.816737] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 138.823979] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 138.852168] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=0 worker1 # [ 138.859596] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 138.869279] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 138.880912] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55668 client # copying 1 paths to 'lb:50051'... worker1 # [ 138.896050] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55668 client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'... worker1 # [ 138.905762] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted) worker1 # [ 138.923378] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=info event=niks3_push_started pid=1406 worker1 # [ 139.062809] nix-grpc-daemon[1406]: time=2026-09-23T16:14:45.400Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 139.174194] nix-daemon[856]: accepted connection from pid 1418, user nix-grpc-daemon (trusted) worker1 # [ 139.194092] nix-daemon[856]: reaped child process 1421, status = succeeded lb # [ 139.197900] niks3-server[1178]: 2026/09/23 16:14:45 INFO Received push request method=POST path=/api/pushes worker1 # [ 139.214326] nix-grpc-daemon[1406]: time=2026-09-23T16:14:45.551Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 139.215892] nix-grpc-daemon[1406]: time=2026-09-23T16:14:45.551Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)" lb # [ 139.248124] niks3-server[1178]: 2026/09/23 16:14:45 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign worker1 # [ 139.259286] nix-grpc-daemon[1406]: time=2026-09-23T16:14:45.596Z level=INFO msg="Uploading 1 narinfos" lb # [ 139.258600] niks3-server[1178]: 2026/09/23 16:14:45 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls lb # [ 139.270168] niks3-server[1178]: 2026/09/23 16:14:45 INFO Signed narinfos id=41 count=1 lb # [ 139.277947] niks3-server[1178]: 2026/09/23 16:14:45 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst lb # [ 139.296172] niks3-server[1178]: 2026/09/23 16:14:45 INFO Received complete push request method=POST path=/api/pushes/41/complete lb # [ 139.300728] envoy[895]: conn peer=[2001:db8:1::1]:52668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9749 tx=4478 ms=6105 lb # [ 139.304455] envoy[895]: conn peer=[2001:db8:1::4]:51300 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=13179 tx=1012 ms=17111 worker1 # [ 139.310896] nix-grpc-daemon[1406]: time=2026-09-23T16:14:45.649Z level=INFO msg="Upload complete. (244ms)" lb # [ 139.310511] niks3-server[1178]: 2026/09/23 16:14:45 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo worker1 # [ 139.320451] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=0 paths=1 nar_bytes_in=520 worker1 # [ 139.341048] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 139.348385] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted) worker1 # [ 139.354251] nix-daemon[856]: reaped child process 1404, status = succeeded client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051... worker1 # [ 139.367239] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted) client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker1 # [ 140.610158] nix-daemon[856]: reaped child process 1425, status = succeeded worker1 # [ 140.728226] nix-daemon[856]: accepted connection from pid 1441, user nix-grpc-daemon (trusted) worker1 # [ 140.746413] nix-daemon[856]: reaped child process 1444, status = succeeded lb # [ 140.750240] niks3-server[1178]: 2026/09/23 16:14:47 INFO Received push request method=POST path=/api/pushes worker1 # [ 140.764403] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.101Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 140.765959] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.101Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)" lb # [ 140.774872] niks3-server[1178]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv lb # [ 140.785608] niks3-server[1178]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls worker1 # [ 140.795971] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.134Z level=INFO msg="Uploading 1 narinfos" lb # [ 140.790922] niks3-server[1178]: 2026/09/23 16:14:47 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign lb # [ 140.796332] niks3-server[1178]: 2026/09/23 16:14:47 INFO Signed narinfos id=42 count=1 lb # [ 140.802365] niks3-server[1178]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst lb # [ 140.811368] niks3-server[1178]: 2026/09/23 16:14:47 INFO Received complete push request method=POST path=/api/pushes/42/complete lb # [ 140.816427] niks3-server[1178]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo worker1 # [ 140.817979] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.156Z level=INFO msg="Upload complete. (233ms)" worker1 # [ 140.835436] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 worker1 # [ 140.849381] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=0 worker1 # [ 140.853371] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 140.857507] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 140.861999] nix-daemon[856]: reaped child process 1423, status = succeeded worker1 # [ 140.865093] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 140.872985] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.211Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'... worker1 # [ 140.879130] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 140.888741] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted) worker1 # [ 141.012463] nix-daemon[856]: accepted connection from pid 1450, user nix-grpc-daemon (trusted) worker1 # [ 141.031341] nix-daemon[856]: reaped child process 1453, status = succeeded lb # [ 141.032964] niks3-server[1178]: 2026/09/23 16:14:47 INFO Received push request method=POST path=/api/pushes worker1 # [ 141.044762] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.383Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 141.046227] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.383Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)" lb # [ 141.062629] niks3-server[1178]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls worker1 # [ 141.077692] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.416Z level=INFO msg="Uploading 1 narinfos" lb # [ 141.076756] niks3-server[1178]: 2026/09/23 16:14:47 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign lb # [ 141.085052] niks3-server[1178]: 2026/09/23 16:14:47 INFO Signed narinfos id=43 count=1 lb # [ 141.090493] niks3-server[1178]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst worker1 # [ 141.105361] nix-grpc-daemon[1406]: time=2026-09-23T16:14:47.444Z level=INFO msg="Upload complete. (206ms)" lb # [ 141.104315] niks3-server[1178]: 2026/09/23 16:14:47 INFO Received complete push request method=POST path=/api/pushes/43/complete worker1 # [ 141.112214] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=0 paths=1 nar_bytes_in=640 lb # [ 141.111813] niks3-server[1178]: 2026/09/23 16:14:47 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo worker1 # [ 141.120698] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 141.130879] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted) worker1 # [ 141.135513] nix-daemon[856]: reaped child process 1448, status = succeeded worker1 # [ 141.145923] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted) client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051... client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker1 # [ 142.650187] nix-daemon[856]: reaped child process 1457, status = succeeded worker1 # [ 142.728354] nix-daemon[856]: accepted connection from pid 1473, user nix-grpc-daemon (trusted) worker1 # [ 142.747707] nix-daemon[856]: reaped child process 1476, status = succeeded lb # [ 142.753311] niks3-server[1178]: 2026/09/23 16:14:49 INFO Received push request method=POST path=/api/pushes worker1 # [ 142.766323] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.103Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 142.767955] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.103Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)" lb # [ 142.789188] niks3-server[1178]: 2026/09/23 16:14:49 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls lb # [ 142.804132] niks3-server[1178]: 2026/09/23 16:14:49 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst worker1 # [ 142.820136] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.158Z level=INFO msg="Uploading 1 narinfos" lb # [ 142.812895] niks3-server[1178]: 2026/09/23 16:14:49 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign lb # [ 142.822722] niks3-server[1178]: 2026/09/23 16:14:49 INFO Signed narinfos id=44 count=1 lb # [ 142.832434] niks3-server[1178]: 2026/09/23 16:14:49 INFO Received complete push request method=POST path=/api/pushes/44/complete worker1 # [ 142.846959] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.185Z level=INFO msg="Upload complete. (288ms)" lb # [ 142.847885] niks3-server[1178]: 2026/09/23 16:14:49 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker1 # [ 142.869896] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 lb # [ 142.876168] niks3-server[1178]: 2026/09/23 16:14:49 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo worker1 # [ 142.888555] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=0 worker1 # [ 142.893345] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 142.901066] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 142.907567] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 142.913364] nix-daemon[856]: reaped child process 1455, status = succeeded worker1 # [ 142.916836] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.255Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'... worker1 # [ 142.927108] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 142.939958] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted) worker1 # [ 143.080496] nix-daemon[856]: accepted connection from pid 1482, user nix-grpc-daemon (trusted) worker1 # [ 143.097524] nix-daemon[856]: reaped child process 1485, status = succeeded lb # [ 143.102939] niks3-server[1178]: 2026/09/23 16:14:49 INFO Received push request method=POST path=/api/pushes worker1 # [ 143.118216] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.456Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 143.119808] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.456Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)" lb # [ 143.139348] niks3-server[1178]: 2026/09/23 16:14:49 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign worker1 # [ 143.149918] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.488Z level=INFO msg="Uploading 1 narinfos" lb # [ 143.154888] niks3-server[1178]: 2026/09/23 16:14:49 INFO Signed narinfos id=45 count=1 lb # [ 143.163403] niks3-server[1178]: 2026/09/23 16:14:49 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls lb # [ 143.171382] niks3-server[1178]: 2026/09/23 16:14:49 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst lb # [ 143.187597] niks3-server[1178]: 2026/09/23 16:14:49 INFO Received complete push request method=POST path=/api/pushes/45/complete worker1 # [ 143.194292] nix-grpc-daemon[1406]: time=2026-09-23T16:14:49.532Z level=INFO msg="Upload complete. (242ms)" lb # [ 143.194849] niks3-server[1178]: 2026/09/23 16:14:49 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo worker1 # [ 143.205398] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 143.220565] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 143.230149] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted) worker1 # [ 143.239409] nix-daemon[856]: reaped child process 1480, status = succeeded client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051... worker1 # [ 143.248679] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted) client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker1 # [ 144.717506] nix-daemon[856]: reaped child process 1489, status = succeeded worker1 # [ 144.831924] nix-daemon[856]: accepted connection from pid 1505, user nix-grpc-daemon (trusted) worker1 # [ 144.854685] nix-daemon[856]: reaped child process 1508, status = succeeded lb # [ 144.857157] niks3-server[1178]: 2026/09/23 16:14:51 INFO Received push request method=POST path=/api/pushes worker1 # [ 144.872574] nix-grpc-daemon[1406]: time=2026-09-23T16:14:51.208Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 144.874257] nix-grpc-daemon[1406]: time=2026-09-23T16:14:51.208Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)" lb # [ 144.903905] niks3-server[1178]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst worker1 # [ 144.918299] nix-grpc-daemon[1406]: time=2026-09-23T16:14:51.256Z level=INFO msg="Uploading 1 narinfos" lb # [ 144.919337] niks3-server[1178]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv lb # [ 144.928240] niks3-server[1178]: 2026/09/23 16:14:51 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign lb # [ 144.940499] niks3-server[1178]: 2026/09/23 16:14:51 INFO Signed narinfos id=46 count=1 lb # [ 144.942868] niks3-server[1178]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls lb # [ 144.950279] niks3-server[1178]: 2026/09/23 16:14:51 INFO Received complete push request method=POST path=/api/pushes/46/complete worker1 # [ 144.957923] nix-grpc-daemon[1406]: time=2026-09-23T16:14:51.296Z level=INFO msg="Upload complete. (297ms)" lb # [ 144.957510] niks3-server[1178]: 2026/09/23 16:14:51 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo worker1 # [ 144.973414] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 worker1 # [ 144.992784] nix-daemon[856]: reaped child process 1487, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 6.43 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds) worker2 # [ 145.289655] systemd[1]: Starting scheduler-token.service... worker2 # [ 145.362833] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 145.371687] systemd[1]: Finished scheduler-token.service. worker2 # [ 145.383038] systemd[1]: scheduler-token.service: Consumed 41ms CPU time over 77ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 145.390759] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 145.497642] nix-grpc-daemon[1365]: ts=2026-09-23T16:14:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 145.499926] nix-grpc-daemon[1365]: ts=2026-09-23T16:14:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 145.503827] nix-grpc-daemon[1365]: ts=2026-09-23T16:14:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 145.510748] nix-grpc-daemon[1365]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 145.532070] nix-grpc-daemon[1365]: E0000 00:00:1790180091.833597 1365 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 145.534422] nix-grpc-daemon[1365]: E0000 00:00:1790180091.835752 1365 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 145.559218] nix-grpc-daemon[1365]: ts=2026-09-23T16:14:51Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 145.561593] nix-grpc-daemon[1365]: E0000 00:00:1790180091.845489 1405 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 145.564193] nix-grpc-daemon[1365]: E0000 00:00:1790180091.867584 1405 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 145.566424] nix-grpc-daemon[1365]: E0000 00:00:1790180091.867752 1405 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 145.568805] nix-grpc-daemon[1365]: E0000 00:00:1790180091.883334 1405 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 145.571363] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 145.585423] nix-grpc-daemon[1365]: ts=2026-09-23T16:14:51Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 145.586365] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:51Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 145.594271] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:40894 worker1 # [ 145.603496] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.90 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs client # [ 145.951543] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: 06ed96ef1c71481cbac461927da43ba3 client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds) worker1 # [ 146.134060] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:55668 client # [ 146.153630] nix[1401]: don't know how to build these paths: client # [ 146.154871] nix[1401]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker1 # [ 146.155347] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=0 worker1 # [ 146.165449] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 146.174484] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 146.182438] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:55668 client # [ 146.202951] nix[1401]: copying 1 paths to 'lb:50051'... client # [ 146.206970] nix[1401]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'... worker1 # [ 146.195548] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 146.210606] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted) worker1 # [ 146.365571] nix-daemon[856]: accepted connection from pid 1521, user nix-grpc-daemon (trusted) worker1 # [ 146.389208] nix-daemon[856]: reaped child process 1524, status = succeeded lb # [ 146.394033] niks3-server[1178]: 2026/09/23 16:14:52 INFO Received push request method=POST path=/api/pushes worker1 # [ 146.405093] nix-grpc-daemon[1406]: time=2026-09-23T16:14:52.742Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 146.406599] nix-grpc-daemon[1406]: time=2026-09-23T16:14:52.743Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)" lb # [ 146.420965] niks3-server[1178]: 2026/09/23 16:14:52 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign worker1 # [ 146.429955] nix-grpc-daemon[1406]: time=2026-09-23T16:14:52.768Z level=INFO msg="Uploading 1 narinfos" lb # [ 146.435425] niks3-server[1178]: 2026/09/23 16:14:52 INFO Signed narinfos id=47 count=1 lb # [ 146.445083] niks3-server[1178]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls lb # [ 146.452842] niks3-server[1178]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst lb # [ 146.466023] niks3-server[1178]: 2026/09/23 16:14:52 INFO Received complete push request method=POST path=/api/pushes/47/complete lb # [ 146.472521] niks3-server[1178]: 2026/09/23 16:14:52 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo worker1 # [ 146.474105] nix-grpc-daemon[1406]: time=2026-09-23T16:14:52.812Z level=INFO msg="Upload complete. (249ms)" worker1 # [ 146.483914] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 146.500395] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 worker1 # [ 146.507373] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted) worker1 # [ 146.513261] nix-daemon[856]: reaped child process 1519, status = succeeded client # [ 146.536678] nix[1401]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051... worker1 # [ 146.528966] nix-daemon[856]: accepted connection from pid 1327, user nix-grpc-daemon (trusted) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker2 # [ 147.630984] systemd-journald[271]: Received client request to rotate journal, rotating. worker2 # [ 147.645929] systemd-journald[271]: Vacuuming done, freed 0B of archived journals from /var/log/journal/200489c2afc34ee798e31e248631b567. worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds) worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker1 # [ 147.661547] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds) client: must succeed: sleep 5.0 worker1 # [ 147.737822] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:54Z level=info event=draining lb # [ 149.267600] envoy[895]: conn peer=[2001:db8:1::1]:37216 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9606 tx=4443 ms=6166 client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker1 # [ 152.963092] nix-daemon[856]: reaped child process 1528, status = succeeded worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker1 # [ 153.054291] nix-daemon[856]: accepted connection from pid 1565, user nix-grpc-daemon (trusted) worker1 # [ 153.073941] nix-daemon[856]: reaped child process 1568, status = succeeded lb # [ 153.077690] niks3-server[1178]: 2026/09/23 16:14:59 INFO Received push request method=POST path=/api/pushes worker1 # [ 153.091282] nix-grpc-daemon[1406]: time=2026-09-23T16:14:59.429Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 153.095478] nix-grpc-daemon[1406]: time=2026-09-23T16:14:59.431Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)" lb # [ 153.122795] niks3-server[1178]: 2026/09/23 16:14:59 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign worker1 # [ 153.134381] nix-grpc-daemon[1406]: time=2026-09-23T16:14:59.472Z level=INFO msg="Uploading 1 narinfos" lb # [ 153.139112] niks3-server[1178]: 2026/09/23 16:14:59 INFO Signed narinfos id=48 count=1 lb # [ 153.148925] niks3-server[1178]: 2026/09/23 16:14:59 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls lb # [ 153.159231] niks3-server[1178]: 2026/09/23 16:14:59 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 153.172537] niks3-server[1178]: 2026/09/23 16:14:59 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv lb # [ 153.175729] niks3-server[1178]: 2026/09/23 16:14:59 INFO Received complete push request method=POST path=/api/pushes/48/complete lb # [ 153.181065] niks3-server[1178]: 2026/09/23 16:14:59 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo worker1 # [ 153.183086] nix-grpc-daemon[1406]: time=2026-09-23T16:14:59.521Z level=INFO msg="Upload complete. (238ms)" worker1 # [ 153.197171] nix-grpc-daemon[1327]: ts=2026-09-23T16:14:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55668 duration_s=6 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 worker1 # [ 153.214785] nix-daemon[856]: reaped child process 1526, status = succeeded client # [ 153.240689] systemd[1]: rs.service: Deactivated successfully. client # [ 153.245070] systemd[1]: rs.service: Consumed 185ms CPU time over 7.293s wall clock time, 39.4M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic. worker1 # [ 153.739775] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:00Z level=info event=signal_exit lb # [ 154.036242] niks3-server[1178]: 2026/09/23 16:15:00 INFO lead: released remote=192.168.1.3:47298 worker1 # [ 155.041122] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:01Z level=info event=scheduler_restarting worker2 # [ 155.058435] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:01Z level=info event=scheduler_restarting addr=lb:50051 lb # [ 155.038497] envoy[895]: [2026-09-23 16:15:01.361][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream} worker1 # [ 155.042831] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:01Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 155.051825] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:40894 worker1 # [ 155.061656] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:01Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 155.066340] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:40894 worker1 # [ 155.071458] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:01Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 155.097464] nix-daemon[856]: reaped child process 1379, status = succeeded worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.17 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } worker2 # [ 155.258859] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 155.264530] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' worker1 # [ 155.266728] nix-grpc-daemon[1327]: ts=2026-09-23T16:15:01Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 155.465661] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 155.470962] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 155.507078] niks3-server[1178]: 2026/09/23 16:15:01 INFO lead: acquired remote=192.168.1.4:54160 worker2 # [ 155.528569] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:01Z level=info event=scheduler_take_over worker2 # [ 155.672170] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 155.677418] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 155.746902] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 155.755558] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 155.761904] systemd[1]: nix-grpc-daemon.service: Consumed 1.560s CPU time over 31.193s wall clock time, 39.3M memory peak, 331.7K incoming IP traffic, 146.9K outgoing IP traffic. worker1 # [ 155.788507] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 155.878689] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 155.884210] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 155.905662] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 155.907891] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 155.911699] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 155.917326] nix-grpc-daemon[1576]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 155.918758] nix-grpc-daemon[1576]: E0000 00:00:1790180102.255151 1576 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 155.922579] nix-grpc-daemon[1576]: E0000 00:00:1790180102.261060 1576 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 155.961624] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 155.963796] nix-grpc-daemon[1576]: E0000 00:00:1790180102.269642 1617 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 155.966659] nix-grpc-daemon[1576]: E0000 00:00:1790180102.284882 1617 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 155.972141] nix-grpc-daemon[1576]: E0000 00:00:1790180102.285034 1617 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 155.976663] nix-grpc-daemon[1576]: E0000 00:00:1790180102.299528 1617 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 155.978974] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 155.979786] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 155.981209] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 156.087315] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 156.091565] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 156.178038] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 156.183316] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 156.288242] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 156.293489] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 156.384196] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 156.389609] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2 # [ 156.494469] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 156.505879] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) worker1 # [ 156.590580] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 156.611567] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:32768 worker2 # [ 156.617097] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:02Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 156.701050] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 156.714262] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:03Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 156.719041] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:03Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:32768 worker2 # [ 156.725099] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:03Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 156.901842] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:03Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:32782 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.76 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid client # [ 157.705438] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: 93fc0abcbbca40929cea39fc562be923 client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds) worker2 # [ 157.905313] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:32796 worker2 # [ 157.912347] nix-daemon[844]: accepted connection from pid 1365, user nix-grpc-daemon (trusted) client # [ 157.919010] nix[1437]: don't know how to build these paths: client # [ 157.920424] nix[1437]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker2 # [ 157.932794] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:04Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker2 # [ 157.937552] nix-grpc-daemon[1365]: ts=2026-09-23T16:15:04Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 157.942654] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47096 worker1 # [ 157.957306] nix-daemon[856]: accepted connection from pid 1576, user nix-grpc-daemon (trusted) worker1 # [ 157.980953] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 93 ms (attempt 1/5) worker1 # [ 158.082598] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 2/5) worker1 # [ 158.140391] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 388 ms (attempt 3/5) worker1 # [ 158.530678] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 672 ms (attempt 4/5) worker1 # [ 159.205020] nix-grpc-daemon[1576]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 159.205248] nix-grpc-daemon[1576]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 159.225629] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47096 client # [ 159.253416] nix[1437]: copying 1 paths to 'lb:50051'... worker1 # [ 159.241606] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47096 client # [ 159.258519] nix[1437]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'... worker1 # [ 159.257081] nix-daemon[856]: accepted connection from pid 1576, user nix-grpc-daemon (trusted) lb # [ 159.267039] envoy[895]: conn peer=[2001:db8:1::1]:59704 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1516 ms=7076 lb # [ 159.272217] envoy[895]: conn peer=[2001:db8:1::3]:39548 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2473 tx=870 ms=30480 worker1 # [ 159.291056] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:05Z level=info event=niks3_push_started pid=1644 worker1 # [ 159.425801] nix-grpc-daemon[1644]: time=2026-09-23T16:15:05.764Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 159.540275] nix-daemon[856]: accepted connection from pid 1656, user nix-grpc-daemon (trusted) worker1 # [ 159.558446] nix-daemon[856]: reaped child process 1659, status = succeeded lb # [ 159.565634] niks3-server[1178]: 2026/09/23 16:15:05 INFO Received push request method=POST path=/api/pushes worker1 # [ 159.580068] nix-grpc-daemon[1644]: time=2026-09-23T16:15:05.917Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 159.581599] nix-grpc-daemon[1644]: time=2026-09-23T16:15:05.917Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)" worker1 # [ 159.624084] nix-grpc-daemon[1644]: time=2026-09-23T16:15:05.962Z level=INFO msg="Uploading 1 narinfos" lb # [ 159.611931] niks3-server[1178]: 2026/09/23 16:15:05 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign lb # [ 159.631747] niks3-server[1178]: 2026/09/23 16:15:05 INFO Signed narinfos id=49 count=1 lb # [ 159.644389] niks3-server[1178]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst lb # [ 159.660572] niks3-server[1178]: 2026/09/23 16:15:05 INFO Received complete push request method=POST path=/api/pushes/49/complete lb # [ 159.672855] niks3-server[1178]: 2026/09/23 16:15:05 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo lb # [ 159.679270] niks3-server[1178]: 2026/09/23 16:15:06 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls worker1 # [ 159.685696] nix-grpc-daemon[1644]: time=2026-09-23T16:15:06.024Z level=INFO msg="Upload complete. (256ms)" worker1 # [ 159.696646] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47096 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 159.712922] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47096 worker1 # [ 159.724806] nix-daemon[856]: accepted connection from pid 1576, user nix-grpc-daemon (trusted) worker1 # [ 159.731334] nix-daemon[856]: reaped child process 1642, status = succeeded worker1 # [ 159.743391] nix-daemon[856]: accepted connection from pid 1576, user nix-grpc-daemon (trusted) client # [ 159.753798] nix[1437]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker2 # [ 161.578800] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1365 (nix-grpc-daemon) on client request. lb # [ 161.573864] niks3-server[1178]: 2026/09/23 16:15:07 INFO lead: released remote=192.168.1.4:54160 worker2 # [ 161.593144] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 161.579548] envoy[895]: [2026-09-23 16:15:07.903][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination} worker1 # [ 161.593051] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:07Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))" client # [ 161.606426] nix[1437]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker2 # [ 161.621092] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker2 # [ 161.623877] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. worker2 # [ 161.648343] systemd[1]: nix-grpc-daemon.service: Consumed 278ms CPU time over 16.228s wall clock time, 14.9M memory peak, 152.5K incoming IP traffic, 50.1K outgoing IP traffic. worker2 # [ 161.651457] systemd[1]: Starting scheduler-token.service... worker2 # [ 161.654986] nix-daemon[844]: reaped child process 1473, status = succeeded worker2 # [ 161.704544] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1. worker2 # [ 161.711726] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 161.713931] systemd[1]: Finished scheduler-token.service. worker2 # [ 161.715091] systemd[1]: scheduler-token.service: Consumed 31ms CPU time over 69ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 161.727006] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 161.793120] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 161.823985] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 161.826272] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 161.829988] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 161.836056] nix-grpc-daemon[1504]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 161.837494] nix-grpc-daemon[1504]: E0000 00:00:1790180108.159545 1504 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 161.840889] nix-grpc-daemon[1504]: E0000 00:00:1790180108.165847 1504 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.34 seconds) client: must succeed: sleep 5.0 worker2 # [ 161.855042] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket lb # [ 161.840862] niks3-server[1178]: 2026/09/23 16:15:08 INFO lead: acquired remote=192.168.1.4:40612 worker2 # [ 161.880107] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 161.880969] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=scheduler_take_over worker2 # [ 161.882062] nix-grpc-daemon[1504]: E0000 00:00:1790180108.184141 1544 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 161.894993] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:58314 worker2 # [ 161.897142] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=1 worker2 # [ 161.899134] nix-grpc-daemon[1504]: E0000 00:00:1790180108.200209 1544 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 161.901470] nix-grpc-daemon[1504]: E0000 00:00:1790180108.200368 1544 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 161.906526] nix-grpc-daemon[1504]: E0000 00:00:1790180108.219333 1544 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 161.911571] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 161.916862] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 161.918154] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:58314 worker2 # [ 161.920041] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 162.344468] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:08Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:58330 client: (finished: must succeed: sleep 5.0, in 5.05 seconds) client: waiting for success: ! systemctl is-active mid worker1 # [ 167.181338] nix-daemon[856]: reaped child process 1663, status = succeeded worker1 # [ 167.291898] nix-daemon[856]: accepted connection from pid 1700, user nix-grpc-daemon (trusted) worker1 # [ 167.310208] nix-daemon[856]: reaped child process 1703, status = succeeded lb # [ 167.316981] niks3-server[1178]: 2026/09/23 16:15:13 INFO Received push request method=POST path=/api/pushes worker1 # [ 167.336434] nix-grpc-daemon[1644]: time=2026-09-23T16:15:13.672Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 167.338070] nix-grpc-daemon[1644]: time=2026-09-23T16:15:13.673Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)" lb # [ 167.367304] niks3-server[1178]: 2026/09/23 16:15:13 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign worker1 # [ 167.377327] nix-grpc-daemon[1644]: time=2026-09-23T16:15:13.715Z level=INFO msg="Uploading 1 narinfos" lb # [ 167.387822] niks3-server[1178]: 2026/09/23 16:15:13 INFO Signed narinfos id=50 count=1 lb # [ 167.403253] niks3-server[1178]: 2026/09/23 16:15:13 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv lb # [ 167.421764] niks3-server[1178]: 2026/09/23 16:15:13 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst lb # [ 167.435968] niks3-server[1178]: 2026/09/23 16:15:13 INFO Received complete push request method=POST path=/api/pushes/50/complete lb # [ 167.441071] niks3-server[1178]: 2026/09/23 16:15:13 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo worker1 # [ 167.438731] nix-grpc-daemon[1644]: time=2026-09-23T16:15:13.777Z level=INFO msg="Upload complete. (276ms)" lb # [ 167.448522] niks3-server[1178]: 2026/09/23 16:15:13 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls worker1 # [ 167.452516] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47096 duration_s=7 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 worker1 # [ 167.473297] nix-daemon[856]: reaped child process 1661, status = succeeded client # [ 167.501477] systemd[1]: mid.service: Deactivated successfully. client # [ 167.505710] systemd[1]: mid.service: Consumed 187ms CPU time over 9.799s wall clock time, 41.4M memory peak, 7.8K incoming IP traffic, 8.7K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.69 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr client # [ 168.368891] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: 461ec0038fdc4ac59b69a6b04ff84c60 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds) client # [ 168.547441] nix[1473]: don't know how to build these paths: client # [ 168.548652] nix[1473]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker1 # [ 168.533624] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:47096 worker2 # [ 168.564712] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:14Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=0 worker2 # [ 168.572786] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:14Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 168.583744] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 168.597081] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 168.616347] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 1/5) worker2 # [ 168.696972] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 2/5) worker2 # [ 168.734554] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 3/5) worker2 # [ 168.774369] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 208 ms (attempt 4/5) worker2 # [ 168.985189] nix-grpc-daemon[1504]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 168.992159] nix-grpc-daemon[1504]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 169.008544] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58344 client # [ 169.020319] nix[1473]: copying 1 paths to 'lb:50051'... worker2 # [ 169.023000] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 client # [ 169.022654] nix[1473]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'... worker2 # [ 169.039538] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 169.069769] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:15Z level=info event=niks3_push_started pid=1578 worker2 # [ 169.223124] nix-grpc-daemon[1578]: time=2026-09-23T16:15:15.546Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" lb # [ 169.266166] envoy[895]: conn peer=[2001:db8:1::4]:44990 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7219 tx=568 ms=16044 lb # [ 169.271416] envoy[895]: conn peer=[2001:db8:1::1]:37192 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3065 tx=1606 ms=9588 worker2 # [ 169.326520] nix-daemon[844]: accepted connection from pid 1591, user nix-grpc-daemon (trusted) worker2 # [ 169.344634] nix-daemon[844]: reaped child process 1594, status = succeeded lb # [ 169.337384] niks3-server[1178]: 2026/09/23 16:15:15 INFO Received push request method=POST path=/api/pushes worker2 # [ 169.368071] nix-grpc-daemon[1578]: time=2026-09-23T16:15:15.692Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 169.369512] nix-grpc-daemon[1578]: time=2026-09-23T16:15:15.692Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)" lb # [ 169.384395] niks3-server[1178]: 2026/09/23 16:15:15 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign worker2 # [ 169.413072] nix-grpc-daemon[1578]: time=2026-09-23T16:15:15.736Z level=INFO msg="Uploading 1 narinfos" lb # [ 169.399173] niks3-server[1178]: 2026/09/23 16:15:15 INFO Signed narinfos id=51 count=1 lb # [ 169.405865] niks3-server[1178]: 2026/09/23 16:15:15 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls lb # [ 169.413399] niks3-server[1178]: 2026/09/23 16:15:15 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst lb # [ 169.424878] niks3-server[1178]: 2026/09/23 16:15:15 INFO Received complete push request method=POST path=/api/pushes/51/complete worker2 # [ 169.444408] nix-grpc-daemon[1578]: time=2026-09-23T16:15:15.769Z level=INFO msg="Upload complete. (222ms)" lb # [ 169.429143] niks3-server[1178]: 2026/09/23 16:15:15 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo worker2 # [ 169.452277] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 169.467469] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 169.476392] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 169.480745] nix-daemon[844]: reaped child process 1576, status = succeeded client # [ 169.488252] nix[1473]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051... worker2 # [ 169.494793] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) client: must succeed: systemctl kill -s INT intr worker2 # [ 172.012554] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:18Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv client # [ 172.008690] nix[1473]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 172.017647] systemd[1]: intr.service: Sent signal SIGINT to main process 1473 (nix) on client request. client # [ 172.028362] nix[1473]: error: interrupted by the user client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds) client # [ 172.057024] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 172.062718] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 172.065548] systemd[1]: intr.service: Consumed 166ms CPU time over 3.691s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic. worker2 # [ 172.139431] nix-daemon[844]: reaped child process 1596, status = succeeded worker2 # [ 172.144437] nix-daemon[1598]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 172.156229] nix-daemon[844]: reaped child process 1598, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 3.83 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw client # [ 172.223970] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: 422122bbe6a1435ab3a89e0e5ccdcb05 client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds) worker2 # [ 172.424076] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:58344 client # [ 172.428838] nix[1494]: don't know how to build these paths: client # [ 172.430467] nix[1494]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker2 # [ 172.442323] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:18Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=0 worker1 # [ 172.433175] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47096 worker2 # [ 172.448950] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:18Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 172.451000] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:47096 client # [ 172.477947] nix[1494]: copying 1 paths to 'lb:50051'... client # [ 172.479815] nix[1494]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'... worker1 # [ 172.466882] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47096 worker1 # [ 172.483371] nix-daemon[856]: accepted connection from pid 1576, user nix-grpc-daemon (trusted) worker1 # [ 172.648531] nix-daemon[856]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker1 # [ 172.671097] nix-daemon[856]: reaped child process 1734, status = succeeded lb # [ 172.667945] niks3-server[1178]: 2026/09/23 16:15:18 INFO Received push request method=POST path=/api/pushes worker1 # [ 172.681245] nix-grpc-daemon[1644]: time=2026-09-23T16:15:19.019Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 172.682708] nix-grpc-daemon[1644]: time=2026-09-23T16:15:19.019Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)" lb # [ 172.708663] niks3-server[1178]: 2026/09/23 16:15:19 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst lb # [ 172.725377] niks3-server[1178]: 2026/09/23 16:15:19 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign worker1 # [ 172.732673] nix-grpc-daemon[1644]: time=2026-09-23T16:15:19.070Z level=INFO msg="Uploading 1 narinfos" lb # [ 172.732681] niks3-server[1178]: 2026/09/23 16:15:19 INFO Signed narinfos id=52 count=1 lb # [ 172.738872] niks3-server[1178]: 2026/09/23 16:15:19 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls lb # [ 172.749273] niks3-server[1178]: 2026/09/23 16:15:19 INFO Received complete push request method=POST path=/api/pushes/52/complete lb # [ 172.762043] niks3-server[1178]: 2026/09/23 16:15:19 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo worker1 # [ 172.764302] nix-grpc-daemon[1644]: time=2026-09-23T16:15:19.102Z level=INFO msg="Upload complete. (262ms)" worker1 # [ 172.775454] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47096 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 172.788579] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47096 worker1 # [ 172.801250] nix-daemon[856]: accepted connection from pid 1576, user nix-grpc-daemon (trusted) worker1 # [ 172.807378] nix-daemon[856]: reaped child process 1729, status = succeeded client # [ 172.828657] nix[1494]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051... worker1 # [ 172.819740] nix-daemon[856]: accepted connection from pid 1576, user nix-grpc-daemon (trusted) worker1: must succeed: systemctl reload nix-grpc-daemon.service worker1 # [ 174.865987] systemd[1]: Reloading Nix worker-protocol over gRPC... worker1 # [ 174.903023] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds) worker1: must succeed: pgrep -f 'read -t [9]0 x' worker1 # [ 174.933826] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:21Z level=info event=draining worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2 worker2 # [ 175.198059] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:58344 client # don't know how to build these paths: client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker2 # [ 175.208275] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 175.234993] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=0 worker2 # [ 175.244317] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 175.252747] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 175.260262] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58344 client # copying 1 paths to 'lb:50051'... worker2 # [ 175.272736] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'... worker2 # [ 175.295150] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 175.446563] nix-daemon[844]: accepted connection from pid 1653, user nix-grpc-daemon (trusted) worker2 # [ 175.466271] nix-daemon[844]: reaped child process 1656, status = succeeded lb # [ 175.455351] niks3-server[1178]: 2026/09/23 16:15:21 INFO Received push request method=POST path=/api/pushes worker2 # [ 175.484956] nix-grpc-daemon[1578]: time=2026-09-23T16:15:21.808Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 175.486987] nix-grpc-daemon[1578]: time=2026-09-23T16:15:21.808Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)" lb # [ 175.489226] niks3-server[1178]: 2026/09/23 16:15:21 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign worker2 # [ 175.514943] nix-grpc-daemon[1578]: time=2026-09-23T16:15:21.839Z level=INFO msg="Uploading 1 narinfos" lb # [ 175.503250] niks3-server[1178]: 2026/09/23 16:15:21 INFO Signed narinfos id=53 count=1 lb # [ 175.510452] niks3-server[1178]: 2026/09/23 16:15:21 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst lb # [ 175.523345] niks3-server[1178]: 2026/09/23 16:15:21 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls worker2 # [ 175.546411] nix-grpc-daemon[1578]: time=2026-09-23T16:15:21.871Z level=INFO msg="Upload complete. (234ms)" lb # [ 175.530345] niks3-server[1178]: 2026/09/23 16:15:21 INFO Received complete push request method=POST path=/api/pushes/53/complete lb # [ 175.535052] niks3-server[1178]: 2026/09/23 16:15:21 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo worker2 # [ 175.556291] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=528 worker2 # [ 175.568658] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 175.583340] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 175.586525] nix-daemon[844]: reaped child process 1651, status = succeeded client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051... worker2 # [ 175.597567] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker2 # [ 176.862186] nix-daemon[844]: reaped child process 1660, status = succeeded worker2 # [ 176.984606] nix-daemon[844]: accepted connection from pid 1676, user nix-grpc-daemon (trusted) worker2 # [ 177.002126] nix-daemon[844]: reaped child process 1679, status = succeeded lb # [ 176.991323] niks3-server[1178]: 2026/09/23 16:15:23 INFO Received push request method=POST path=/api/pushes worker2 # [ 177.020407] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.343Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 177.022031] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.343Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)" lb # [ 177.041290] niks3-server[1178]: 2026/09/23 16:15:23 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst worker2 # [ 177.068121] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.392Z level=INFO msg="Uploading 1 narinfos" lb # [ 177.056835] niks3-server[1178]: 2026/09/23 16:15:23 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign lb # [ 177.065912] niks3-server[1178]: 2026/09/23 16:15:23 INFO Signed narinfos id=54 count=1 lb # [ 177.071835] niks3-server[1178]: 2026/09/23 16:15:23 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv lb # [ 177.083691] niks3-server[1178]: 2026/09/23 16:15:23 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls lb # [ 177.088384] niks3-server[1178]: 2026/09/23 16:15:23 INFO Received complete push request method=POST path=/api/pushes/54/complete worker2 # [ 177.104373] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.429Z level=INFO msg="Upload complete. (294ms)" lb # [ 177.092090] niks3-server[1178]: 2026/09/23 16:15:23 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo worker2 # [ 177.122855] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=1 worker2 # [ 177.141458] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=0 worker2 # [ 177.149307] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 177.156634] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 177.163163] nix-daemon[844]: reaped child process 1658, status = succeeded worker2 # [ 177.167226] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 177.173154] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.497Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 177.184604] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'... worker2 # [ 177.206042] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 177.359670] nix-daemon[844]: accepted connection from pid 1685, user nix-grpc-daemon (trusted) worker2 # [ 177.375997] nix-daemon[844]: reaped child process 1688, status = succeeded lb # [ 177.365750] niks3-server[1178]: 2026/09/23 16:15:23 INFO Received push request method=POST path=/api/pushes worker2 # [ 177.390978] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.715Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 177.392635] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.715Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)" lb # [ 177.394689] niks3-server[1178]: 2026/09/23 16:15:23 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst worker2 # [ 177.429825] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.754Z level=INFO msg="Uploading 1 narinfos" lb # [ 177.404667] niks3-server[1178]: 2026/09/23 16:15:23 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign lb # [ 177.419051] niks3-server[1178]: 2026/09/23 16:15:23 INFO Signed narinfos id=55 count=1 lb # [ 177.425993] niks3-server[1178]: 2026/09/23 16:15:23 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls lb # [ 177.439865] niks3-server[1178]: 2026/09/23 16:15:23 INFO Received complete push request method=POST path=/api/pushes/55/complete worker2 # [ 177.463812] nix-grpc-daemon[1578]: time=2026-09-23T16:15:23.788Z level=INFO msg="Upload complete. (242ms)" lb # [ 177.449485] niks3-server[1178]: 2026/09/23 16:15:23 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo worker2 # [ 177.474110] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=640 worker2 # [ 177.490755] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 177.501223] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 177.504467] nix-daemon[844]: reaped child process 1683, status = succeeded client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051... client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker2 # [ 177.519969] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker2 # [ 178.196353] nix-daemon[844]: reaped child process 1692, status = succeeded worker2 # [ 178.296921] nix-daemon[844]: accepted connection from pid 1708, user nix-grpc-daemon (trusted) worker2 # [ 178.317027] nix-daemon[844]: reaped child process 1711, status = succeeded lb # [ 178.300707] niks3-server[1178]: 2026/09/23 16:15:24 INFO Received push request method=POST path=/api/pushes worker2 # [ 178.328626] nix-grpc-daemon[1578]: time=2026-09-23T16:15:24.652Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 178.330755] nix-grpc-daemon[1578]: time=2026-09-23T16:15:24.652Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)" lb # [ 178.349536] niks3-server[1178]: 2026/09/23 16:15:24 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign worker2 # [ 178.375863] nix-grpc-daemon[1578]: time=2026-09-23T16:15:24.700Z level=INFO msg="Uploading 1 narinfos" lb # [ 178.363765] niks3-server[1178]: 2026/09/23 16:15:24 INFO Signed narinfos id=56 count=1 lb # [ 178.371085] niks3-server[1178]: 2026/09/23 16:15:24 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls lb # [ 178.378043] niks3-server[1178]: 2026/09/23 16:15:24 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker2 # [ 178.403765] nix-grpc-daemon[1578]: time=2026-09-23T16:15:24.728Z level=INFO msg="Upload complete. (274ms)" lb # [ 178.387823] niks3-server[1178]: 2026/09/23 16:15:24 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst lb # [ 178.396263] niks3-server[1178]: 2026/09/23 16:15:24 INFO Received complete push request method=POST path=/api/pushes/56/complete lb # [ 178.400320] niks3-server[1178]: 2026/09/23 16:15:24 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo worker2 # [ 178.420000] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=1 worker2 # [ 178.441231] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:24Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=0 worker2 # [ 178.447025] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:24Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 178.453191] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 178.458520] nix-daemon[844]: reaped child process 1690, status = succeeded worker2 # [ 178.462170] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 178.475123] nix-grpc-daemon[1578]: time=2026-09-23T16:15:24.800Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'... worker2 # [ 178.485563] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 178.497061] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 178.636004] nix-daemon[844]: accepted connection from pid 1717, user nix-grpc-daemon (trusted) worker2 # [ 178.657607] nix-daemon[844]: reaped child process 1720, status = succeeded lb # [ 178.644487] niks3-server[1178]: 2026/09/23 16:15:24 INFO Received push request method=POST path=/api/pushes worker2 # [ 178.669999] nix-grpc-daemon[1578]: time=2026-09-23T16:15:24.994Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 178.671493] nix-grpc-daemon[1578]: time=2026-09-23T16:15:24.994Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)" lb # [ 178.679557] niks3-server[1178]: 2026/09/23 16:15:25 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls lb # [ 178.692511] niks3-server[1178]: 2026/09/23 16:15:25 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign worker2 # [ 178.713822] nix-grpc-daemon[1578]: time=2026-09-23T16:15:25.038Z level=INFO msg="Uploading 1 narinfos" lb # [ 178.701330] niks3-server[1178]: 2026/09/23 16:15:25 INFO Signed narinfos id=57 count=1 lb # [ 178.703829] niks3-server[1178]: 2026/09/23 16:15:25 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst lb # [ 178.719487] niks3-server[1178]: 2026/09/23 16:15:25 INFO Received complete push request method=POST path=/api/pushes/57/complete lb # [ 178.724376] niks3-server[1178]: 2026/09/23 16:15:25 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo worker2 # [ 178.739954] nix-grpc-daemon[1578]: time=2026-09-23T16:15:25.065Z level=INFO msg="Upload complete. (229ms)" worker2 # [ 178.752721] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=768 worker2 # [ 178.764293] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 178.773968] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 178.776924] nix-daemon[844]: reaped child process 1715, status = succeeded client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051... worker2 # [ 178.790971] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds lb # [ 179.267463] envoy[895]: conn peer=[2001:db8:1::1]:45326 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3000 tx=828 ms=3493 client # farm-top> LOG-farm-top-during-drain worker2 # [ 179.658587] nix-daemon[844]: reaped child process 1724, status = succeeded worker2 # [ 179.788190] nix-daemon[844]: accepted connection from pid 1740, user nix-grpc-daemon (trusted) worker2 # [ 179.809402] nix-daemon[844]: reaped child process 1743, status = succeeded lb # [ 179.791862] niks3-server[1178]: 2026/09/23 16:15:26 INFO Received push request method=POST path=/api/pushes worker2 # [ 179.820580] nix-grpc-daemon[1578]: time=2026-09-23T16:15:26.144Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 179.824134] nix-grpc-daemon[1578]: time=2026-09-23T16:15:26.144Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)" lb # [ 179.828233] niks3-server[1178]: 2026/09/23 16:15:26 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign worker2 # [ 179.849216] nix-grpc-daemon[1578]: time=2026-09-23T16:15:26.174Z level=INFO msg="Uploading 1 narinfos" lb # [ 179.836087] niks3-server[1178]: 2026/09/23 16:15:26 INFO Signed narinfos id=58 count=1 lb # [ 179.840983] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst lb # [ 179.847426] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls lb # [ 179.852828] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv lb # [ 179.860547] niks3-server[1178]: 2026/09/23 16:15:26 INFO Received complete push request method=POST path=/api/pushes/58/complete lb # [ 179.869775] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo worker2 # [ 179.885784] nix-grpc-daemon[1578]: time=2026-09-23T16:15:26.210Z level=INFO msg="Upload complete. (282ms)" worker2 # [ 179.901117] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=1 worker2 # [ 179.916723] nix-daemon[844]: reaped child process 1722, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 4.97 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.19 seconds) worker1: must succeed: pkill -f 'read -t [9]0 x' worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds) client: waiting for success: ! systemctl is-active sw worker1 # [ 180.296457] nix-daemon[856]: reaped child process 1738, status = succeeded worker1 # [ 180.358243] nix-daemon[856]: accepted connection from pid 1779, user nix-grpc-daemon (trusted) worker1 # [ 180.375650] nix-daemon[856]: reaped child process 1782, status = succeeded lb # [ 180.379536] niks3-server[1178]: 2026/09/23 16:15:26 INFO Received push request method=POST path=/api/pushes worker1 # [ 180.393419] nix-grpc-daemon[1644]: time=2026-09-23T16:15:26.731Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 180.394904] nix-grpc-daemon[1644]: time=2026-09-23T16:15:26.731Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)" lb # [ 180.421637] niks3-server[1178]: 2026/09/23 16:15:26 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign worker1 # [ 180.429512] nix-grpc-daemon[1644]: time=2026-09-23T16:15:26.767Z level=INFO msg="Uploading 1 narinfos" lb # [ 180.433678] niks3-server[1178]: 2026/09/23 16:15:26 INFO Signed narinfos id=59 count=1 lb # [ 180.435818] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv lb # [ 180.444431] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst lb # [ 180.456322] niks3-server[1178]: 2026/09/23 16:15:26 INFO Received complete push request method=POST path=/api/pushes/59/complete worker1 # [ 180.458484] nix-grpc-daemon[1644]: time=2026-09-23T16:15:26.797Z level=INFO msg="Upload complete. (223ms)" lb # [ 180.459185] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls lb # [ 180.464416] niks3-server[1178]: 2026/09/23 16:15:26 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo worker1 # [ 180.468875] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47096 duration_s=7 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=1 worker1 # [ 180.485467] nix-daemon[856]: reaped child process 1736, status = succeeded client # [ 180.518246] systemd[1]: sw.service: Deactivated successfully. client # [ 180.524331] systemd[1]: sw.service: Consumed 192ms CPU time over 8.294s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic. worker1 # [ 180.936150] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:27Z level=info event=signal_exit client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds) worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service worker1 # [ 182.943686] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:29Z level=info event=scheduler_restarting worker2 # [ 182.957320] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:58314 worker2 # [ 182.965788] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:29Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 182.968439] nix-grpc-daemon[1576]: ts=2026-09-23T16:15:29Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 182.996916] nix-daemon[856]: reaped child process 1632, status = succeeded worker1 # [ 183.955584] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 183.962863] systemd[1]: nix-grpc-daemon.service: Consumed 933ms CPU time over 28.180s wall clock time, 36.5M memory peak, 56.5K incoming IP traffic, 67.2K outgoing IP traffic. worker1 # [ 183.978049] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker1 # [ 183.999904] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 184.122262] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 184.124637] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 184.128413] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 184.134812] nix-grpc-daemon[1797]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 184.156155] nix-grpc-daemon[1797]: E0000 00:00:1790180130.471353 1797 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 184.158534] nix-grpc-daemon[1797]: E0000 00:00:1790180130.473343 1797 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.86 seconds) lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 184.183925] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:30Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 184.185919] nix-grpc-daemon[1797]: E0000 00:00:1790180130.482344 1837 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 184.188231] nix-grpc-daemon[1797]: E0000 00:00:1790180130.503925 1837 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 184.193565] nix-grpc-daemon[1797]: E0000 00:00:1790180130.504127 1837 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 184.197885] nix-grpc-daemon[1797]: E0000 00:00:1790180130.520330 1837 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 184.200361] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 184.201203] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:30Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 184.216905] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:30Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:58314 worker2 # [ 184.217209] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:30Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 12.23 seconds) subtest: queue gauges show a burst while every build blocks lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]' client # [ 184.626549] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]". client # Running as unit: burst.service; invocation ID: 9c4fafbb486e481ba50306fbe0438ea7 client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2 # [ 184.842025] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:58344 client # [ 184.846490] nix[1550]: don't know how to build these paths: client # [ 184.847495] nix[1550]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv client # [ 184.848676] nix[1550]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv client # [ 184.849835] nix[1550]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv client # [ 184.850898] nix[1550]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker2 # [ 184.865095] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=want client=5 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=0 worker2 # [ 184.872511] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.4:50051 clients=1 worker1 # [ 184.866026] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker2 # [ 184.881807] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=want client=5 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=0 worker2 # [ 184.889706] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 184.881478] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker2 # [ 184.898547] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=want client=5 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=0 client # [ 184.904397] nix[1550]: copying 1 paths to 'lb:50051'... worker2 # [ 184.904915] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=want client=5 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=0 client # [ 184.907505] nix[1550]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'... worker2 # [ 184.913380] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker1 # [ 184.905714] nix-grpc-daemon[1797]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 1/5) worker2 # [ 184.924767] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 184.934405] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 184.941310] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker1 # [ 184.927593] nix-grpc-daemon[1797]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 43 ms (attempt 2/5) worker1 # [ 184.972510] nix-grpc-daemon[1797]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 230 ms (attempt 3/5) worker2 # [ 185.067128] nix-daemon[844]: accepted connection from pid 1758, user nix-grpc-daemon (trusted) worker2 # [ 185.083854] nix-daemon[844]: reaped child process 1761, status = succeeded lb # [ 185.075221] niks3-server[1178]: 2026/09/23 16:15:31 INFO Received push request method=POST path=/api/pushes worker2 # [ 185.117314] nix-grpc-daemon[1578]: time=2026-09-23T16:15:31.441Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 185.119138] nix-grpc-daemon[1578]: time=2026-09-23T16:15:31.441Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)" lb # [ 185.120654] niks3-server[1178]: 2026/09/23 16:15:31 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign worker2 # [ 185.143820] nix-grpc-daemon[1578]: time=2026-09-23T16:15:31.468Z level=INFO msg="Uploading 1 narinfos" lb # [ 185.134462] niks3-server[1178]: 2026/09/23 16:15:31 INFO Signed narinfos id=60 count=1 lb # [ 185.142023] niks3-server[1178]: 2026/09/23 16:15:31 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls lb # [ 185.154005] niks3-server[1178]: 2026/09/23 16:15:31 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst worker2 # [ 185.179340] nix-grpc-daemon[1578]: time=2026-09-23T16:15:31.504Z level=INFO msg="Upload complete. (235ms)" lb # [ 185.164501] niks3-server[1178]: 2026/09/23 16:15:31 INFO Received complete push request method=POST path=/api/pushes/60/complete lb # [ 185.167567] niks3-server[1178]: 2026/09/23 16:15:31 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo worker2 # [ 185.188401] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 185.208831] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 185.215785] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 185.221517] nix-daemon[844]: reaped child process 1756, status = succeeded worker1 # [ 185.205644] nix-grpc-daemon[1797]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 334 ms (attempt 4/5) client # [ 185.229000] nix[1550]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051... worker2 # [ 185.235112] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker1 # [ 185.542009] nix-grpc-daemon[1797]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 185.548005] nix-grpc-daemon[1797]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 185.565730] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39762 client # [ 185.592289] nix[1550]: copying 1 paths to 'lb:50051'... worker1 # [ 185.581602] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 client # [ 185.597002] nix[1550]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'... worker1 # [ 185.596104] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 185.624513] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:31Z level=info event=niks3_push_started pid=1857 worker1 # [ 185.772687] nix-grpc-daemon[1857]: time=2026-09-23T16:15:32.110Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 185.880041] nix-daemon[856]: accepted connection from pid 1870, user nix-grpc-daemon (trusted) lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 185.899285] nix-daemon[856]: reaped child process 1873, status = succeeded lb # [ 185.902100] niks3-server[1178]: 2026/09/23 16:15:32 INFO Received push request method=POST path=/api/pushes worker1 # [ 185.920073] nix-grpc-daemon[1857]: time=2026-09-23T16:15:32.258Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 185.921591] nix-grpc-daemon[1857]: time=2026-09-23T16:15:32.258Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)" lb # [ 185.957167] niks3-server[1178]: 2026/09/23 16:15:32 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst worker1 # [ 185.965619] nix-grpc-daemon[1857]: time=2026-09-23T16:15:32.303Z level=INFO msg="Uploading 1 narinfos" lb # [ 185.968069] niks3-server[1178]: 2026/09/23 16:15:32 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign lb # [ 185.970159] niks3-server[1178]: 2026/09/23 16:15:32 INFO Signed narinfos id=61 count=1 lb # [ 185.971604] niks3-server[1178]: 2026/09/23 16:15:32 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls lb # [ 185.985569] niks3-server[1178]: 2026/09/23 16:15:32 INFO Received complete push request method=POST path=/api/pushes/61/complete lb # [ 185.989842] niks3-server[1178]: 2026/09/23 16:15:32 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo worker1 # [ 185.990557] nix-grpc-daemon[1857]: time=2026-09-23T16:15:32.329Z level=INFO msg="Upload complete. (214ms)" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 185.999762] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 186.014628] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 186.020835] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 186.026010] nix-daemon[856]: reaped child process 1855, status = succeeded worker1 # [ 186.037183] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) client # [ 186.048750] nix[1550]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051... worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker2 # [ 187.469924] nix-daemon[844]: reaped child process 1765, status = succeeded worker1 # [ 187.504890] nix-daemon[856]: reaped child process 1877, status = succeeded worker2 # [ 187.562773] nix-daemon[844]: accepted connection from pid 1793, user nix-grpc-daemon (trusted) worker2 # [ 187.585781] nix-daemon[844]: reaped child process 1796, status = succeeded lb # [ 187.568886] niks3-server[1178]: 2026/09/23 16:15:33 INFO Received push request method=POST path=/api/pushes worker2 # [ 187.594163] nix-grpc-daemon[1578]: time=2026-09-23T16:15:33.918Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 187.598341] nix-grpc-daemon[1578]: time=2026-09-23T16:15:33.920Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)" lb # [ 187.608745] niks3-server[1178]: 2026/09/23 16:15:33 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst worker1 # [ 187.620196] nix-daemon[856]: accepted connection from pid 1902, user nix-grpc-daemon (trusted) worker2 # [ 187.641907] nix-grpc-daemon[1578]: time=2026-09-23T16:15:33.966Z level=INFO msg="Uploading 1 narinfos" lb # [ 187.625191] niks3-server[1178]: 2026/09/23 16:15:33 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign lb # [ 187.635140] niks3-server[1178]: 2026/09/23 16:15:33 INFO Signed narinfos id=62 count=1 worker1 # [ 187.643511] nix-daemon[856]: reaped child process 1905, status = succeeded lb # [ 187.642269] niks3-server[1178]: 2026/09/23 16:15:33 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls worker1 # [ 187.658662] nix-grpc-daemon[1857]: time=2026-09-23T16:15:33.995Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" lb # [ 187.655615] niks3-server[1178]: 2026/09/23 16:15:33 INFO Received push request method=POST path=/api/pushes worker1 # [ 187.662111] nix-grpc-daemon[1857]: time=2026-09-23T16:15:33.995Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)" lb # [ 187.664301] niks3-server[1178]: 2026/09/23 16:15:33 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv lb # [ 187.677082] niks3-server[1178]: 2026/09/23 16:15:33 INFO Received complete push request method=POST path=/api/pushes/62/complete worker2 # [ 187.701968] nix-grpc-daemon[1578]: time=2026-09-23T16:15:34.027Z level=INFO msg="Upload complete. (255ms)" lb # [ 187.693834] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv lb # [ 187.703985] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo worker2 # [ 187.718765] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=2 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=6 outputs=1 lb # [ 187.710012] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst worker1 # [ 187.722152] nix-grpc-daemon[1857]: time=2026-09-23T16:15:34.059Z level=INFO msg="Uploading 1 narinfos" lb # [ 187.722570] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign worker2 # [ 187.740372] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.4:50051 clients=1 lb # [ 187.728161] niks3-server[1178]: 2026/09/23 16:15:34 INFO Signed narinfos id=63 count=1 worker2 # [ 187.749076] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 lb # [ 187.735702] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls worker2 # [ 187.754087] nix-daemon[844]: reaped child process 1763, status = succeeded worker1 # [ 187.744696] nix-grpc-daemon[1857]: time=2026-09-23T16:15:34.083Z level=INFO msg="Upload complete. (294ms)" lb # [ 187.744529] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received complete push request method=POST path=/api/pushes/63/complete worker2 # [ 187.759720] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58344 lb # [ 187.748147] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo client # [ 187.769209] nix[1550]: copying 1 paths to 'lb:50051'... worker2 # [ 187.769027] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.3:50051 clients=1 client # [ 187.772905] nix[1550]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'... worker1 # [ 187.757657] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=7 outputs=1 worker2 # [ 187.776224] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 187.788449] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker1 # [ 187.779341] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 187.785798] nix-daemon[856]: reaped child process 1875, status = succeeded worker1 # [ 187.793541] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39762 client # [ 187.816175] nix[1550]: copying 1 paths to 'lb:50051'... worker1 # [ 187.805104] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 client # [ 187.818497] nix[1550]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'... worker1 # [ 187.818507] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker2 # [ 187.938193] nix-daemon[844]: accepted connection from pid 1800, user nix-grpc-daemon (trusted) worker2 # [ 187.962864] nix-daemon[844]: reaped child process 1803, status = succeeded lb # [ 187.944209] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received push request method=POST path=/api/pushes worker2 # [ 187.973023] nix-grpc-daemon[1578]: time=2026-09-23T16:15:34.297Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 187.959881] nix-daemon[856]: accepted connection from pid 1909, user nix-grpc-daemon (trusted) worker2 # [ 187.974898] nix-grpc-daemon[1578]: time=2026-09-23T16:15:34.297Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)" worker1 # [ 187.982325] nix-daemon[856]: reaped child process 1912, status = succeeded lb # [ 187.977991] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign worker2 # [ 188.001899] nix-grpc-daemon[1578]: time=2026-09-23T16:15:34.326Z level=INFO msg="Uploading 1 narinfos" lb # [ 187.997469] niks3-server[1178]: 2026/09/23 16:15:34 INFO Signed narinfos id=64 count=1 lb # [ 188.005989] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received push request method=POST path=/api/pushes lb # [ 188.010825] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls worker1 # [ 188.024456] nix-grpc-daemon[1857]: time=2026-09-23T16:15:34.362Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 188.026443] nix-grpc-daemon[1857]: time=2026-09-23T16:15:34.362Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)" lb # [ 188.022663] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received complete push request method=POST path=/api/pushes/64/complete lb # [ 188.034821] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst worker2 # [ 188.062640] nix-grpc-daemon[1578]: time=2026-09-23T16:15:34.387Z level=INFO msg="Upload complete. (260ms)" lb # [ 188.054480] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo worker2 # [ 188.072361] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 188.064910] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign worker1 # [ 188.067848] nix-grpc-daemon[1857]: time=2026-09-23T16:15:34.405Z level=INFO msg="Uploading 1 narinfos" lb # [ 188.069378] niks3-server[1178]: 2026/09/23 16:15:34 INFO Signed narinfos id=65 count=1 lb # [ 188.076889] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls worker2 # [ 188.092899] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 lb # [ 188.084260] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst worker2 # [ 188.102286] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 188.108904] nix-daemon[844]: reaped child process 1798, status = succeeded lb # [ 188.094786] niks3-server[1178]: 2026/09/23 16:15:34 INFO Received complete push request method=POST path=/api/pushes/65/complete worker1 # [ 188.104131] nix-grpc-daemon[1857]: time=2026-09-23T16:15:34.442Z level=INFO msg="Upload complete. (273ms)" lb # [ 188.103153] niks3-server[1178]: 2026/09/23 16:15:34 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo client # [ 188.121392] nix[1550]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051... worker2 # [ 188.122357] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker1 # [ 188.114752] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 188.133244] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 188.144485] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 188.153818] nix-daemon[856]: reaped child process 1907, status = succeeded worker1 # [ 188.167506] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) client # [ 188.178295] nix[1550]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051... lb # [ 189.268296] envoy[895]: conn peer=[2001:db8:1::1]:36302 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9608 tx=4385 ms=4719 lb # [ 189.268677] envoy[895]: conn peer=[2001:db8:1::1]:36296 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=1517 ms=8072 lb # [ 189.268787] envoy[895]: conn peer=[2001:db8:1::3]:34416 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1478 tx=537 ms=26985 worker1 # [ 189.836863] nix-daemon[856]: reaped child process 1916, status = succeeded worker1 # [ 189.954867] nix-daemon[856]: accepted connection from pid 1941, user nix-grpc-daemon (trusted) worker1 # [ 189.976984] nix-daemon[856]: reaped child process 1944, status = succeeded lb # [ 189.981626] niks3-server[1178]: 2026/09/23 16:15:36 INFO Received push request method=POST path=/api/pushes worker1 # [ 189.995659] nix-grpc-daemon[1857]: time=2026-09-23T16:15:36.334Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 189.998136] nix-grpc-daemon[1857]: time=2026-09-23T16:15:36.336Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)" worker2 # [ 190.016975] nix-daemon[844]: reaped child process 1807, status = succeeded lb # [ 190.031586] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst worker1 # [ 190.044133] nix-grpc-daemon[1857]: time=2026-09-23T16:15:36.382Z level=INFO msg="Uploading 1 narinfos" lb # [ 190.043374] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls lb # [ 190.051351] niks3-server[1178]: 2026/09/23 16:15:36 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign lb # [ 190.057509] niks3-server[1178]: 2026/09/23 16:15:36 INFO Signed narinfos id=66 count=1 lb # [ 190.061645] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv lb # [ 190.070300] niks3-server[1178]: 2026/09/23 16:15:36 INFO Received complete push request method=POST path=/api/pushes/66/complete worker1 # [ 190.075829] nix-grpc-daemon[1857]: time=2026-09-23T16:15:36.414Z level=INFO msg="Upload complete. (292ms)" lb # [ 190.075142] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo worker1 # [ 190.088770] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=9 outputs=1 worker2 # [ 190.118701] nix-daemon[844]: accepted connection from pid 1832, user nix-grpc-daemon (trusted) worker1 # [ 190.107271] nix-daemon[856]: reaped child process 1914, status = succeeded worker2 # [ 190.139277] nix-daemon[844]: reaped child process 1835, status = succeeded lb # [ 190.127403] niks3-server[1178]: 2026/09/23 16:15:36 INFO Received push request method=POST path=/api/pushes worker2 # [ 190.152398] nix-grpc-daemon[1578]: time=2026-09-23T16:15:36.475Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 190.153861] nix-grpc-daemon[1578]: time=2026-09-23T16:15:36.475Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)" lb # [ 190.168969] niks3-server[1178]: 2026/09/23 16:15:36 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign worker2 # [ 190.190548] nix-grpc-daemon[1578]: time=2026-09-23T16:15:36.515Z level=INFO msg="Uploading 1 narinfos" lb # [ 190.179297] niks3-server[1178]: 2026/09/23 16:15:36 INFO Signed narinfos id=67 count=1 lb # [ 190.184224] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst lb # [ 190.194125] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv lb # [ 190.201591] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls lb # [ 190.211342] niks3-server[1178]: 2026/09/23 16:15:36 INFO Received complete push request method=POST path=/api/pushes/67/complete worker2 # [ 190.228907] nix-grpc-daemon[1578]: time=2026-09-23T16:15:36.553Z level=INFO msg="Upload complete. (286ms)" lb # [ 190.218013] niks3-server[1178]: 2026/09/23 16:15:36 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo worker2 # [ 190.241569] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=2 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=8 outputs=1 worker2 # [ 190.261124] nix-daemon[844]: reaped child process 1805, status = succeeded client # [ 190.275195] systemd[1]: burst.service: Deactivated successfully. client # [ 190.279833] systemd[1]: burst.service: Consumed 233ms CPU time over 5.650s wall clock time, 49.6M memory peak, 14K incoming IP traffic, 18.6K outgoing IP traffic. client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; } client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: queue gauges show a burst while every build blocks, in 6.88 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker1 # [ 191.479669] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 191.490307] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39762 worker2 # [ 191.546713] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:37Z level=debug event=want client=6 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=0 worker2 # [ 191.552908] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:37Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 191.541992] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker2 # [ 191.558988] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:37Z level=debug event=want client=6 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=0 worker2 # [ 191.564388] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:37Z level=debug event=want client=6 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=0 worker1 # [ 191.557419] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 191.571794] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 191.587298] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 191.747525] nix-daemon[856]: accepted connection from pid 1952, user nix-grpc-daemon (trusted) worker1 # [ 191.769133] nix-daemon[856]: reaped child process 1955, status = succeeded lb # [ 191.772696] niks3-server[1178]: 2026/09/23 16:15:38 INFO Received push request method=POST path=/api/pushes worker1 # [ 191.786851] nix-grpc-daemon[1857]: time=2026-09-23T16:15:38.124Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 191.788432] nix-grpc-daemon[1857]: time=2026-09-23T16:15:38.124Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)" lb # [ 191.803108] niks3-server[1178]: 2026/09/23 16:15:38 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign worker1 # [ 191.813534] nix-grpc-daemon[1857]: time=2026-09-23T16:15:38.151Z level=INFO msg="Uploading 1 narinfos" lb # [ 191.814603] niks3-server[1178]: 2026/09/23 16:15:38 INFO Signed narinfos id=68 count=1 lb # [ 191.823970] niks3-server[1178]: 2026/09/23 16:15:38 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls worker1 # [ 191.846481] nix-grpc-daemon[1857]: time=2026-09-23T16:15:38.185Z level=INFO msg="Upload complete. (242ms)" lb # [ 191.838603] niks3-server[1178]: 2026/09/23 16:15:38 INFO Received complete push request method=POST path=/api/pushes/68/complete lb # [ 191.850012] niks3-server[1178]: 2026/09/23 16:15:38 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst worker1 # [ 191.856254] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 191.862295] niks3-server[1178]: 2026/09/23 16:15:38 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo worker1 # [ 191.867138] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 191.874267] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 191.879406] nix-daemon[856]: reaped child process 1950, status = succeeded worker1 # [ 191.892250] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 193.119461] nix-daemon[856]: reaped child process 1959, status = succeeded worker1 # [ 193.224994] nix-daemon[856]: accepted connection from pid 1975, user nix-grpc-daemon (trusted) worker1 # [ 193.252451] nix-daemon[856]: reaped child process 1978, status = succeeded lb # [ 193.249193] niks3-server[1178]: 2026/09/23 16:15:39 INFO Received push request method=POST path=/api/pushes worker1 # [ 193.287227] nix-grpc-daemon[1857]: time=2026-09-23T16:15:39.625Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 193.290728] nix-grpc-daemon[1857]: time=2026-09-23T16:15:39.627Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)" lb # [ 193.319052] niks3-server[1178]: 2026/09/23 16:15:39 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign worker1 # [ 193.331418] nix-grpc-daemon[1857]: time=2026-09-23T16:15:39.669Z level=INFO msg="Uploading 1 narinfos" lb # [ 193.335520] niks3-server[1178]: 2026/09/23 16:15:39 INFO Signed narinfos id=69 count=1 lb # [ 193.344724] niks3-server[1178]: 2026/09/23 16:15:39 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst lb # [ 193.356775] niks3-server[1178]: 2026/09/23 16:15:39 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls lb # [ 193.368393] niks3-server[1178]: 2026/09/23 16:15:39 INFO Received complete push request method=POST path=/api/pushes/69/complete worker1 # [ 193.375922] nix-grpc-daemon[1857]: time=2026-09-23T16:15:39.714Z level=INFO msg="Upload complete. (271ms)" lb # [ 193.374763] niks3-server[1178]: 2026/09/23 16:15:39 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv lb # [ 193.382764] niks3-server[1178]: 2026/09/23 16:15:39 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo worker2 # [ 193.400697] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:39Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 193.389267] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=10 outputs=1 worker2 # [ 193.415235] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:39Z level=debug event=want client=6 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=0 worker1 # [ 193.412391] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 193.417931] nix-daemon[856]: reaped child process 1957, status = succeeded worker1 # [ 193.426169] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 193.438742] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 193.449903] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 193.594839] nix-daemon[856]: accepted connection from pid 1982, user nix-grpc-daemon (trusted) worker1 # [ 193.616519] nix-daemon[856]: reaped child process 1985, status = succeeded lb # [ 193.618993] niks3-server[1178]: 2026/09/23 16:15:39 INFO Received push request method=POST path=/api/pushes worker1 # [ 193.630563] nix-grpc-daemon[1857]: time=2026-09-23T16:15:39.968Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 193.632144] nix-grpc-daemon[1857]: time=2026-09-23T16:15:39.968Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)" lb # [ 193.649278] niks3-server[1178]: 2026/09/23 16:15:39 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign worker1 # [ 193.661263] nix-grpc-daemon[1857]: time=2026-09-23T16:15:39.999Z level=INFO msg="Uploading 1 narinfos" lb # [ 193.663681] niks3-server[1178]: 2026/09/23 16:15:39 INFO Signed narinfos id=70 count=1 lb # [ 193.672894] niks3-server[1178]: 2026/09/23 16:15:39 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst lb # [ 193.688335] niks3-server[1178]: 2026/09/23 16:15:39 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls lb # [ 193.691835] niks3-server[1178]: 2026/09/23 16:15:40 INFO Received complete push request method=POST path=/api/pushes/70/complete worker1 # [ 193.698463] nix-grpc-daemon[1857]: time=2026-09-23T16:15:40.037Z level=INFO msg="Upload complete. (235ms)" lb # [ 193.701205] niks3-server[1178]: 2026/09/23 16:15:40 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo worker1 # [ 193.707339] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 193.723751] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 193.731367] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 193.737254] nix-daemon[856]: reaped child process 1980, status = succeeded worker1 # [ 193.751962] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 195.002794] nix-daemon[856]: reaped child process 1989, status = succeeded worker1 # [ 195.132759] nix-daemon[856]: accepted connection from pid 2005, user nix-grpc-daemon (trusted) worker1 # [ 195.153464] nix-daemon[856]: reaped child process 2008, status = succeeded lb # [ 195.157027] niks3-server[1178]: 2026/09/23 16:15:41 INFO Received push request method=POST path=/api/pushes worker1 # [ 195.180389] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.518Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 195.184569] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.518Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)" lb # [ 195.207622] niks3-server[1178]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst worker1 # [ 195.234293] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.572Z level=INFO msg="Uploading 1 narinfos" lb # [ 195.222341] niks3-server[1178]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv lb # [ 195.237119] niks3-server[1178]: 2026/09/23 16:15:41 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign lb # [ 195.246826] niks3-server[1178]: 2026/09/23 16:15:41 INFO Signed narinfos id=71 count=1 lb # [ 195.253320] niks3-server[1178]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls lb # [ 195.261526] niks3-server[1178]: 2026/09/23 16:15:41 INFO Received complete push request method=POST path=/api/pushes/71/complete worker1 # [ 195.268546] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.607Z level=INFO msg="Upload complete. (305ms)" lb # [ 195.270427] niks3-server[1178]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo worker2 # [ 195.293847] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:41Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 195.281269] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=11 outputs=1 worker2 # [ 195.308828] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:41Z level=debug event=want client=6 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=0 worker1 # [ 195.304419] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 195.311429] nix-daemon[856]: reaped child process 1987, status = succeeded worker1 # [ 195.315397] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 195.328111] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 195.339453] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 195.484176] nix-daemon[856]: accepted connection from pid 2012, user nix-grpc-daemon (trusted) worker1 # [ 195.505407] nix-daemon[856]: reaped child process 2015, status = succeeded lb # [ 195.502232] niks3-server[1178]: 2026/09/23 16:15:41 INFO Received push request method=POST path=/api/pushes worker1 # [ 195.516876] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.855Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 195.518432] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.855Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)" lb # [ 195.533081] niks3-server[1178]: 2026/09/23 16:15:41 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign worker1 # [ 195.543837] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.882Z level=INFO msg="Uploading 1 narinfos" lb # [ 195.546069] niks3-server[1178]: 2026/09/23 16:15:41 INFO Signed narinfos id=72 count=1 lb # [ 195.558824] niks3-server[1178]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls lb # [ 195.569575] niks3-server[1178]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst lb # [ 195.584447] niks3-server[1178]: 2026/09/23 16:15:41 INFO Received complete push request method=POST path=/api/pushes/72/complete lb # [ 195.594891] niks3-server[1178]: 2026/09/23 16:15:41 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo worker1 # [ 195.603646] nix-grpc-daemon[1857]: time=2026-09-23T16:15:41.942Z level=INFO msg="Upload complete. (248ms)" worker1 # [ 195.613312] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 195.630901] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 195.638348] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 195.651008] nix-daemon[856]: reaped child process 2010, status = succeeded worker1 # [ 195.659292] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 196.944779] nix-daemon[856]: reaped child process 2019, status = succeeded worker1 # [ 197.039058] nix-daemon[856]: accepted connection from pid 2035, user nix-grpc-daemon (trusted) worker1 # [ 197.064457] nix-daemon[856]: reaped child process 2038, status = succeeded lb # [ 197.059499] niks3-server[1178]: 2026/09/23 16:15:43 INFO Received push request method=POST path=/api/pushes worker1 # [ 197.080499] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.415Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 197.082520] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.415Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)" lb # [ 197.109311] niks3-server[1178]: 2026/09/23 16:15:43 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign worker1 # [ 197.121818] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.460Z level=INFO msg="Uploading 1 narinfos" lb # [ 197.125303] niks3-server[1178]: 2026/09/23 16:15:43 INFO Signed narinfos id=73 count=1 lb # [ 197.133138] niks3-server[1178]: 2026/09/23 16:15:43 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 197.143188] niks3-server[1178]: 2026/09/23 16:15:43 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv lb # [ 197.155414] niks3-server[1178]: 2026/09/23 16:15:43 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls lb # [ 197.159861] niks3-server[1178]: 2026/09/23 16:15:43 INFO Received complete push request method=POST path=/api/pushes/73/complete worker1 # [ 197.167565] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.506Z level=INFO msg="Upload complete. (297ms)" lb # [ 197.168630] niks3-server[1178]: 2026/09/23 16:15:43 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo worker2 # [ 197.192917] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:43Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 197.180749] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=12 outputs=1 worker2 # [ 197.199846] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:43Z level=debug event=want client=6 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=0 worker1 # [ 197.196646] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 197.205061] nix-daemon[856]: reaped child process 2017, status = succeeded worker1 # [ 197.209284] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 197.219845] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.558Z level=INFO msg="All 1 paths already cached" worker1 # [ 197.229399] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 197.238211] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 197.369947] nix-daemon[856]: accepted connection from pid 2042, user nix-grpc-daemon (trusted) worker1 # [ 197.393500] nix-daemon[856]: reaped child process 2045, status = succeeded lb # [ 197.390320] niks3-server[1178]: 2026/09/23 16:15:43 INFO Received push request method=POST path=/api/pushes worker1 # [ 197.406355] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.744Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 197.407825] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.744Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)" lb # [ 197.427754] niks3-server[1178]: 2026/09/23 16:15:43 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign worker1 # [ 197.436930] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.775Z level=INFO msg="Uploading 1 narinfos" lb # [ 197.441728] niks3-server[1178]: 2026/09/23 16:15:43 INFO Signed narinfos id=74 count=1 lb # [ 197.449477] niks3-server[1178]: 2026/09/23 16:15:43 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls lb # [ 197.457672] niks3-server[1178]: 2026/09/23 16:15:43 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst worker1 # [ 197.472577] nix-grpc-daemon[1857]: time=2026-09-23T16:15:43.811Z level=INFO msg="Upload complete. (222ms)" lb # [ 197.472742] niks3-server[1178]: 2026/09/23 16:15:43 INFO Received complete push request method=POST path=/api/pushes/74/complete lb # [ 197.477144] niks3-server[1178]: 2026/09/23 16:15:43 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo worker1 # [ 197.484716] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 197.496244] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 197.502947] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 197.507568] nix-daemon[856]: reaped child process 2040, status = succeeded worker1 # [ 197.517612] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 198.612749] nix-daemon[856]: reaped child process 2049, status = succeeded worker1 # [ 198.700868] nix-daemon[856]: accepted connection from pid 2065, user nix-grpc-daemon (trusted) worker1 # [ 198.723604] nix-daemon[856]: reaped child process 2068, status = succeeded lb # [ 198.725844] niks3-server[1178]: 2026/09/23 16:15:45 INFO Received push request method=POST path=/api/pushes worker1 # [ 198.736387] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.074Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 198.738057] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.074Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)" lb # [ 198.768053] niks3-server[1178]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv lb # [ 198.781319] niks3-server[1178]: 2026/09/23 16:15:45 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign worker1 # [ 198.788650] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.126Z level=INFO msg="Uploading 1 narinfos" lb # [ 198.790364] niks3-server[1178]: 2026/09/23 16:15:45 INFO Signed narinfos id=75 count=1 lb # [ 198.795753] niks3-server[1178]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst lb # [ 198.811031] niks3-server[1178]: 2026/09/23 16:15:45 INFO Received complete push request method=POST path=/api/pushes/75/complete worker1 # [ 198.819260] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.157Z level=INFO msg="Upload complete. (290ms)" lb # [ 198.818908] niks3-server[1178]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls lb # [ 198.822862] niks3-server[1178]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo worker2 # [ 198.849804] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:45Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 198.834092] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=13 outputs=1 worker1 # [ 198.852441] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 198.860816] nix-daemon[856]: reaped child process 2047, status = succeeded worker1 # [ 198.869082] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 198.881463] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.220Z level=INFO msg="All 1 paths already cached" worker1 # [ 198.890676] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 198.905902] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 199.045302] nix-daemon[856]: accepted connection from pid 2072, user nix-grpc-daemon (trusted) worker1 # [ 199.063336] nix-daemon[856]: reaped child process 2075, status = succeeded lb # [ 199.067087] niks3-server[1178]: 2026/09/23 16:15:45 INFO Received push request method=POST path=/api/pushes worker1 # [ 199.077356] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.415Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 199.079345] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.415Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)" lb # [ 199.098048] niks3-server[1178]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls worker1 # [ 199.108106] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.446Z level=INFO msg="Uploading 1 narinfos" lb # [ 199.111745] niks3-server[1178]: 2026/09/23 16:15:45 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign lb # [ 199.121079] niks3-server[1178]: 2026/09/23 16:15:45 INFO Signed narinfos id=76 count=1 lb # [ 199.129813] niks3-server[1178]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst worker1 # [ 199.137848] nix-grpc-daemon[1857]: time=2026-09-23T16:15:45.476Z level=INFO msg="Upload complete. (219ms)" lb # [ 199.137912] niks3-server[1178]: 2026/09/23 16:15:45 INFO Received complete push request method=POST path=/api/pushes/76/complete lb # [ 199.142902] niks3-server[1178]: 2026/09/23 16:15:45 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo worker1 # [ 199.146524] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 199.162844] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 199.170461] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 199.175421] nix-daemon[856]: reaped child process 2070, status = succeeded worker1 # [ 199.189280] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) lb # [ 199.270151] envoy[895]: conn peer=[2001:db8:1::1]:34588 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9697 tx=5215 ms=5416 worker2 # [ 198.858296] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:45Z level=debug event=want client=6 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=0 worker1 # [ 200.249921] nix-daemon[856]: reaped child process 2079, status = succeeded worker1 # [ 200.372408] nix-daemon[856]: accepted connection from pid 2095, user nix-grpc-daemon (trusted) worker1 # [ 200.391845] nix-daemon[856]: reaped child process 2098, status = succeeded lb # [ 200.394830] niks3-server[1178]: 2026/09/23 16:15:46 INFO Received push request method=POST path=/api/pushes worker1 # [ 200.407539] nix-grpc-daemon[1857]: time=2026-09-23T16:15:46.744Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 200.410948] nix-grpc-daemon[1857]: time=2026-09-23T16:15:46.744Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)" lb # [ 200.438206] niks3-server[1178]: 2026/09/23 16:15:46 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls worker1 # [ 200.450113] nix-grpc-daemon[1857]: time=2026-09-23T16:15:46.788Z level=INFO msg="Uploading 1 narinfos" lb # [ 200.448865] niks3-server[1178]: 2026/09/23 16:15:46 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst lb # [ 200.460655] niks3-server[1178]: 2026/09/23 16:15:46 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign lb # [ 200.471129] niks3-server[1178]: 2026/09/23 16:15:46 INFO Signed narinfos id=77 count=1 worker1 # [ 200.479147] nix-grpc-daemon[1857]: time=2026-09-23T16:15:46.817Z level=INFO msg="Upload complete. (280ms)" lb # [ 200.478047] niks3-server[1178]: 2026/09/23 16:15:46 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv lb # [ 200.481897] niks3-server[1178]: 2026/09/23 16:15:46 INFO Received complete push request method=POST path=/api/pushes/77/complete lb # [ 200.487936] niks3-server[1178]: 2026/09/23 16:15:46 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo worker2 # [ 200.507776] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:46Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 200.491675] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=14 outputs=1 worker2 # [ 200.517294] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:46Z level=debug event=want client=6 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=0 worker1 # [ 200.512768] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 200.520767] nix-daemon[856]: reaped child process 2077, status = succeeded worker1 # [ 200.528680] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 200.541077] nix-grpc-daemon[1857]: time=2026-09-23T16:15:46.879Z level=INFO msg="All 1 paths already cached" worker1 # [ 200.550275] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 200.563306] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 200.702380] nix-daemon[856]: accepted connection from pid 2102, user nix-grpc-daemon (trusted) worker1 # [ 200.721280] nix-daemon[856]: reaped child process 2105, status = succeeded lb # [ 200.725067] niks3-server[1178]: 2026/09/23 16:15:47 INFO Received push request method=POST path=/api/pushes worker1 # [ 200.735290] nix-grpc-daemon[1857]: time=2026-09-23T16:15:47.073Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 200.736915] nix-grpc-daemon[1857]: time=2026-09-23T16:15:47.073Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)" lb # [ 200.756387] niks3-server[1178]: 2026/09/23 16:15:47 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign worker1 # [ 200.764130] nix-grpc-daemon[1857]: time=2026-09-23T16:15:47.102Z level=INFO msg="Uploading 1 narinfos" lb # [ 200.769838] niks3-server[1178]: 2026/09/23 16:15:47 INFO Signed narinfos id=78 count=1 lb # [ 200.777543] niks3-server[1178]: 2026/09/23 16:15:47 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst lb # [ 200.786402] niks3-server[1178]: 2026/09/23 16:15:47 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls lb # [ 200.801834] niks3-server[1178]: 2026/09/23 16:15:47 INFO Received complete push request method=POST path=/api/pushes/78/complete worker1 # [ 200.808110] nix-grpc-daemon[1857]: time=2026-09-23T16:15:47.146Z level=INFO msg="Upload complete. (232ms)" lb # [ 200.807793] niks3-server[1178]: 2026/09/23 16:15:47 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo worker1 # [ 200.817643] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 200.835254] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 200.841804] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 200.847344] nix-daemon[856]: reaped child process 2100, status = succeeded worker1 # [ 200.857898] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 202.089085] nix-daemon[856]: reaped child process 2109, status = succeeded worker1 # [ 202.186129] nix-daemon[856]: accepted connection from pid 2125, user nix-grpc-daemon (trusted) worker1 # [ 202.203455] nix-daemon[856]: reaped child process 2128, status = succeeded lb # [ 202.211109] niks3-server[1178]: 2026/09/23 16:15:48 INFO Received push request method=POST path=/api/pushes worker1 # [ 202.223872] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.561Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 202.225568] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.561Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)" lb # [ 202.255666] niks3-server[1178]: 2026/09/23 16:15:48 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign worker1 # [ 202.265405] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.603Z level=INFO msg="Uploading 1 narinfos" lb # [ 202.268971] niks3-server[1178]: 2026/09/23 16:15:48 INFO Signed narinfos id=79 count=1 lb # [ 202.277230] niks3-server[1178]: 2026/09/23 16:15:48 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv lb # [ 202.292121] niks3-server[1178]: 2026/09/23 16:15:48 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls worker1 # [ 202.303174] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.641Z level=INFO msg="Upload complete. (233ms)" lb # [ 202.302881] niks3-server[1178]: 2026/09/23 16:15:48 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst lb # [ 202.309833] niks3-server[1178]: 2026/09/23 16:15:48 INFO Received complete push request method=POST path=/api/pushes/79/complete worker2 # [ 202.328255] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:48Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=1 lb # [ 202.315615] niks3-server[1178]: 2026/09/23 16:15:48 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo worker1 # [ 202.313080] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=15 outputs=1 worker1 # [ 202.329077] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker2 # [ 202.342142] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:48Z level=debug event=want client=6 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=0 worker1 # [ 202.339207] nix-daemon[856]: reaped child process 2107, status = succeeded worker1 # [ 202.348470] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 202.361027] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.699Z level=INFO msg="All 2 paths already cached" worker1 # [ 202.370268] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 202.380595] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 202.513289] nix-daemon[856]: accepted connection from pid 2132, user nix-grpc-daemon (trusted) worker1 # [ 202.534820] nix-daemon[856]: reaped child process 2135, status = succeeded lb # [ 202.530543] niks3-server[1178]: 2026/09/23 16:15:48 INFO Received push request method=POST path=/api/pushes worker1 # [ 202.541795] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.880Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 202.543284] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.880Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)" lb # [ 202.568543] niks3-server[1178]: 2026/09/23 16:15:48 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign worker1 # [ 202.579220] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.917Z level=INFO msg="Uploading 1 narinfos" lb # [ 202.582187] niks3-server[1178]: 2026/09/23 16:15:48 INFO Signed narinfos id=80 count=1 lb # [ 202.589304] niks3-server[1178]: 2026/09/23 16:15:48 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls lb # [ 202.601327] niks3-server[1178]: 2026/09/23 16:15:48 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst worker1 # [ 202.608962] nix-grpc-daemon[1857]: time=2026-09-23T16:15:48.947Z level=INFO msg="Upload complete. (217ms)" lb # [ 202.607650] niks3-server[1178]: 2026/09/23 16:15:48 INFO Received complete push request method=POST path=/api/pushes/80/complete lb # [ 202.611322] niks3-server[1178]: 2026/09/23 16:15:48 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo worker1 # [ 202.620513] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 202.628248] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 202.641725] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 202.644417] nix-daemon[856]: reaped child process 2130, status = succeeded worker1 # [ 202.656219] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 203.565629] nix-daemon[856]: reaped child process 2139, status = succeeded worker1 # [ 203.634131] nix-daemon[856]: accepted connection from pid 2155, user nix-grpc-daemon (trusted) worker1 # [ 203.655461] nix-daemon[856]: reaped child process 2158, status = succeeded lb # [ 203.658434] niks3-server[1178]: 2026/09/23 16:15:49 INFO Received push request method=POST path=/api/pushes worker1 # [ 203.672433] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.009Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 203.674119] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.009Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)" worker1 # [ 203.715148] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.053Z level=INFO msg="Uploading 1 narinfos" lb # [ 203.701126] niks3-server[1178]: 2026/09/23 16:15:50 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign lb # [ 203.720892] niks3-server[1178]: 2026/09/23 16:15:50 INFO Signed narinfos id=81 count=1 lb # [ 203.723980] niks3-server[1178]: 2026/09/23 16:15:50 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst lb # [ 203.738856] niks3-server[1178]: 2026/09/23 16:15:50 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls lb # [ 203.750294] niks3-server[1178]: 2026/09/23 16:15:50 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv lb # [ 203.753732] niks3-server[1178]: 2026/09/23 16:15:50 INFO Received complete push request method=POST path=/api/pushes/81/complete worker1 # [ 203.759048] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.097Z level=INFO msg="Upload complete. (292ms)" lb # [ 203.758357] niks3-server[1178]: 2026/09/23 16:15:50 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo worker2 # [ 203.782954] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:50Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 203.771711] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=16 outputs=1 worker1 # [ 203.792727] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 203.800273] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 203.805843] nix-daemon[856]: reaped child process 2137, status = succeeded worker1 # [ 203.808558] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.147Z level=INFO msg="All 2 paths already cached" worker1 # [ 203.816164] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 203.826401] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 203.962688] nix-daemon[856]: accepted connection from pid 2162, user nix-grpc-daemon (trusted) worker1 # [ 203.980610] nix-daemon[856]: reaped child process 2165, status = succeeded lb # [ 203.983594] niks3-server[1178]: 2026/09/23 16:15:50 INFO Received push request method=POST path=/api/pushes worker1 # [ 203.994152] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.332Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 203.995670] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.332Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)" lb # [ 204.015280] niks3-server[1178]: 2026/09/23 16:15:50 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls lb # [ 204.027553] niks3-server[1178]: 2026/09/23 16:15:50 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign worker1 # [ 204.035066] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.373Z level=INFO msg="Uploading 1 narinfos" lb # [ 204.038993] niks3-server[1178]: 2026/09/23 16:15:50 INFO Signed narinfos id=82 count=1 lb # [ 204.044296] niks3-server[1178]: 2026/09/23 16:15:50 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst lb # [ 204.056284] niks3-server[1178]: 2026/09/23 16:15:50 INFO Received complete push request method=POST path=/api/pushes/82/complete worker1 # [ 204.068405] nix-grpc-daemon[1857]: time=2026-09-23T16:15:50.406Z level=INFO msg="Upload complete. (231ms)" lb # [ 204.067741] niks3-server[1178]: 2026/09/23 16:15:50 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo worker1 # [ 204.078620] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 204.094435] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 204.101423] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 204.107084] nix-daemon[856]: reaped child process 2160, status = succeeded worker1 # [ 204.122015] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 205.370673] nix-daemon[856]: reaped child process 2169, status = succeeded worker1 # [ 205.482743] nix-daemon[856]: accepted connection from pid 2185, user nix-grpc-daemon (trusted) worker1 # [ 205.501816] nix-daemon[856]: reaped child process 2188, status = succeeded lb # [ 205.507926] niks3-server[1178]: 2026/09/23 16:15:51 INFO Received push request method=POST path=/api/pushes worker1 # [ 205.522657] nix-grpc-daemon[1857]: time=2026-09-23T16:15:51.859Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 205.524788] nix-grpc-daemon[1857]: time=2026-09-23T16:15:51.859Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)" lb # [ 205.549570] niks3-server[1178]: 2026/09/23 16:15:51 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls worker1 # [ 205.564304] nix-grpc-daemon[1857]: time=2026-09-23T16:15:51.902Z level=INFO msg="Uploading 1 narinfos" lb # [ 205.562969] niks3-server[1178]: 2026/09/23 16:15:51 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign lb # [ 205.574462] niks3-server[1178]: 2026/09/23 16:15:51 INFO Signed narinfos id=83 count=1 lb # [ 205.579277] niks3-server[1178]: 2026/09/23 16:15:51 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv lb # [ 205.590172] niks3-server[1178]: 2026/09/23 16:15:51 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst lb # [ 205.595506] niks3-server[1178]: 2026/09/23 16:15:51 INFO Received complete push request method=POST path=/api/pushes/83/complete lb # [ 205.599114] niks3-server[1178]: 2026/09/23 16:15:51 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo worker1 # [ 205.600857] nix-grpc-daemon[1857]: time=2026-09-23T16:15:51.939Z level=INFO msg="Upload complete. (268ms)" worker2 # [ 205.624939] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:51Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 205.617249] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=17 outputs=1 worker1 # [ 205.636913] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 205.640664] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 205.645460] nix-daemon[856]: reaped child process 2167, status = succeeded worker1 # [ 205.652300] nix-grpc-daemon[1857]: time=2026-09-23T16:15:51.990Z level=INFO msg="All 2 paths already cached" worker1 # [ 205.659667] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 205.668662] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 205.801805] nix-daemon[856]: accepted connection from pid 2192, user nix-grpc-daemon (trusted) worker1 # [ 205.821786] nix-daemon[856]: reaped child process 2195, status = succeeded lb # [ 205.825351] niks3-server[1178]: 2026/09/23 16:15:52 INFO Received push request method=POST path=/api/pushes worker1 # [ 205.835964] nix-grpc-daemon[1857]: time=2026-09-23T16:15:52.174Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 205.837668] nix-grpc-daemon[1857]: time=2026-09-23T16:15:52.174Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)" lb # [ 205.855315] niks3-server[1178]: 2026/09/23 16:15:52 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls worker1 # [ 205.867366] nix-grpc-daemon[1857]: time=2026-09-23T16:15:52.205Z level=INFO msg="Uploading 1 narinfos" lb # [ 205.868632] niks3-server[1178]: 2026/09/23 16:15:52 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign lb # [ 205.879566] niks3-server[1178]: 2026/09/23 16:15:52 INFO Signed narinfos id=84 count=1 lb # [ 205.892667] niks3-server[1178]: 2026/09/23 16:15:52 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst lb # [ 205.899596] niks3-server[1178]: 2026/09/23 16:15:52 INFO Received complete push request method=POST path=/api/pushes/84/complete worker1 # [ 205.906671] nix-grpc-daemon[1857]: time=2026-09-23T16:15:52.245Z level=INFO msg="Upload complete. (227ms)" lb # [ 205.906578] niks3-server[1178]: 2026/09/23 16:15:52 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo worker1 # [ 205.915708] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 205.932805] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 worker1 # [ 205.939079] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 205.943985] nix-daemon[856]: reaped child process 2190, status = succeeded worker1 # [ 205.958661] nix-daemon[856]: accepted connection from pid 1797, user nix-grpc-daemon (trusted) worker1 # [ 207.240638] nix-daemon[856]: reaped child process 2199, status = succeeded worker1 # [ 207.343221] nix-daemon[856]: accepted connection from pid 2215, user nix-grpc-daemon (trusted) worker1 # [ 207.362570] nix-daemon[856]: reaped child process 2218, status = succeeded lb # [ 207.369260] niks3-server[1178]: 2026/09/23 16:15:53 INFO Received push request method=POST path=/api/pushes worker1 # [ 207.384445] nix-grpc-daemon[1857]: time=2026-09-23T16:15:53.720Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 207.386130] nix-grpc-daemon[1857]: time=2026-09-23T16:15:53.720Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)" lb # [ 207.416930] niks3-server[1178]: 2026/09/23 16:15:53 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker1 # [ 207.432113] nix-grpc-daemon[1857]: time=2026-09-23T16:15:53.770Z level=INFO msg="Uploading 1 narinfos" lb # [ 207.432042] niks3-server[1178]: 2026/09/23 16:15:53 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign lb # [ 207.435667] niks3-server[1178]: 2026/09/23 16:15:53 INFO Signed narinfos id=85 count=1 lb # [ 207.447412] niks3-server[1178]: 2026/09/23 16:15:53 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls lb # [ 207.459749] niks3-server[1178]: 2026/09/23 16:15:53 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst worker1 # [ 207.467619] nix-grpc-daemon[1857]: time=2026-09-23T16:15:53.806Z level=INFO msg="Upload complete. (297ms)" lb # [ 207.467363] niks3-server[1178]: 2026/09/23 16:15:53 INFO Received complete push request method=POST path=/api/pushes/85/complete lb # [ 207.471326] niks3-server[1178]: 2026/09/23 16:15:53 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo worker1 # [ 207.481286] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39762 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=18 outputs=1 worker1 # [ 207.500926] nix-daemon[856]: reaped child process 2197, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.29 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 207.685614] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: cfb8fc73c3c44e09825ba57aa4dfa9b9 client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics client # [ 207.895256] nix[1620]: don't know how to build these paths: client # [ 207.896419] nix[1620]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv worker2 # [ 207.900644] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:58344 worker1 # [ 207.888543] nix-grpc-daemon[1797]: ts=2026-09-23T16:15:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39762 worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker2 # [ 207.925777] nix-grpc-daemon[1504]: ts=2026-09-23T16:15:54Z level=debug event=want client=7 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=0 client # [ 207.930550] nix[1620]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code lb # [ 209.269968] envoy[895]: conn peer=[2001:db8:1::1]:52714 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27213 tx=12359 ms=16018 client # [ 237.937133] nix[1620]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip} client # [ 237.953019] nix[1620]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed client # [ 237.985228] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 237.993015] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 237.997447] systemd[1]: gpu.service: Consumed 198ms CPU time over 30.305s wall clock time, 35.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.09 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.10 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill lb # [ 239.273888] envoy[895]: conn peer=[2001:db8:1::1]:57428 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=30067 worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.22 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1 # [ 240.174651] nix-grpc-daemon[1797]: ts=2026-09-23T16:16:26Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2 worker2 # [ 240.913838] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:58344 client # don't know how to build these paths: client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker2 # [ 240.923610] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 240.951543] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=want client=8 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=0 worker2 # [ 240.960998] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 240.969583] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 240.977169] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58344 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'... worker2 # [ 240.989122] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 241.004414] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 241.146965] nix-daemon[844]: accepted connection from pid 1873, user nix-grpc-daemon (trusted) worker2 # [ 241.162874] nix-daemon[844]: reaped child process 1876, status = succeeded lb # [ 241.155607] niks3-server[1178]: 2026/09/23 16:16:27 INFO Received push request method=POST path=/api/pushes worker2 # [ 241.183748] nix-grpc-daemon[1578]: time=2026-09-23T16:16:27.508Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 241.185858] nix-grpc-daemon[1578]: time=2026-09-23T16:16:27.508Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)" lb # [ 241.199430] niks3-server[1178]: 2026/09/23 16:16:27 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst lb # [ 241.225396] niks3-server[1178]: 2026/09/23 16:16:27 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign worker2 # [ 241.240800] nix-grpc-daemon[1578]: time=2026-09-23T16:16:27.565Z level=INFO msg="Uploading 1 narinfos" lb # [ 241.230930] niks3-server[1178]: 2026/09/23 16:16:27 INFO Signed narinfos id=86 count=1 lb # [ 241.237312] niks3-server[1178]: 2026/09/23 16:16:27 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls lb # [ 241.244413] niks3-server[1178]: 2026/09/23 16:16:27 INFO Received complete push request method=POST path=/api/pushes/86/complete lb # [ 241.253636] niks3-server[1178]: 2026/09/23 16:16:27 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo worker2 # [ 241.276708] nix-grpc-daemon[1578]: time=2026-09-23T16:16:27.601Z level=INFO msg="Upload complete. (257ms)" worker2 # [ 241.288365] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 241.300577] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 241.314514] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 241.328402] nix-daemon[844]: reaped child process 1871, status = succeeded client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051... client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker2 # [ 241.338831] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 242.416876] nix-daemon[844]: reaped child process 1880, status = succeeded worker2 # [ 242.513077] nix-daemon[844]: accepted connection from pid 1896, user nix-grpc-daemon (trusted) worker2 # [ 242.532174] nix-daemon[844]: reaped child process 1899, status = succeeded lb # [ 242.523000] niks3-server[1178]: 2026/09/23 16:16:28 INFO Received push request method=POST path=/api/pushes worker2 # [ 242.559370] nix-grpc-daemon[1578]: time=2026-09-23T16:16:28.882Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 242.561426] nix-grpc-daemon[1578]: time=2026-09-23T16:16:28.883Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)" lb # [ 242.571111] niks3-server[1178]: 2026/09/23 16:16:28 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign worker2 # [ 242.594622] nix-grpc-daemon[1578]: time=2026-09-23T16:16:28.919Z level=INFO msg="Uploading 1 narinfos" lb # [ 242.582229] niks3-server[1178]: 2026/09/23 16:16:28 INFO Signed narinfos id=87 count=1 lb # [ 242.589103] niks3-server[1178]: 2026/09/23 16:16:28 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 242.599840] niks3-server[1178]: 2026/09/23 16:16:28 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv lb # [ 242.611613] niks3-server[1178]: 2026/09/23 16:16:28 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls worker2 # [ 242.633732] nix-grpc-daemon[1578]: time=2026-09-23T16:16:28.958Z level=INFO msg="Upload complete. (282ms)" lb # [ 242.620323] niks3-server[1178]: 2026/09/23 16:16:28 INFO Received complete push request method=POST path=/api/pushes/87/complete lb # [ 242.623619] niks3-server[1178]: 2026/09/23 16:16:28 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo worker2 # [ 242.645124] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=19 outputs=1 worker2 # [ 242.669055] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:28Z level=debug event=want client=8 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=0 worker2 # [ 242.678711] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:28Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 242.684395] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 242.689418] nix-daemon[844]: reaped child process 1878, status = succeeded worker2 # [ 242.692997] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 242.705430] nix-grpc-daemon[1578]: time=2026-09-23T16:16:29.030Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'... worker2 # [ 242.715364] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 242.727412] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 242.869012] nix-daemon[844]: accepted connection from pid 1905, user nix-grpc-daemon (trusted) worker2 # [ 242.888653] nix-daemon[844]: reaped child process 1908, status = succeeded lb # [ 242.877281] niks3-server[1178]: 2026/09/23 16:16:29 INFO Received push request method=POST path=/api/pushes worker2 # [ 242.902672] nix-grpc-daemon[1578]: time=2026-09-23T16:16:29.227Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 242.904748] nix-grpc-daemon[1578]: time=2026-09-23T16:16:29.229Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)" lb # [ 242.911115] niks3-server[1178]: 2026/09/23 16:16:29 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst worker2 # [ 242.937512] nix-grpc-daemon[1578]: time=2026-09-23T16:16:29.262Z level=INFO msg="Uploading 1 narinfos" lb # [ 242.925691] niks3-server[1178]: 2026/09/23 16:16:29 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign lb # [ 242.933939] niks3-server[1178]: 2026/09/23 16:16:29 INFO Signed narinfos id=88 count=1 lb # [ 242.939139] niks3-server[1178]: 2026/09/23 16:16:29 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls lb # [ 242.947312] niks3-server[1178]: 2026/09/23 16:16:29 INFO Received complete push request method=POST path=/api/pushes/88/complete worker2 # [ 242.970460] nix-grpc-daemon[1578]: time=2026-09-23T16:16:29.295Z level=INFO msg="Upload complete. (229ms)" lb # [ 242.954929] niks3-server[1178]: 2026/09/23 16:16:29 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo worker2 # [ 242.981075] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=632 worker2 # [ 242.995090] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 243.001588] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 243.006675] nix-daemon[844]: reaped child process 1903, status = succeeded client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051... worker2 # [ 243.017143] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 243.881881] nix-daemon[844]: reaped child process 1912, status = succeeded worker2 # [ 244.017765] nix-daemon[844]: accepted connection from pid 1928, user nix-grpc-daemon (trusted) worker2 # [ 244.034926] nix-daemon[844]: reaped child process 1931, status = succeeded lb # [ 244.027391] niks3-server[1178]: 2026/09/23 16:16:30 INFO Received push request method=POST path=/api/pushes worker2 # [ 244.056485] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.378Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 244.058164] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.378Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)" worker2 # [ 244.102064] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.426Z level=INFO msg="Uploading 1 narinfos" lb # [ 244.075325] niks3-server[1178]: 2026/09/23 16:16:30 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign lb # [ 244.091385] niks3-server[1178]: 2026/09/23 16:16:30 INFO Signed narinfos id=89 count=1 lb # [ 244.098689] niks3-server[1178]: 2026/09/23 16:16:30 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst lb # [ 244.108677] niks3-server[1178]: 2026/09/23 16:16:30 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls lb # [ 244.121951] niks3-server[1178]: 2026/09/23 16:16:30 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv lb # [ 244.127666] niks3-server[1178]: 2026/09/23 16:16:30 INFO Received complete push request method=POST path=/api/pushes/89/complete worker2 # [ 244.143833] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.468Z level=INFO msg="Upload complete. (314ms)" lb # [ 244.131898] niks3-server[1178]: 2026/09/23 16:16:30 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo worker2 # [ 244.161027] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=20 outputs=1 worker2 # [ 244.184472] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=debug event=want client=8 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=0 worker2 # [ 244.191762] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 244.199353] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 244.205717] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 244.211589] nix-daemon[844]: reaped child process 1910, status = succeeded worker2 # [ 244.215823] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.540Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'... worker2 # [ 244.226430] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 244.240390] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 244.381348] nix-daemon[844]: accepted connection from pid 1937, user nix-grpc-daemon (trusted) worker2 # [ 244.400466] nix-daemon[844]: reaped child process 1940, status = succeeded lb # [ 244.388902] niks3-server[1178]: 2026/09/23 16:16:30 INFO Received push request method=POST path=/api/pushes worker2 # [ 244.413802] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.738Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 244.415349] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.738Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)" lb # [ 244.419327] niks3-server[1178]: 2026/09/23 16:16:30 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign worker2 # [ 244.442474] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.767Z level=INFO msg="Uploading 1 narinfos" lb # [ 244.432643] niks3-server[1178]: 2026/09/23 16:16:30 INFO Signed narinfos id=90 count=1 lb # [ 244.435834] niks3-server[1178]: 2026/09/23 16:16:30 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls lb # [ 244.450155] niks3-server[1178]: 2026/09/23 16:16:30 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst lb # [ 244.462251] niks3-server[1178]: 2026/09/23 16:16:30 INFO Received complete push request method=POST path=/api/pushes/90/complete worker2 # [ 244.483952] nix-grpc-daemon[1578]: time=2026-09-23T16:16:30.809Z level=INFO msg="Upload complete. (231ms)" lb # [ 244.468880] niks3-server[1178]: 2026/09/23 16:16:30 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo worker2 # [ 244.496340] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=0 paths=1 nar_bytes_in=760 worker2 # [ 244.510951] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 worker2 # [ 244.518151] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) worker2 # [ 244.523141] nix-daemon[844]: reaped child process 1935, status = succeeded client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051... worker2 # [ 244.536954] nix-daemon[844]: accepted connection from pid 1504, user nix-grpc-daemon (trusted) client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 245.812546] nix-daemon[844]: reaped child process 1944, status = succeeded worker2 # [ 245.918086] nix-daemon[844]: accepted connection from pid 1960, user nix-grpc-daemon (trusted) worker2 # [ 245.939758] nix-daemon[844]: reaped child process 1963, status = succeeded lb # [ 245.928621] niks3-server[1178]: 2026/09/23 16:16:32 INFO Received push request method=POST path=/api/pushes worker2 # [ 245.955210] nix-grpc-daemon[1578]: time=2026-09-23T16:16:32.278Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 245.956954] nix-grpc-daemon[1578]: time=2026-09-23T16:16:32.279Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)" lb # [ 245.960202] niks3-server[1178]: 2026/09/23 16:16:32 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv lb # [ 245.980535] niks3-server[1178]: 2026/09/23 16:16:32 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst worker2 # [ 246.009098] nix-grpc-daemon[1578]: time=2026-09-23T16:16:32.333Z level=INFO msg="Uploading 1 narinfos" lb # [ 245.994771] niks3-server[1178]: 2026/09/23 16:16:32 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign lb # [ 246.004759] niks3-server[1178]: 2026/09/23 16:16:32 INFO Signed narinfos id=91 count=1 lb # [ 246.011063] niks3-server[1178]: 2026/09/23 16:16:32 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls lb # [ 246.017866] niks3-server[1178]: 2026/09/23 16:16:32 INFO Received complete push request method=POST path=/api/pushes/91/complete lb # [ 246.028106] niks3-server[1178]: 2026/09/23 16:16:32 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo worker2 # [ 246.044225] nix-grpc-daemon[1578]: time=2026-09-23T16:16:32.369Z level=INFO msg="Upload complete. (296ms)" worker2 # [ 246.057298] nix-grpc-daemon[1504]: ts=2026-09-23T16:16:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58344 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=21 outputs=1 worker2 # [ 246.076572] nix-daemon[844]: reaped child process 1942, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.41 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 247.185846] nix-grpc-daemon[1797]: ts=2026-09-23T16:16:33Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 9.26 seconds) (finished: run the VM test script, in 248.74 seconds) test script finished in 248.98s cleanup kill QemuMachine (pid 46) client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) client # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 48) lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) lb # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker1 # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T16:16:34Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker2 # [2026-09-23T16:16:35Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T16:16:35Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T16:16:35Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.20 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh