tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm worker2 # Disk image does not exist, creating the virtualisation disk image... lb # Disk image does not exist, creating the virtualisation disk image... client # Disk image does not exist, creating the virtualisation disk image... client: QEMU running (pid 49) worker2: QEMU running (pid 52) worker1 # Disk image does not exist, creating the virtualisation disk image... worker1: QEMU running (pid 51) lb: QEMU running (pid 50) worker2 # Formatting '/build/vm-state-worker2/tmp.UBUbkLo2Ao', fmt=raw size=1073741824 client # Formatting '/build/vm-state-client/tmp.q4xRT4xwti', fmt=raw size=1073741824 worker1 # Formatting '/build/vm-state-worker1/tmp.pLnZvU534n', fmt=raw size=1073741824 lb # Formatting '/build/vm-state-lb/tmp.AA5qMkLJgX', fmt=raw size=1073741824 worker2 # mke2fs 1.47.4 (6-Mar-2025) worker1 # mke2fs 1.47.4 (6-Mar-2025) lb # mke2fs 1.47.4 (6-Mar-2025) worker2 # Discarding device blocks: 0/262144 done worker1 # Discarding device blocks: 0/262144 done lb # Discarding device blocks: 0/262144 done worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes lb # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # Filesystem UUID: 1aed990d-7dc4-4d98-a511-11af0c3377ab worker1 # Filesystem UUID: 5d94f2b0-e56f-404b-be05-4831c77c48d8 lb # Filesystem UUID: 17035f7d-332b-4eff-bed7-2108b75626d2 worker2 # Superblock backups stored on blocks: worker1 # Superblock backups stored on blocks: lb # Superblock backups stored on blocks: worker2 # 32768, 98304, 163840, 229376 worker1 # 32768, 98304, 163840, 229376 lb # 32768, 98304, 163840, 229376 worker2 # worker1 # lb # worker2 # Allocating group tables: 0/8 done worker1 # Allocating group tables: 0/8 done lb # Allocating group tables: 0/8 done worker2 # Writing inode tables: 0/8 done worker1 # Writing inode tables: 0/8 done lb # Writing inode tables: 0/8 done worker2 # Creating journal (8192 blocks): done worker1 # Creating journal (8192 blocks): done lb # Creating journal (8192 blocks): done worker2 # Writing superblocks and filesystem accounting information: 0/8 done worker1 # Writing superblocks and filesystem accounting information: 0/8 done lb # Writing superblocks and filesystem accounting information: 0/8 done worker2 # worker1 # lb # worker2 # Virtualisation disk image created. worker1 # Virtualisation disk image created. lb # Virtualisation disk image created. worker2 # Starting virtiofs daemons... worker1 # Starting virtiofs daemons... lb # Starting virtiofs daemons... worker2 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests (finished: start all VMs, in 0.53 seconds) client # mke2fs 1.47.4 (6-Mar-2025) client: waiting for unit network-addresses-eth1.service client # Discarding device blocks: 0/262144 done client: waiting for the VM to finish booting client # Creating filesystem with 262144 4k blocks and 65536 inodes client # Filesystem UUID: 8a2019a4-cdd5-4625-8535-d98bd9f80ddd client # Superblock backups stored on blocks: client # 32768, 98304, 163840, 229376 client # client # Allocating group tables: 0/8 done client # Writing inode tables: 0/8 done client # Creating journal (8192 blocks): done client # Writing superblocks and filesystem accounting information: 0/8 done client # client # Virtualisation disk image created. client # Starting virtiofs daemons... client # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] KASLR enabled worker1 # [ 0.000000] random: crng init done worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] Machine model: linux,dummy-virt worker2 # [ 0.000000] KASLR enabled client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] random: crng init done worker2 # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker2 # [ 0.000000] efi: UEFI not found. client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] KASLR enabled worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] random: crng init done worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker1 # [ 0.000000] Zone ranges: worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] Zone ranges: worker1 # [ 0.000000] DMA32 empty client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000000] Normal empty worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] Device empty worker2 # [ 0.000000] DMA32 empty client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] Movable zone start for each node worker2 # [ 0.000000] Normal empty worker1 # [ 0.000000] Early memory node ranges worker2 # [ 0.000000] Device empty client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] Zone ranges: lb # [ 0.000000] KASLR enabled worker2 # [ 0.000000] Movable zone start for each node worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] random: crng init done worker2 # [ 0.000000] Early memory node ranges client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] DMA32 empty worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] efi: UEFI not found. client # [ 0.000000] Normal empty worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] Device empty worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] Movable zone start for each node client # [ 0.000000] Early memory node ranges worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware. lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] psci: probing for conduit method from DT. worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.000000] Zone ranges: client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] psci: SMC Calling Convention v1.1 worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.000000] DMA32 empty client # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.000000] Normal empty worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker2 # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.000000] Device empty client # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 lb # [ 0.000000] Movable zone start for each node worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs lb # [ 0.000000] Early memory node ranges worker1 # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] psci: Trusted OS migration not required worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker2 # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] CPU features: detected: Spectre-v4 lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] CPU features: detected: Spectre-BHB worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 lb # [ 0.000000] psci: probing for conduit method from DT. worker2 # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker2 # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.000000] psci: Trusted OS migration not required worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker1 # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] CPU features: detected: GICv3 CPU interface lb # [ 0.000000] psci: SMC Calling Convention v1.1 worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] CPU features: detected: Spectre-v4 worker2 # [ 0.000000] alternatives: applying boot alternatives lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 lb # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] alternatives: applying boot alternatives lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ry7bqhhhnwa7dlpa75fylilll15fakpb-nixos-system-worker1-test/init regInfo=/nix/store/94nix7rmj7g6brapzr4gg95i44rfy50d-closure-info/registration console=ttyAMA0,115200n8 console=tty0 lb # [ 0.000000] CPU features: detected: GICv3 CPU interface lb # [ 0.000000] CPU features: detected: Spectre-v4 worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/30asfw61910plj74np6pz2ka43zplm0r-nixos-system-worker2-test/init regInfo=/nix/store/cpmzrhcw14bjzfxvmig0nm1xvcis5jif-closure-info/registration console=ttyAMA0,115200n8 console=tty0 lb # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/94nix7rmj7g6brapzr4gg95i44rfy50d-closure-info/registration", will be passed to user space. lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/cpmzrhcw14bjzfxvmig0nm1xvcis5jif-closure-info/registration", will be passed to user space. lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test/init regInfo=/nix/store/149kn4k15jb67kmb28r78yiz2vw4r88d-closure-info/registration console=ttyAMA0,115200n8 console=tty0 lb # [ 0.000000] alternatives: applying boot alternatives worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/149kn4k15jb67kmb28r78yiz2vw4r88d-closure-info/registration", will be passed to user space. worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker1 # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] software IO TLB: area num 1. worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kgjzg78npi7z07am2szqxkr41f06jp5w-nixos-system-lb-test/init regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration", will be passed to user space. worker2 # [ 0.000000] Fallback order for Node 0: 0 client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] Policy zone: DMA client # [ 0.000000] software IO TLB: area num 1. lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker2 # [ 0.000000] Policy zone: DMA client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker1 # [ 0.000000] allocated 3145728 bytes of page_ext lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] allocated 3145728 bytes of page_ext client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages client # [ 0.000000] Policy zone: DMA lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker1 # [ 0.000000] rcu: Hierarchical RCU implementation. worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups lb # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.000000] allocated 3145728 bytes of page_ext worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled. lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages lb # [ 0.000000] Fallback order for Node 0: 0 worker1 # [ 0.000000] Rude variant of Tasks RCU enabled. worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled. lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.000000] rcu: Hierarchical RCU implementation. lb # [ 0.000000] Policy zone: DMA worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.000000] rcu: RCU event tracing is enabled. lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 client # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 lb # [ 0.000000] allocated 3145728 bytes of page_ext worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages client # [ 0.000000] Tracing variant of Tasks RCU enabled. worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.000000] rcu: Hierarchical RCU implementation. worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] rcu: RCU event tracing is enabled. client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000000] GICv3: 256 SPIs implemented client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented worker2 # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] Rude variant of Tasks RCU enabled. worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] client # [ 0.000000] GICv3: 256 SPIs implemented worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) client # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 lb # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] GICv3: 0 Extended SPIs implemented worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 lb # [ 0.000000] Root IRQ handler: gic_handle_irq worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker1 # [ 0.000028] arm-pv: using stolen time PV client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker2 # [ 0.000029] arm-pv: using stolen time PV client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker1 # [ 0.000385] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker1 # [ 0.000551] Console: colour dummy device 80x25 lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker1 # [ 0.000558] printk: legacy console [tty0] enabled worker2 # [ 0.000375] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.000541] Console: colour dummy device 80x25 lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.000548] printk: legacy console [tty0] enabled worker1 # [ 0.000759] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) client # [ 0.000030] arm-pv: using stolen time PV lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.000766] pid_max: default: 32768 minimum: 301 worker2 # [ 0.000740] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker1 # [ 0.000849] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.000403] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.000747] pid_max: default: 32768 minimum: 301 worker1 # [ 0.000978] landlock: Up and running. client # [ 0.000619] Console: colour dummy device 80x25 worker1 # [ 0.000981] Yama: becoming mindful. worker2 # [ 0.000828] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.000627] printk: legacy console [tty0] enabled worker1 # [ 0.001404] LSM support for eBPF active worker2 # [ 0.000963] landlock: Up and running. worker2 # [ 0.000966] Yama: becoming mindful. worker1 # [ 0.001577] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.001379] LSM support for eBPF active client # [ 0.000845] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) lb # [ 0.000029] arm-pv: using stolen time PV client # [ 0.000852] pid_max: default: 32768 minimum: 301 worker1 # [ 0.001619] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.001533] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.003476] rcu: Hierarchical SRCU implementation. client # [ 0.000923] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.000396] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.001053] landlock: Up and running. worker1 # [ 0.003481] rcu: Max phase no-delay instances is 1000. lb # [ 0.000560] Console: colour dummy device 80x25 worker2 # [ 0.001566] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.001056] Yama: becoming mindful. worker1 # [ 0.004748] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.003426] rcu: Hierarchical SRCU implementation. lb # [ 0.000567] printk: legacy console [tty0] enabled client # [ 0.001510] LSM support for eBPF active worker1 # [ 0.004839] EFI services will not be available. worker2 # [ 0.003431] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.004918] smp: Bringing up secondary CPUs ... worker2 # [ 0.004650] fsl-mc MSI: its@8080000 domain created client # [ 0.001667] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000753] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.004926] smp: Brought up 1 node, 1 CPU worker2 # [ 0.004740] EFI services will not be available. lb # [ 0.000759] pid_max: default: 32768 minimum: 301 worker1 # [ 0.004929] SMP: Total of 1 processors activated. client # [ 0.001700] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.004806] smp: Bringing up secondary CPUs ... worker1 # [ 0.004935] CPU: All CPU(s) started at EL1 lb # [ 0.000830] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.003495] rcu: Hierarchical SRCU implementation. worker2 # [ 0.004815] smp: Brought up 1 node, 1 CPU lb # [ 0.000969] landlock: Up and running. client # [ 0.003500] rcu: Max phase no-delay instances is 1000. worker2 # [ 0.004818] SMP: Total of 1 processors activated. lb # [ 0.000972] Yama: becoming mindful. worker1 # [ 0.004945] CPU features: detected: Branch Target Identification worker2 # [ 0.004823] CPU: All CPU(s) started at EL1 client # [ 0.004793] fsl-mc MSI: its@8080000 domain created lb # [ 0.001406] LSM support for eBPF active worker1 # [ 0.004950] CPU features: detected: ARMv8.4 Translation Table Level client # [ 0.004884] EFI services will not be available. worker2 # [ 0.004833] CPU features: detected: Branch Target Identification lb # [ 0.001570] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.004957] smp: Bringing up secondary CPUs ... worker1 # [ 0.004953] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.004838] CPU features: detected: ARMv8.4 Translation Table Level client # [ 0.004965] smp: Brought up 1 node, 1 CPU lb # [ 0.001602] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.004968] SMP: Total of 1 processors activated. worker1 # [ 0.004957] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.003448] rcu: Hierarchical SRCU implementation. worker2 # [ 0.004841] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.004974] CPU: All CPU(s) started at EL1 worker1 # [ 0.004961] CPU features: detected: Common not Private translations lb # [ 0.003452] rcu: Max phase no-delay instances is 1000. client # [ 0.004984] CPU features: detected: Branch Target Identification worker2 # [ 0.004845] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.004715] fsl-mc MSI: its@8080000 domain created worker1 # [ 0.004964] CPU features: detected: CRC32 instructions client # [ 0.004988] CPU features: detected: ARMv8.4 Translation Table Level lb # [ 0.004813] EFI services will not be available. worker2 # [ 0.004849] CPU features: detected: Common not Private translations worker1 # [ 0.004967] CPU features: detected: Data cache clean to Point of Deep Persistence lb # [ 0.004880] smp: Bringing up secondary CPUs ... worker2 # [ 0.004852] CPU features: detected: CRC32 instructions lb # [ 0.004888] smp: Brought up 1 node, 1 CPU client # [ 0.004991] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker1 # [ 0.004970] CPU features: detected: Data cache clean to Point of Persistence lb # [ 0.004892] SMP: Total of 1 processors activated. worker2 # [ 0.004855] CPU features: detected: Data cache clean to Point of Deep Persistence worker1 # [ 0.004973] CPU features: detected: Data independent timing control (DIT) lb # [ 0.004896] CPU: All CPU(s) started at EL1 client # [ 0.004995] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.004858] CPU features: detected: Data cache clean to Point of Persistence worker1 # [ 0.004976] CPU features: detected: E0PD lb # [ 0.004907] CPU features: detected: Branch Target Identification client # [ 0.004999] CPU features: detected: Common not Private translations worker1 # [ 0.004979] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.004861] CPU features: detected: Data independent timing control (DIT) client # [ 0.005002] CPU features: detected: CRC32 instructions lb # [ 0.004911] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.004864] CPU features: detected: E0PD worker1 # [ 0.004982] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.005005] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.004866] CPU features: detected: Enhanced Counter Virtualization lb # [ 0.004915] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker1 # [ 0.004985] CPU features: detected: Enhanced Virtualization Traps client # [ 0.005008] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.004869] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker1 # [ 0.004988] CPU features: detected: Fine Grained Traps lb # [ 0.004918] CPU features: detected: Data cache clean to the PoU not required for I/D coherence client # [ 0.005011] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.004873] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.004991] CPU features: detected: Generic authentication (architected QARMA5 algorithm) lb # [ 0.004922] CPU features: detected: Common not Private translations client # [ 0.005015] CPU features: detected: E0PD worker2 # [ 0.004876] CPU features: detected: Fine Grained Traps lb # [ 0.004926] CPU features: detected: CRC32 instructions worker1 # [ 0.004997] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.005017] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.004879] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker1 # [ 0.005000] CPU features: detected: LSE atomic instructions lb # [ 0.004928] CPU features: detected: Data cache clean to Point of Deep Persistence client # [ 0.005020] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker2 # [ 0.004884] CPU features: detected: RCpc load-acquire (LDAPR) worker1 # [ 0.005002] CPU features: detected: Privileged Access Never lb # [ 0.004932] CPU features: detected: Data cache clean to Point of Persistence worker1 # [ 0.005005] CPU features: detected: PMUv3 client # [ 0.005023] CPU features: detected: Enhanced Virtualization Traps worker2 # [ 0.004887] CPU features: detected: LSE atomic instructions lb # [ 0.004935] CPU features: detected: Data independent timing control (DIT) worker1 # [ 0.005008] CPU features: detected: RAS Extension Support worker2 # [ 0.004890] CPU features: detected: Privileged Access Never client # [ 0.005027] CPU features: detected: Fine Grained Traps lb # [ 0.004938] CPU features: detected: E0PD worker1 # [ 0.005011] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.004892] CPU features: detected: PMUv3 client # [ 0.005031] CPU features: detected: Generic authentication (architected QARMA5 algorithm) lb # [ 0.004941] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.004895] CPU features: detected: RAS Extension Support worker1 # [ 0.005013] CPU features: detected: Random Number Generator client # [ 0.005036] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.004898] CPU features: detected: RASv1p1 Extension Support worker1 # [ 0.005016] CPU features: detected: Speculation barrier (SB) lb # [ 0.004944] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.005039] CPU features: detected: LSE atomic instructions worker2 # [ 0.004901] CPU features: detected: Random Number Generator worker1 # [ 0.005019] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.004947] CPU features: detected: Enhanced Virtualization Traps client # [ 0.005041] CPU features: detected: Privileged Access Never worker2 # [ 0.004903] CPU features: detected: Speculation barrier (SB) lb # [ 0.004950] CPU features: detected: Fine Grained Traps client # [ 0.005044] CPU features: detected: PMUv3 worker2 # [ 0.004906] CPU features: detected: Stage-2 Force Write-Back client # [ 0.005047] CPU features: detected: RAS Extension Support lb # [ 0.004954] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.004909] CPU features: detected: TLB range maintenance instructions client # [ 0.005050] CPU features: detected: RASv1p1 Extension Support lb # [ 0.004959] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.005053] CPU features: detected: Random Number Generator worker2 # [ 0.004914] CPU features: detected: Speculative Store Bypassing Safe (SSBS) lb # [ 0.004962] CPU features: detected: LSE atomic instructions worker2 # [ 0.004954] alternatives: applying system-wide alternatives lb # [ 0.004965] CPU features: detected: Privileged Access Never lb # [ 0.004967] CPU features: detected: PMUv3 worker2 # [ 0.007825] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.004970] CPU features: detected: RAS Extension Support lb # [ 0.004973] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.007993] Memory: 1405656K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved) lb # [ 0.004976] CPU features: detected: Random Number Generator worker2 # [ 0.008372] devtmpfs: initialized lb # [ 0.004978] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.010132] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.004981] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.004984] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.010154] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.005022] CPU features: detected: TLB range maintenance instructions lb # [ 0.004988] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker2 # [ 0.010335] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker1 # [ 0.005027] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker2 # [ 0.010339] 0 pages in range for non-PLT usage worker1 # [ 0.005065] alternatives: applying system-wide alternatives worker2 # [ 0.010340] 508272 pages in range for PLT usage worker2 # [ 0.010445] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.007987] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.011126] DMI not present or invalid. client # [ 0.005055] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.014372] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.005058] CPU features: detected: Stage-2 Force Write-Back worker1 # [ 0.008126] Memory: 1405880K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved) client # [ 0.005061] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.016530] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.008517] devtmpfs: initialized client # [ 0.005066] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.010356] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.005103] alternatives: applying system-wide alternatives worker2 # [ 0.016696] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker1 # [ 0.010378] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.008070] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.016889] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker1 # [ 0.010551] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker2 # [ 0.016909] audit: initializing netlink subsys (disabled) worker1 # [ 0.010555] 0 pages in range for non-PLT usage worker2 # [ 0.017436] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.010556] 508272 pages in range for PLT usage client # [ 0.008241] Memory: 1405652K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved) client # [ 0.008639] devtmpfs: initialized worker1 # [ 0.010651] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.017438] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.011365] DMI not present or invalid. lb # [ 0.005023] alternatives: applying system-wide alternatives client # [ 0.010401] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.017442] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.014578] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.007957] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.017445] thermal_sys: Registered thermal governor 'user_space' client # [ 0.010422] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.016750] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.017450] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.010618] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker1 # [ 0.016921] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.008191] Memory: 1405676K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved) client # [ 0.010622] 0 pages in range for non-PLT usage worker2 # [ 0.017472] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.008586] devtmpfs: initialized client # [ 0.010624] 508272 pages in range for PLT usage worker2 # [ 0.017482] cpuidle: using governor ladder worker1 # [ 0.017115] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.010719] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.017488] cpuidle: using governor menu lb # [ 0.010409] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.017135] audit: initializing netlink subsys (disabled) client # [ 0.011498] DMI not present or invalid. worker2 # [ 0.017675] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.017709] thermal_sys: Registered thermal governor 'fair_share' lb # [ 0.010430] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.014726] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.017690] ASID allocator initialised with 65536 entries worker1 # [ 0.017711] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.010618] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.016894] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.018826] Serial: AMBA PL011 UART driver lb # [ 0.010622] 0 pages in range for non-PLT usage worker1 # [ 0.017715] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.010623] 508272 pages in range for PLT usage client # [ 0.017068] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.023967] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker1 # [ 0.017717] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.010731] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.024094] printk: console [ttyAMA0] enabled client # [ 0.017263] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.011454] DMI not present or invalid. worker1 # [ 0.017723] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.017283] audit: initializing netlink subsys (disabled) lb # [ 0.014571] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.017747] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 client # [ 0.017824] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.017755] cpuidle: using governor ladder lb # [ 0.016782] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.017761] cpuidle: using governor menu client # [ 0.017826] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.016951] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.017830] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.017945] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.017960] ASID allocator initialised with 65536 entries lb # [ 0.017148] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.017832] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.019127] Serial: AMBA PL011 UART driver lb # [ 0.017168] audit: initializing netlink subsys (disabled) client # [ 0.017837] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.017685] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.024326] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 client # [ 0.017862] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.024477] printk: console [ttyAMA0] enabled lb # [ 0.017687] thermal_sys: Registered thermal governor 'bang_bang' client # [ 0.017870] cpuidle: using governor ladder client # [ 0.017876] cpuidle: using governor menu lb # [ 0.017691] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.141199] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.142917] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.017694] thermal_sys: Registered thermal governor 'user_space' client # [ 0.018070] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.141215] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker1 # [ 0.142935] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.018086] ASID allocator initialised with 65536 entries lb # [ 0.017699] thermal_sys: Registered thermal governor 'power_allocator' worker2 # [ 0.141220] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.019322] Serial: AMBA PL011 UART driver worker1 # [ 0.142940] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages lb # [ 0.017724] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.141225] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.024418] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker1 # [ 0.142944] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.017732] cpuidle: using governor ladder lb # [ 0.017737] cpuidle: using governor menu worker2 # [ 0.141229] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.024575] printk: console [ttyAMA0] enabled worker1 # [ 0.142949] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.017913] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.141233] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.142953] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.017928] ASID allocator initialised with 65536 entries client # [ 0.143963] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker2 # [ 0.141238] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.142958] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.019110] Serial: AMBA PL011 UART driver client # [ 0.143980] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.141242] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker1 # [ 0.142962] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.024388] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 client # [ 0.143985] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages lb # [ 0.024507] printk: console [ttyAMA0] enabled client # [ 0.143990] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.143994] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.148346] fbcon: Taking over console lb # [ 0.143051] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.150137] fbcon: Taking over console worker2 # [ 0.148359] ACPI: Interpreter disabled. client # [ 0.143998] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.150150] ACPI: Interpreter disabled. lb # [ 0.143068] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.144003] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.143074] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.144007] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.143078] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.143083] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.143087] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.151295] fbcon: Taking over console lb # [ 0.143091] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages client # [ 0.151307] ACPI: Interpreter disabled. lb # [ 0.143095] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page client # [ 0.153168] iommu: Default domain type: Translated client # [ 0.153178] iommu: DMA domain TLB invalidation policy: strict mode client # [ 0.154888] SCSI subsystem initialized lb # [ 0.150476] fbcon: Taking over console worker1 # [ 0.158915] iommu: Default domain type: Translated lb # [ 0.150488] ACPI: Interpreter disabled. worker2 # [ 0.157244] iommu: Default domain type: Translated worker1 # [ 0.158925] iommu: DMA domain TLB invalidation policy: strict mode worker2 # [ 0.157254] iommu: DMA domain TLB invalidation policy: strict mode worker2 # [ 0.157587] SCSI subsystem initialized worker1 # [ 0.160613] SCSI subsystem initialized worker2 # [ 0.159557] usbcore: registered new interface driver usbfs worker1 # [ 0.161292] usbcore: registered new interface driver usbfs worker2 # [ 0.159593] usbcore: registered new interface driver hub worker1 # [ 0.161316] usbcore: registered new interface driver hub worker2 # [ 0.159608] usbcore: registered new device driver usb worker1 # [ 0.161339] usbcore: registered new device driver usb worker2 # [ 0.159861] pps_core: LinuxPPS API ver. 1 registered worker1 # [ 0.161610] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.159867] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker1 # [ 0.161615] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.157512] iommu: Default domain type: Translated worker2 # [ 0.159876] PTP clock support registered worker1 # [ 0.161626] PTP clock support registered worker2 # [ 0.159925] EDAC MC: Ver: 3.0.0 lb # [ 0.157521] iommu: DMA domain TLB invalidation policy: strict mode worker1 # [ 0.161665] EDAC MC: Ver: 3.0.0 worker2 # [ 0.164523] scmi_core: SCMI protocol bus registered client # [ 0.162535] usbcore: registered new interface driver usbfs lb # [ 0.159141] SCSI subsystem initialized worker1 # [ 0.166231] scmi_core: SCMI protocol bus registered client # [ 0.162576] usbcore: registered new interface driver hub worker2 # [ 0.165488] FPGA manager framework client # [ 0.162592] usbcore: registered new device driver usb worker1 # [ 0.167178] FPGA manager framework lb # [ 0.159805] usbcore: registered new interface driver usbfs worker2 # [ 0.166408] vgaarb: loaded worker1 # [ 0.168165] vgaarb: loaded client # [ 0.162872] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.159836] usbcore: registered new interface driver hub worker2 # [ 0.167006] clocksource: Switched to clocksource arch_sys_counter lb # [ 0.159851] usbcore: registered new device driver usb client # [ 0.162878] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.160138] pps_core: LinuxPPS API ver. 1 registered client # [ 0.162887] PTP clock support registered worker1 # [ 0.168749] clocksource: Switched to clocksource arch_sys_counter client # [ 0.162927] EDAC MC: Ver: 3.0.0 worker1 # [ 0.169317] VFS: Disk quotas dquot_6.6.0 lb # [ 0.160159] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti client # [ 0.167511] scmi_core: SCMI protocol bus registered worker1 # [ 0.169347] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.160169] PTP clock support registered lb # [ 0.160215] EDAC MC: Ver: 3.0.0 worker1 # [ 0.171694] netfs: FS-Cache loaded client # [ 0.168439] FPGA manager framework worker1 # [ 0.171812] pnp: PnP ACPI: disabled client # [ 0.169348] vgaarb: loaded lb # [ 0.164675] scmi_core: SCMI protocol bus registered lb # [ 0.165638] FPGA manager framework lb # [ 0.166568] vgaarb: loaded client # [ 0.169947] clocksource: Switched to clocksource arch_sys_counter client # [ 0.170514] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.172961] VFS: Disk quotas dquot_6.6.0 lb # [ 0.167186] clocksource: Switched to clocksource arch_sys_counter client # [ 0.170538] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.173000] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) client # [ 0.172870] netfs: FS-Cache loaded worker2 # [ 0.174595] netfs: FS-Cache loaded client # [ 0.172981] pnp: PnP ACPI: disabled worker2 # [ 0.174708] pnp: PnP ACPI: disabled lb # [ 0.170155] VFS: Disk quotas dquot_6.6.0 lb # [ 0.170190] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.178442] NET: Registered PF_INET protocol family worker1 # [ 0.177697] NET: Registered PF_INET protocol family lb # [ 0.173808] netfs: FS-Cache loaded worker2 # [ 0.178695] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.173924] pnp: PnP ACPI: disabled worker1 # [ 0.177957] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) client # [ 0.178764] NET: Registered PF_INET protocol family client # [ 0.179028] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.177639] NET: Registered PF_INET protocol family lb # [ 0.177898] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.206975] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.207015] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.206924] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.207042] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.206964] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.207114] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker1 # [ 0.207255] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.206989] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.207360] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.207680] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker1 # [ 0.207334] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.207065] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.207400] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.207368] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.207719] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.211039] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.207424] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.207403] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.207743] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.207498] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.211120] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.207502] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.207813] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.211153] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.207642] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.207523] NET: Registered PF_XDP protocol family worker1 # [ 0.207539] PCI: CLS 0 bytes, default 64 client # [ 0.207955] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.211188] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.207719] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.207764] Trying to unpack rootfs image as initramfs... client # [ 0.208033] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.207753] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.208065] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.207788] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.207875] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.208100] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.207901] NET: Registered PF_XDP protocol family client # [ 0.208197] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.207917] PCI: CLS 0 bytes, default 64 client # [ 0.208236] NET: Registered PF_XDP protocol family worker2 # [ 0.208144] Trying to unpack rootfs image as initramfs... client # [ 0.208253] PCI: CLS 0 bytes, default 64 client # [ 0.208491] Trying to unpack rootfs image as initramfs... worker1 # [ 0.222147] kvm [1]: HYP mode not available lb # [ 0.214485] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.214505] NET: Registered PF_XDP protocol family lb # [ 0.214522] PCI: CLS 0 bytes, default 64 lb # [ 0.214757] Trying to unpack rootfs image as initramfs... worker2 # [ 0.224494] kvm [1]: HYP mode not available client # [ 0.223745] kvm [1]: HYP mode not available lb # [ 0.225633] kvm [1]: HYP mode not available worker1 # [ 0.313336] Initialise system trusted keyrings worker1 # [ 0.314075] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.315344] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker1 # [ 0.316106] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.315555] Initialise system trusted keyrings client # [ 0.314469] Initialise system trusted keyrings worker2 # [ 0.316325] workingset: timestamp_bits=42 max_order=19 bucket_order=0 client # [ 0.315211] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker2 # [ 0.317621] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.316477] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker2 # [ 0.318419] 9p: Installing v9fs 9p2000 file system support client # [ 0.317271] 9p: Installing v9fs 9p2000 file system support lb # [ 0.319730] Initialise system trusted keyrings lb # [ 0.320474] workingset: timestamp_bits=42 max_order=19 bucket_order=0 lb # [ 0.321749] squashfs: version 4.0 (2009/01/31) Phillip Lougher lb # [ 0.322512] 9p: Installing v9fs 9p2000 file system support client # [ 0.337932] Key type asymmetric registered worker1 # [ 0.344919] Key type asymmetric registered worker1 # [ 0.344947] Asymmetric key parser 'x509' registered worker1 # [ 0.345009] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker1 # [ 0.347087] io scheduler mq-deadline registered worker1 # [ 0.347097] io scheduler kyber registered worker2 # [ 0.347128] Key type asymmetric registered client # [ 0.345988] Asymmetric key parser 'x509' registered worker2 # [ 0.347152] Asymmetric key parser 'x509' registered client # [ 0.346061] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker2 # [ 0.347216] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) lb # [ 0.343145] Key type asymmetric registered client # [ 0.347697] io scheduler mq-deadline registered worker2 # [ 0.349367] io scheduler mq-deadline registered lb # [ 0.343163] Asymmetric key parser 'x509' registered client # [ 0.347708] io scheduler kyber registered worker2 # [ 0.349378] io scheduler kyber registered worker1 # [ 0.356903] pl061_gpio 9030000.pl061: PL061 GPIO chip registered lb # [ 0.351288] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) lb # [ 0.352327] io scheduler mq-deadline registered lb # [ 0.352337] io scheduler kyber registered worker1 # [ 0.358273] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.359150] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker1 # [ 0.358681] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.358080] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker1 # [ 0.358698] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker1 # [ 0.358710] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.360496] ledtrig-cpu: registered to indicate activity on CPUs worker1 # [ 0.358718] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.359435] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.360861] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker1 # [ 0.358744] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.359820] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.360878] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.359836] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker1 # [ 0.358767] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker2 # [ 0.360890] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker1 # [ 0.358836] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.359850] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker1 # [ 0.358845] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.360899] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.359858] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker1 # [ 0.358851] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker2 # [ 0.360920] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker1 # [ 0.358856] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.359878] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.360944] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker1 # [ 0.358861] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.359901] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker2 # [ 0.361020] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker1 # [ 0.358914] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.359972] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker2 # [ 0.361030] pci_bus 0000:00: root bus resource [bus 00-ff] client # [ 0.359981] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.359346] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.361036] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.359987] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.359558] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.361041] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker1 # [ 0.359575] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.359992] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.361046] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker1 # [ 0.359604] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.359997] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.361099] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.359620] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.360085] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.361543] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.360084] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.360533] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.361728] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.360265] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.360719] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.361747] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.360281] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.360735] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.363358] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker1 # [ 0.360310] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.361778] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.360765] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.363960] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.361795] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.360780] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.364349] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.362270] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.361234] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.362452] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.379281] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.364366] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.361419] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.362469] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.379474] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.361435] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.364378] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.362500] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.379491] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.361464] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.379520] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.364386] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.362959] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.361918] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.379982] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.364406] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.382599] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker1 # [ 0.380163] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.381562] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker2 # [ 0.382620] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.364430] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker1 # [ 0.380179] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.381582] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.382651] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.364502] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker1 # [ 0.380208] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.381611] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.364512] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.380227] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.364517] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.380673] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint lb # [ 0.364523] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.385184] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.364527] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.384157] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.385371] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.384344] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.364604] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.385388] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.384360] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.385419] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.365040] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.384389] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.385438] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.365224] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.384407] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.365241] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.385889] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint lb # [ 0.365270] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.384863] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.386079] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.365286] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.385050] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.386110] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.385079] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.365737] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.386559] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.365921] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.385535] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.386746] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.365936] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.385721] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.386777] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.392901] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.365965] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.385751] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.392941] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.366410] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.366590] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker1 # [ 0.393421] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.366606] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.393614] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.366635] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.393644] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.367091] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.394045] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.367291] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.394227] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.367307] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.394500] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.367336] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.394687] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.367355] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.394716] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.399257] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.367817] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.398202] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker1 # [ 0.395159] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.399449] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.368001] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.398397] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker1 # [ 0.395345] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.368031] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.399700] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.395375] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.398691] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.399887] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.368494] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.398880] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.395831] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.399919] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.368679] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.396017] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.398909] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.368708] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.400366] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.396048] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.399365] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.400553] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.369137] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.399552] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.396505] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.400584] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.369320] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] client # [ 0.399582] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.401033] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.369569] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.400034] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.401220] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.369756] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.400220] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.401251] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.369787] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.400249] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.401711] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.370239] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.400713] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.401978] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.370423] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.400973] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.401996] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.370452] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.400991] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.402032] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.370903] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.401020] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.402486] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.371089] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.401478] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.402668] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.371118] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.401662] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.402685] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.401678] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.402716] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.401707] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.416827] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.416854] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.416884] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.417366] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.417552] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.417567] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.417597] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.418177] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker1 # [ 0.418188] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker2 # [ 0.423363] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker1 # [ 0.418194] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned client # [ 0.422369] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.423383] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.418238] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned client # [ 0.422390] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker2 # [ 0.423389] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.418285] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned client # [ 0.422396] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.419697] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.423436] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.419983] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.418332] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned client # [ 0.422444] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.423485] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.420002] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.418380] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned client # [ 0.422492] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.420032] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.423533] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.418428] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned client # [ 0.422539] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned lb # [ 0.420503] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.423581] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker1 # [ 0.418475] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.420686] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.422587] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.423630] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.420702] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.418522] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.422634] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.423678] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.420732] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.418569] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned client # [ 0.422682] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.421305] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker2 # [ 0.423726] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker1 # [ 0.418616] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned client # [ 0.422729] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned lb # [ 0.421316] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker2 # [ 0.423773] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker1 # [ 0.418721] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned lb # [ 0.421322] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned client # [ 0.422776] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.423820] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker1 # [ 0.418768] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.422822] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.421366] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.423925] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.418789] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned client # [ 0.422929] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned lb # [ 0.421411] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker2 # [ 0.423973] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker1 # [ 0.418810] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned client # [ 0.422983] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned lb # [ 0.421458] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker2 # [ 0.423995] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker1 # [ 0.418832] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned client # [ 0.423004] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.424018] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.418853] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.421505] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned client # [ 0.423026] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker2 # [ 0.424041] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.418875] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned lb # [ 0.421552] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned client # [ 0.423046] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.424063] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.418896] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.421599] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned client # [ 0.423067] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.424086] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.418922] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned lb # [ 0.421646] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.423088] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.424108] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.418944] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned client # [ 0.423109] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.421726] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.424136] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.418966] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned client # [ 0.423134] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.424159] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.421772] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker1 # [ 0.418988] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned client # [ 0.423156] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker2 # [ 0.424183] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker1 # [ 0.419011] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.421844] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned client # [ 0.423178] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.424206] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.419032] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.421890] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.423200] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.424230] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.419053] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.421912] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.424252] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned client # [ 0.423221] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker1 # [ 0.419073] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.421933] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.419095] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned client # [ 0.423242] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.424275] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.421954] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.419115] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.424297] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned client # [ 0.423263] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.421975] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.419137] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.424319] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned client # [ 0.423284] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker1 # [ 0.419163] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.421997] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.424341] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned client # [ 0.423304] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker1 # [ 0.419172] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.422018] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.424363] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned client # [ 0.423325] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.419177] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker2 # [ 0.424390] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] client # [ 0.423346] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.422043] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.419981] pci 0000:00:07.0: enabling device (0000 -> 0002) worker2 # [ 0.424401] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] client # [ 0.423372] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.422065] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned client # [ 0.423382] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker2 # [ 0.424405] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.422086] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.425204] pci 0000:00:07.0: enabling device (0000 -> 0002) client # [ 0.423386] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.422109] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned client # [ 0.424215] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.422131] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.422152] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.422173] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.422194] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.422215] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.422235] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.422257] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.422283] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.422292] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.422297] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.423112] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.458242] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 34312 usecs worker1 # [ 0.473931] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker1 # [ 0.476111] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker2 # [ 0.480932] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) client # [ 0.480080] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.482937] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker2 # [ 0.486480] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker1 # [ 0.487052] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) client # [ 0.483232] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.485190] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.481929] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.492847] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) client # [ 0.492014] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.492922] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.494886] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) lb # [ 0.488213] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.490145] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.500031] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker2 # [ 0.501997] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker1 # [ 0.504900] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) client # [ 0.503218] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker1 # [ 0.506859] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) lb # [ 0.500267] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.508624] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) lb # [ 0.502346] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker2 # [ 0.512066] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker2 # [ 0.513938] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker1 # [ 0.518551] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker1 # [ 0.520543] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) lb # [ 0.512955] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) lb # [ 0.514850] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.514120] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) client # [ 0.515917] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) lb # [ 0.518161] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) client # [ 0.517688] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) client # [ 0.519492] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) client # [ 0.521284] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker2 # [ 0.523654] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker1 # [ 0.530607] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.526379] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker2 # [ 0.528801] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) lb # [ 0.524101] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.526802] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) client # [ 0.535440] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker1 # [ 0.545437] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.538633] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.544452] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker1 # [ 0.547992] msm_serial: driver initialized worker1 # [ 0.548132] SuperH (H)SCI(F) driver initialized worker2 # [ 0.547460] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker1 # [ 0.548186] STM32 USART driver initialized worker2 # [ 0.549935] msm_serial: driver initialized worker2 # [ 0.550103] SuperH (H)SCI(F) driver initialized worker2 # [ 0.550156] STM32 USART driver initialized client # [ 0.551198] msm_serial: driver initialized client # [ 0.551336] SuperH (H)SCI(F) driver initialized client # [ 0.551387] STM32 USART driver initialized lb # [ 0.551084] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.553578] msm_serial: driver initialized lb # [ 0.553715] SuperH (H)SCI(F) driver initialized lb # [ 0.553771] STM32 USART driver initialized worker1 # [ 0.579698] loop: module loaded worker1 # [ 0.579877] virtio_blk virtio2: 1/0/0 default/read/poll queues worker1 # [ 0.580692] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker2 # [ 0.581432] loop: module loaded worker2 # [ 0.581625] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 0.582546] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 0.582531] loop: module loaded client # [ 0.582698] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 0.583434] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker1 # [ 0.585309] megasas: 07.734.00.00-rc1 worker1 # [ 0.586017] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.587553] megasas: 07.734.00.00-rc1 worker2 # [ 0.588219] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.585913] loop: module loaded lb # [ 0.586100] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 0.586878] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker1 # [ 0.596793] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.596818] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.599606] Using buffer write method client # [ 0.594534] megasas: 07.734.00.00-rc1 worker1 # [ 0.599686] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.599658] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.595287] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.599724] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.601346] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.601264] Using buffer write method client # [ 0.597510] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.601367] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.597537] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.601326] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker1 # [ 0.602953] Using buffer write method client # [ 0.599120] Using buffer write method worker1 # [ 0.602976] Concatenating MTD devices: client # [ 0.599192] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker1 # [ 0.602980] (0): "0.flash" worker1 # [ 0.602984] (1): "0.flash" worker1 # [ 0.602994] into device "0.flash" worker2 # [ 0.607539] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.600862] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.607564] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.600884] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.608801] Using buffer write method worker2 # [ 0.608826] Concatenating MTD devices: worker2 # [ 0.608830] (0): "0.flash" worker2 # [ 0.608833] (1): "0.flash" worker2 # [ 0.608844] into device "0.flash" lb # [ 0.599771] megasas: 07.734.00.00-rc1 lb # [ 0.600555] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.602624] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.602648] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.604149] Using buffer write method lb # [ 0.604213] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.605834] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.605856] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.615297] Using buffer write method client # [ 0.619711] Using buffer write method lb # [ 0.615321] Concatenating MTD devices: client # [ 0.619744] Concatenating MTD devices: lb # [ 0.615325] (0): "0.flash" client # [ 0.619748] (0): "0.flash" lb # [ 0.615334] (1): "0.flash" client # [ 0.619752] (1): "0.flash" lb # [ 0.615338] into device "0.flash" client # [ 0.619761] into device "0.flash" worker2 # [ 0.843133] Freeing initrd memory: 26944K worker1 # [ 0.846182] Freeing initrd memory: 26976K worker2 # [ 0.849128] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.848686] Freeing initrd memory: 26972K worker1 # [ 0.852262] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 0.855892] thunder_xcv, ver 1.0 worker1 # [ 0.855936] thunder_bgx, ver 1.0 worker1 # [ 0.855958] nicpf, ver 1.0 worker2 # [ 0.852655] thunder_xcv, ver 1.0 worker2 # [ 0.852697] thunder_bgx, ver 1.0 client # [ 0.854611] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 0.856521] e1000: Intel(R) PRO/1000 Network Driver worker2 # [ 0.852718] nicpf, ver 1.0 worker1 # [ 0.856527] e1000: Copyright (c) 1999-2006 Intel Corporation. worker2 # [ 0.853255] e1000: Intel(R) PRO/1000 Network Driver worker1 # [ 0.856550] e1000e: Intel(R) PRO/1000 Network Driver worker2 # [ 0.853263] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 0.856558] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 0.853287] e1000e: Intel(R) PRO/1000 Network Driver worker1 # [ 0.856585] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 0.853295] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. lb # [ 0.851971] Freeing initrd memory: 26948K worker1 # [ 0.856590] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 0.853323] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 0.853329] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 0.856613] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 0.853370] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.856618] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 0.853376] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 0.853507] sky2: driver version 1.30 worker2 # [ 0.855706] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.858279] thunder_xcv, ver 1.0 lb # [ 0.857871] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.858317] thunder_bgx, ver 1.0 worker2 # [ 0.855730] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.863491] sky2: driver version 1.30 client # [ 0.858340] nicpf, ver 1.0 worker2 # [ 0.856005] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.865367] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.858895] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.858902] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 0.865394] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 client # [ 0.858926] e1000e: Intel(R) PRO/1000 Network Driver worker1 # [ 0.865588] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 client # [ 0.858933] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 0.865340] usbcore: registered new interface driver usb-storage worker1 # [ 0.868669] usbcore: registered new interface driver usb-storage client # [ 0.858962] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 0.865428] usbcore: registered new interface driver usbserial_generic client # [ 0.858968] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 0.865442] usbserial: USB Serial support registered for generic lb # [ 0.861422] thunder_xcv, ver 1.0 client # [ 0.858992] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 0.866100] hv_vmbus: registering driver hyperv_keyboard lb # [ 0.861459] thunder_bgx, ver 1.0 worker1 # [ 0.869524] usbcore: registered new interface driver usbserial_generic client # [ 0.858997] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. lb # [ 0.861481] nicpf, ver 1.0 client # [ 0.859144] sky2: driver version 1.30 lb # [ 0.862056] e1000: Intel(R) PRO/1000 Network Driver worker1 # [ 0.869541] usbserial: USB Serial support registered for generic worker2 # [ 0.869657] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.860699] usbcore: registered new interface driver usb-storage worker2 # [ 0.869934] hub 1-0:1.0: USB hub found worker1 # [ 0.870147] hv_vmbus: registering driver hyperv_keyboard lb # [ 0.862063] e1000: Copyright (c) 1999-2006 Intel Corporation. worker2 # [ 0.869953] hub 1-0:1.0: 6 ports detected lb # [ 0.862087] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.860783] usbcore: registered new interface driver usbserial_generic lb # [ 0.862095] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 0.860797] usbserial: USB Serial support registered for generic lb # [ 0.862124] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 0.873205] rtc-pl031 9010000.pl031: registered as rtc0 client # [ 0.861375] hv_vmbus: registering driver hyperv_keyboard worker2 # [ 0.872179] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 0.862129] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 0.873231] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T22:20:13 UTC (1790115613) lb # [ 0.862156] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 0.872205] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T22:20:13 UTC (1790115613) worker1 # [ 0.873534] i2c_dev: i2c /dev entries driver client # [ 0.872635] ehci-pci 0000:00:07.0: EHCI Host Controller worker2 # [ 0.872513] i2c_dev: i2c /dev entries driver lb # [ 0.862162] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. lb # [ 0.862293] sky2: driver version 1.30 client # [ 0.872665] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 client # [ 0.872916] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 lb # [ 0.870752] usbcore: registered new interface driver usb-storage lb # [ 0.870874] usbcore: registered new interface driver usbserial_generic client # [ 0.875535] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 0.870887] usbserial: USB Serial support registered for generic worker2 # [ 0.877606] sdhci: Secure Digital Host Controller Interface driver client # [ 0.875565] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T22:20:13 UTC (1790115613) worker1 # [ 0.878865] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 0.877616] sdhci: Copyright(c) Pierre Ossman client # [ 0.875890] i2c_dev: i2c /dev entries driver worker1 # [ 0.878876] sdhci: Copyright(c) Pierre Ossman worker2 # [ 0.877866] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.874141] ehci-pci 0000:00:07.0: EHCI Host Controller worker1 # [ 0.879147] Synopsys Designware Multimedia Card Interface Driver worker2 # [ 0.878235] sdhci-pltfm: SDHCI platform and OF driver helper lb # [ 0.874169] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.879535] sdhci-pltfm: SDHCI platform and OF driver helper lb # [ 0.874455] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.883461] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker2 # [ 0.882360] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.883750] hub 1-0:1.0: USB hub found lb # [ 0.877138] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 0.883780] hub 1-0:1.0: 6 ports detected worker2 # [ 0.882612] usbcore: registered new interface driver usbhid client # [ 0.881148] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 0.882618] usbhid: USB HID core driver client # [ 0.881158] sdhci: Copyright(c) Pierre Ossman client # [ 0.881422] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.878596] rtc-pl031 9010000.pl031: registered as rtc0 client # [ 0.881781] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.886099] hid: raw HID events driver (C) Jiri Kosina lb # [ 0.878623] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T22:20:13 UTC (1790115613) client # [ 0.884625] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker1 # [ 0.886350] usbcore: registered new interface driver usbhid lb # [ 0.878921] i2c_dev: i2c /dev entries driver worker1 # [ 0.886356] usbhid: USB HID core driver client # [ 0.885642] hub 1-0:1.0: USB hub found worker2 # [ 0.885439] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available client # [ 0.886146] hub 1-0:1.0: 6 ports detected worker2 # [ 0.886919] drop_monitor: Initializing network drop monitor service worker2 # [ 0.888854] NET: Registered PF_INET6 protocol family worker2 # [ 0.890043] Segment Routing with IPv6 worker1 # [ 0.889271] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.890064] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.890790] drop_monitor: Initializing network drop monitor service worker2 # [ 0.890095] NET: Registered PF_PACKET protocol family client # [ 0.888380] hid: raw HID events driver (C) Jiri Kosina worker1 # [ 0.890926] NET: Registered PF_INET6 protocol family client # [ 0.888641] usbcore: registered new interface driver usbhid client # [ 0.888647] usbhid: USB HID core driver worker1 # [ 0.894027] Segment Routing with IPv6 worker1 # [ 0.894061] In-situ OAM (IOAM) with IPv6 lb # [ 0.884167] sdhci: Secure Digital Host Controller Interface driver worker1 # [ 0.894104] NET: Registered PF_PACKET protocol family lb # [ 0.884177] sdhci: Copyright(c) Pierre Ossman lb # [ 0.884442] Synopsys Designware Multimedia Card Interface Driver worker2 # [ 0.891917] 9pnet: Installing 9P2000 support lb # [ 0.884811] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.895788] 9pnet: Installing 9P2000 support client # [ 0.891535] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.891962] Key type dns_resolver registered worker1 # [ 0.895833] Key type dns_resolver registered lb # [ 0.886288] hid: raw HID events driver (C) Jiri Kosina client # [ 0.893057] drop_monitor: Initializing network drop monitor service lb # [ 0.886524] usbcore: registered new interface driver usbhid client # [ 0.893182] NET: Registered PF_INET6 protocol family lb # [ 0.886530] usbhid: USB HID core driver client # [ 0.896184] Segment Routing with IPv6 lb # [ 0.891675] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.896204] In-situ OAM (IOAM) with IPv6 lb # [ 0.891984] hub 1-0:1.0: USB hub found client # [ 0.896230] NET: Registered PF_PACKET protocol family worker2 # [ 0.898501] registered taskstats version 1 lb # [ 0.892003] hub 1-0:1.0: 6 ports detected client # [ 0.897864] 9pnet: Installing 9P2000 support worker2 # [ 0.898636] Loading compiled-in X.509 certificates client # [ 0.897914] Key type dns_resolver registered lb # [ 0.894376] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available lb # [ 0.896979] drop_monitor: Initializing network drop monitor service worker1 # [ 0.902451] registered taskstats version 1 lb # [ 0.897152] NET: Registered PF_INET6 protocol family worker1 # [ 0.902598] Loading compiled-in X.509 certificates lb # [ 0.899116] Segment Routing with IPv6 lb # [ 0.899135] In-situ OAM (IOAM) with IPv6 lb # [ 0.899174] NET: Registered PF_PACKET protocol family client # [ 0.904468] registered taskstats version 1 worker2 # [ 0.907184] Demotion targets for Node 0: null worker2 # [ 0.907282] Key type .fscrypt registered client # [ 0.904620] Loading compiled-in X.509 certificates worker2 # [ 0.907288] Key type fscrypt-provisioning registered lb # [ 0.900819] 9pnet: Installing 9P2000 support worker2 # [ 0.907372] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.900868] Key type dns_resolver registered worker2 # [ 0.907391] ima: Allocated hash algorithm: sha1 worker2 # [ 0.907417] ima: No architecture policies found worker1 # [ 0.911462] Demotion targets for Node 0: null worker1 # [ 0.911557] Key type .fscrypt registered worker1 # [ 0.911564] Key type fscrypt-provisioning registered worker2 # [ 0.911490] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker1 # [ 0.911654] ima: No TPM chip found, activating TPM-bypass! worker1 # [ 0.911673] ima: Allocated hash algorithm: sha1 worker1 # [ 0.911694] ima: No architecture policies found worker1 # [ 0.915692] input: gpio-keys as /devices/platform/gpio-keys/input/input0 client # [ 0.913242] Demotion targets for Node 0: null client # [ 0.913355] Key type .fscrypt registered client # [ 0.913363] Key type fscrypt-provisioning registered client # [ 0.913487] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.907512] registered taskstats version 1 client # [ 0.913507] ima: Allocated hash algorithm: sha1 lb # [ 0.907656] Loading compiled-in X.509 certificates client # [ 0.913527] ima: No architecture policies found client # [ 0.917623] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 0.916274] Demotion targets for Node 0: null lb # [ 0.916382] Key type .fscrypt registered lb # [ 0.916388] Key type fscrypt-provisioning registered lb # [ 0.916489] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.916508] ima: Allocated hash algorithm: sha1 lb # [ 0.916528] ima: No architecture policies found lb # [ 0.920442] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.929058] clk: Disabling unused clocks worker2 # [ 0.929080] PM: genpd: Disabling unused power domains worker2 # [ 0.933153] Freeing unused kernel memory: 4736K worker2 # [ 0.933338] Run /init as init process worker1 # [ 0.933560] clk: Disabling unused clocks worker1 # [ 0.933582] PM: genpd: Disabling unused power domains worker1 # [ 0.937672] Freeing unused kernel memory: 4736K worker1 # [ 0.937884] Run /init as init process client # [ 0.935500] clk: Disabling unused clocks client # [ 0.935524] PM: genpd: Disabling unused power domains client # [ 0.939628] Freeing unused kernel memory: 4736K client # [ 0.939835] Run /init as init process lb # [ 0.938832] clk: Disabling unused clocks lb # [ 0.938856] PM: genpd: Disabling unused power domains lb # [ 0.942984] Freeing unused kernel memory: 4736K worker2 # [ 0.949153] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.943181] Run /init as init process worker1 # [ 0.954912] systemd[1]: Successfully made /usr/ read-only. client # [ 0.956112] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.957376] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 1.115083] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.128821] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.134022] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 1.139267] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 1.267592] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker1 # [ 1.281269] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 client # [ 1.286426] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.283977] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.296142] systemd[1]: Detected virtualization qemu. worker2 # [ 1.298152] systemd[1]: Detected architecture arm64. worker1 # [ 1.289718] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.300133] systemd[1]: Running in initrd. worker1 # [ 1.302169] systemd[1]: Detected virtualization qemu. lb # [ 1.291637] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker1 # [ 1.304286] systemd[1]: Detected architecture arm64. worker2 # [ 1.302707] systemd[1]: Initializing machine ID from random generator. client # [ 1.292369] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.306282] systemd[1]: Running in initrd. worker2 # [ 1.305697] systemd[1]: Hostname set to . client # [ 1.304824] systemd[1]: Detected virtualization qemu. worker1 # [ 1.308916] systemd[1]: Initializing machine ID from random generator. client # [ 1.306980] systemd[1]: Detected architecture arm64. client # [ 1.308887] systemd[1]: Running in initrd. worker1 # [ 1.311788] systemd[1]: Hostname set to . client # [ 1.311481] systemd[1]: Initializing machine ID from random generator. lb # [ 1.297521] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 1.314367] systemd[1]: Hostname set to . lb # [ 1.297579] systemd[1]: Detected virtualization qemu. lb # [ 1.297650] systemd[1]: Detected architecture arm64. lb # [ 1.297673] systemd[1]: Running in initrd. lb # [ 1.298558] systemd[1]: Initializing machine ID from random generator. lb # [ 1.298825] systemd[1]: Hostname set to . worker2 # [ 1.351309] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker1 # [ 1.369010] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 client # [ 1.370197] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 lb # [ 1.375436] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker2 # [ 1.471095] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.488794] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.489993] usb 1-2: new high-speed USB device number 3 using ehci-pci lb # [ 1.499234] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.607788] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 1.607603] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 1.618792] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.623158] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 1.630152] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 1.636513] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.641557] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker1 # [ 1.646183] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 client # [ 1.648674] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.654439] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 lb # [ 1.658766] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 lb # [ 1.664051] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.717417] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.721726] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.727122] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.728403] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 1.729808] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.730867] systemd[1]: Reached target Path Units. worker1 # [ 1.731686] systemd[1]: Reached target Slice Units. worker1 # [ 1.732509] systemd[1]: Reached target Swaps. worker1 # [ 1.733312] systemd[1]: Reached target Timer Units. worker1 # [ 1.734303] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 1.731877] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.735547] systemd[1]: Listening on Journal Socket (/dev/log). worker2 # [ 1.734793] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.736695] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.736100] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 1.737766] systemd[1]: Listening on udev Control Socket. worker2 # [ 1.737549] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.737885] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.738668] systemd[1]: Reached target Path Units. worker1 # [ 1.737909] systemd[1]: Reached target Socket Units. worker2 # [ 1.739581] systemd[1]: Reached target Slice Units. worker2 # [ 1.740444] systemd[1]: Reached target Swaps. worker1 # [ 1.741995] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 1.741231] systemd[1]: Reached target Timer Units. client # [ 1.740612] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.742291] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.743148] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 1.742065] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.743617] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.743723] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 1.743772] systemd[1]: Listening on Journal Sockets. client # [ 1.745004] systemd[1]: Reached target Path Units. lb # [ 1.740454] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.743912] systemd[1]: Listening on udev Control Socket. client # [ 1.746019] systemd[1]: Reached target Slice Units. worker2 # [ 1.744028] systemd[1]: Listening on udev Kernel Socket. client # [ 1.747013] systemd[1]: Reached target Swaps. worker2 # [ 1.744052] systemd[1]: Reached target Socket Units. client # [ 1.747912] systemd[1]: Reached target Timer Units. worker2 # [ 1.750082] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.749111] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.752896] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 1.751300] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 1.750645] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.750823] systemd[1]: Listening on Journal Sockets. lb # [ 1.749615] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.750980] systemd[1]: Listening on udev Control Socket. lb # [ 1.750955] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.751109] systemd[1]: Listening on udev Kernel Socket. client # [ 1.751134] systemd[1]: Reached target Socket Units. lb # [ 1.752450] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 1.753550] systemd[1]: Reached target Path Units. worker1 # [ 1.761021] systemd[1]: Starting Journal Service... client # [ 1.758101] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.754387] systemd[1]: Reached target Slice Units. lb # [ 1.755274] systemd[1]: Reached target Swaps. lb # [ 1.756044] systemd[1]: Reached target Timer Units. client # [ 1.759457] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 1.762393] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.757213] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.758463] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.759673] systemd[1]: Listening on Journal Sockets. lb # [ 1.759818] systemd[1]: Listening on udev Control Socket. lb # [ 1.759938] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.759963] systemd[1]: Reached target Socket Units. lb # [ 1.765186] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.770135] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.766377] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 1.777166] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.778704] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.778237] systemd[1]: Starting Journal Service... lb # [ 1.775361] systemd[1]: Mounting Kernel Configuration File System... lb # [ 1.783501] systemd[1]: Starting Journal Service... worker2 # [ 1.791938] systemd[1]: Starting Journal Service... worker2 # [ 1.794663] systemd[1]: Starting Load Kernel Modules... worker2 # [ 1.795872] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.802527] systemd[1]: Starting Load Kernel Modules... worker2 # [ 1.805391] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.803564] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.808961] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.815023] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.807459] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.816524] systemd[1]: Mounted Kernel Configuration File System. lb # [ 1.807573] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.817300] systemd[1]: Finished Create List of Static Device Nodes. worker2 # [ 1.819798] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 1.822181] systemd-journald[72]: Collecting audit messages is disabled. worker1 # [ 1.823339] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 1.823927] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.823357] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.828887] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 1.830650] systemd[1]: Starting Coldplug All udev Devices... lb # [ 1.837080] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.840464] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.841946] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.850056] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.850799] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.859624] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.858678] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.863362] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.869786] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.870593] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.874544] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.875841] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.879683] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 1.869636] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.882140] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.886087] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.890000] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 1.895867] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 1.890305] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.897858] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker1 # [ 1.898095] [drm] features: -virgl +edid -resource_blob -host_visible lb # [ 1.892767] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.898105] [drm] features: -context_init worker2 # [ 1.899088] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.899434] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 client # [ 1.899668] [drm] features: -virgl +edid -resource_blob -host_visible worker1 # [ 1.898805] [drm] number of scanouts: 1 client # [ 1.899679] [drm] features: -context_init worker1 # [ 1.898822] [drm] number of cap sets: 0 client # [ 1.900356] [drm] number of scanouts: 1 client # [ 1.900373] [drm] number of cap sets: 0 worker2 # [ 1.912129] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker2 # [ 1.912368] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 1.912378] [drm] features: -context_init worker2 # [ 1.913142] [drm] number of scanouts: 1 worker2 # [ 1.913162] [drm] number of cap sets: 0 lb # [ 1.904979] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.911086] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker1 # [ 1.921352] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.921665] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.921692] systemd[1]: Reached target Local File Systems. client # [ 1.922574] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 1.924935] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.925536] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker1 # [ 1.925549] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker2 # [ 1.931624] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.931948] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 1.927417] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 1.931974] systemd[1]: Reached target Local File Systems. lb # [ 1.927435] [drm] features: -context_init lb # [ 1.928177] [drm] number of scanouts: 1 worker1 # [ 1.933605] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 1.929383] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic lb # [ 1.928196] [drm] number of cap sets: 0 client # [ 1.929402] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker2 # [ 1.939318] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker2 # [ 1.939331] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 lb # [ 1.938049] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic lb # [ 1.938069] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker2 # [ 1.947454] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 1.949230] Console: switching to colour frame buffer device 160x50 lb # [ 1.947791] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 1.948112] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 1.948152] systemd[1]: Reached target Local File Systems. lb # [ 1.951974] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 1.954242] Console: switching to colour frame buffer device 160x50 client # [ 1.960682] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker2 # [ 1.963293] Console: switching to colour frame buffer device 160x50 lb # [ 1.964115] Console: switching to colour frame buffer device 160x50 client # [ 1.974599] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 1.974942] systemd[1]: Reached target Preparation for Local File Systems. client # [ 1.974976] systemd[1]: Reached target Local File Systems. worker1 # [ 1.981338] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device client # [ 1.982348] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.987618] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device lb # [ 1.987791] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 1.998172] systemd[1]: Finished Load Kernel Modules. worker1 # [ 2.001206] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.000340] systemd[1]: Finished Load Kernel Modules. client # [ 1.998563] systemd[1]: Finished Load Kernel Modules. lb # [ 2.000634] systemd[1]: Finished Load Kernel Modules. worker2 # [ 2.007445] systemd[1]: Starting Apply Kernel Variables... client # [ 2.006434] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 2.000913] systemd-modules-load[73]: Inserted module 'dm_mod' worker1 # [ 2.014716] systemd[1]: Started Journal Service. worker1 # [ 2.004801] systemd-modules-load[73]: Module 'virtio_balloon' is built in worker1 # [ 2.005894] systemd-modules-load[73]: Module 'virtio_console' is built in worker1 # [ 2.006957] systemd-modules-load[73]: Inserted module 'virtio_gpu' worker1 # [ 2.007921] systemd-modules-load[73]: Module 'virtio_rng' is built in lb # [ 2.011805] systemd[1]: Starting Apply Kernel Variables... client # [ 2.007096] systemd-modules-load[74]: Inserted module 'dm_mod' client # [ 2.008225] systemd-modules-load[74]: Module 'virtio_balloon' is built in lb # [ 2.002686] systemd-modules-load[74]: Inserted module 'dm_mod' lb # [ 2.003764] systemd-modules-load[74]: Module 'virtio_balloon' is built in client # [ 2.009332] systemd-modules-load[74]: Module 'virtio_console' is built in client # [ 2.010403] systemd-modules-load[74]: Inserted module 'virtio_gpu' client # [ 2.011410] systemd-modules-load[74]: Module 'virtio_rng' is built in client # [ 2.026467] systemd[1]: Started Journal Service. lb # [ 2.012337] systemd-modules-load[74]: Module 'virtio_console' is built in lb # [ 2.013432] systemd-modules-load[74]: Inserted module 'virtio_gpu' lb # [ 2.014388] systemd-modules-load[74]: Module 'virtio_rng' is built in worker1 # [ 2.024596] systemd-udevd[79]: Using default interface naming scheme 'v261'. lb # [ 2.031061] systemd[1]: Started Journal Service. worker1 # [ 2.032166] systemd[1]: Starting Create System Files and Directories... worker2 # [ 2.029004] systemd-modules-load[73]: Inserted module 'dm_mod' worker2 # [ 2.045088] systemd[1]: Started Journal Service. worker2 # [ 2.044197] systemd-modules-load[73]: Module 'virtio_balloon' is built in client # [ 2.044097] systemd[1]: Starting Create System Files and Directories... worker2 # [ 2.045348] systemd-modules-load[73]: Module 'virtio_console' is built in worker2 # [ 2.046426] systemd-modules-load[73]: Inserted module 'virtio_gpu' worker2 # [ 2.047444] systemd-modules-load[73]: Module 'virtio_rng' is built in worker1 # [ 2.056795] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.048909] systemd[1]: Starting Create System Files and Directories... lb # [ 2.051289] systemd-udevd[78]: Using default interface naming scheme 'v261'. client # [ 2.057375] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 2.058597] systemd[1]: Starting Create System Files and Directories... lb # [ 2.053232] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 2.062365] systemd-udevd[79]: Using default interface naming scheme 'v261'. worker1 # [ 2.070387] systemd[1]: Finished Create System Files and Directories. client # [ 2.072226] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.073012] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 2.081166] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 2.077383] systemd-udevd[78]: Using default interface naming scheme 'v261'. worker2 # [ 2.082901] systemd[1]: Finished Create System Files and Directories. lb # [ 2.087029] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.098155] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.092682] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 2.104528] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 2.133237] systemd[1]: Starting Virtual Console Setup... lb # [ 2.139464] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.148113] systemd[1]: Starting Virtual Console Setup... client # [ 2.156879] systemd[1]: Starting Virtual Console Setup... worker1 # [ 2.185895] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.187571] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.196416] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.201769] systemd[1]: Finished Virtual Console Setup. lb # [ 2.196470] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.199772] systemd[1]: Finished Virtual Console Setup. client # [ 2.217818] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.219671] systemd[1]: Finished Virtual Console Setup. worker1 # [ 2.785248] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 2.786198] systemd[1]: Reached target System Initialization. worker1 # [ 2.788094] systemd[1]: Reached target Basic System. worker2 # [ 2.805312] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 2.806288] systemd[1]: Reached target System Initialization. worker2 # [ 2.807129] systemd[1]: Reached target Basic System. client # [ 2.821264] systemd[1]: Finished Coldplug All udev Devices. lb # [ 2.816080] systemd[1]: Finished Coldplug All udev Devices. client # [ 2.822229] systemd[1]: Reached target System Initialization. lb # [ 2.817424] systemd[1]: Reached target System Initialization. lb # [ 2.818291] systemd[1]: Reached target Basic System. client # [ 2.824098] systemd[1]: Reached target Basic System. worker1 # [ 2.928168] (udev-worker)[91]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 2.935938] (udev-worker)[104]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.954563] (udev-worker)[104]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.968374] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.967301] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 2.975341] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line. client # [ 2.973981] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 2.984875] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. client # [ 2.989584] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 2.993034] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line. lb # [ 2.994376] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 3.000264] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.047574] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 3.051176] systemd[1]: Reached target Initrd Root Device. worker1 # [ 3.054141] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 3.051936] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.054737] systemd[1]: Reached target Initrd Root Device. worker2 # [ 3.058399] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 3.064529] systemd[1]: Found device /dev/disk/by-label/nixos. lb # [ 3.061921] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.070852] systemd[1]: Reached target Initrd Root Device. lb # [ 3.068283] systemd[1]: Reached target Initrd Root Device. client # [ 3.078646] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... lb # [ 3.073199] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 3.096967] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 3.103908] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 3.105517] systemd[1]: Mounting /sysroot... worker2 # [ 3.112499] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.119876] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.124142] systemd[1]: Mounting /sysroot... client # [ 3.123323] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 3.125484] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks client # [ 3.132208] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. client # [ 3.136198] systemd[1]: Mounting /sysroot... lb # [ 3.132206] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 3.137392] systemd[1]: Mounting /sysroot... worker1 # [ 3.158281] EXT4-fs (vda): mounted filesystem 5d94f2b0-e56f-404b-be05-4831c77c48d8 r/w with ordered data mode. Quota mode: none. worker1 # [ 3.151885] systemd[1]: Mounted /sysroot. worker1 # [ 3.156088] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.161354] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 3.172964] EXT4-fs (vda): mounted filesystem 1aed990d-7dc4-4d98-a511-11af0c3377ab r/w with ordered data mode. Quota mode: none. worker2 # [ 3.163071] systemd[1]: Mounted /sysroot. worker2 # [ 3.164067] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.166171] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 3.182612] EXT4-fs (vda): mounted filesystem 8a2019a4-cdd5-4625-8535-d98bd9f80ddd r/w with ordered data mode. Quota mode: none. client # [ 3.173773] systemd[1]: Mounted /sysroot. client # [ 3.175921] systemd[1]: Reached target Initrd Root File System. client # [ 3.181096] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.189327] EXT4-fs (vda): mounted filesystem 17035f7d-332b-4eff-bed7-2108b75626d2 r/w with ordered data mode. Quota mode: none. worker1 # [ 3.187810] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.179616] systemd[1]: Mounted /sysroot. lb # [ 3.182703] systemd[1]: Reached target Initrd Root File System. lb # [ 3.183969] systemd[1]: Starting Mountpoints Configured in the Real Root... worker1 # [ 3.193381] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker2 # [ 3.191543] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. worker1 # [ 3.198562] systemd[1]: Reloading... worker2 # [ 3.196750] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker2 # [ 3.201620] systemd[1]: Reloading... client # [ 3.208371] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.210643] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 3.216093] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.219244] systemd[1]: Reloading... lb # [ 3.219228] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.222102] systemd[1]: Reloading... worker1 # [ 3.403779] systemd[1]: Reloading finished in 206 ms. worker2 # [ 3.404117] systemd[1]: Reloading finished in 203 ms. client # [ 3.417151] systemd[1]: Reloading finished in 200 ms. worker2 # [ 3.431674] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... lb # [ 3.428149] systemd[1]: Reloading finished in 210 ms. worker1 # [ 3.438094] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.436242] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... worker1 # [ 3.441913] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... worker2 # [ 3.442702] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.445354] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.449306] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.447474] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.452131] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.454131] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.452717] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... client # [ 3.457228] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... client # [ 3.462833] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 3.465378] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.460865] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... client # [ 3.467987] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. lb # [ 3.465250] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... lb # [ 3.471560] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.474661] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.479432] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.806088] systemd[1]: Mounting /sysroot/nix/.ro-store... worker2 # [ 3.806723] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.823503] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.822929] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.823488] systemd[1]: Mounting /sysroot/nix/.ro-store... worker2 # [ 3.828093] systemd[1]: Mounting /sysroot/run... lb # [ 3.824414] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.842583] systemd[1]: Mounting /sysroot/run... client # [ 3.838509] systemd[1]: Mounting /sysroot/nix/.rw-store... lb # [ 3.843300] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.852773] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.852877] systemd[1]: Mounting /sysroot/run... lb # [ 3.849780] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.864184] systemd[1]: Mounting /sysroot/tmp/shared... lb # [ 3.861448] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.878993] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.884435] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.900687] fuse: init (API version 7.45) worker1 # [ 3.891910] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.893892] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 3.914439] fuse: init (API version 7.45) lb # [ 3.895556] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.917206] virtiofs virtio6: discovered new tag: nix-store worker1 # [ 3.918034] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 3.908057] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.907719] systemd[1]: Mounting /sysroot/tmp/xchg... worker2 # [ 3.929735] virtiofs virtio6: discovered new tag: nix-store worker2 # [ 3.930598] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 3.934637] virtiofs virtio7: discovered new tag: shared worker1 # [ 3.935389] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 3.936717] fuse: init (API version 7.45) worker1 # [ 3.942923] virtiofs virtio8: discovered new tag: xchg client # [ 3.927683] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 3.943727] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker2 # [ 3.944731] virtiofs virtio7: discovered new tag: shared worker2 # [ 3.945491] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 3.951886] virtiofs virtio8: discovered new tag: xchg worker2 # [ 3.952674] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 3.955216] virtiofs virtio6: discovered new tag: nix-store client # [ 3.955992] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 3.950749] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 3.962161] fuse: init (API version 7.45) worker1 # [ 3.957888] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 3.955742] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 3.972323] virtiofs virtio7: discovered new tag: shared client # [ 3.973131] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 3.969524] virtiofs virtio6: discovered new tag: nix-store lb # [ 3.970365] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker1 # [ 3.969752] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 3.969747] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 3.966003] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.983571] virtiofs virtio8: discovered new tag: xchg client # [ 3.984323] virtiofs virtio8: virtio_fs_setup_dax: No cache capability lb # [ 3.984993] virtiofs virtio7: discovered new tag: shared worker1 # [ 3.981717] systemd[1]: Mounted /sysroot/run. lb # [ 3.985750] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 3.973502] systemd[1]: Mounted /sysroot/run. worker2 # [ 3.980942] systemd[1]: Mounted /sysroot/run. worker2 # [ 3.982865] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 3.996476] virtiofs virtio8: discovered new tag: xchg lb # [ 3.997244] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker1 # [ 3.995455] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 3.993190] systemd[1]: Mounted /sysroot/tmp/xchg. worker1 # [ 4.001422] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 4.001430] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 4.003512] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 3.997569] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 4.007243] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 3.999670] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 4.005550] systemd[1]: Mounted /sysroot/run. client # [ 4.007075] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 4.007324] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.010537] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.009264] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.011370] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 4.019506] systemd[1]: Mounted /sysroot/tmp/xchg. client # [ 4.026932] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 4.028367] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.033849] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.035243] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 4.325322] (udev-worker)[100]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.331621] (udev-worker)[100]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory lb # [ 4.336707] (udev-worker)[100]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.357025] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.360150] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.348753] (udev-worker)[100]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.364413] systemd[1]: Stopping Virtual Console Setup... worker1 # [ 4.365272] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.373799] (udev-worker)[93]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.382224] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.384339] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.381704] (udev-worker)[93]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.387925] systemd[1]: Starting Virtual Console Setup... lb # [ 4.379443] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.383993] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.385047] systemd[1]: Stopping Virtual Console Setup... lb # [ 4.385843] systemd[1]: Starting Virtual Console Setup... client # [ 4.393591] (udev-worker)[96]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory client # [ 4.400889] (udev-worker)[96]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.417056] systemd-vconsole-setup[153]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 4.414318] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.416457] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.420397] systemd[1]: Finished Virtual Console Setup. worker2 # [ 4.420163] systemd[1]: Stopping Virtual Console Setup... lb # [ 4.417285] systemd-vconsole-setup[154]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 4.424228] systemd[1]: Starting Virtual Console Setup... lb # [ 4.420537] systemd[1]: Finished Virtual Console Setup. client # [ 4.428253] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.429983] systemd[1]: Stopped Virtual Console Setup. client # [ 4.432167] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.432654] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.433697] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.434453] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.440556] systemd[1]: Starting Virtual Console Setup... client # [ 4.463872] systemd-vconsole-setup[152]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 4.467753] systemd[1]: Finished Virtual Console Setup. worker2 # [ 4.469810] systemd-vconsole-setup[152]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 4.473547] systemd[1]: Finished Virtual Console Setup. worker1 # [ 4.808312] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 4.809601] systemd[1]: Mounting /sysroot/nix/store... client # [ 4.821863] systemd[1]: Mounting /sysroot/nix/store... lb # [ 4.827349] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.876457] systemd[1]: Mounted /sysroot/nix/store. worker2 # [ 4.875608] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.880257] systemd[1]: Reached target Initrd File Systems. worker2 # [ 4.880347] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.884190] systemd[1]: Starting Find NixOS closure... worker2 # [ 4.884168] systemd[1]: Starting Find NixOS closure... client # [ 4.887139] systemd[1]: Mounted /sysroot/nix/store. worker2 # [ 4.889498] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.890232] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.896307] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.895212] systemd[1]: Starting Find NixOS closure... lb # [ 4.894456] systemd[1]: Mounted /sysroot/nix/store. lb # [ 4.897340] systemd[1]: Reached target Initrd File Systems. lb # [ 4.901965] systemd[1]: Starting Find NixOS closure... client # [ 4.908418] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.909787] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 4.942212] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 4.940238] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 4.948373] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. lb # [ 4.942280] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 4.947262] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 4.949371] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 4.953110] systemd[1]: Finished Find NixOS closure. worker1 # [ 4.956952] systemd[1]: Finished Find NixOS closure. lb # [ 4.946796] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. client # [ 4.953289] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 4.959734] systemd[1]: Reached target Initrd Default Target. worker2 # [ 4.955968] systemd[1]: Reached target Initrd Default Target. worker2 # [ 4.957860] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 4.961586] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 4.960204] systemd[1]: Finished Find NixOS closure. client # [ 4.962063] systemd[1]: Reached target Initrd Default Target. lb # [ 4.957521] systemd[1]: Finished Find NixOS closure. client # [ 4.965108] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 4.960268] systemd[1]: Reached target Initrd Default Target. lb # [ 4.962039] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 4.986282] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 4.988669] systemd[1]: Stopped target Basic System. worker1 # [ 4.990262] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 4.986481] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 4.992711] systemd[1]: Stopped target Path Units. worker2 # [ 4.988999] systemd[1]: Stopped target Basic System. worker2 # [ 4.990344] systemd[1]: Stopped target Initrd Root Device. lb # [ 4.985208] systemd[1]: Stopped target Initrd Default Target. worker2 # [ 4.991736] systemd[1]: Stopped target Path Units. lb # [ 4.986664] systemd[1]: Stopped target Basic System. worker2 # [ 4.993056] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 4.996564] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 4.988250] systemd[1]: Stopped target Initrd Root Device. client # [ 4.994473] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker2 # [ 4.996336] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 4.999932] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. lb # [ 4.992005] systemd[1]: Stopped target Path Units. lb # [ 4.993175] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 4.998693] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 5.000191] systemd[1]: Stopped target Slice Units. client # [ 4.999947] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 5.004853] systemd[1]: Stopped target Slice Units. worker2 # [ 5.001550] systemd[1]: Stopped target Socket Units. lb # [ 4.996151] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. lb # [ 4.997432] systemd[1]: Stopped target Slice Units. worker2 # [ 5.004212] systemd[1]: Stopped target System Initialization. lb # [ 4.998841] systemd[1]: Stopped target Socket Units. client # [ 5.004268] systemd[1]: Stopped target Basic System. worker1 # [ 5.008718] systemd[1]: Stopped target Socket Units. worker2 # [ 5.005943] systemd[1]: Stopped target Swaps. client # [ 5.005874] systemd[1]: Stopped target Initrd Root Device. worker2 # [ 5.007887] systemd[1]: Stopped target Timer Units. client # [ 5.007953] systemd[1]: Stopped target Path Units. lb # [ 5.002497] systemd[1]: Stopped target System Initialization. lb # [ 5.003545] systemd[1]: Stopped target Swaps. worker1 # [ 5.013127] systemd[1]: Stopped target System Initialization. lb # [ 5.004369] systemd[1]: Stopped target Timer Units. worker1 # [ 5.015181] systemd[1]: Stopped target Swaps. lb # [ 5.006064] systemd[1]: dbus.socket: Deactivated successfully. client # [ 5.012294] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker2 # [ 5.013599] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 5.018012] systemd[1]: Stopped target Timer Units. client # [ 5.013611] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.014776] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 5.014914] systemd[1]: Stopped target Slice Units. client # [ 5.015756] systemd[1]: Stopped target Socket Units. worker2 # [ 5.017192] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 5.012158] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 5.025615] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 5.022197] systemd[1]: Stopped Find NixOS closure. worker1 # [ 5.026727] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 5.017294] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker2 # [ 5.025105] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 5.024717] systemd[1]: Stopped target System Initialization. worker1 # [ 5.029293] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 5.021432] systemd[1]: Stopped Find NixOS closure. client # [ 5.028305] systemd[1]: Stopped target Swaps. client # [ 5.029054] systemd[1]: Stopped target Timer Units. worker1 # [ 5.033384] systemd[1]: Stopped Find NixOS closure. worker2 # [ 5.029521] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 5.024632] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 5.030467] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 5.035152] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 5.031408] systemd[1]: Closed D-Bus System Message Bus Socket. lb # [ 5.029085] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 5.036423] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker2 # [ 5.037444] systemd[1]: Stopped Apply Kernel Variables. worker1 # [ 5.041306] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 5.033307] systemd[1]: Stopped Apply Kernel Variables. client # [ 5.038969] systemd[1]: Stopped Find NixOS closure. worker1 # [ 5.044417] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 5.040384] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 5.035561] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 5.045895] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 5.037818] systemd[1]: Stopped Load Kernel Modules. client # [ 5.044425] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 5.038966] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 5.045456] systemd[1]: Stopped Load Kernel Modules. lb # [ 5.040308] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 5.041483] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 5.047140] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 5.047376] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 5.042490] systemd[1]: Stopped Create System Files and Directories. worker1 # [ 5.052612] systemd[1]: Stopped Load Kernel Modules. lb # [ 5.043366] systemd[1]: Stopped target Local File Systems. worker1 # [ 5.053479] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 5.055746] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 5.052446] systemd[1]: Stopped Apply Kernel Variables. lb # [ 5.046969] systemd[1]: Stopped target Preparation for Local File Systems. worker2 # [ 5.052833] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 5.053309] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 5.054322] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 5.058474] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 5.055045] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 5.049594] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 5.055057] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 5.060236] systemd[1]: Stopped Create System Files and Directories. lb # [ 5.051396] systemd[1]: Stopped Coldplug All udev Devices. client # [ 5.056827] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 5.057933] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 5.058955] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 5.060284] systemd[1]: Stopped Create System Files and Directories. client # [ 5.059841] systemd[1]: Stopped target Local File Systems. worker1 # [ 5.064224] systemd[1]: Stopped target Local File Systems. worker2 # [ 5.061304] systemd[1]: Stopped target Local File Systems. worker1 # [ 5.065107] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 5.066074] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 5.056288] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 5.063301] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 5.057367] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 5.068307] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 5.058660] systemd[1]: Stopped Virtual Console Setup. lb # [ 5.059392] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 5.069275] systemd[1]: Stopping Rule-based Manager for Device Events and Files... client # [ 5.065313] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 5.060660] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 5.070323] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 5.071371] systemd[1]: Stopped Virtual Console Setup. client # [ 5.067024] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 5.061607] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 5.062575] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 5.072217] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 5.068551] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 5.069572] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 5.073124] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 5.063366] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 5.070329] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 5.064551] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 5.074113] systemd[1]: systemd-udevd.service: Consumed 1.396s CPU time over 3.119s wall clock time, 22.3M memory peak. worker2 # [ 5.071328] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 5.075509] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 5.065527] systemd[1]: systemd-udevd.service: Consumed 1.385s CPU time over 3.109s wall clock time, 22.3M memory peak. worker2 # [ 5.072831] systemd[1]: Stopped Virtual Console Setup. client # [ 5.072358] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 5.066903] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 5.073578] systemd[1]: initrd-cleanup.service: Deactivated successfully. lb # [ 5.067853] systemd[1]: Closed udev Control Socket. client # [ 5.073350] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 5.074498] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 5.074395] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 5.075421] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.076531] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 5.076263] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 5.080412] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 5.077345] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 5.078242] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 5.082408] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 5.079256] systemd[1]: systemd-udevd.service: Consumed 1.439s CPU time over 3.126s wall clock time, 22.3M memory peak. client # [ 5.080166] systemd[1]: systemd-udevd.service: Deactivated successfully. client # [ 5.081133] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 5.076236] systemd[1]: Starting Cleanup udev Database... client # [ 5.082128] systemd[1]: systemd-udevd.service: Consumed 1.435s CPU time over 3.072s wall clock time, 22.4M memory peak. lb # [ 5.077107] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 5.084257] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 5.078464] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 5.088554] systemd[1]: Closed udev Control Socket. worker2 # [ 5.085335] systemd[1]: Closed udev Control Socket. worker1 # [ 5.089407] systemd[1]: Starting Cleanup udev Database... lb # [ 5.079359] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 5.090404] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 5.087225] systemd[1]: Starting Cleanup udev Database... worker1 # [ 5.091590] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 5.088354] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 5.089387] systemd[1]: Closed udev Control Socket. lb # [ 5.084220] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 5.090106] systemd[1]: Starting Cleanup udev Database... lb # [ 5.085250] systemd[1]: kmod-static-nodes.service: Deactivated successfully. client # [ 5.090886] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 5.086215] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 5.092246] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. client # [ 5.091957] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 5.096334] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 5.093372] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 5.097489] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 5.094239] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 5.098504] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 5.095394] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 5.100121] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 5.096492] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 5.097638] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 5.098624] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 5.100237] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 5.104423] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.101210] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 5.105554] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 5.104122] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 5.105079] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 5.106080] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 5.114864] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 5.121959] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 5.123119] systemd[1]: Finished Cleanup udev Database. client # [ 5.124688] systemd[1]: Reached target Switch Root. lb # [ 5.120708] systemd[1]: Finished Cleanup udev Database. worker1 # [ 5.130551] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 5.121560] systemd[1]: Reached target Switch Root. lb # [ 5.122276] systemd[1]: Starting NixOS Activation... client # [ 5.128217] systemd[1]: Starting NixOS Activation... worker1 # [ 5.133913] systemd[1]: Finished Cleanup udev Database. worker2 # [ 5.130437] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 5.134815] systemd[1]: Reached target Switch Root. worker2 # [ 5.131873] systemd[1]: Finished Cleanup udev Database. worker1 # [ 5.136385] systemd[1]: Starting NixOS Activation... worker2 # [ 5.136394] systemd[1]: Reached target Switch Root. worker2 # [ 5.137133] systemd[1]: Starting NixOS Activation... client # [ 5.226967] initrd-nixos-activation-start[175]: booting system configuration /nix/store/rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test worker2 # [ 5.227741] initrd-nixos-activation-start[175]: booting system configuration /nix/store/30asfw61910plj74np6pz2ka43zplm0r-nixos-system-worker2-test lb # [ 5.223412] initrd-nixos-activation-start[178]: booting system configuration /nix/store/kgjzg78npi7z07am2szqxkr41f06jp5w-nixos-system-lb-test worker1 # [ 5.234259] initrd-nixos-activation-start[177]: booting system configuration /nix/store/ry7bqhhhnwa7dlpa75fylilll15fakpb-nixos-system-worker1-test client # [ 5.263970] initrd-nixos-activation-start[175]: running activation script... lb # [ 5.258796] initrd-nixos-activation-start[178]: running activation script... worker1 # [ 5.268866] initrd-nixos-activation-start[177]: running activation script... worker2 # [ 5.265827] initrd-nixos-activation-start[175]: running activation script... lb # [ 5.533681] initrd-nixos-activation-start[201]: setting up /etc... client # [ 5.540504] initrd-nixos-activation-start[198]: setting up /etc... worker1 # [ 5.544864] initrd-nixos-activation-start[200]: setting up /etc... worker2 # [ 5.541888] initrd-nixos-activation-start[198]: setting up /etc... client # [ 5.665892] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 5.661518] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 5.671540] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 5.668400] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 5.668857] systemd[1]: Finished NixOS Activation. lb # [ 5.664209] systemd[1]: Finished NixOS Activation. client # [ 5.669989] systemd[1]: Starting Switch Root... worker1 # [ 5.674621] systemd[1]: Finished NixOS Activation. worker2 # [ 5.671294] systemd[1]: Finished NixOS Activation. worker1 # [ 5.675906] systemd[1]: Starting Switch Root... worker2 # [ 5.672561] systemd[1]: Starting Switch Root... lb # [ 5.668236] systemd[1]: Starting Switch Root... client # [ 5.692371] systemd[1]: Switching root. lb # [ 5.687575] systemd[1]: Switching root. worker2 # [ 5.695671] systemd[1]: Switching root. worker1 # [ 5.700357] systemd[1]: Switching root. client # [ 5.874773] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). lb # [ 5.879891] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker2 # [ 5.888526] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker1 # [ 5.896351] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker1 # [ 6.548641] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 6.555301] systemd[1]: Detected virtualization qemu. worker2 # [ 6.548659] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 6.557330] systemd[1]: Detected architecture arm64. worker2 # [ 6.556049] systemd[1]: Detected virtualization qemu. worker1 # [ 6.560154] systemd[1]: Detected first boot. lb # [ 6.542369] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 6.560575] systemd[1]: Detected architecture arm64. lb # [ 6.555143] systemd[1]: Detected virtualization qemu. worker1 # [ 6.565014] systemd[1]: Initializing machine ID from random generator. worker2 # [ 6.564227] systemd[1]: Detected first boot. client # [ 6.553386] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 6.559270] systemd[1]: Detected architecture arm64. client # [ 6.566698] systemd[1]: Detected virtualization qemu. lb # [ 6.563694] systemd[1]: Detected first boot. worker2 # [ 6.572575] systemd[1]: Initializing machine ID from random generator. client # [ 6.571095] systemd[1]: Detected architecture arm64. client # [ 6.574727] systemd[1]: Detected first boot. lb # [ 6.571748] systemd[1]: Initializing machine ID from random generator. client # [ 6.581522] systemd[1]: Initializing machine ID from random generator. worker1 # [ 6.901796] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.914010] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 6.941265] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.945463] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 7.150370] systemd[1]: Applying preset policy. client # [ 7.184637] systemd[1]: Applying preset policy. worker2 # [ 7.193010] systemd[1]: Applying preset policy. lb # [ 7.221109] systemd[1]: Applying preset policy. client # [ 7.456497] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 7.474502] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 7.505803] systemd[1]: Populated /etc with preset unit settings. lb # [ 7.502079] systemd[1]: Populated /etc with preset unit settings. client # [ 7.723771] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 7.725575] systemd[1]: Stopped initrd-switch-root.service. client # [ 7.730600] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 7.734918] systemd[1]: Created slice Slice /system/getty. client # [ 7.736972] systemd[1]: Created slice User and Session Slice. client # [ 7.739262] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 7.741711] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 7.744032] systemd[1]: Expecting device /dev/hvc0... client # [ 7.745923] systemd[1]: Expecting device /dev/ttyAMA0... client # [ 7.747847] systemd[1]: Reached target Local Encrypted Volumes. client # [ 7.749830] systemd[1]: Stopped target initrd-fs.target. client # [ 7.751812] systemd[1]: Stopped target initrd-root-fs.target. client # [ 7.752960] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.754687] systemd[1]: Reached target Virtual Machines and Containers. client # [ 7.757150] systemd[1]: Reached target Path Units. client # [ 7.758237] systemd[1]: Reached target Remote File Systems. client # [ 7.760579] systemd[1]: Reached target Slice Units. client # [ 7.761560] systemd[1]: Reached target Swaps. client # [ 7.764803] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 7.768470] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 7.768296] systemd[1]: Listening on Process Core Dump Socket. worker2 # [ 7.769923] systemd[1]: Stopped initrd-switch-root.service. client # [ 7.770992] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 7.773595] systemd[1]: Listening on Factory Reset Management. worker2 # [ 7.774494] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 7.775028] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 7.779389] systemd[1]: Created slice Slice /system/getty. client # [ 7.779926] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 7.781746] systemd[1]: Created slice User and Session Slice. worker2 # [ 7.783207] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 7.782431] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.786781] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 7.785240] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 7.787211] systemd[1]: Expecting device /dev/hvc0... client # [ 7.785885] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 7.787670] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 7.788310] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 7.788688] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.791281] systemd[1]: Reached target Local Encrypted Volumes. client # [ 7.790485] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.792424] systemd[1]: Stopped target initrd-fs.target. lb # [ 7.787329] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 7.794858] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 7.792203] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 7.793996] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 7.788765] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 7.796642] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 7.799143] systemd[1]: Created slice Slice /system/getty. worker2 # [ 7.797808] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 7.793605] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 7.799627] systemd[1]: Reached target Path Units. worker1 # [ 7.801580] systemd[1]: Created slice User and Session Slice. client # [ 7.799542] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 7.801945] systemd[1]: Reached target Remote File Systems. lb # [ 7.797050] systemd[1]: Created slice Slice /system/getty. client # [ 7.800997] systemd[1]: Listening on udev Control Socket. worker2 # [ 7.803072] systemd[1]: Reached target Slice Units. worker1 # [ 7.803897] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 7.804591] systemd[1]: Reached target Swaps. lb # [ 7.799155] systemd[1]: Created slice User and Session Slice. client # [ 7.802971] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 7.806511] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 7.800936] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 7.807924] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 7.808908] systemd[1]: Expecting device /dev/hvc0... lb # [ 7.802779] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 7.810899] systemd[1]: Expecting device /dev/ttyAMA0... client # [ 7.808989] systemd[1]: Mounting Huge Pages File System... worker2 # [ 7.811452] systemd[1]: Listening on Process Core Dump Socket. lb # [ 7.805437] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.813012] systemd[1]: Reached target Local Encrypted Volumes. lb # [ 7.807505] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.814031] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 7.815115] systemd[1]: Stopped target initrd-fs.target. lb # [ 7.808624] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 7.816661] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.817241] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 7.810338] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.817891] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 7.819296] systemd[1]: Stopped target initrd-switch-root.target. lb # [ 7.812215] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 7.814759] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.818317] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 7.821511] systemd[1]: Reached target Virtual Machines and Containers. lb # [ 7.816100] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 7.822870] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 7.823722] systemd[1]: Reached target Path Units. lb # [ 7.817882] systemd[1]: Reached target Path Units. worker1 # [ 7.825789] systemd[1]: Reached target Remote File Systems. worker2 # [ 7.825528] systemd[1]: Listening on Journal Audit Socket. lb # [ 7.819493] systemd[1]: Reached target Remote File Systems. worker1 # [ 7.827898] systemd[1]: Reached target Slice Units. lb # [ 7.821179] systemd[1]: Reached target Slice Units. worker2 # [ 7.828840] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 7.822756] systemd[1]: Reached target Swaps. worker1 # [ 7.830065] systemd[1]: Reached target Swaps. client # [ 7.827077] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 7.830554] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 7.826196] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 7.833643] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 7.832881] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.829520] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.837045] systemd[1]: Listening on Process Core Dump Socket. worker2 # [ 7.834622] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 7.832058] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 7.839625] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 7.834771] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.842365] systemd[1]: Listening on Factory Reset Management. worker2 # [ 7.841979] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.836196] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 7.843703] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 7.843536] systemd[1]: Listening on udev Control Socket. worker2 # [ 7.845242] systemd[1]: Listening on udev Varlink Socket. lb # [ 7.841288] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 7.848680] systemd[1]: Starting Journal Log Access Socket... client # [ 7.846170] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 7.849617] systemd[1]: Mounting Huge Pages File System... lb # [ 7.843889] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.851011] systemd[1]: Listening on Journal Audit Socket. lb # [ 7.846893] systemd[1]: Listening on Console Output Muting Service Socket. worker1 # [ 7.855269] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 7.848905] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 7.857013] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 7.854899] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 7.859169] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.851115] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.857531] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 7.861260] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 7.854506] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 7.859903] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 7.867520] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.860919] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.862288] systemd[1]: Listening on udev Control Socket. worker2 # [ 7.867510] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 7.870115] systemd[1]: Listening on udev Control Socket. worker1 # [ 7.871742] systemd[1]: Listening on udev Varlink Socket. lb # [ 7.863939] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 7.875244] systemd[1]: Mounting Huge Pages File System... lb # [ 7.868434] systemd[1]: Mounting Huge Pages File System... lb # [ 7.872002] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 7.880907] systemd[1]: Mounting Kernel Trace File System... client # [ 7.880071] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.885152] systemd[1]: Mounting POSIX Message Queue File System... client # [ 7.886753] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 7.884370] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 7.892954] systemd[1]: Mounting Kernel Debug File System... client # [ 7.890316] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 7.895552] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 7.896015] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 7.898889] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.905125] systemd[1]: Mounting Kernel Trace File System... lb # [ 7.902898] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 7.917906] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.921514] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 7.918455] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.923100] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 7.918858] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 7.919000] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 7.924236] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 7.923566] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 7.930400] systemd[1]: Mounting FUSE Control File System... client # [ 7.932510] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 7.944469] systemd[1]: Mounting FUSE Control File System... worker1 # [ 7.944985] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.945450] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 7.946151] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.945812] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 7.942969] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.946104] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 7.945522] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 7.957110] systemd[1]: Starting Journal Service... lb # [ 7.952711] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 7.956861] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.970076] systemd[1]: Mounting FUSE Control File System... worker1 # [ 7.971331] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 7.980614] systemd[1]: Starting Load Kernel Modules... worker2 # [ 7.985408] systemd[1]: Starting Journal Service... lb # [ 7.988353] systemd[1]: Mounting FUSE Control File System... lb # [ 7.990879] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 8.004707] systemd[1]: Starting Journal Service... worker2 # [ 8.008897] systemd[1]: Starting Load Kernel Modules... client # [ 8.014419] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 8.024322] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 8.027585] systemd[1]: Starting Journal Service... worker1 # [ 8.037821] systemd[1]: Starting Load Kernel Modules... client # [ 8.038332] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 8.049593] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 8.048378] systemd-journald[268]: Collecting audit messages is enabled. worker2 # [ 8.051621] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 8.051760] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 8.058359] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 8.053895] systemd[1]: Starting Load Kernel Modules... worker1 # [ 8.064143] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 8.064216] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 8.065933] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 8.069214] systemd[1]: Listening on Journal Log Access Socket. client # [ 8.051931] systemd[1]: Queued start job for default target Multi-User System. client # [ 8.055397] systemd[1]: systemd-journald.service: Deactivated successfully. worker1 # [ 8.073165] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 8.073980] systemd[1]: Mounted Huge Pages File System. lb # [ 8.067520] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 8.078299] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 8.079888] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 8.081690] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 8.085180] systemd[1]: Mounted Huge Pages File System. worker2 # [ 8.087448] systemd[1]: Mounted Kernel Trace File System. client # [ 8.087344] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 8.089190] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 8.089850] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 8.089733] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 8.090166] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 8.090634] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 8.091051] systemd[1]: Mounted FUSE Control File System. worker2 # [ 8.092409] systemd[1]: Mounted FUSE Control File System. lb # [ 8.089340] systemd-journald[271]: Collecting audit messages is enabled. client # [ 8.097478] systemd[1]: Started Journal Service. client # [ 8.087004] systemd-modules-load[269]: Module 'atkbd' is built in client # [ 8.089270] systemd-modules-load[269]: Module 'loop' is built in lb # [ 8.089742] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 8.110446] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 8.091227] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 8.110047] systemd[1]: Starting Remount Root and Kernel File Systems... lb # [ 8.111689] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 8.120155] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 8.111832] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 8.129964] systemd[1]: Finished Create List of Static Device Nodes. client # [ 8.117866] systemd[1]: Mounted Huge Pages File System. client # [ 8.118750] systemd[1]: Mounted POSIX Message Queue File System. client # [ 8.119607] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 8.137139] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 8.132664] systemd[1]: Starting Coldplug All udev Devices... client # [ 8.130067] systemd[1]: Mounted Kernel Trace File System. client # [ 8.130908] systemd[1]: Finished Create List of Static Device Nodes. client # [ 8.131800] systemd[1]: Mounted Kernel Configuration File System. lb # [ 8.142210] systemd[1]: Started Journal Service. client # [ 8.149441] EXT4-fs (vda): re-mounted 8a2019a4-cdd5-4625-8535-d98bd9f80ddd. client # [ 8.143682] systemd[1]: Mounted FUSE Control File System. client # [ 8.147727] systemd[1]: Finished Load Kernel Modules. lb # [ 8.148836] systemd-modules-load[273]: Module 'atkbd' is built in lb # [ 8.151592] systemd-modules-load[273]: Module 'loop' is built in client # [ 8.160259] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 8.161270] systemd[1]: Listening on Disk Image Download Service Socket. lb # [ 8.158027] systemd[1]: Listening on Journal Log Access Socket. lb # [ 8.164151] systemd[1]: Mounted Huge Pages File System. client # [ 8.172073] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 8.173098] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 8.168867] systemd[1]: Mounted POSIX Message Queue File System. lb # [ 8.169859] systemd[1]: Mounted Kernel Debug File System. lb # [ 8.170811] systemd[1]: Mounted Kernel Trace File System. client # [ 8.180661] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 8.180509] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 8.181541] systemd[1]: Mounted Kernel Configuration File System. lb # [ 8.182454] systemd[1]: Finished Load Kernel Modules. worker1 # [ 8.206055] systemd[1]: Finished Load Kernel Modules. client # [ 8.192137] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 8.208673] systemd[1]: Finished Load Kernel Modules. worker1 # [ 8.212271] systemd[1]: Starting Firewall... worker2 # [ 8.215694] systemd[1]: Starting Firewall... worker1 # [ 8.221841] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 8.221743] systemd[1]: Starting Apply Kernel Variables... lb # [ 8.204268] systemd[1]: Starting Firewall... worker2 # [ 8.225574] systemd-journald[268]: Collecting audit messages is enabled. lb # [ 8.214145] systemd[1]: Starting Apply Kernel Variables... lb # [ 8.222889] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 8.244773] EXT4-fs (vda): re-mounted 1aed990d-7dc4-4d98-a511-11af0c3377ab. client # [ 8.232845] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 8.234090] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 8.250257] EXT4-fs (vda): re-mounted 5d94f2b0-e56f-404b-be05-4831c77c48d8. worker1 # [ 8.255855] systemd-journald[270]: Collecting audit messages is enabled. worker2 # [ 8.260218] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 8.262988] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 8.265398] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 8.263995] EXT4-fs (vda): re-mounted 17035f7d-332b-4eff-bed7-2108b75626d2. worker1 # [ 8.270541] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 8.271371] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 8.275017] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 8.260809] systemd-oomd[271]: No swap; memory pressure usage will be degraded worker2 # [ 8.276921] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 8.282521] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 8.287441] systemd[1]: Starting Load/Save OS Random Seed... worker1 # [ 8.288847] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 8.273327] systemd[1]: Finished Remount Root and Kernel File Systems. lb # [ 8.276710] systemd-oomd[274]: No swap; memory pressure usage will be degraded worker2 # [ 8.298881] systemd[1]: Started Journal Service. worker2 # [ 8.288121] systemd[1]: Queued start job for default target Multi-User System. client # [ 8.290772] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 8.291884] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 8.287543] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 8.319108] systemd[1]: Started Journal Service. lb # [ 8.302749] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 8.303835] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 8.321159] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 8.317984] systemd-modules-load[269]: Module 'atkbd' is built in worker1 # [ 8.322399] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 8.335057] systemd-journald[268]: Received client request to flush runtime journal. worker2 # [ 8.328525] systemd-modules-load[269]: Module 'loop' is built in lb # [ 8.322662] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 8.323543] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 8.334285] systemd-modules-load[271]: Module 'atkbd' is built in worker2 # [ 8.335088] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 8.333537] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. lb # [ 8.334539] systemd[1]: Mounted FUSE Control File System. worker1 # [ 8.350100] systemd-modules-load[271]: Module 'loop' is built in worker1 # [ 8.359380] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 8.380789] systemd[1]: Finished Load/Save OS Random Seed. client # [ 8.381779] systemd[1]: Reached target First Boot Complete. worker2 # [ 8.388913] systemd-oomd[270]: No swap; memory pressure usage will be degraded client # [ 8.390590] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 8.431177] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 8.431760] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 8.452253] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 8.445818] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 8.446814] systemd[1]: Reached target First Boot Complete. worker2 # [ 8.452905] systemd[1]: Finished Apply Kernel Variables. lb # [ 8.464550] systemd-journald[271]: Received client request to flush runtime journal. worker1 # [ 8.461934] systemd[1]: Reached target First Boot Complete. worker1 # [ 8.463258] systemd-oomd[272]: No swap; memory pressure usage will be degraded worker2 # [ 8.467756] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 8.476321] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 8.473240] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 8.474385] systemd[1]: Reached target First Boot Complete. worker1 # [ 8.492488] systemd-journald[270]: Received client request to flush runtime journal. client # [ 8.476797] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 8.499921] systemd-journald[268]: Received client request to flush runtime journal. lb # [ 8.504553] systemd[1]: Finished Apply Kernel Variables. lb # [ 8.512831] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 8.549786] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 8.551032] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 8.549646] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 8.564362] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 8.569790] systemd[1]: Finished Flush Journal to Persistent Storage. lb # [ 8.577166] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 8.586108] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 8.651238] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 8.656982] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 8.660733] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 8.695296] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 8.701648] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 8.708679] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 8.756106] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 8.761413] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 8.766803] systemd[1]: Mounting /run/wrappers... worker1 # [ 8.778215] systemd[1]: Mounting /run/wrappers... worker1 # [ 8.813289] systemd-udevd[308]: Using default interface naming scheme 'v261'. worker2 # [ 8.840696] systemd[1]: Mounted /run/wrappers. worker2 # [ 8.843885] systemd[1]: Reached target Local File Systems. worker1 # [ 8.850219] systemd[1]: Mounted /run/wrappers. worker2 # [ 8.852331] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 8.856738] systemd[1]: Reached target Local File Systems. worker1 # [ 8.863755] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 8.863463] systemd[1]: Starting register-nix-paths.service... lb # [ 8.862438] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 8.872894] systemd[1]: Starting register-nix-paths.service... lb # [ 8.868220] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 8.880598] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 8.884405] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 8.876107] systemd[1]: Mounting /run/wrappers... worker2 # [ 8.884577] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 8.882509] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 8.893236] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 8.893753] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 8.898586] systemd[1]: Starting Create System Files and Directories... worker1 # [ 8.902876] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 8.906696] systemd-udevd[308]: Using default interface naming scheme 'v261'. worker1 # [ 8.913451] systemd[1]: Starting Create System Files and Directories... lb # [ 8.986765] systemd[1]: Mounted /run/wrappers. lb # [ 8.987629] systemd[1]: Reached target Local File Systems. lb # [ 8.997670] systemd[1]: Listening on Boot Loader Control Service Socket. lb # [ 9.006355] systemd[1]: Starting register-nix-paths.service... lb # [ 9.015041] systemd[1]: Starting Create SUID/SGID Wrappers... worker2 # [ 9.025152] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 9.016006] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 9.033353] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 9.030017] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 9.036124] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 9.043661] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 9.065089] systemd[1]: Starting Create System Files and Directories... client # [ 9.084339] systemd[1]: Finished Coldplug All udev Devices. lb # [ 9.085051] systemd-udevd[311]: Using default interface naming scheme 'v261'. client # [ 9.094666] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 9.097687] systemd[1]: Reached target Preparation for Local File Systems. client # [ 9.102588] systemd[1]: Mounting /run/wrappers... client # [ 9.104698] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 9.137074] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 9.159459] systemd[1]: Mounted /run/wrappers. client # [ 9.161127] systemd[1]: Reached target Local File Systems. client # [ 9.164127] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 9.168297] systemd[1]: Starting register-nix-paths.service... client # [ 9.177798] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 9.179248] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 9.187291] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 9.189864] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 9.190548] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 9.200310] systemd[1]: Finished Create System Files and Directories. lb # [ 9.200444] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 9.213611] systemd[1]: Finished Create System Files and Directories. worker2 # [ 9.211575] systemd[1]: Starting Rebuild Journal Catalog... client # [ 9.213200] systemd[1]: Starting Create System Files and Directories... worker1 # [ 9.221494] systemd[1]: Starting Rebuild Journal Catalog... client # [ 9.217420] systemd-udevd[300]: Using default interface naming scheme 'v261'. worker2 # [ 9.224251] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 9.228199] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 9.303917] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 9.343127] systemd[1]: Finished Create System Files and Directories. lb # [ 9.355455] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 9.367857] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker2 # [ 9.371123] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 9.379684] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 9.388464] systemd[1]: Finished Create System Files and Directories. lb # [ 9.388676] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 9.396608] systemd[1]: Starting Rebuild Journal Catalog... client # [ 9.408328] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 9.416421] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 9.439661] systemd[1]: Finished Rebuild Journal Catalog. worker2 # [ 9.442695] systemd[1]: Starting Update is Completed... worker1 # [ 9.448582] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 9.451308] systemd[1]: Starting Update is Completed... client # [ 9.523771] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker2 # [ 9.528661] systemd[1]: Finished Update is Completed. worker1 # [ 9.543147] systemd[1]: Finished Update is Completed. lb # [ 9.541234] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 9.546747] systemd[1]: Finished Rebuild Journal Catalog. client # [ 9.554241] systemd[1]: Starting Update is Completed... lb # [ 9.553557] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 9.561589] systemd[1]: Starting Update is Completed... client # [ 9.626563] systemd[1]: Finished Update is Completed. lb # [ 9.654979] systemd[1]: Finished Update is Completed. client # [ 9.710700] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 9.805788] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 9.859756] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 9.967228] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 9.973450] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 10.031671] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 10.036948] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 10.069412] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. lb # [ 10.102587] systemd[1]: Finished Coldplug All udev Devices. client # [ 10.123123] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 10.127309] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 10.155924] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. lb # [ 10.166809] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 10.218521] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 10.279777] (udev-worker)[351]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 10.285310] (udev-worker)[351]: Network interface NamePolicy= disabled on kernel command line. client # [ 10.286850] (udev-worker)[347]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.370399] systemd[1]: Finished Firewall. lb # [ 10.427841] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 10.449312] systemd[1]: Finished Firewall. lb # [ 10.484758] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 10.487812] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 10.497811] systemd[1]: Finished register-nix-paths.service. lb # [ 10.498752] systemd[1]: Reached target System Initialization. lb # [ 10.499582] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 10.506118] systemd[1]: Started niks3 garbage collection timer. lb # [ 10.507000] systemd[1]: Started Daily Cleanup of Temporary Directories. lb # [ 10.507936] systemd[1]: Reached target Timer Units. worker2 # [ 10.517677] systemd[1]: Finished Coldplug All udev Devices. lb # [ 10.515850] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 10.519382] systemd[1]: Listening on niks3 server socket. lb # [ 10.524389] systemd[1]: Listening on Nix Daemon Socket. lb # [ 10.525218] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 10.526387] systemd[1]: Reached target Socket Units. lb # [ 10.527108] systemd[1]: Reached target Basic System. lb # [ 10.527834] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 10.541786] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.542736] systemd[1]: Starting Post-Boot Actions... lb # [ 10.555374] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.565688] systemd[1]: Finished Coldplug All udev Devices. client # [ 10.564456] systemd[1]: Condition check resulted in Virtio network device being skipped. client # [ 10.569217] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 10.573788] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 10.574730] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 10.575952] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 10.573430] systemd[1]: Starting resolvconf update... worker2 # [ 10.580761] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 10.591585] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 10.599151] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 10.599089] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 10.604679] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 10.600915] systemd[1]: Started rustfs.service. worker1 # [ 10.622222] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 10.638943] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 10.631977] systemd[1]: Starting rustfs-bucket.service... client # [ 10.676764] mousedev: PS/2 mouse device common for all mice lb # [ 10.672098] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. client # [ 10.707415] systemd[1]: Finished register-nix-paths.service. lb # [ 10.727266] nsncd[448]: Sep 22 22:20:23.362 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.754486] systemd[1]: Finished register-nix-paths.service. worker1 # [ 10.758671] systemd[1]: Reached target System Initialization. lb # [ 10.749338] systemd[1]: Started backdoor.service. worker1 # [ 10.760401] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 10.762978] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 10.763216] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 10.768687] systemd[1]: Reached target Timer Units. worker1 # [ 10.772262] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 10.782732] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 10.783561] systemd[1]: Listening on Nix worker-protocol over gRPC. lb # [ 10.773895] systemd[1]: Starting D-Bus System Message Bus... lb # [ 10.781921] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.794059] systemd[1]: Finished register-nix-paths.service. worker2 # [ 10.798001] systemd[1]: Reached target System Initialization. worker2 # [ 10.800734] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 10.803422] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 10.807461] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 10.806387] systemd[1]: Reached target Timer Units. lb # [ 10.801897] systemd[1]: Finished Post-Boot Actions. worker1 # [ 10.814594] systemd[1]: Reached target Socket Units. worker1 # [ 10.815867] systemd[1]: Reached target Basic System. worker2 # [ 10.812279] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 10.809735] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.819717] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 10.810786] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.820219] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 10.830333] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 10.827675] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 10.838862] systemd[1]: Starting Post-Boot Actions... worker2 # [ 10.835378] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 10.844782] systemd[1]: Reached target Socket Units. worker1 # [ 10.849418] systemd[1]: Started Reset console on configuration changes. worker2 # [ 10.852444] systemd[1]: Reached target Basic System. worker1 # [ 10.863723] systemd[1]: Starting resolvconf update... worker2 # [ 10.860262] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 10.856592] systemd[1]: Starting User Login Management... worker1 # [ 10.869115] systemd[1]: Starting D-Bus System Message Bus... lb # [ 10.861498] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 10.873634] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 10.885423] systemd[1]: Starting Post-Boot Actions... worker2 # [ 10.896795] systemd[1]: Started Reset console on configuration changes. worker2 # [ 10.904761] systemd[1]: Starting resolvconf update... worker1 # [ 10.917485] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.915714] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 10.926630] systemd[1]: Started backdoor.service. worker2 # [ 10.938532] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 10.943344] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.948365] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.958353] nsncd[487]: Sep 22 22:20:23.584 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.964568] systemd[1]: Finished Post-Boot Actions. worker1 # [ 10.969644] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.976373] systemd[1]: Starting User Login Management... worker1 # [ 10.988946] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 10.999305] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 11.004086] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 11.005514] nsncd[487]: Sep 22 22:20:23.638 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 11.022791] dbus-broker-launch[491]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 11.024177] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 11.018437] (udev-worker)[383]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 11.035262] dbus-broker-launch[491]: NSS returned no entry for 'systemd-timesync' worker2 # [ 11.033945] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 11.041324] dbus-broker-launch[491]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # connecting to host... worker2 # [ 11.041233] systemd[1]: Starting User Login Management... lb # [ 11.041782] (udev-worker)[361]: Network interface NamePolicy= disabled on kernel command line. worker2 # connecting to host... worker1 # [ 11.058219] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 11.066870] systemd[1]: Finished Import lastlog data into lastlog2 database. lb # [ 11.076734] (udev-worker)[383]: Network interface NamePolicy= disabled on kernel command line. client # [ 11.089454] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 11.095723] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. client # [ 11.092739] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 11.095035] systemd[1]: Reached target System Initialization. worker2 # [ 11.096398] dbus-broker-launch[493]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 11.101999] systemd[1]: Started backdoor.service. client # [ 11.099779] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 11.104193] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 11.106190] systemd[1]: Reached target Timer Units. worker2 # [ 11.107417] dbus-broker-launch[493]: NSS returned no entry for 'systemd-timesync' client # [ 11.108857] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 11.112306] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 11.119607] dbus-broker-launch[491]: Ready worker2 # [ 11.115454] dbus-broker-launch[493]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client # [ 11.117320] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 11.119625] systemd[1]: Reached target Socket Units. client # [ 11.121273] systemd[1]: Reached target Basic System. client # [ 11.122014] systemd[1]: Started backdoor.service. client # [ 11.122714] systemd[1]: Starting Import lastlog data into lastlog2 database... client # [ 11.129302] systemd[1]: Starting nftables firewall... worker2 # [ 11.132243] systemd[1]: Started D-Bus System Message Bus. lb # [ 11.135210] dbus-broker-launch[475]: Looking up NSS user entry for 'systemd-timesync'... client # [ 11.150293] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 11.151745] dbus-broker-launch[475]: NSS returned no entry for 'systemd-timesync' client # [ 11.158345] systemd[1]: Starting Post-Boot Actions... worker1 # [ 11.164818] systemd-logind[499]: New seat seat0. worker1 # [ 11.175537] systemd[1]: Started User Login Management. lb # [ 11.157852] dbus-broker-launch[475]: Invalid user-name in /nix/store/4yiprls1sll47jid3xzq0m9nmi1qz2ww-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker1 # [ 11.179630] systemd[1]: Starting linger-users.service... client # [ 11.186110] systemd[1]: Started Reset console on configuration changes. worker2 # [ 11.188825] dbus-broker-launch[493]: Ready worker1 # [ 11.210151] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 11.217349] systemd[1]: Stopping Host and Network Name Lookups... client # [ 11.215588] systemd[1]: Starting resolvconf update... lb # [ 11.214637] systemd-logind[486]: New seat seat0. lb # [ 11.220573] systemd[1]: Started User Login Management. worker1 # [ 11.230207] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 11.224181] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 11.236759] systemd[1]: Stopping User and Group Name Lookups... lb # [ 11.228510] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 11.229400] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 11.230259] systemd[1]: Stopping User and Group Name Lookups... worker1 # connecting to host... worker2 # [ 11.232449] systemd-logind[511]: New seat seat0. worker2 # [ 11.240992] systemd[1]: Started User Login Management. worker2 # [ 11.241763] systemd[1]: Starting linger-users.service... worker1 # [ 11.245866] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 11.246864] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 11.247659] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 11.241191] systemd[1]: Starting linger-users.service... lb # [ 11.245635] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 11.260552] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 11.252248] systemd[1]: Started D-Bus System Message Bus. client # [ 11.266478] systemd[1]: Starting D-Bus System Message Bus... lb # [ 11.262274] systemd[1]: nscd.service: Deactivated successfully. client # connecting to host... worker2 # [ 11.269844] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 11.269383] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 11.278241] systemd[1]: Stopping Host and Network Name Lookups... worker2 # [ 11.279262] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 11.286814] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 11.291989] systemd[1]: Finished linger-users.service. worker2 # [ 11.294013] systemd[1]: Stopping User and Group Name Lookups... worker2 # [ 11.294887] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 11.295754] systemd[1]: nscd.service: Deactivated successfully. client # [ 11.305633] nsncd[452]: Sep 22 22:20:23.934 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 11.308992] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 11.309986] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 11.316736] dbus-broker-launch[475]: Ready worker2 # [ 11.322586] (udev-worker)[366]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 11.324653] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 11.333424] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 11.334750] nsncd[560]: Sep 22 22:20:23.971 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 11.335612] systemd[1]: linger-users.service: Deactivated successfully. lb # [ 11.323657] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client: Guest shell says: b'Spawning backdoor root shell...\n' worker2 # [ 11.339967] systemd[1]: Finished linger-users.service. worker1 # [ 11.343793] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 11.347223] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 11.345143] (udev-worker)[366]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 11.352280] (udev-worker)[357]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 11.363702] systemd[1]: Finished resolvconf update. worker1 # [ 11.368217] systemd[1]: Reached target Preparation for Network. client # [ 11.355174] systemd[1]: Finished Post-Boot Actions. worker1 # [ 11.374791] systemd[1]: Starting DHCP Client... client # [ 11.381805] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 11.375497] systemd[1]: Starting Extra networking commands.... client # [ 11.388980] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 11.433860] (udev-worker)[354]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 11.383609] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 11.431207] systemd[1]: linger-users.service: Deactivated successfully. client # [ 11.415937] systemd[1]: Starting User Login Management... worker1 # [ 11.439593] (udev-worker)[365]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 11.389267] nsncd[561]: Sep 22 22:20:24.026 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 11.450908] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 11.449838] (udev-worker)[365]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 11.391151] systemd[1]: Reached target Host and Network Name Lookups. client # [ 11.474003] dbus-broker-launch[460]: Looking up NSS user entry for 'systemd-timesync'... client: connected to guest root shell lb # [ 11.436284] systemd[1]: Finished linger-users.service. worker2 # [ 11.391953] systemd[1]: Reached target User and Group Name Lookups. client # [ 11.485023] dbus-broker-launch[460]: NSS returned no entry for 'systemd-timesync' lb # [ 11.468811] systemd[1]: Started Name Service Cache Daemon (nsncd). client: (connecting took 11.91 seconds) lb # [ 11.469777] systemd[1]: Reached target Host and Network Name Lookups. client: (finished: waiting for the VM to finish booting, in 11.91 seconds) lb # [ 11.470575] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 11.410637] systemd[1]: Finished resolvconf update. lb # [ 11.479865] nsncd[557]: Sep 22 22:20:24.108 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 11.411461] systemd[1]: Reached target Preparation for Network. worker2 # [ 11.425150] systemd[1]: Starting DHCP Client... worker2 # [ 11.425876] systemd[1]: Starting Extra networking commands.... client # [ 11.493609] dbus-broker-launch[460]: Invalid user-name in /nix/store/6wvgzhp3xj1v1xybf1ixpbqi5gif5fbv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 11.494123] systemd[1]: Finished resolvconf update. lb # [ 11.509429] systemd[1]: Starting DHCP Client... client # [ 11.554290] systemd[1]: Started D-Bus System Message Bus. lb # [ 11.592906] systemd-logind[486]: Watching system buttons on /dev/input/event0 (gpio-keys) worker1 # [ 11.616430] dhcpcd[594]: dhcpcd-10.3.2 starting client # [ 11.612813] dbus-broker-launch[460]: Ready worker1 # [ 11.631008] dhcpcd[634]: dev: loaded udev client # [ 11.645273] systemd-logind[482]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker2 # [ 11.647585] dhcpcd[595]: dhcpcd-10.3.2 starting client # [ 11.651559] systemd-logind[482]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.653655] systemd-logind[482]: New seat seat0. worker2 # [ 11.658548] dhcpcd[636]: dev: loaded udev worker1 # [ 11.678225] 8021q: 802.1Q VLAN Support v1.8 client # [ 11.682858] systemd[1]: Started User Login Management. client # [ 11.688248] systemd[1]: Starting linger-users.service... worker2 # [ 11.705221] 8021q: 802.1Q VLAN Support v1.8 lb # [ 11.690405] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 11.732411] systemd[1]: Finished Extra networking commands.. worker1 # [ 11.740927] systemd[1]: Reached target Network. worker1 # [ 11.743697] systemd[1]: Starting Permit User Sessions... client # [ 11.741525] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 11.742521] systemd[1]: Stopping Host and Network Name Lookups... client # [ 11.743331] systemd[1]: Stopped target User and Group Name Lookups. worker2 # [ 11.748975] systemd[1]: Finished Extra networking commands.. client # [ 11.759174] systemd[1]: Stopping User and Group Name Lookups... worker2 # [ 11.761007] systemd[1]: Reached target Network. worker2 # [ 11.765781] systemd[1]: Starting Permit User Sessions... client # [ 11.765972] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client # [ 11.766919] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 11.782784] cfg80211: Loading compiled-in X.509 certificates for regulatory database client # [ 11.767712] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 11.795326] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 11.790212] dhcpcd[603]: dhcpcd-10.3.2 starting worker2 # [ 11.801413] systemd[1]: Starting Address configuration of eth1... client # [ 11.803006] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 11.806741] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 11.823064] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 11.823614] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker2 # [ 11.821268] cfg80211: Loading compiled-in X.509 certificates for regulatory database client # [ 11.810877] systemd[1]: Finished linger-users.service. lb # [ 11.806165] dhcpcd[616]: dev: loaded udev worker1 # [ 11.826666] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 11.827003] cfg80211: failed to load regulatory.db worker1 # [ 11.832853] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 11.841463] systemd[1]: Starting Address configuration of eth1... worker2 # [ 11.861822] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 11.862393] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker1 # [ 11.854839] systemd[1]: Finished Permit User Sessions. worker1 # [ 11.855743] systemd[1]: Started Getty on tty1. worker2 # [ 11.865291] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 11.865619] cfg80211: failed to load regulatory.db worker1 # [ 11.864674] systemd[1]: Reached target Login Prompts. worker2 # [ 11.868421] systemd-logind[511]: Watching system buttons on /dev/input/event0 (gpio-keys) lb # [ 11.884186] mousedev: PS/2 mouse device common for all mice worker2 # [ 11.880698] systemd[1]: Finished Permit User Sessions. worker2 # [ 11.888235] systemd[1]: Started Getty on tty1. lb # [ 11.897464] 8021q: 802.1Q VLAN Support v1.8 client # [ 11.890288] systemd[1]: Finished resolvconf update. worker2 # [ 11.891271] systemd[1]: Reached target Login Prompts. client # [ 11.903656] nsncd[548]: Sep 22 22:20:24.540 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 11.911304] systemd[1]: Starting DHCP Client... client # [ 11.917183] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 11.918118] systemd[1]: Reached target Host and Network Name Lookups. client # [ 11.919039] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 11.945645] 8021q: adding VLAN 0 to HW filter on device eth0 worker1 # [ 11.935576] dhcpcd[634]: eth0: waiting for carrier worker1 # [ 11.940365] dhcpcd[634]: eth0: waiting for carrier worker1 # [ 11.941527] dhcpcd[634]: eth0: carrier acquired worker2 # [ 11.958151] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 11.956381] dhcpcd[634]: DUID 00:01:00:01:32:45:bb:a8:52:54:00:12:34:56 worker1 # [ 11.957449] dhcpcd[634]: eth0: IAID 00:12:34:56 worker1 # [ 11.958438] dhcpcd[634]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 11.976965] 8021q: adding VLAN 0 to HW filter on device eth1 worker2 # [ 11.964794] network-addresses-eth1-start[650]: adding address 192.168.1.4/24... done worker2 # [ 11.983362] 8021q: adding VLAN 0 to HW filter on device eth0 worker1 # [ 11.972669] dhcpcd[634]: eth0: soliciting a DHCP lease worker2 # [ 11.972558] dhcpcd[636]: eth0: waiting for carrier worker1 # [ 11.979310] network-addresses-eth1-start[656]: adding address 192.168.1.3/24... done worker2 # [ 11.976192] dhcpcd[636]: eth0: waiting for carrier worker2 # [ 11.979572] dhcpcd[636]: eth0: carrier acquired worker1 # [ 11.983824] dhcpcd[634]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 11.988240] dhcpcd[634]: eth0: probing address 10.0.2.15/24 worker2 # [ 11.986161] network-addresses-eth1-start[650]: adding address 2001:db8:1::4/64... done worker1 # [ 11.993023] network-addresses-eth1-start[656]: adding address 2001:db8:1::3/64... done worker2 # [ 11.996806] dhcpcd[636]: DUID 00:01:00:01:32:45:bb:a8:52:54:00:12:34:56 worker2 # [ 11.997811] dhcpcd[636]: eth0: IAID 00:12:34:56 worker2 # [ 11.998436] dhcpcd[636]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 12.009762] systemd[1]: Finished Address configuration of eth1. worker2 # [ 12.008933] systemd[1]: Finished Address configuration of eth1. worker2 # [ 12.032768] dhcpcd[636]: eth0: soliciting a DHCP lease worker2 # [ 12.040851] dhcpcd[636]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 12.048287] dhcpcd[636]: eth0: probing address 10.0.2.15/24 lb # [ 12.067270] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 12.062527] systemd-logind[499]: Watching system buttons on /dev/input/event0 (gpio-keys) lb # [ 12.067601] systemd-logind[486]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker2 # [ 12.133455] mousedev: PS/2 mouse device common for all mice client # [ 12.121633] dhcpcd[577]: dhcpcd-10.3.2 starting worker1 # [ 12.140971] mousedev: PS/2 mouse device common for all mice client # [ 12.131860] dhcpcd[585]: dev: loaded udev lb # [ 12.142784] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 12.145417] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 12.147767] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 12.140399] systemd-logind[511]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 12.148111] cfg80211: failed to load regulatory.db lb # [ 12.141606] systemd[1]: Finished Firewall. lb # [ 12.150823] systemd[1]: Reached target Preparation for Network. lb # [ 12.160363] systemd[1]: Starting Address configuration of eth1... client # [ 12.186955] 8021q: 802.1Q VLAN Support v1.8 lb # [ 12.174629] systemd[1]: Starting Extra networking commands.... client # [ 12.278884] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 12.290311] systemd-logind[499]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 12.298710] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 12.304940] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 12.305437] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 12.287607] dhcpcd[616]: eth0: waiting for carrier lb # [ 12.290107] dhcpcd[616]: libudev: received NULL device client # [ 12.311083] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 12.311404] cfg80211: failed to load regulatory.db lb # [ 12.294420] dhcpcd[616]: libudev: received NULL device lb # [ 12.299770] dhcpcd[616]: eth0: carrier acquired lb # [ 12.325562] dhcpcd[616]: DUID 00:01:00:01:32:45:bb:a8:52:54:00:12:34:56 lb # [ 12.326582] dhcpcd[616]: eth0: IAID 00:12:34:56 lb # [ 12.327212] dhcpcd[616]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 12.357983] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 12.388302] systemd[1]: Finished nftables firewall. client # [ 12.392185] systemd[1]: Reached target Preparation for Network. lb # [ 12.386736] network-addresses-eth1-start[642]: adding address 192.168.1.2/24... done client # [ 12.399663] systemd[1]: Starting Address configuration of eth1... client # [ 12.405534] systemd[1]: Starting Extra networking commands.... client # [ 12.440844] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 12.431445] dhcpcd[585]: eth0: waiting for carrier lb # [ 12.426609] network-addresses-eth1-start[642]: adding address 2001:db8:1::2/64... done client # [ 12.433526] dhcpcd[585]: libudev: received NULL device client # [ 12.437846] dhcpcd[585]: libudev: received NULL device client # [ 12.438754] dhcpcd[585]: eth0: carrier acquired client # [ 12.454297] dhcpcd[585]: DUID 00:01:00:01:32:45:bb:a9:52:54:00:12:34:56 client # [ 12.455320] dhcpcd[585]: eth0: IAID 00:12:34:56 client # [ 12.455945] dhcpcd[585]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 12.455139] systemd[1]: Finished Address configuration of eth1. client # [ 12.529843] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 12.542832] network-addresses-eth1-start[618]: adding address 192.168.1.1/24... done lb # [ 12.555994] systemd[1]: Finished Extra networking commands.. lb # [ 12.561619] systemd[1]: Reached target Network. client # [ 12.572108] network-addresses-eth1-start[618]: adding address 2001:db8:1::1/64... done lb # [ 12.568681] systemd[1]: Starting PostgreSQL Server... lb # [ 12.580658] systemd[1]: Starting Permit User Sessions... client # [ 12.598020] systemd[1]: Finished Address configuration of eth1. lb # [ 12.660641] systemd[1]: Finished Permit User Sessions. lb # [ 12.675838] systemd[1]: Started Getty on tty1. lb # [ 12.676821] systemd[1]: Reached target Login Prompts. client # [ 12.690933] systemd[1]: Finished Extra networking commands.. client # [ 12.694231] systemd[1]: Reached target Network. client # [ 12.698779] systemd[1]: Starting Permit User Sessions... client # [ 12.752818] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client # [ 12.780723] systemd[1]: Finished Permit User Sessions. client # [ 12.786960] systemd[1]: Started Getty on tty1. client # [ 12.788749] systemd[1]: Reached target Login Prompts. worker2 # [ 12.833734] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client # [ 12.836392] dhcpcd[585]: eth0: soliciting a DHCP lease client # [ 12.844743] dhcpcd[585]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 12.852219] dhcpcd[585]: eth0: probing address 10.0.2.15/24 lb # [ 12.862484] postgresql-pre-start[726]: The files belonging to this database system will be owned by user "postgres". lb # [ 12.863943] postgresql-pre-start[726]: This user must also own the server process. worker1 # [ 12.894516] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 12.880092] postgresql-pre-start[726]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 12.881432] postgresql-pre-start[726]: The default database encoding has accordingly been set to "UTF8". lb # [ 12.882613] postgresql-pre-start[726]: The default text search configuration will be set to "english". lb # [ 12.883769] postgresql-pre-start[726]: Data page checksums are enabled. lb # [ 12.890025] postgresql-pre-start[726]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 12.891387] postgresql-pre-start[726]: creating subdirectories ... ok lb # [ 12.894900] postgresql-pre-start[726]: selecting dynamic shared memory implementation ... posix client: (finished: waiting for unit network-addresses-eth1.service, in 13.39 seconds) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.05 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb # [ 13.013122] postgresql-pre-start[726]: selecting default "max_connections" ... 100 client # [ 13.108928] systemd[1]: Starting Virtual Console Setup... client # [ 13.133495] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 13.138892] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 13.140959] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. client # [ 13.141870] systemd[1]: Starting Virtual Console Setup... lb: (finished: waiting for unit network-addresses-eth1.service, in 0.13 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } worker2 # [ 13.148622] systemd[1]: Starting Virtual Console Setup... client # [ 13.163114] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 13.160735] postgresql-pre-start[726]: selecting default "shared_buffers" ... 128MB worker1 # [ 13.178876] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 13.181330] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 13.182450] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 13.189124] systemd[1]: Starting Virtual Console Setup... worker1 # [ 13.193893] systemd[1]: Starting Virtual Console Setup... lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.06 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker1 # [ 13.216435] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 13.217579] systemd[1]: Stopped Virtual Console Setup. client # [ 13.214761] systemd-logind[482]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 13.227305] systemd[1]: Starting Virtual Console Setup... lb # [ 13.252617] dhcpcd[616]: eth0: soliciting a DHCP lease lb # [ 13.260628] dhcpcd[616]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 13.266780] systemd-logind[511]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) lb # [ 13.268302] dhcpcd[616]: eth0: probing address 10.0.2.15/24 worker1 # [ 13.304620] systemd-logind[499]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 13.302709] systemd-vconsole-setup[696]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 13.306042] systemd[1]: Finished Virtual Console Setup. worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.10 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } worker2 # [ 13.337195] systemd-vconsole-setup[692]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 13.341179] systemd[1]: Finished Virtual Console Setup. worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.03 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.03 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.02 seconds) lb: waiting for unit niks3.service worker1 # [ 13.413596] systemd-vconsole-setup[694]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 13.417574] systemd[1]: Finished Virtual Console Setup. lb # [ 13.634940] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 13.984683] systemd[1]: Starting Virtual Console Setup... lb # [ 14.015580] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 14.018421] systemd[1]: Stopped Virtual Console Setup. lb # [ 14.039117] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 14.045946] systemd[1]: Starting Virtual Console Setup... worker2 # [ 14.057165] dhcpcd[636]: eth0: soliciting an IPv6 router worker2 # [ 14.059193] dhcpcd[636]: eth0: Router Advertisement from fe80::2 worker2 # [ 14.060658] dhcpcd[636]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 14.062001] dhcpcd[636]: eth0: adding route to fec0::/64 worker2 # [ 14.063131] dhcpcd[636]: eth0: adding default route via fe80::2 lb # [ 14.100210] systemd-logind[486]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 14.166035] dhcpcd[634]: eth0: soliciting an IPv6 router worker1 # [ 14.169649] dhcpcd[634]: eth0: Router Advertisement from fe80::2 worker1 # [ 14.172669] dhcpcd[634]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 14.175701] dhcpcd[634]: eth0: adding route to fec0::/64 worker1 # [ 14.178194] dhcpcd[634]: eth0: adding default route via fe80::2 lb # [ 14.279889] dhcpcd[616]: eth0: soliciting an IPv6 router lb # [ 14.282110] dhcpcd[616]: eth0: Router Advertisement from fe80::2 lb # [ 14.282998] dhcpcd[616]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 14.283919] dhcpcd[616]: eth0: adding route to fec0::/64 lb # [ 14.284916] dhcpcd[616]: eth0: adding default route via fe80::2 lb # [ 14.287457] systemd-vconsole-setup[757]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 14.291190] systemd[1]: Finished Virtual Console Setup. client # [ 14.522176] dhcpcd[585]: eth0: soliciting an IPv6 router client # [ 14.525794] dhcpcd[585]: eth0: Router Advertisement from fe80::2 client # [ 14.528691] dhcpcd[585]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 14.531436] dhcpcd[585]: eth0: adding route to fec0::/64 client # [ 14.533839] dhcpcd[585]: eth0: adding default route via fe80::2 lb # [ 14.716638] postgresql-pre-start[726]: selecting default time zone ... UTC lb # [ 14.719698] postgresql-pre-start[726]: creating configuration files ... ok lb # [ 14.958825] postgresql-pre-start[726]: running bootstrap script ... ok lb # [ 15.612970] postgresql-pre-start[726]: performing post-bootstrap initialization ... ok lb # [ 15.813283] postgresql-pre-start[726]: syncing data to disk ... ok lb # [ 15.815401] postgresql-pre-start[726]: initdb: warning: enabling "trust" authentication for local connections lb # [ 15.816993] postgresql-pre-start[726]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 15.819053] postgresql-pre-start[726]: Success. You can now start the database server using: lb # [ 15.820185] postgresql-pre-start[726]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 15.920513] postgres[791]: [791] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 15.923840] postgres[791]: [791] LOG: listening on IPv6 address "::1", port 5432 lb # [ 15.925397] postgres[791]: [791] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 15.927210] postgres[791]: [791] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 15.940618] postgres[800]: [800] LOG: database system was shut down at 2026-09-22 22:20:28 GMT lb # [ 15.948659] postgres[791]: [791] LOG: database system is ready to accept connections lb # [ 15.953996] systemd[1]: Started PostgreSQL Server. lb # [ 15.965533] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 16.158932] postgresql-setup-start[811]: CREATE DATABASE lb # [ 16.206844] postgresql-setup-start[816]: CREATE ROLE lb # [ 16.228624] postgresql-setup-start[818]: ALTER DATABASE lb # [ 16.234946] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 16.237090] systemd[1]: Reached target PostgreSQL. worker1 # [ 16.571482] dhcpcd[634]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 16.574719] dhcpcd[634]: eth0: adding route to 10.0.2.0/24 worker1 # [ 16.580330] dhcpcd[634]: eth0: adding default route via 10.0.2.2 worker1 # [ 16.730422] systemd[1]: Started DHCP Client. worker1 # [ 16.732864] systemd[1]: Reached target Multi-User System. worker1 # [ 16.733843] systemd[1]: Startup finished in 929ms (kernel) + 5.155s (initrd) + 10.646s (userspace) = 16.732s. worker2 # [ 16.769105] dhcpcd[636]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 16.772415] dhcpcd[636]: eth0: adding route to 10.0.2.0/24 worker2 # [ 16.774807] dhcpcd[636]: eth0: adding default route via 10.0.2.2 worker2 # [ 16.925972] systemd[1]: Started DHCP Client. worker2 # [ 16.928706] systemd[1]: Reached target Multi-User System. worker2 # [ 16.929727] systemd[1]: Startup finished in 922ms (kernel) + 5.159s (initrd) + 10.846s (userspace) = 16.928s. client # [ 18.077489] dhcpcd[585]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 18.082882] dhcpcd[585]: eth0: adding route to 10.0.2.0/24 client # [ 18.089924] dhcpcd[585]: eth0: adding default route via 10.0.2.2 client # [ 18.393148] systemd[1]: Started DHCP Client. client # [ 18.400670] systemd[1]: Reached target Multi-User System. client # [ 18.401572] systemd[1]: Startup finished in 930ms (kernel) + 5.170s (initrd) + 12.299s (userspace) = 18.400s. lb # [ 18.773126] dhcpcd[616]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 18.809799] dhcpcd[616]: eth0: adding route to 10.0.2.0/24 lb # [ 18.811089] dhcpcd[616]: eth0: adding default route via 10.0.2.2 lb # [ 18.995505] systemd[1]: Started DHCP Client. lb # [ 18.999177] systemd[1]: Reached target Network is Online. lb # [ 19.004987] systemd[1]: Started Envoy reverse proxy. lb # [ 19.013986] systemd[1]: Started mock-oidc.service. lb # [ 19.385871] mock-oidc-server[896]: Mock OIDC Server running lb # [ 19.386834] mock-oidc-server[896]: OIDC Address: 192.168.1.2:8080 lb # [ 19.387701] mock-oidc-server[896]: Issue Address: 0.0.0.0:8081 lb # [ 19.392321] mock-oidc-server[896]: Issuer: http://192.168.1.2:8080/oidc lb # [ 19.397726] mock-oidc-server[896]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 19.400919] mock-oidc-server[896]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 19.403716] mock-oidc-server[896]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 19.637640] envoy[895]: [2026-09-22 22:20:32.273][895][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120) lb # [ 19.640867] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 19.642751] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 19.645847] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 19.649491] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 19.653272] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 19.655706] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 19.660338] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 19.690344] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 19.696313] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 19.705731] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 19.707613] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 19.716342] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 19.726231] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 19.731497] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 19.737893] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 19.740925] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 19.744128] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 19.749429] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 19.751769] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 19.756626] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 19.759905] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd lb # [ 19.765931] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 19.780500] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 19.782403] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 19.785183] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 19.789735] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 19.792392] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 19.794437] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 19.796583] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 19.798442] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 19.801190] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default lb # [ 19.803580] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 19.805658] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 19.807501] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 19.812658] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 19.815156] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 19.817071] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 19.819709] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 19.821772] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 19.825447] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 19.827309] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 19.829671] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 19.831729] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 19.834826] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 19.843528] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 19.846188] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 19.848568] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 19.850298] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 19.853309] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 19.856435] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 19.858972] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 19.862579] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher lb # [ 19.865273] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 19.871763] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 19.874821] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 19.877317] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 19.880106] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 19.882271] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 19.884863] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 19.890076] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 19.894458] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder lb # [ 19.898462] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 19.902192] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 19.906621] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 19.913312] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 19.921673] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 19.923322] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap worker1 # [ 19.932687] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 19.933479] systemd[1]: Reached target Network is Online. lb # [ 19.928617] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 19.930561] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service worker2 # [ 19.945827] systemd[1]: Starting scheduler-token.service... lb # [ 19.937170] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 19.950585] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router lb # [ 19.984149] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 19.986043] envoy[895]: [2026-09-22 22:20:32.276][895][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 19.987796] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 19.989820] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 19.993433] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 19.995340] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 19.998158] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 20.000003] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 20.001986] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 20.004504] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 20.013287] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 20.020285] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 20.023136] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 20.025416] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 20.027726] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 20.030144] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 20.033376] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 20.035785] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 20.037797] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 20.039488] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 20.041222] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 20.043096] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 20.046153] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 20.051635] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 20.053594] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 20.059747] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 20.061728] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 20.064130] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 20.066088] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 20.068160] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 20.071250] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 20.078746] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 20.081369] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 20.086082] envoy[895]: [2026-09-22 22:20:32.277][895][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 20.090459] envoy[895]: [2026-09-22 22:20:32.314][895][info][main] [source/server/server.cc:531] HTTP header map info: lb # [ 20.092401] envoy[895]: [2026-09-22 22:20:32.314][895][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 20.108474] envoy[895]: [2026-09-22 22:20:32.314][895][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: worker2 # [ 20.114054] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 20.119392] systemd[1]: Finished scheduler-token.service. lb # [ 20.110102] envoy[895]: [2026-09-22 22:20:32.314][895][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 20.119218] envoy[895]: [2026-09-22 22:20:32.314][895][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 20.122445] envoy[895]: [2026-09-22 22:20:32.385][895][info][main] [source/server/server.cc:968] runtime: {} worker2 # [ 20.125471] systemd[1]: scheduler-token.service: Consumed 103ms CPU time over 173ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb # [ 20.123782] envoy[895]: [2026-09-22 22:20:32.392][895][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. lb # [ 20.125624] envoy[895]: [2026-09-22 22:20:32.392][895][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 20.127257] envoy[895]: [2026-09-22 22:20:32.392][895][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 lb # [ 20.128935] envoy[895]: [2026-09-22 22:20:32.392][895][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 20.130574] envoy[895]: [2026-09-22 22:20:32.392][895][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 20.132168] envoy[895]: [2026-09-22 22:20:32.392][895][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) worker2 # [ 20.138417] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 20.133761] envoy[895]: [2026-09-22 22:20:32.547][895][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) lb # [ 20.135277] envoy[895]: [2026-09-22 22:20:32.583][895][info][config] [source/server/configuration_impl.cc:164] loading stats configuration lb # [ 20.139610] envoy[895]: [2026-09-22 22:20:32.619][895][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 20.146216] envoy[895]: [2026-09-22 22:20:32.619][895][info][main] [source/server/server.cc:1077] starting main dispatch loop worker1 # [ 20.219723] nix-grpc-daemon[766]: ts=2026-09-22T22:20:32Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 20.244070] nix-grpc-daemon[766]: ts=2026-09-22T22:20:32Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 20.247683] nix-grpc-daemon[766]: ts=2026-09-22T22:20:32Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 20.287704] nix-grpc-daemon[766]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 20.293633] nix-grpc-daemon[766]: E0000 00:00:1790115632.925373 766 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 20.305054] nix-grpc-daemon[766]: E0000 00:00:1790115632.943498 766 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 20.321376] nix-grpc-daemon[766]: ts=2026-09-22T22:20:32Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 20.323646] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 20.375294] nix-grpc-daemon[766]: E0000 00:00:1790115633.014423 806 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 20.415447] nix-grpc-daemon[766]: E0000 00:00:1790115633.054188 806 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 20.421041] nix-grpc-daemon[769]: ts=2026-09-22T22:20:33Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 20.424120] nix-grpc-daemon[766]: E0000 00:00:1790115633.058868 806 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 20.452949] nix-grpc-daemon[769]: ts=2026-09-22T22:20:33Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 20.455028] nix-grpc-daemon[769]: ts=2026-09-22T22:20:33Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 20.457548] nix-grpc-daemon[769]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 20.461311] nix-grpc-daemon[766]: E0000 00:00:1790115633.099242 806 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 20.478503] nix-grpc-daemon[769]: E0000 00:00:1790115633.090897 769 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 20.481926] nix-grpc-daemon[769]: E0000 00:00:1790115633.103425 769 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 20.485459] nix-grpc-daemon[769]: ts=2026-09-22T22:20:33Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 20.489573] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 20.492086] nix-grpc-daemon[769]: E0000 00:00:1790115633.134672 810 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 20.524359] nix-grpc-daemon[769]: E0000 00:00:1790115633.166507 810 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 20.527099] envoy[895]: [2026-09-22 22:20:33.163][895][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization lb # [ 20.529267] envoy[895]: [2026-09-22 22:20:33.163][895][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized lb # [ 20.531080] envoy[895]: [2026-09-22 22:20:33.163][895][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager worker1 # [ 20.538470] nix-grpc-daemon[766]: ts=2026-09-22T22:20:33Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" lb # [ 20.541661] envoy[895]: [2026-09-22 22:20:33.176][895][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers worker2 # [ 20.549418] nix-grpc-daemon[769]: E0000 00:00:1790115633.167070 810 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 20.552853] nix-grpc-daemon[769]: E0000 00:00:1790115633.190116 810 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 20.562489] nix-grpc-daemon[769]: ts=2026-09-22T22:20:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.564691] nix-grpc-daemon[769]: ts=2026-09-22T22:20:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.739541] nix-grpc-daemon[766]: ts=2026-09-22T22:20:33Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 20.767045] nix-grpc-daemon[769]: ts=2026-09-22T22:20:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.788953] nix-grpc-daemon[769]: ts=2026-09-22T22:20:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.990992] nix-grpc-daemon[769]: ts=2026-09-22T22:20:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.997894] nix-grpc-daemon[769]: ts=2026-09-22T22:20:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.140682] nix-grpc-daemon[766]: ts=2026-09-22T22:20:33Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused" worker2 # [ 21.200496] nix-grpc-daemon[769]: ts=2026-09-22T22:20:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.212730] nix-grpc-daemon[769]: ts=2026-09-22T22:20:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.414285] nix-grpc-daemon[769]: ts=2026-09-22T22:20:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.422264] nix-grpc-daemon[769]: ts=2026-09-22T22:20:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.622421] nix-grpc-daemon[769]: ts=2026-09-22T22:20:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.629031] nix-grpc-daemon[769]: ts=2026-09-22T22:20:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.829702] nix-grpc-daemon[769]: ts=2026-09-22T22:20:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.832810] nix-grpc-daemon[769]: ts=2026-09-22T22:20:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.943678] nix-grpc-daemon[766]: ts=2026-09-22T22:20:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.952815] nix-grpc-daemon[766]: ts=2026-09-22T22:20:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.034367] nix-grpc-daemon[769]: ts=2026-09-22T22:20:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.037798] nix-grpc-daemon[769]: ts=2026-09-22T22:20:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.154719] nix-grpc-daemon[766]: ts=2026-09-22T22:20:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.162072] nix-grpc-daemon[766]: ts=2026-09-22T22:20:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.240593] nix-grpc-daemon[769]: ts=2026-09-22T22:20:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.255645] nix-grpc-daemon[769]: ts=2026-09-22T22:20:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.362491] nix-grpc-daemon[766]: ts=2026-09-22T22:20:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.367492] nix-grpc-daemon[766]: ts=2026-09-22T22:20:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.457653] nix-grpc-daemon[769]: ts=2026-09-22T22:20:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.460582] nix-grpc-daemon[769]: ts=2026-09-22T22:20:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.568526] nix-grpc-daemon[766]: ts=2026-09-22T22:20:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.573335] nix-grpc-daemon[766]: ts=2026-09-22T22:20:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.663920] nix-grpc-daemon[769]: ts=2026-09-22T22:20:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.672812] nix-grpc-daemon[769]: ts=2026-09-22T22:20:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.774291] nix-grpc-daemon[766]: ts=2026-09-22T22:20:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.778284] nix-grpc-daemon[766]: ts=2026-09-22T22:20:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.877569] nix-grpc-daemon[769]: ts=2026-09-22T22:20:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.880171] nix-grpc-daemon[769]: ts=2026-09-22T22:20:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.979903] nix-grpc-daemon[766]: ts=2026-09-22T22:20:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.983620] nix-grpc-daemon[766]: ts=2026-09-22T22:20:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.081347] nix-grpc-daemon[769]: ts=2026-09-22T22:20:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.084476] nix-grpc-daemon[769]: ts=2026-09-22T22:20:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.185618] nix-grpc-daemon[766]: ts=2026-09-22T22:20:35Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.193742] nix-grpc-daemon[766]: ts=2026-09-22T22:20:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.285466] nix-grpc-daemon[769]: ts=2026-09-22T22:20:35Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.288986] nix-grpc-daemon[769]: ts=2026-09-22T22:20:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 23.309445] rustfs-bucket-start[949]: mb s3://farm lb # [ 23.325408] systemd[1]: Finished rustfs-bucket.service. lb # [ 23.336773] systemd[1]: Starting niks3 server... worker1 # [ 23.394128] nix-grpc-daemon[766]: ts=2026-09-22T22:20:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.397892] nix-grpc-daemon[766]: ts=2026-09-22T22:20:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.502296] nix-grpc-daemon[769]: ts=2026-09-22T22:20:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.505778] nix-grpc-daemon[769]: ts=2026-09-22T22:20:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 23.554302] postgres[965]: [965] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 23.559362] postgres[965]: [965] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC worker1 # [ 23.599003] nix-grpc-daemon[766]: ts=2026-09-22T22:20:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.602228] nix-grpc-daemon[766]: ts=2026-09-22T22:20:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 23.649851] niks3-server[959]: 2026/09/22 22:20:36 OK 20241026095416_initial_model.sql (68.91ms) lb # [ 23.659826] niks3-server[959]: 2026/09/22 22:20:36 OK 20251210153512_drop_unused_gin_index.sql (10.19ms) lb # [ 23.672232] niks3-server[959]: 2026/09/22 22:20:36 OK 20251218171726_add_pins.sql (12.24ms) lb # [ 23.680185] niks3-server[959]: 2026/09/22 22:20:36 OK 20260628120000_add_object_size_and_stats.sql (7.88ms) lb # [ 23.689331] niks3-server[959]: 2026/09/22 22:20:36 OK 20260905000000_add_claims.sql (9.03ms) lb # [ 23.694874] niks3-server[959]: 2026/09/22 22:20:36 OK 20260920000000_drop_claims.sql (5.38ms) worker2 # [ 23.703127] nix-grpc-daemon[769]: ts=2026-09-22T22:20:36Z level=info event=scheduler_connected addr=lb:50051 lb # [ 23.698736] niks3-server[959]: 2026/09/22 22:20:36 goose: successfully migrated database to version: 20260920000000 worker2 # [ 23.706240] nix-grpc-daemon[769]: ts=2026-09-22T22:20:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 23.707429] niks3-server[959]: 2026/09/22 22:20:36 OK 1_commit_pending_closure.sql (12.53ms) lb # [ 23.713763] niks3-server[959]: 2026/09/22 22:20:36 OK 2_object_stats_trigger.sql (5.13ms) lb # [ 23.714992] niks3-server[959]: 2026/09/22 22:20:36 goose: up to current file version: 2 lb # [ 23.741009] niks3-server[959]: 2026/09/22 22:20:36 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key worker1 # [ 23.803093] nix-grpc-daemon[766]: ts=2026-09-22T22:20:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.806564] nix-grpc-daemon[766]: ts=2026-09-22T22:20:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 23.824150] niks3-server[959]: 2026/09/22 22:20:36 INFO Read proxy enabled — serving cache objects from S3 lb # [ 23.827481] niks3-server[959]: 2026/09/22 22:20:36 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 23.832945] systemd[1]: Started niks3 server. lb # [ 23.841276] niks3-server[959]: 2026/09/22 22:20:36 INFO systemd watchdog enabled interval=15s lb # [ 23.849481] niks3-server[959]: 2026/09/22 22:20:36 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 23.850871] systemd[1]: Reached target Multi-User System. lb # [ 23.860096] systemd[1]: Startup finished in 932ms (kernel) + 5.152s (initrd) + 17.771s (userspace) = 23.856s. worker2 # [ 23.908795] nix-grpc-daemon[769]: ts=2026-09-22T22:20:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.916567] nix-grpc-daemon[769]: ts=2026-09-22T22:20:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: waiting for unit niks3.service, in 10.55 seconds) lb: waiting for TCP port 5751 on localhost worker1 # [ 24.010232] nix-grpc-daemon[766]: ts=2026-09-22T22:20:36Z level=info event=scheduler_connected addr=lb:50051 lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! worker1 # [ 24.015002] nix-grpc-daemon[766]: ts=2026-09-22T22:20:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.05 seconds) lb: waiting for TCP port 50051 on localhost lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 24.097167] envoy[895]: conn peer=[::1]:45298 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=3 lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds) worker2 # [ 24.119147] nix-grpc-daemon[769]: ts=2026-09-22T22:20:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.125311] nix-grpc-daemon[769]: ts=2026-09-22T22:20:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.210381] nix-grpc-daemon[766]: ts=2026-09-22T22:20:36Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.217726] nix-grpc-daemon[766]: ts=2026-09-22T22:20:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.305137] systemd[1]: Starting scheduler-token.service... worker2 # [ 24.326210] nix-grpc-daemon[769]: ts=2026-09-22T22:20:36Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.330425] nix-grpc-daemon[769]: ts=2026-09-22T22:20:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.399489] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 24.402461] systemd[1]: Finished scheduler-token.service. worker2 # [ 24.418961] systemd[1]: scheduler-token.service: Consumed 49ms CPU time over 99ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker1 # [ 24.424097] nix-grpc-daemon[766]: ts=2026-09-22T22:20:37Z level=info event=scheduler_connected addr=lb:50051 lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 24.430393] nix-grpc-daemon[766]: ts=2026-09-22T22:20:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.05 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2 # [ 24.532815] nix-grpc-daemon[769]: ts=2026-09-22T22:20:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.535894] nix-grpc-daemon[769]: ts=2026-09-22T22:20:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.631342] nix-grpc-daemon[766]: ts=2026-09-22T22:20:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.642429] nix-grpc-daemon[766]: ts=2026-09-22T22:20:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.734505] nix-grpc-daemon[769]: ts=2026-09-22T22:20:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.737428] nix-grpc-daemon[769]: ts=2026-09-22T22:20:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.842468] nix-grpc-daemon[766]: ts=2026-09-22T22:20:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.845095] nix-grpc-daemon[766]: ts=2026-09-22T22:20:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.938529] nix-grpc-daemon[769]: ts=2026-09-22T22:20:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.946215] nix-grpc-daemon[769]: ts=2026-09-22T22:20:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.046516] nix-grpc-daemon[766]: ts=2026-09-22T22:20:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.049866] nix-grpc-daemon[766]: ts=2026-09-22T22:20:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.147595] nix-grpc-daemon[769]: ts=2026-09-22T22:20:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.153674] nix-grpc-daemon[769]: ts=2026-09-22T22:20:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.252095] nix-grpc-daemon[766]: ts=2026-09-22T22:20:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.255178] nix-grpc-daemon[766]: ts=2026-09-22T22:20:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.355676] nix-grpc-daemon[769]: ts=2026-09-22T22:20:37Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.361967] nix-grpc-daemon[769]: ts=2026-09-22T22:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.456739] nix-grpc-daemon[766]: ts=2026-09-22T22:20:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.462767] nix-grpc-daemon[766]: ts=2026-09-22T22:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 25.564555] nix-grpc-daemon[769]: ts=2026-09-22T22:20:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.573116] nix-grpc-daemon[769]: ts=2026-09-22T22:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) worker1 # [ 25.663323] nix-grpc-daemon[766]: ts=2026-09-22T22:20:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.666675] nix-grpc-daemon[766]: ts=2026-09-22T22:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.774332] nix-grpc-daemon[769]: ts=2026-09-22T22:20:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.778341] nix-grpc-daemon[769]: ts=2026-09-22T22:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.867612] nix-grpc-daemon[766]: ts=2026-09-22T22:20:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.872927] nix-grpc-daemon[766]: ts=2026-09-22T22:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.986093] nix-grpc-daemon[769]: ts=2026-09-22T22:20:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.039687] nix-grpc-daemon[769]: ts=2026-09-22T22:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.183159] nix-grpc-daemon[766]: ts=2026-09-22T22:20:38Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.185470] nix-grpc-daemon[766]: ts=2026-09-22T22:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.240849] nix-grpc-daemon[769]: ts=2026-09-22T22:20:38Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.249443] nix-grpc-daemon[769]: ts=2026-09-22T22:20:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.386344] nix-grpc-daemon[766]: ts=2026-09-22T22:20:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.393875] nix-grpc-daemon[766]: ts=2026-09-22T22:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.450556] nix-grpc-daemon[769]: ts=2026-09-22T22:20:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.453879] nix-grpc-daemon[769]: ts=2026-09-22T22:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.594756] nix-grpc-daemon[766]: ts=2026-09-22T22:20:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.599123] nix-grpc-daemon[766]: ts=2026-09-22T22:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 26.656555] nix-grpc-daemon[769]: ts=2026-09-22T22:20:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.660538] nix-grpc-daemon[769]: ts=2026-09-22T22:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1 # [ 26.800228] nix-grpc-daemon[766]: ts=2026-09-22T22:20:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.807527] nix-grpc-daemon[766]: ts=2026-09-22T22:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.858370] nix-grpc-daemon[769]: ts=2026-09-22T22:20:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.871187] nix-grpc-daemon[769]: ts=2026-09-22T22:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.008926] nix-grpc-daemon[766]: ts=2026-09-22T22:20:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.015827] nix-grpc-daemon[766]: ts=2026-09-22T22:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.073069] nix-grpc-daemon[769]: ts=2026-09-22T22:20:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.080809] nix-grpc-daemon[769]: ts=2026-09-22T22:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.218582] nix-grpc-daemon[766]: ts=2026-09-22T22:20:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.221746] nix-grpc-daemon[766]: ts=2026-09-22T22:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.282392] nix-grpc-daemon[769]: ts=2026-09-22T22:20:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.288581] nix-grpc-daemon[769]: ts=2026-09-22T22:20:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.419821] nix-grpc-daemon[766]: ts=2026-09-22T22:20:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.426324] nix-grpc-daemon[766]: ts=2026-09-22T22:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.490187] nix-grpc-daemon[769]: ts=2026-09-22T22:20:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.495959] nix-grpc-daemon[769]: ts=2026-09-22T22:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.626950] nix-grpc-daemon[766]: ts=2026-09-22T22:20:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.632920] nix-grpc-daemon[766]: ts=2026-09-22T22:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 27.699314] nix-grpc-daemon[769]: ts=2026-09-22T22:20:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.703785] nix-grpc-daemon[769]: ts=2026-09-22T22:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 27.834143] nix-grpc-daemon[766]: ts=2026-09-22T22:20:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.839831] nix-grpc-daemon[766]: ts=2026-09-22T22:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.900800] nix-grpc-daemon[769]: ts=2026-09-22T22:20:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.906696] nix-grpc-daemon[769]: ts=2026-09-22T22:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.041011] nix-grpc-daemon[766]: ts=2026-09-22T22:20:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.046681] nix-grpc-daemon[766]: ts=2026-09-22T22:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.108073] nix-grpc-daemon[769]: ts=2026-09-22T22:20:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.113548] nix-grpc-daemon[769]: ts=2026-09-22T22:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.249707] nix-grpc-daemon[766]: ts=2026-09-22T22:20:40Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.254125] nix-grpc-daemon[766]: ts=2026-09-22T22:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.314907] nix-grpc-daemon[769]: ts=2026-09-22T22:20:40Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.320654] nix-grpc-daemon[769]: ts=2026-09-22T22:20:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.450588] nix-grpc-daemon[766]: ts=2026-09-22T22:20:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.455410] nix-grpc-daemon[766]: ts=2026-09-22T22:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.521843] nix-grpc-daemon[769]: ts=2026-09-22T22:20:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.527556] nix-grpc-daemon[769]: ts=2026-09-22T22:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.655676] nix-grpc-daemon[766]: ts=2026-09-22T22:20:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.661312] nix-grpc-daemon[766]: ts=2026-09-22T22:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.730229] nix-grpc-daemon[769]: ts=2026-09-22T22:20:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.737591] nix-grpc-daemon[769]: ts=2026-09-22T22:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1 # [ 28.861666] nix-grpc-daemon[766]: ts=2026-09-22T22:20:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.864995] nix-grpc-daemon[766]: ts=2026-09-22T22:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.938723] nix-grpc-daemon[769]: ts=2026-09-22T22:20:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.944307] nix-grpc-daemon[769]: ts=2026-09-22T22:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.066408] nix-grpc-daemon[766]: ts=2026-09-22T22:20:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.072237] nix-grpc-daemon[766]: ts=2026-09-22T22:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.145776] nix-grpc-daemon[769]: ts=2026-09-22T22:20:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.151664] nix-grpc-daemon[769]: ts=2026-09-22T22:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.275403] nix-grpc-daemon[766]: ts=2026-09-22T22:20:41Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.280113] nix-grpc-daemon[766]: ts=2026-09-22T22:20:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.353046] nix-grpc-daemon[769]: ts=2026-09-22T22:20:41Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.358528] nix-grpc-daemon[769]: ts=2026-09-22T22:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.476361] nix-grpc-daemon[766]: ts=2026-09-22T22:20:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.483388] nix-grpc-daemon[766]: ts=2026-09-22T22:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.560158] nix-grpc-daemon[769]: ts=2026-09-22T22:20:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.566721] nix-grpc-daemon[769]: ts=2026-09-22T22:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.683614] nix-grpc-daemon[766]: ts=2026-09-22T22:20:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.690259] nix-grpc-daemon[766]: ts=2026-09-22T22:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.770174] nix-grpc-daemon[769]: ts=2026-09-22T22:20:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.774447] nix-grpc-daemon[769]: ts=2026-09-22T22:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 29.891458] nix-grpc-daemon[766]: ts=2026-09-22T22:20:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.898144] nix-grpc-daemon[766]: ts=2026-09-22T22:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.971679] nix-grpc-daemon[769]: ts=2026-09-22T22:20:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.977450] nix-grpc-daemon[769]: ts=2026-09-22T22:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.098522] nix-grpc-daemon[766]: ts=2026-09-22T22:20:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.105049] nix-grpc-daemon[766]: ts=2026-09-22T22:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.178694] nix-grpc-daemon[769]: ts=2026-09-22T22:20:42Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.184343] nix-grpc-daemon[769]: ts=2026-09-22T22:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.307451] nix-grpc-daemon[766]: ts=2026-09-22T22:20:42Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.313018] nix-grpc-daemon[766]: ts=2026-09-22T22:20:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.385610] nix-grpc-daemon[769]: ts=2026-09-22T22:20:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.391349] nix-grpc-daemon[769]: ts=2026-09-22T22:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.514076] nix-grpc-daemon[766]: ts=2026-09-22T22:20:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.520104] nix-grpc-daemon[766]: ts=2026-09-22T22:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.592873] nix-grpc-daemon[769]: ts=2026-09-22T22:20:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.598737] nix-grpc-daemon[769]: ts=2026-09-22T22:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.721149] nix-grpc-daemon[766]: ts=2026-09-22T22:20:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.728825] nix-grpc-daemon[766]: ts=2026-09-22T22:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.802129] nix-grpc-daemon[769]: ts=2026-09-22T22:20:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.806592] nix-grpc-daemon[769]: ts=2026-09-22T22:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 30.929800] nix-grpc-daemon[766]: ts=2026-09-22T22:20:43Z level=info event=scheduler_connected addr=lb:50051 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker1 # [ 30.935587] nix-grpc-daemon[766]: ts=2026-09-22T22:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.003442] nix-grpc-daemon[769]: ts=2026-09-22T22:20:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.009142] nix-grpc-daemon[769]: ts=2026-09-22T22:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.136930] nix-grpc-daemon[766]: ts=2026-09-22T22:20:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.142718] nix-grpc-daemon[766]: ts=2026-09-22T22:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.210883] nix-grpc-daemon[769]: ts=2026-09-22T22:20:43Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.217273] nix-grpc-daemon[769]: ts=2026-09-22T22:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.345978] nix-grpc-daemon[766]: ts=2026-09-22T22:20:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.350396] nix-grpc-daemon[766]: ts=2026-09-22T22:20:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.418657] nix-grpc-daemon[769]: ts=2026-09-22T22:20:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.424431] nix-grpc-daemon[769]: ts=2026-09-22T22:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.547135] nix-grpc-daemon[766]: ts=2026-09-22T22:20:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.553152] nix-grpc-daemon[766]: ts=2026-09-22T22:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.625451] nix-grpc-daemon[769]: ts=2026-09-22T22:20:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.631253] nix-grpc-daemon[769]: ts=2026-09-22T22:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.754506] nix-grpc-daemon[766]: ts=2026-09-22T22:20:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.766073] nix-grpc-daemon[766]: ts=2026-09-22T22:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.834879] nix-grpc-daemon[769]: ts=2026-09-22T22:20:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.839307] nix-grpc-daemon[769]: ts=2026-09-22T22:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 31.963736] nix-grpc-daemon[766]: ts=2026-09-22T22:20:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.970634] nix-grpc-daemon[766]: ts=2026-09-22T22:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.036333] nix-grpc-daemon[769]: ts=2026-09-22T22:20:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.041760] nix-grpc-daemon[769]: ts=2026-09-22T22:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.13 seconds) worker1 # [ 32.171431] nix-grpc-daemon[766]: ts=2026-09-22T22:20:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.178897] nix-grpc-daemon[766]: ts=2026-09-22T22:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.243275] nix-grpc-daemon[769]: ts=2026-09-22T22:20:44Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.250646] nix-grpc-daemon[769]: ts=2026-09-22T22:20:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.382019] nix-grpc-daemon[766]: ts=2026-09-22T22:20:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.386398] nix-grpc-daemon[766]: ts=2026-09-22T22:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.452232] nix-grpc-daemon[769]: ts=2026-09-22T22:20:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.457917] nix-grpc-daemon[769]: ts=2026-09-22T22:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.583430] nix-grpc-daemon[766]: ts=2026-09-22T22:20:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.587283] nix-grpc-daemon[766]: ts=2026-09-22T22:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.659160] nix-grpc-daemon[769]: ts=2026-09-22T22:20:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.666691] nix-grpc-daemon[769]: ts=2026-09-22T22:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.791810] nix-grpc-daemon[766]: ts=2026-09-22T22:20:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.799327] nix-grpc-daemon[766]: ts=2026-09-22T22:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.867982] nix-grpc-daemon[769]: ts=2026-09-22T22:20:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.870797] nix-grpc-daemon[769]: ts=2026-09-22T22:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.000198] nix-grpc-daemon[766]: ts=2026-09-22T22:20:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.002254] nix-grpc-daemon[766]: ts=2026-09-22T22:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 33.071931] nix-grpc-daemon[769]: ts=2026-09-22T22:20:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.076558] nix-grpc-daemon[769]: ts=2026-09-22T22:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1 # [ 33.204055] nix-grpc-daemon[766]: ts=2026-09-22T22:20:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.211937] nix-grpc-daemon[766]: ts=2026-09-22T22:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.278487] nix-grpc-daemon[769]: ts=2026-09-22T22:20:45Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.284153] nix-grpc-daemon[769]: ts=2026-09-22T22:20:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.414705] nix-grpc-daemon[766]: ts=2026-09-22T22:20:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.420217] nix-grpc-daemon[766]: ts=2026-09-22T22:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.485808] nix-grpc-daemon[769]: ts=2026-09-22T22:20:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.491535] nix-grpc-daemon[769]: ts=2026-09-22T22:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.620731] nix-grpc-daemon[766]: ts=2026-09-22T22:20:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.623694] nix-grpc-daemon[766]: ts=2026-09-22T22:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.692898] nix-grpc-daemon[769]: ts=2026-09-22T22:20:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.698847] nix-grpc-daemon[769]: ts=2026-09-22T22:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.824956] nix-grpc-daemon[766]: ts=2026-09-22T22:20:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.830665] nix-grpc-daemon[766]: ts=2026-09-22T22:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.862887] nix-grpc-daemon[766]: ts=2026-09-22T22:20:46Z level=info event=scheduler_take_over lb # [ 33.853712] niks3-server[959]: 2026/09/22 22:20:46 INFO lead: acquired remote=192.168.1.3:41082 worker2 # [ 33.902291] nix-grpc-daemon[769]: ts=2026-09-22T22:20:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.906754] nix-grpc-daemon[769]: ts=2026-09-22T22:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.031821] nix-grpc-daemon[766]: ts=2026-09-22T22:20:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.037486] nix-grpc-daemon[766]: ts=2026-09-22T22:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.103540] nix-grpc-daemon[769]: ts=2026-09-22T22:20:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.109089] nix-grpc-daemon[769]: ts=2026-09-22T22:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker1 # [ 34.238360] nix-grpc-daemon[766]: ts=2026-09-22T22:20:46Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.245104] nix-grpc-daemon[766]: ts=2026-09-22T22:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.310645] nix-grpc-daemon[769]: ts=2026-09-22T22:20:46Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.316683] nix-grpc-daemon[769]: ts=2026-09-22T22:20:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.448808] nix-grpc-daemon[766]: ts=2026-09-22T22:20:47Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.453175] nix-grpc-daemon[766]: ts=2026-09-22T22:20:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.518049] nix-grpc-daemon[769]: ts=2026-09-22T22:20:47Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.525390] nix-grpc-daemon[769]: ts=2026-09-22T22:20:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.652157] nix-grpc-daemon[766]: ts=2026-09-22T22:20:47Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.677909] nix-grpc-daemon[766]: ts=2026-09-22T22:20:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:40002 worker1 # [ 34.681170] nix-grpc-daemon[766]: ts=2026-09-22T22:20:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 34.726347] nix-grpc-daemon[769]: ts=2026-09-22T22:20:47Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.742202] nix-grpc-daemon[766]: ts=2026-09-22T22:20:47Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 34.746645] nix-grpc-daemon[766]: ts=2026-09-22T22:20:47Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:40002 worker1 # [ 34.749946] nix-grpc-daemon[766]: ts=2026-09-22T22:20:47Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 11.54 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker1 # [ 36.319013] nix-grpc-daemon[766]: ts=2026-09-22T22:20:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 36.326472] systemd[1]: Started Nix Daemon. worker1 # [ 36.501563] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 36.532720] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 1/5) worker1 # [ 36.603386] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 110 ms (attempt 2/5) worker1 # [ 36.730551] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 155 ms (attempt 3/5) worker1 # [ 36.887629] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 567 ms (attempt 4/5) worker1 # [ 37.459845] nix-grpc-daemon[766]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 37.465748] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 37.482347] nix-grpc-daemon[766]: ts=2026-09-22T22:20:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 37.537550] nix-grpc-daemon[766]: ts=2026-09-22T22:20:50Z level=debug event=path_info_miss path=rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test error="error: path '/nix/store/rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 37.563256] nix-grpc-daemon[769]: ts=2026-09-22T22:20:50Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 37.585962] systemd[1]: Started Nix Daemon. worker2 # [ 37.783036] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 37.811786] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 1/5) worker2 # [ 37.827769] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 144 ms (attempt 2/5) worker2 # [ 37.978174] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 102 ms (attempt 3/5) worker2 # [ 38.083113] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 537 ms (attempt 4/5) worker2 # [ 38.626291] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 38.632805] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 38.648200] nix-grpc-daemon[769]: ts=2026-09-22T22:20:51Z level=debug event=path_info_miss path=rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test error="error: path '/nix/store/rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.11 seconds) client: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds) worker1 # [ 39.111010] nix-grpc-daemon[766]: ts=2026-09-22T22:20:51Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:40012 worker2 # [ 39.136198] nix-grpc-daemon[769]: ts=2026-09-22T22:20:51Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 39.142933] nix-grpc-daemon[769]: ts=2026-09-22T22:20:51Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:53690 worker2 # [ 39.152088] nix-grpc-daemon[769]: ts=2026-09-22T22:20:51Z level=debug event=path_info_miss path=rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test error="error: path '/nix/store/rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 39.168389] nix-grpc-daemon[769]: ts=2026-09-22T22:20:51Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:53690 worker2 # [ 39.178866] nix-grpc-daemon[769]: ts=2026-09-22T22:20:51Z level=debug event=path_info_miss path=rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test error="error: path '/nix/store/rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds) worker2 # [ 39.455902] nix-grpc-daemon[769]: ts=2026-09-22T22:20:52Z level=info event=denied method=QueryMissing cn=- role=none worker1 # [ 39.684393] nix-grpc-daemon[766]: ts=2026-09-22T22:20:52Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.69 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 worker2 # [ 39.902613] nix-grpc-daemon[769]: ts=2026-09-22T22:20:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53690 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 39.924531] nix-grpc-daemon[766]: ts=2026-09-22T22:20:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 lb # [ 39.918118] envoy[895]: conn peer=[2001:db8:1::1]:40258 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=749 tx=447 ms=2389 worker1 # [ 40.004349] nix-grpc-daemon[766]: ts=2026-09-22T22:20:52Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=0 worker1 # [ 40.012478] nix-grpc-daemon[766]: ts=2026-09-22T22:20:52Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 40.058764] nix-grpc-daemon[766]: ts=2026-09-22T22:20:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 40.081998] nix-grpc-daemon[766]: ts=2026-09-22T22:20:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 client # copying 1 paths to 'lb:50051'... worker1 # [ 40.103109] nix-grpc-daemon[766]: ts=2026-09-22T22:20:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'... worker1 # [ 40.134050] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 40.189150] nix-grpc-daemon[766]: ts=2026-09-22T22:20:52Z level=info event=niks3_push_started pid=863 worker1 # [ 40.446697] nix-grpc-daemon[863]: time=2026-09-22T22:20:53.084Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" lb # [ 39.922201] envoy[895]: conn peer=[2001:db8:1::1]:40264 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=91 lb # [ 39.922337] envoy[895]: conn peer=[2001:db8:1::1]:40266 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=21 lb # [ 39.922408] envoy[895]: conn peer=[2001:db8:1::1]:40282 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=20 worker1 # [ 40.608906] nix-daemon[848]: accepted connection from pid 876, user nix-grpc-daemon (trusted) worker1 # [ 40.647097] nix-daemon[848]: reaped child process 879, status = succeeded lb # [ 40.634223] niks3-server[959]: 2026/09/22 22:20:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 40.676142] nix-grpc-daemon[863]: time=2026-09-22T22:20:53.315Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 40.677958] nix-grpc-daemon[863]: time=2026-09-22T22:20:53.315Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)" lb # [ 40.743498] niks3-server[959]: 2026/09/22 22:20:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign lb # [ 40.770945] niks3-server[959]: 2026/09/22 22:20:53 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls lb # [ 40.801251] niks3-server[959]: 2026/09/22 22:20:53 INFO Signed narinfos id=1 count=1 worker1 # [ 40.816405] nix-grpc-daemon[863]: time=2026-09-22T22:20:53.455Z level=INFO msg="Uploading 1 narinfos" lb # [ 40.819916] niks3-server[959]: 2026/09/22 22:20:53 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst lb # [ 40.832201] niks3-server[959]: 2026/09/22 22:20:53 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 40.846444] niks3-server[959]: 2026/09/22 22:20:53 INFO Completed upload id=1 worker1 # [ 40.856944] nix-grpc-daemon[863]: time=2026-09-22T22:20:53.496Z level=INFO msg="Upload complete. (407ms)" lb # [ 40.857975] niks3-server[959]: 2026/09/22 22:20:53 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo worker1 # [ 40.868711] nix-grpc-daemon[766]: ts=2026-09-22T22:20:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 40.881739] nix-grpc-daemon[766]: ts=2026-09-22T22:20:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 40.885460] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 40.889995] nix-daemon[848]: reaped child process 861, status = succeeded client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051... client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker1 # [ 40.901067] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker1 # [ 42.139650] nix-daemon[848]: reaped child process 883, status = succeeded worker1 # [ 42.286763] nix-daemon[848]: accepted connection from pid 900, user nix-grpc-daemon (trusted) worker1 # [ 42.309460] nix-daemon[848]: reaped child process 903, status = succeeded lb # [ 42.307982] niks3-server[959]: 2026/09/22 22:20:54 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 42.332559] nix-grpc-daemon[863]: time=2026-09-22T22:20:54.969Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 42.334128] nix-grpc-daemon[863]: time=2026-09-22T22:20:54.969Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)" lb # [ 42.379558] niks3-server[959]: 2026/09/22 22:20:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign worker1 # [ 42.412248] nix-grpc-daemon[863]: time=2026-09-22T22:20:55.050Z level=INFO msg="Uploading 1 narinfos" lb # [ 42.404002] niks3-server[959]: 2026/09/22 22:20:55 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv lb # [ 42.419598] niks3-server[959]: 2026/09/22 22:20:55 INFO Signed narinfos id=2 count=1 lb # [ 42.432301] niks3-server[959]: 2026/09/22 22:20:55 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 42.434699] niks3-server[959]: 2026/09/22 22:20:55 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls lb # [ 42.445923] niks3-server[959]: 2026/09/22 22:20:55 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 42.454250] niks3-server[959]: 2026/09/22 22:20:55 INFO Completed upload id=2 worker1 # [ 42.473163] nix-grpc-daemon[863]: time=2026-09-22T22:20:55.112Z level=INFO msg="Upload complete. (361ms)" lb # [ 42.463388] niks3-server[959]: 2026/09/22 22:20:55 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo worker1 # [ 42.484635] nix-grpc-daemon[766]: ts=2026-09-22T22:20:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=1 worker1 # [ 42.498003] nix-daemon[848]: reaped child process 881, status = succeeded worker1 # [ 42.532257] nix-grpc-daemon[766]: ts=2026-09-22T22:20:55Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=0 worker1 # [ 42.537290] nix-grpc-daemon[766]: ts=2026-09-22T22:20:55Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 42.590016] nix-grpc-daemon[766]: ts=2026-09-22T22:20:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 42.725050] nix-grpc-daemon[766]: ts=2026-09-22T22:20:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 42.795490] nix-grpc-daemon[863]: time=2026-09-22T22:20:55.434Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 42.805382] nix-grpc-daemon[766]: ts=2026-09-22T22:20:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'... worker1 # [ 42.845609] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 42.999093] nix-daemon[848]: accepted connection from pid 909, user nix-grpc-daemon (trusted) worker1 # [ 43.041380] nix-daemon[848]: reaped child process 912, status = succeeded lb # [ 43.038621] niks3-server[959]: 2026/09/22 22:20:55 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 43.096888] nix-grpc-daemon[863]: time=2026-09-22T22:20:55.736Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 43.099251] nix-grpc-daemon[863]: time=2026-09-22T22:20:55.738Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)" lb # [ 43.136799] niks3-server[959]: 2026/09/22 22:20:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign worker1 # [ 43.154218] nix-grpc-daemon[863]: time=2026-09-22T22:20:55.793Z level=INFO msg="Uploading 1 narinfos" lb # [ 43.148453] niks3-server[959]: 2026/09/22 22:20:55 INFO Signed narinfos id=3 count=1 lb # [ 43.226190] niks3-server[959]: 2026/09/22 22:20:55 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 43.241983] niks3-server[959]: 2026/09/22 22:20:55 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst lb # [ 43.255475] niks3-server[959]: 2026/09/22 22:20:55 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls worker1 # [ 43.319490] nix-grpc-daemon[863]: time=2026-09-22T22:20:55.958Z level=INFO msg="Upload complete. (463ms)" lb # [ 43.309914] niks3-server[959]: 2026/09/22 22:20:55 INFO Completed upload id=3 lb # [ 43.318351] niks3-server[959]: 2026/09/22 22:20:55 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo worker1 # [ 43.332626] nix-grpc-daemon[766]: ts=2026-09-22T22:20:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 43.361148] nix-daemon[848]: reaped child process 907, status = succeeded worker1 # [ 43.376706] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 43.379868] nix-grpc-daemon[766]: ts=2026-09-22T22:20:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 43.463671] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051... client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker1 # [ 44.549155] nix-daemon[848]: reaped child process 916, status = succeeded worker1 # [ 44.668467] nix-daemon[848]: accepted connection from pid 932, user nix-grpc-daemon (trusted) worker1 # [ 44.696836] nix-daemon[848]: reaped child process 935, status = succeeded lb # [ 44.685479] niks3-server[959]: 2026/09/22 22:20:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 44.716581] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.355Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 44.718487] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.355Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)" lb # [ 44.798579] niks3-server[959]: 2026/09/22 22:20:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign lb # [ 44.802159] niks3-server[959]: 2026/09/22 22:20:57 INFO Signed narinfos id=4 count=1 lb # [ 44.807901] niks3-server[959]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst worker1 # [ 44.833583] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.472Z level=INFO msg="Uploading 1 narinfos" lb # [ 44.824753] niks3-server[959]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv lb # [ 44.834791] niks3-server[959]: 2026/09/22 22:20:57 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete lb # [ 44.844229] niks3-server[959]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls worker1 # [ 44.858004] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.497Z level=INFO msg="Upload complete. (370ms)" lb # [ 44.849591] niks3-server[959]: 2026/09/22 22:20:57 INFO Completed upload id=4 lb # [ 44.862384] niks3-server[959]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo worker1 # [ 44.867361] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=1 worker1 # [ 44.885342] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=0 worker1 # [ 44.887854] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 44.892218] nix-daemon[848]: reaped child process 914, status = succeeded worker1 # [ 44.931018] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 44.941807] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 44.952915] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.592Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'... worker1 # [ 44.966277] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 44.975427] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 45.133869] nix-daemon[848]: accepted connection from pid 942, user nix-grpc-daemon (trusted) lb # [ 45.160452] niks3-server[959]: 2026/09/22 22:20:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 45.178737] nix-daemon[848]: reaped child process 945, status = succeeded worker1 # [ 45.187597] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.826Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 45.190157] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.826Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)" lb # [ 45.203198] niks3-server[959]: 2026/09/22 22:20:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign worker1 # [ 45.219577] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.858Z level=INFO msg="Uploading 1 narinfos" lb # [ 45.216513] niks3-server[959]: 2026/09/22 22:20:57 INFO Signed narinfos id=5 count=1 lb # [ 45.224418] niks3-server[959]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls lb # [ 45.234273] niks3-server[959]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst lb # [ 45.250626] niks3-server[959]: 2026/09/22 22:20:57 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete lb # [ 45.261549] niks3-server[959]: 2026/09/22 22:20:57 INFO Completed upload id=5 worker1 # [ 45.272054] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.911Z level=INFO msg="Upload complete. (285ms)" worker1 # [ 45.277009] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 45.266879] niks3-server[959]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo worker1 # [ 45.282514] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 45.292687] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 45.296557] nix-daemon[848]: reaped child process 940, status = succeeded worker1 # [ 45.305870] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051... client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker1 # [ 46.154743] nix-daemon[848]: reaped child process 950, status = succeeded worker1 # [ 46.271591] nix-daemon[848]: accepted connection from pid 966, user nix-grpc-daemon (trusted) worker1 # [ 46.297894] nix-daemon[848]: reaped child process 969, status = succeeded lb # [ 46.293310] niks3-server[959]: 2026/09/22 22:20:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 46.310294] nix-grpc-daemon[863]: time=2026-09-22T22:20:58.948Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 46.314615] nix-grpc-daemon[863]: time=2026-09-22T22:20:58.948Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)" lb # [ 46.318836] niks3-server[959]: 2026/09/22 22:20:58 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst lb # [ 46.330022] niks3-server[959]: 2026/09/22 22:20:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign worker1 # [ 46.343492] nix-grpc-daemon[863]: time=2026-09-22T22:20:58.982Z level=INFO msg="Uploading 1 narinfos" lb # [ 46.336083] niks3-server[959]: 2026/09/22 22:20:58 INFO Signed narinfos id=6 count=1 lb # [ 46.339530] niks3-server[959]: 2026/09/22 22:20:58 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls lb # [ 46.349578] niks3-server[959]: 2026/09/22 22:20:58 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete lb # [ 46.354953] niks3-server[959]: 2026/09/22 22:20:58 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv lb # [ 46.366216] niks3-server[959]: 2026/09/22 22:20:59 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo worker1 # [ 46.379802] nix-grpc-daemon[863]: time=2026-09-22T22:20:59.019Z level=INFO msg="Upload complete. (243ms)" lb # [ 46.370413] niks3-server[959]: 2026/09/22 22:20:59 INFO Completed upload id=6 worker1 # [ 46.386031] nix-grpc-daemon[766]: ts=2026-09-22T22:20:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=1 worker1 # [ 46.396293] nix-daemon[848]: reaped child process 948, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 6.71 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.17 seconds) client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client # copying 3 paths... client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.37 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.39 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.25 seconds) worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.23 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 worker1 # [ 47.835592] nix-grpc-daemon[766]: ts=2026-09-22T22:21:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40012 client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked): client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # [ 47.852703] nix-grpc-daemon[769]: ts=2026-09-22T22:21:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53690 worker1 # [ 47.933146] nix-grpc-daemon[766]: ts=2026-09-22T22:21:00Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.38 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.15 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.11 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 1.40 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2 client # don't know how to build these paths: client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 48.920172] nix-grpc-daemon[769]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 48.927364] nix-grpc-daemon[769]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53690 worker1 # [ 48.953222] nix-grpc-daemon[766]: ts=2026-09-22T22:21:01Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=0 worker1 # [ 48.953691] nix-grpc-daemon[766]: ts=2026-09-22T22:21:01Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 49.000747] nix-grpc-daemon[766]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 49.019823] nix-grpc-daemon[766]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 49.039576] nix-grpc-daemon[863]: time=2026-09-22T22:21:01.677Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'... worker1 # [ 49.052668] nix-grpc-daemon[766]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 49.069131] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 49.239528] nix-daemon[848]: accepted connection from pid 998, user nix-grpc-daemon (trusted) worker1 # [ 49.268314] nix-daemon[848]: reaped child process 1001, status = succeeded lb # [ 49.261107] niks3-server[959]: 2026/09/22 22:21:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 49.286838] nix-grpc-daemon[863]: time=2026-09-22T22:21:01.926Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 49.290588] nix-grpc-daemon[863]: time=2026-09-22T22:21:01.928Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)" lb # [ 49.310211] niks3-server[959]: 2026/09/22 22:21:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign worker1 # [ 49.330790] nix-grpc-daemon[863]: time=2026-09-22T22:21:01.969Z level=INFO msg="Uploading 1 narinfos" lb # [ 49.325644] niks3-server[959]: 2026/09/22 22:21:01 INFO Signed narinfos id=7 count=1 lb # [ 49.333119] niks3-server[959]: 2026/09/22 22:21:01 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst lb # [ 49.344335] niks3-server[959]: 2026/09/22 22:21:01 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls lb # [ 49.353653] niks3-server[959]: 2026/09/22 22:21:01 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete lb # [ 49.365291] niks3-server[959]: 2026/09/22 22:21:01 INFO Completed upload id=7 worker1 # [ 49.372959] nix-grpc-daemon[863]: time=2026-09-22T22:21:02.012Z level=INFO msg="Upload complete. (283ms)" lb # [ 49.371887] niks3-server[959]: 2026/09/22 22:21:02 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo worker1 # [ 49.384711] nix-grpc-daemon[766]: ts=2026-09-22T22:21:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 49.400656] nix-grpc-daemon[766]: ts=2026-09-22T22:21:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 49.417336] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 49.426092] nix-daemon[848]: reaped child process 996, status = succeeded client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051... worker1 # [ 49.442812] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv lb # [ 49.913716] envoy[895]: conn peer=[2001:db8:1::1]:40296 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9549 tx=4390 ms=6519 lb # [ 49.916086] envoy[895]: conn peer=[2001:db8:1::1]:45098 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=128 client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 50.381691] nix-grpc-daemon[766]: E0000 00:00:1790115663.017992 808 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 50.412110] nix-grpc-daemon[766]: E0000 00:00:1790115663.043100 808 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 50.414629] nix-grpc-daemon[766]: E0000 00:00:1790115663.044312 808 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 50.430109] nix-grpc-daemon[766]: E0000 00:00:1790115663.065065 808 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. client # farm-top> LOG-farm-top-t1b worker1 # [ 50.473709] nix-daemon[848]: reaped child process 1005, status = succeeded worker1 # [ 50.657571] nix-daemon[848]: accepted connection from pid 1021, user nix-grpc-daemon (trusted) worker1 # [ 50.709653] nix-daemon[848]: reaped child process 1024, status = succeeded lb # [ 50.703105] niks3-server[959]: 2026/09/22 22:21:03 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 50.830596] nix-grpc-daemon[863]: time=2026-09-22T22:21:03.469Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 50.837285] nix-grpc-daemon[863]: time=2026-09-22T22:21:03.471Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)" lb # [ 50.885878] niks3-server[959]: 2026/09/22 22:21:03 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv lb # [ 50.908327] niks3-server[959]: 2026/09/22 22:21:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign worker1 # [ 50.922349] nix-grpc-daemon[863]: time=2026-09-22T22:21:03.561Z level=INFO msg="Uploading 1 narinfos" lb # [ 50.914447] niks3-server[959]: 2026/09/22 22:21:03 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst lb # [ 50.920863] niks3-server[959]: 2026/09/22 22:21:03 INFO Signed narinfos id=8 count=1 lb # [ 50.922109] niks3-server[959]: 2026/09/22 22:21:03 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls lb # [ 50.929602] niks3-server[959]: 2026/09/22 22:21:03 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete lb # [ 50.933880] niks3-server[959]: 2026/09/22 22:21:03 INFO Completed upload id=8 lb # [ 50.935327] niks3-server[959]: 2026/09/22 22:21:03 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo worker1 # [ 50.943514] nix-grpc-daemon[863]: time=2026-09-22T22:21:03.582Z level=INFO msg="Upload complete. (487ms)" worker1 # [ 50.956411] nix-grpc-daemon[766]: ts=2026-09-22T22:21:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 worker1 # [ 50.974777] nix-daemon[848]: reaped child process 1003, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.30 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.20 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.87 seconds) subtest: outputs read back through any worker carry the cache signature client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2 worker2 # [ 51.606672] nix-grpc-daemon[769]: ts=2026-09-22T22:21:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53690 client # don't know how to build these paths: client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker1 # [ 51.627360] nix-grpc-daemon[766]: ts=2026-09-22T22:21:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 51.653220] nix-grpc-daemon[766]: ts=2026-09-22T22:21:04Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=0 worker1 # [ 51.659620] nix-grpc-daemon[766]: ts=2026-09-22T22:21:04Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 51.659866] nix-grpc-daemon[769]: ts=2026-09-22T22:21:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 51.677703] nix-grpc-daemon[769]: ts=2026-09-22T22:21:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53690 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'... worker2 # [ 51.696243] nix-grpc-daemon[769]: ts=2026-09-22T22:21:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 51.713230] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 51.747045] nix-grpc-daemon[769]: ts=2026-09-22T22:21:04Z level=info event=niks3_push_started pid=865 worker2 # [ 51.933451] nix-grpc-daemon[865]: time=2026-09-22T22:21:04.574Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 52.055615] nix-daemon[837]: accepted connection from pid 878, user nix-grpc-daemon (trusted) worker2 # [ 52.078211] nix-daemon[837]: reaped child process 881, status = succeeded lb # [ 52.081768] niks3-server[959]: 2026/09/22 22:21:04 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 52.110135] nix-grpc-daemon[865]: time=2026-09-22T22:21:04.751Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 52.114936] nix-grpc-daemon[865]: time=2026-09-22T22:21:04.751Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)" lb # [ 52.131661] niks3-server[959]: 2026/09/22 22:21:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign lb # [ 52.138120] niks3-server[959]: 2026/09/22 22:21:04 INFO Signed narinfos id=9 count=1 worker2 # [ 52.145486] nix-grpc-daemon[865]: time=2026-09-22T22:21:04.787Z level=INFO msg="Uploading 1 narinfos" lb # [ 52.148356] niks3-server[959]: 2026/09/22 22:21:04 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete lb # [ 52.155108] niks3-server[959]: 2026/09/22 22:21:04 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst lb # [ 52.161760] niks3-server[959]: 2026/09/22 22:21:04 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls lb # [ 52.163439] niks3-server[959]: 2026/09/22 22:21:04 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo lb # [ 52.166253] niks3-server[959]: 2026/09/22 22:21:04 INFO Completed upload id=9 worker2 # [ 52.172752] nix-grpc-daemon[865]: time=2026-09-22T22:21:04.815Z level=INFO msg="Upload complete. (236ms)" worker2 # [ 52.181008] nix-grpc-daemon[769]: ts=2026-09-22T22:21:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 52.183984] nix-grpc-daemon[769]: ts=2026-09-22T22:21:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 52.195546] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 52.199634] nix-daemon[837]: reaped child process 863, status = succeeded client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051... worker2 # [ 52.211031] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-sig worker2 # [ 53.244615] nix-daemon[837]: reaped child process 885, status = succeeded worker2 # [ 53.364167] nix-daemon[837]: accepted connection from pid 901, user nix-grpc-daemon (trusted) worker2 # [ 53.390990] nix-daemon[837]: reaped child process 904, status = succeeded lb # [ 53.381832] niks3-server[959]: 2026/09/22 22:21:06 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 53.404537] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.045Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 53.406130] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.045Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)" lb # [ 53.423829] niks3-server[959]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv lb # [ 53.443970] niks3-server[959]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls lb # [ 53.454287] niks3-server[959]: 2026/09/22 22:21:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign worker2 # [ 53.467351] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.109Z level=INFO msg="Uploading 1 narinfos" lb # [ 53.463573] niks3-server[959]: 2026/09/22 22:21:06 INFO Signed narinfos id=10 count=1 lb # [ 53.469288] niks3-server[959]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst lb # [ 53.481334] niks3-server[959]: 2026/09/22 22:21:06 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete worker2 # [ 53.492802] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.135Z level=INFO msg="Upload complete. (273ms)" lb # [ 53.488197] niks3-server[959]: 2026/09/22 22:21:06 INFO Completed upload id=10 lb # [ 53.492882] niks3-server[959]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo worker2 # [ 53.499082] nix-grpc-daemon[769]: ts=2026-09-22T22:21:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=1 worker1 # [ 53.511336] nix-grpc-daemon[766]: ts=2026-09-22T22:21:06Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=0 worker2 # [ 53.511785] nix-daemon[837]: reaped child process 883, status = succeeded worker1 # [ 53.518204] nix-grpc-daemon[766]: ts=2026-09-22T22:21:06Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 53.554789] nix-grpc-daemon[769]: ts=2026-09-22T22:21:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 53.567214] nix-grpc-daemon[769]: ts=2026-09-22T22:21:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 53.582144] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.224Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 53.592816] nix-grpc-daemon[769]: ts=2026-09-22T22:21:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'... worker2 # [ 53.603722] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 53.751710] nix-daemon[837]: accepted connection from pid 908, user nix-grpc-daemon (trusted) lb # [ 53.769709] niks3-server[959]: 2026/09/22 22:21:06 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 53.779396] nix-daemon[837]: reaped child process 911, status = succeeded worker2 # [ 53.788103] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.430Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 53.789803] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.430Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)" lb # [ 53.804888] niks3-server[959]: 2026/09/22 22:21:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign worker2 # [ 53.818446] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.460Z level=INFO msg="Uploading 1 narinfos" lb # [ 53.820549] niks3-server[959]: 2026/09/22 22:21:06 INFO Signed narinfos id=11 count=1 lb # [ 53.829712] niks3-server[959]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls lb # [ 53.838271] niks3-server[959]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst worker2 # [ 53.858991] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.501Z level=INFO msg="Upload complete. (245ms)" lb # [ 53.852474] niks3-server[959]: 2026/09/22 22:21:06 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete lb # [ 53.859896] niks3-server[959]: 2026/09/22 22:21:06 INFO Completed upload id=11 worker2 # [ 53.868485] nix-grpc-daemon[769]: ts=2026-09-22T22:21:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 53.865820] niks3-server[959]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo worker2 # [ 53.877298] nix-grpc-daemon[769]: ts=2026-09-22T22:21:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 53.890269] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 53.896560] nix-daemon[837]: reaped child process 906, status = succeeded client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051... client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker2 # [ 53.910680] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-sig worker2 # [ 54.545666] nix-daemon[837]: reaped child process 915, status = succeeded worker2 # [ 54.666765] nix-daemon[837]: accepted connection from pid 931, user nix-grpc-daemon (trusted) worker2 # [ 54.692324] nix-daemon[837]: reaped child process 934, status = succeeded lb # [ 54.693541] niks3-server[959]: 2026/09/22 22:21:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 54.713349] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.353Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 54.715009] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.353Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)" lb # [ 54.732351] niks3-server[959]: 2026/09/22 22:21:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign worker2 # [ 54.745618] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.388Z level=INFO msg="Uploading 1 narinfos" lb # [ 54.741264] niks3-server[959]: 2026/09/22 22:21:07 INFO Signed narinfos id=12 count=1 lb # [ 54.749222] niks3-server[959]: 2026/09/22 22:21:07 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv lb # [ 54.756750] niks3-server[959]: 2026/09/22 22:21:07 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls lb # [ 54.758972] niks3-server[959]: 2026/09/22 22:21:07 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst lb # [ 54.771364] niks3-server[959]: 2026/09/22 22:21:07 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete lb # [ 54.778896] niks3-server[959]: 2026/09/22 22:21:07 INFO Completed upload id=12 worker2 # [ 54.784925] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.427Z level=INFO msg="Upload complete. (264ms)" lb # [ 54.781861] niks3-server[959]: 2026/09/22 22:21:07 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo worker2 # [ 54.797362] nix-grpc-daemon[769]: ts=2026-09-22T22:21:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=1 worker1 # [ 54.806392] nix-grpc-daemon[766]: ts=2026-09-22T22:21:07Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=0 worker1 # [ 54.812954] nix-grpc-daemon[766]: ts=2026-09-22T22:21:07Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 54.815577] nix-daemon[837]: reaped child process 913, status = succeeded worker2 # [ 54.847390] nix-grpc-daemon[769]: ts=2026-09-22T22:21:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 54.865299] nix-grpc-daemon[769]: ts=2026-09-22T22:21:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 54.882631] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.525Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'... worker2 # [ 54.897595] nix-grpc-daemon[769]: ts=2026-09-22T22:21:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 54.918711] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 55.091908] nix-daemon[837]: accepted connection from pid 938, user nix-grpc-daemon (trusted) worker2 # [ 55.114180] nix-daemon[837]: reaped child process 941, status = succeeded lb # [ 55.114829] niks3-server[959]: 2026/09/22 22:21:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 55.133073] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.775Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 55.134748] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.775Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)" lb # [ 55.155438] niks3-server[959]: 2026/09/22 22:21:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign worker2 # [ 55.172590] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.814Z level=INFO msg="Uploading 1 narinfos" lb # [ 55.173052] niks3-server[959]: 2026/09/22 22:21:07 INFO Signed narinfos id=13 count=1 lb # [ 55.182211] niks3-server[959]: 2026/09/22 22:21:07 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls lb # [ 55.190056] niks3-server[959]: 2026/09/22 22:21:07 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst lb # [ 55.205970] niks3-server[959]: 2026/09/22 22:21:07 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete lb # [ 55.212671] niks3-server[959]: 2026/09/22 22:21:07 INFO Completed upload id=13 lb # [ 55.216139] niks3-server[959]: 2026/09/22 22:21:07 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo worker2 # [ 55.217594] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.860Z level=INFO msg="Upload complete. (278ms)" worker2 # [ 55.228946] nix-grpc-daemon[769]: ts=2026-09-22T22:21:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 55.244580] nix-grpc-daemon[769]: ts=2026-09-22T22:21:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 55.260156] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 55.266700] nix-daemon[837]: reaped child process 936, status = succeeded client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051... worker2 # [ 55.279837] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-sig worker2 # [ 56.114140] nix-daemon[837]: reaped child process 945, status = succeeded worker2 # [ 56.231659] nix-daemon[837]: accepted connection from pid 961, user nix-grpc-daemon (trusted) worker2 # [ 56.257106] nix-daemon[837]: reaped child process 964, status = succeeded lb # [ 56.260000] niks3-server[959]: 2026/09/22 22:21:08 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 56.280267] nix-grpc-daemon[865]: time=2026-09-22T22:21:08.920Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 56.281890] nix-grpc-daemon[865]: time=2026-09-22T22:21:08.920Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)" lb # [ 56.310847] niks3-server[959]: 2026/09/22 22:21:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign worker2 # [ 56.325654] nix-grpc-daemon[865]: time=2026-09-22T22:21:08.968Z level=INFO msg="Uploading 1 narinfos" lb # [ 56.326262] niks3-server[959]: 2026/09/22 22:21:08 INFO Signed narinfos id=14 count=1 lb # [ 56.332895] niks3-server[959]: 2026/09/22 22:21:08 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls lb # [ 56.342788] niks3-server[959]: 2026/09/22 22:21:08 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv lb # [ 56.352483] niks3-server[959]: 2026/09/22 22:21:08 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst worker2 # [ 56.368129] nix-grpc-daemon[865]: time=2026-09-22T22:21:09.010Z level=INFO msg="Upload complete. (274ms)" lb # [ 56.364640] niks3-server[959]: 2026/09/22 22:21:08 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 56.368486] niks3-server[959]: 2026/09/22 22:21:08 INFO Completed upload id=14 lb # [ 56.373126] niks3-server[959]: 2026/09/22 22:21:09 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo worker2 # [ 56.381438] nix-grpc-daemon[769]: ts=2026-09-22T22:21:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=1 worker2 # [ 56.405109] nix-daemon[837]: reaped child process 943, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.06 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 56.782005] nix-grpc-daemon[766]: ts=2026-09-22T22:21:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:55298 client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked): client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 56.823215] nix-grpc-daemon[766]: ts=2026-09-22T22:21:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:55298 worker1 # [ 56.833322] nix-grpc-daemon[766]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'... worker1 # [ 56.858446] nix-grpc-daemon[766]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'... worker1 # [ 56.877357] nix-grpc-daemon[766]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.30 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker2 # [ 57.109197] nix-grpc-daemon[769]: ts=2026-09-22T22:21:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:53390 worker2 # [ 57.152424] nix-grpc-daemon[769]: ts=2026-09-22T22:21:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:53390 client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.30 seconds) client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top client # finding garbage collector roots... client # removing stale temporary roots file "/nix/var/nix/temproots/883" client # removing stale temporary roots file "/nix/var/nix/temproots/903" client # removing stale temporary roots file "/nix/var/nix/temproots/845" client # removing stale temporary roots file "/nix/var/nix/temproots/895" client # removing stale temporary roots file "/nix/var/nix/temproots/916" client # removing stale temporary roots file "/nix/var/nix/temproots/938" client # removing stale temporary roots file "/nix/var/nix/temproots/862" client # removing stale temporary roots file "/nix/var/nix/temproots/924" client # deleting specified paths... client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' client # deleting unused links... client # note: hard linking is currently saving 0.0 KiB worker2 # [ 57.594237] nix-grpc-daemon[769]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53690 worker1 # [ 57.608962] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 57.622784] nix-grpc-daemon[863]: time=2026-09-22T22:21:10.261Z level=INFO msg="All 1 paths already cached" worker1 # [ 57.633713] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 57.645269] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 57.656516] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40012 client # copying 3 paths... worker2 # [ 57.672175] nix-grpc-daemon[769]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53690 client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'... worker1 # [ 57.713527] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 57.722164] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_out=232 worker2 # [ 57.738844] nix-grpc-daemon[769]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 57.747281] nix-grpc-daemon[769]: ts=2026-09-22T22:21:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_out=176 worker1 # [ 57.761943] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 57.769934] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_out=128 client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'... client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'... client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.60 seconds) (finished: subtest: outputs read back through any worker carry the cache signature, in 6.46 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 57.883181] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: b13591785c854ee78757e801bbfd41f9 client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup worker2 # [ 58.091807] nix-grpc-daemon[769]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53690 client # [ 58.099504] nix[991]: don't know how to build these paths: client # [ 58.101670] nix[991]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 58.120157] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker1 # [ 58.125982] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 58.163252] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 58.181895] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 client # [ 58.192268] nix[991]: copying 1 paths to 'lb:50051'... client # [ 58.198890] nix[991]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'... worker1 # [ 58.202541] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 58.220190] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 58.396748] nix-daemon[848]: accepted connection from pid 1044, user nix-grpc-daemon (trusted) worker1 # [ 58.424235] nix-daemon[848]: reaped child process 1047, status = succeeded lb # [ 58.419788] niks3-server[959]: 2026/09/22 22:21:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 58.440830] nix-grpc-daemon[863]: time=2026-09-22T22:21:11.079Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 58.442717] nix-grpc-daemon[863]: time=2026-09-22T22:21:11.079Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)" lb # [ 58.460899] niks3-server[959]: 2026/09/22 22:21:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign worker1 # [ 58.484025] nix-grpc-daemon[863]: time=2026-09-22T22:21:11.123Z level=INFO msg="Uploading 1 narinfos" lb # [ 58.474119] niks3-server[959]: 2026/09/22 22:21:11 INFO Signed narinfos id=15 count=1 lb # [ 58.484778] niks3-server[959]: 2026/09/22 22:21:11 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst lb # [ 58.498463] niks3-server[959]: 2026/09/22 22:21:11 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls lb # [ 58.513633] niks3-server[959]: 2026/09/22 22:21:11 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete worker1 # [ 58.530843] nix-grpc-daemon[863]: time=2026-09-22T22:21:11.170Z level=INFO msg="Upload complete. (291ms)" lb # [ 58.522865] niks3-server[959]: 2026/09/22 22:21:11 INFO Completed upload id=15 lb # [ 58.527150] niks3-server[959]: 2026/09/22 22:21:11 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo worker1 # [ 58.542848] nix-grpc-daemon[766]: ts=2026-09-22T22:21:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=432 worker1 # [ 58.559262] nix-grpc-daemon[766]: ts=2026-09-22T22:21:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 58.576510] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 58.589162] nix-daemon[848]: reaped child process 1042, status = succeeded client # [ 58.591038] nix[991]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker1 # [ 58.600081] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) client # [ 58.985817] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: 185f35890eb74405b3f289db5b4526aa client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.12 seconds) client # [ 59.244724] nix[1007]: don't know how to build these paths: worker2 # [ 59.241067] nix-grpc-daemon[769]: ts=2026-09-22T22:21:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53690 client # [ 59.245688] nix[1007]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 59.262783] nix-grpc-daemon[766]: ts=2026-09-22T22:21:11Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker1 # [ 59.276650] nix-grpc-daemon[766]: ts=2026-09-22T22:21:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 client # [ 59.280087] nix[1007]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... lb # [ 59.915204] envoy[895]: conn peer=[2001:db8:1::1]:45100 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4215 tx=1911 ms=2050 lb # [ 59.920507] envoy[895]: conn peer=[2001:db8:1::1]:54570 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9515 tx=4354 ms=4794 lb # [ 59.925420] envoy[895]: conn peer=[2001:db8:1::1]:54580 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2926 ms=215 lb # [ 59.930289] envoy[895]: conn peer=[2001:db8:1::1]:54602 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=54 lb # [ 59.935005] envoy[895]: conn peer=[2001:db8:1::1]:54588 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=83 lb # [ 59.939726] envoy[895]: conn peer=[2001:db8:1::1]:54584 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=104 client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.06 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker1 # [ 62.671370] nix-daemon[848]: reaped child process 1051, status = succeeded worker1 # [ 62.800588] nix-daemon[848]: accepted connection from pid 1083, user nix-grpc-daemon (trusted) worker1 # [ 62.823890] nix-daemon[848]: reaped child process 1086, status = succeeded lb # [ 62.822991] niks3-server[959]: 2026/09/22 22:21:15 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 62.844636] nix-grpc-daemon[863]: time=2026-09-22T22:21:15.482Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 62.846878] nix-grpc-daemon[863]: time=2026-09-22T22:21:15.482Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)" lb # [ 62.879974] niks3-server[959]: 2026/09/22 22:21:15 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst worker1 # [ 62.905022] nix-grpc-daemon[863]: time=2026-09-22T22:21:15.544Z level=INFO msg="Uploading 1 narinfos" lb # [ 62.894688] niks3-server[959]: 2026/09/22 22:21:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign lb # [ 62.907294] niks3-server[959]: 2026/09/22 22:21:15 INFO Signed narinfos id=16 count=1 lb # [ 62.913739] niks3-server[959]: 2026/09/22 22:21:15 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv lb # [ 62.923045] niks3-server[959]: 2026/09/22 22:21:15 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls lb # [ 62.932892] niks3-server[959]: 2026/09/22 22:21:15 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 62.940120] niks3-server[959]: 2026/09/22 22:21:15 INFO Completed upload id=16 lb # [ 62.944309] niks3-server[959]: 2026/09/22 22:21:15 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo worker1 # [ 62.949463] nix-grpc-daemon[863]: time=2026-09-22T22:21:15.588Z level=INFO msg="Upload complete. (333ms)" worker1 # [ 62.965744] nix-grpc-daemon[766]: ts=2026-09-22T22:21:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 62.987256] nix-grpc-daemon[766]: ts=2026-09-22T22:21:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker1 # [ 62.995196] nix-daemon[848]: reaped child process 1049, status = succeeded client # [ 62.997647] systemd[1]: dedup1.service: Deactivated successfully. client # [ 63.003444] systemd[1]: dedup1.service: Consumed 193ms CPU time over 5.115s wall clock time, 41.5M memory peak, 7.4K incoming IP traffic, 8.4K outgoing IP traffic. client # [ 63.051475] systemd[1]: dedup2.service: Deactivated successfully. client # [ 63.055856] systemd[1]: dedup2.service: Consumed 230ms CPU time over 4.069s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: two clients wanting the same drv build it once, in 6.16 seconds) subtest: --timeout and --max-silent-time reach the builder worker2 # [ 64.221519] nix-grpc-daemon[769]: ts=2026-09-22T22:21:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53690 worker1 # [ 64.238056] nix-grpc-daemon[766]: ts=2026-09-22T22:21:16Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=0 worker1 # [ 64.244561] nix-grpc-daemon[766]: ts=2026-09-22T22:21:16Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 64.244760] nix-grpc-daemon[769]: ts=2026-09-22T22:21:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 64.263134] nix-grpc-daemon[769]: ts=2026-09-22T22:21:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 64.281974] nix-grpc-daemon[769]: ts=2026-09-22T22:21:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 64.296829] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 64.466351] nix-daemon[837]: accepted connection from pid 989, user nix-grpc-daemon (trusted) worker2 # [ 64.494743] nix-daemon[837]: reaped child process 992, status = succeeded lb # [ 64.487221] niks3-server[959]: 2026/09/22 22:21:17 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 64.504529] nix-grpc-daemon[865]: time=2026-09-22T22:21:17.146Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 64.506957] nix-grpc-daemon[865]: time=2026-09-22T22:21:17.146Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)" lb # [ 64.525073] niks3-server[959]: 2026/09/22 22:21:17 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst worker2 # [ 64.543122] nix-grpc-daemon[865]: time=2026-09-22T22:21:17.185Z level=INFO msg="Uploading 1 narinfos" lb # [ 64.537809] niks3-server[959]: 2026/09/22 22:21:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign lb # [ 64.545792] niks3-server[959]: 2026/09/22 22:21:17 INFO Signed narinfos id=17 count=1 lb # [ 64.557581] niks3-server[959]: 2026/09/22 22:21:17 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls lb # [ 64.566132] niks3-server[959]: 2026/09/22 22:21:17 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete lb # [ 64.572293] niks3-server[959]: 2026/09/22 22:21:17 INFO Completed upload id=17 worker2 # [ 64.577879] nix-grpc-daemon[865]: time=2026-09-22T22:21:17.220Z level=INFO msg="Upload complete. (262ms)" lb # [ 64.578667] niks3-server[959]: 2026/09/22 22:21:17 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo worker2 # [ 64.588423] nix-grpc-daemon[769]: ts=2026-09-22T22:21:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 64.609088] nix-grpc-daemon[769]: ts=2026-09-22T22:21:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 64.617563] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 64.624384] nix-daemon[837]: reaped child process 987, status = succeeded worker2 # [ 64.641834] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) lb # [ 69.914871] envoy[895]: conn peer=[2001:db8:1::1]:54612 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1563 ms=4884 lb # [ 69.915162] envoy[895]: conn peer=[2001:db8:1::1]:54622 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=3790 worker2 # [ 71.060869] nix-grpc-daemon[769]: ts=2026-09-22T22:21:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=0 worker2 # [ 71.097751] nix-daemon[837]: reaped child process 994, status = succeeded worker2 # [ 71.100673] nix-daemon[837]: reaped child process 996, status = succeeded worker1 # [ 71.309022] nix-grpc-daemon[766]: ts=2026-09-22T22:21:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 71.326677] nix-grpc-daemon[766]: ts=2026-09-22T22:21:23Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=0 worker1 # [ 71.334284] nix-grpc-daemon[766]: ts=2026-09-22T22:21:23Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 71.343951] nix-grpc-daemon[766]: ts=2026-09-22T22:21:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 71.355750] nix-grpc-daemon[766]: ts=2026-09-22T22:21:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 71.371896] nix-grpc-daemon[766]: ts=2026-09-22T22:21:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 71.387571] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 71.548921] nix-daemon[848]: accepted connection from pid 1103, user nix-grpc-daemon (trusted) worker1 # [ 71.572678] nix-daemon[848]: reaped child process 1106, status = succeeded lb # [ 71.561250] niks3-server[959]: 2026/09/22 22:21:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 71.584438] nix-grpc-daemon[863]: time=2026-09-22T22:21:24.223Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 71.585974] nix-grpc-daemon[863]: time=2026-09-22T22:21:24.223Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)" lb # [ 71.601349] niks3-server[959]: 2026/09/22 22:21:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign worker1 # [ 71.622490] nix-grpc-daemon[863]: time=2026-09-22T22:21:24.261Z level=INFO msg="Uploading 1 narinfos" lb # [ 71.614308] niks3-server[959]: 2026/09/22 22:21:24 INFO Signed narinfos id=18 count=1 lb # [ 71.624847] niks3-server[959]: 2026/09/22 22:21:24 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst lb # [ 71.635260] niks3-server[959]: 2026/09/22 22:21:24 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls lb # [ 71.646912] niks3-server[959]: 2026/09/22 22:21:24 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete lb # [ 71.653051] niks3-server[959]: 2026/09/22 22:21:24 INFO Completed upload id=18 worker1 # [ 71.662795] nix-grpc-daemon[863]: time=2026-09-22T22:21:24.302Z level=INFO msg="Upload complete. (257ms)" lb # [ 71.657907] niks3-server[959]: 2026/09/22 22:21:24 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo worker1 # [ 71.675206] nix-grpc-daemon[766]: ts=2026-09-22T22:21:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 71.692468] nix-grpc-daemon[766]: ts=2026-09-22T22:21:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 71.700245] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 71.706564] nix-daemon[848]: reaped child process 1101, status = succeeded worker1 # [ 71.723792] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 77.542414] nix-grpc-daemon[766]: ts=2026-09-22T22:21:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=5 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=0 worker1 # [ 77.568537] nix-daemon[848]: reaped child process 1108, status = succeeded worker1 # [ 77.570223] nix-daemon[848]: reaped child process 1110, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 13.77 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 77.830581] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: b16d88b03a2447dc835347e06b779733 client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.10 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 78.062348] nix[1079]: don't know how to build these paths: client # [ 78.063479] nix[1079]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker2 # [ 78.061290] nix-grpc-daemon[769]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53690 worker1 # [ 78.078843] nix-grpc-daemon[766]: ts=2026-09-22T22:21:30Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker1 # [ 78.085460] nix-grpc-daemon[766]: ts=2026-09-22T22:21:30Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 78.085098] nix-grpc-daemon[769]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 78.102399] nix-grpc-daemon[769]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53690 client # [ 78.114934] nix[1079]: copying 1 paths to 'lb:50051'... client # [ 78.118985] nix[1079]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'... worker2 # [ 78.119894] nix-grpc-daemon[769]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 78.135559] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 78.307333] nix-daemon[837]: accepted connection from pid 1019, user nix-grpc-daemon (trusted) worker2 # [ 78.330303] nix-daemon[837]: reaped child process 1022, status = succeeded lb # [ 78.323400] niks3-server[959]: 2026/09/22 22:21:30 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 78.340712] nix-grpc-daemon[865]: time=2026-09-22T22:21:30.983Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 78.342232] nix-grpc-daemon[865]: time=2026-09-22T22:21:30.983Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)" lb # [ 78.363220] niks3-server[959]: 2026/09/22 22:21:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign worker2 # [ 78.381180] nix-grpc-daemon[865]: time=2026-09-22T22:21:31.023Z level=INFO msg="Uploading 1 narinfos" lb # [ 78.377691] niks3-server[959]: 2026/09/22 22:21:31 INFO Signed narinfos id=19 count=1 lb # [ 78.385473] niks3-server[959]: 2026/09/22 22:21:31 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls lb # [ 78.396559] niks3-server[959]: 2026/09/22 22:21:31 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst lb # [ 78.411562] niks3-server[959]: 2026/09/22 22:21:31 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete worker2 # [ 78.426595] nix-grpc-daemon[865]: time=2026-09-22T22:21:31.069Z level=INFO msg="Upload complete. (273ms)" lb # [ 78.422739] niks3-server[959]: 2026/09/22 22:21:31 INFO Completed upload id=19 lb # [ 78.427758] niks3-server[959]: 2026/09/22 22:21:31 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo worker2 # [ 78.436577] nix-grpc-daemon[769]: ts=2026-09-22T22:21:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 78.460287] nix-grpc-daemon[769]: ts=2026-09-22T22:21:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 78.468000] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 78.474871] nix-daemon[837]: reaped child process 1017, status = succeeded client # [ 78.486204] nix[1079]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... worker2 # [ 78.491418] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # [ 78.950485] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 3610e887514745d9b6db841dcc44fc4f client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.12 seconds) client # [ 79.149866] nix[1095]: don't know how to build these paths: client # [ 79.150899] nix[1095]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 79.153129] nix-grpc-daemon[766]: ts=2026-09-22T22:21:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 79.168230] nix-grpc-daemon[766]: ts=2026-09-22T22:21:31Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker2 # [ 79.218231] nix-grpc-daemon[769]: ts=2026-09-22T22:21:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 client # [ 79.227713] nix[1095]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... lb # [ 79.915991] envoy[895]: conn peer=[2001:db8:1::1]:37698 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3001 tx=978 ms=6863 lb # [ 79.916484] envoy[895]: conn peer=[2001:db8:1::1]:35736 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3015 tx=977 ms=6292 client: must succeed: sleep 2; systemctl kill -s INT share1 client # [ 82.343047] systemd[1]: share1.service: Sent signal SIGINT to main process 1079 (nix) on client request. client # [ 82.350769] nix[1079]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 82.366451] nix[1079]: error: interrupted by the user client # [ 82.425552] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 82.428608] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 82.431129] systemd[1]: share1.service: Consumed 190ms CPU time over 4.598s wall clock time, 41.4M memory peak, 6.2K incoming IP traffic, 8.3K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 1.12 seconds) client: waiting for success: ! systemctl is-active share2 worker2 # [ 83.975068] nix-daemon[837]: reaped child process 1026, status = succeeded worker2 # [ 84.083420] nix-daemon[837]: accepted connection from pid 1060, user nix-grpc-daemon (trusted) worker2 # [ 84.108678] nix-daemon[837]: reaped child process 1063, status = succeeded lb # [ 84.100825] niks3-server[959]: 2026/09/22 22:21:36 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 84.120479] nix-grpc-daemon[865]: time=2026-09-22T22:21:36.762Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 84.122087] nix-grpc-daemon[865]: time=2026-09-22T22:21:36.762Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)" lb # [ 84.153558] niks3-server[959]: 2026/09/22 22:21:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign worker2 # [ 84.168696] nix-grpc-daemon[865]: time=2026-09-22T22:21:36.811Z level=INFO msg="Uploading 1 narinfos" lb # [ 84.164674] niks3-server[959]: 2026/09/22 22:21:36 INFO Signed narinfos id=20 count=1 lb # [ 84.172849] niks3-server[959]: 2026/09/22 22:21:36 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 84.181917] niks3-server[959]: 2026/09/22 22:21:36 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv lb # [ 84.190411] niks3-server[959]: 2026/09/22 22:21:36 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls lb # [ 84.199563] niks3-server[959]: 2026/09/22 22:21:36 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete lb # [ 84.206748] niks3-server[959]: 2026/09/22 22:21:36 INFO Completed upload id=20 worker2 # [ 84.211643] nix-grpc-daemon[865]: time=2026-09-22T22:21:36.854Z level=INFO msg="Upload complete. (302ms)" lb # [ 84.209526] niks3-server[959]: 2026/09/22 22:21:36 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo worker2 # [ 84.228551] nix-grpc-daemon[769]: ts=2026-09-22T22:21:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker2 # [ 84.244486] nix-grpc-daemon[769]: ts=2026-09-22T22:21:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker2 # [ 84.251222] nix-daemon[837]: reaped child process 1024, status = succeeded client # [ 84.303509] systemd[1]: share2.service: Deactivated successfully. client # [ 84.308120] systemd[1]: share2.service: Consumed 180ms CPU time over 5.354s wall clock time, 33.3M memory peak, 6.2K incoming IP traffic, 5.6K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.10 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.04 seconds) (finished: subtest: the first client leaving does not fail the second, in 7.12 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 85.008385] systemd[1]: Started Nix Daemon. client # [ 85.133064] nix-daemon[1147]: accepted connection from pid 1144, user root (trusted) worker1 # [ 85.294350] nix-grpc-daemon[766]: ts=2026-09-22T22:21:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 85.302311] nix-grpc-daemon[766]: ts=2026-09-22T22:21:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 85.316142] nix-grpc-daemon[766]: ts=2026-09-22T22:21:37Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=0 worker1 # [ 85.322906] nix-grpc-daemon[766]: ts=2026-09-22T22:21:37Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 85.333601] nix-grpc-daemon[766]: ts=2026-09-22T22:21:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 85.345932] nix-grpc-daemon[766]: ts=2026-09-22T22:21:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 85.362277] nix-grpc-daemon[766]: ts=2026-09-22T22:21:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 85.379152] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 85.534183] nix-daemon[848]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) worker1 # [ 85.558667] nix-daemon[848]: reaped child process 1158, status = succeeded lb # [ 85.547468] niks3-server[959]: 2026/09/22 22:21:38 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 85.567509] nix-grpc-daemon[863]: time=2026-09-22T22:21:38.206Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 85.569397] nix-grpc-daemon[863]: time=2026-09-22T22:21:38.206Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)" lb # [ 85.584610] niks3-server[959]: 2026/09/22 22:21:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign worker1 # [ 85.602101] nix-grpc-daemon[863]: time=2026-09-22T22:21:38.241Z level=INFO msg="Uploading 1 narinfos" lb # [ 85.594722] niks3-server[959]: 2026/09/22 22:21:38 INFO Signed narinfos id=21 count=1 lb # [ 85.603583] niks3-server[959]: 2026/09/22 22:21:38 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst lb # [ 85.614711] niks3-server[959]: 2026/09/22 22:21:38 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls lb # [ 85.624261] niks3-server[959]: 2026/09/22 22:21:38 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete lb # [ 85.627162] niks3-server[959]: 2026/09/22 22:21:38 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo lb # [ 85.633075] niks3-server[959]: 2026/09/22 22:21:38 INFO Completed upload id=21 worker1 # [ 85.642284] nix-grpc-daemon[863]: time=2026-09-22T22:21:38.281Z level=INFO msg="Upload complete. (247ms)" worker1 # [ 85.653208] nix-grpc-daemon[766]: ts=2026-09-22T22:21:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 85.664592] nix-grpc-daemon[766]: ts=2026-09-22T22:21:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 85.677032] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 85.682210] nix-daemon[848]: reaped child process 1153, status = succeeded worker1 # [ 85.695336] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 86.596303] nix-daemon[848]: reaped child process 1162, status = succeeded worker1 # [ 86.679126] nix-daemon[848]: accepted connection from pid 1178, user nix-grpc-daemon (trusted) worker1 # [ 86.703993] nix-daemon[848]: reaped child process 1181, status = succeeded lb # [ 86.693527] niks3-server[959]: 2026/09/22 22:21:39 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 86.715496] nix-grpc-daemon[863]: time=2026-09-22T22:21:39.353Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 86.717172] nix-grpc-daemon[863]: time=2026-09-22T22:21:39.353Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)" lb # [ 86.737024] niks3-server[959]: 2026/09/22 22:21:39 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker1 # [ 86.760470] nix-grpc-daemon[863]: time=2026-09-22T22:21:39.399Z level=INFO msg="Uploading 1 narinfos" lb # [ 86.751978] niks3-server[959]: 2026/09/22 22:21:39 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst lb # [ 86.763422] niks3-server[959]: 2026/09/22 22:21:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign lb # [ 86.771846] niks3-server[959]: 2026/09/22 22:21:39 INFO Signed narinfos id=22 count=1 lb # [ 86.777347] niks3-server[959]: 2026/09/22 22:21:39 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls lb # [ 86.779852] niks3-server[959]: 2026/09/22 22:21:39 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete lb # [ 86.784223] niks3-server[959]: 2026/09/22 22:21:39 INFO Completed upload id=22 lb # [ 86.786361] niks3-server[959]: 2026/09/22 22:21:39 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo worker1 # [ 86.785993] nix-grpc-daemon[863]: time=2026-09-22T22:21:39.425Z level=INFO msg="Upload complete. (280ms)" worker1 # [ 86.801620] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=1 worker1 # [ 86.820760] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 86.827011] nix-daemon[848]: reaped child process 1160, status = succeeded worker1 # [ 86.838550] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 86.845176] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_out=128 worker1 # [ 87.124136] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 87.134550] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 87.144694] nix-grpc-daemon[863]: time=2026-09-22T22:21:39.784Z level=INFO msg="All 1 paths already cached" worker1 # [ 87.164618] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=0 worker1 # [ 87.171411] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 87.180582] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 87.192741] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 87.206905] nix-grpc-daemon[863]: time=2026-09-22T22:21:39.846Z level=INFO msg="All 1 paths already cached" worker1 # [ 87.217215] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 87.231864] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 87.394881] nix-daemon[848]: accepted connection from pid 1187, user nix-grpc-daemon (trusted) worker1 # [ 87.416750] nix-daemon[848]: reaped child process 1190, status = succeeded lb # [ 87.412782] niks3-server[959]: 2026/09/22 22:21:40 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 87.433578] nix-grpc-daemon[863]: time=2026-09-22T22:21:40.072Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 87.435161] nix-grpc-daemon[863]: time=2026-09-22T22:21:40.072Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)" lb # [ 87.456369] niks3-server[959]: 2026/09/22 22:21:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign worker1 # [ 87.475856] nix-grpc-daemon[863]: time=2026-09-22T22:21:40.115Z level=INFO msg="Uploading 1 narinfos" lb # [ 87.471873] niks3-server[959]: 2026/09/22 22:21:40 INFO Signed narinfos id=23 count=1 lb # [ 87.477325] niks3-server[959]: 2026/09/22 22:21:40 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst lb # [ 87.484868] niks3-server[959]: 2026/09/22 22:21:40 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls worker1 # [ 87.502000] nix-grpc-daemon[863]: time=2026-09-22T22:21:40.141Z level=INFO msg="Upload complete. (252ms)" lb # [ 87.493759] niks3-server[959]: 2026/09/22 22:21:40 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete lb # [ 87.501485] niks3-server[959]: 2026/09/22 22:21:40 INFO Completed upload id=23 lb # [ 87.504504] niks3-server[959]: 2026/09/22 22:21:40 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo worker1 # [ 87.512321] nix-grpc-daemon[766]: ts=2026-09-22T22:21:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 87.524696] nix-grpc-daemon[766]: ts=2026-09-22T22:21:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 87.539686] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 87.545905] nix-daemon[848]: reaped child process 1185, status = succeeded worker1 # [ 87.558798] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 88.644705] nix-daemon[848]: reaped child process 1194, status = succeeded worker1 # [ 88.770541] nix-daemon[848]: accepted connection from pid 1210, user nix-grpc-daemon (trusted) worker1 # [ 88.797628] nix-daemon[848]: reaped child process 1213, status = succeeded lb # [ 88.787600] niks3-server[959]: 2026/09/22 22:21:41 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 88.815803] nix-grpc-daemon[863]: time=2026-09-22T22:21:41.451Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 88.817553] nix-grpc-daemon[863]: time=2026-09-22T22:21:41.451Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)" lb # [ 88.853668] niks3-server[959]: 2026/09/22 22:21:41 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst lb # [ 88.866277] niks3-server[959]: 2026/09/22 22:21:41 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker1 # [ 88.892325] nix-grpc-daemon[863]: time=2026-09-22T22:21:41.531Z level=INFO msg="Uploading 1 narinfos" lb # [ 88.875879] niks3-server[959]: 2026/09/22 22:21:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign lb # [ 88.888509] niks3-server[959]: 2026/09/22 22:21:41 INFO Signed narinfos id=24 count=1 lb # [ 88.894426] niks3-server[959]: 2026/09/22 22:21:41 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls lb # [ 88.912802] niks3-server[959]: 2026/09/22 22:21:41 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete lb # [ 88.919615] niks3-server[959]: 2026/09/22 22:21:41 INFO Completed upload id=24 worker1 # [ 88.929715] nix-grpc-daemon[863]: time=2026-09-22T22:21:41.568Z level=INFO msg="Upload complete. (355ms)" lb # [ 88.922410] niks3-server[959]: 2026/09/22 22:21:41 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo worker1 # [ 88.945681] nix-grpc-daemon[766]: ts=2026-09-22T22:21:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=1 worker1 # [ 88.967100] nix-daemon[848]: reaped child process 1192, status = succeeded worker2 # [ 88.993674] nix-grpc-daemon[769]: ts=2026-09-22T22:21:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 89.026581] nix-grpc-daemon[769]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'... worker2 # [ 89.048463] nix-grpc-daemon[769]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'... worker1 # [ 89.104813] nix-grpc-daemon[766]: ts=2026-09-22T22:21:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 89.113512] nix-grpc-daemon[766]: ts=2026-09-22T22:21:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_out=176 worker2 # [ 89.403802] nix-grpc-daemon[769]: ts=2026-09-22T22:21:42Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:53690 worker1 # [ 89.419732] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 89.432791] nix-grpc-daemon[863]: time=2026-09-22T22:21:42.072Z level=INFO msg="All 2 paths already cached" worker1 # [ 89.459674] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=0 worker1 # [ 89.467911] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 89.478296] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 89.492981] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 89.511335] nix-grpc-daemon[863]: time=2026-09-22T22:21:42.150Z level=INFO msg="All 2 paths already cached" worker1 # [ 89.525279] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 89.543141] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 89.727631] nix-daemon[848]: accepted connection from pid 1219, user nix-grpc-daemon (trusted) worker1 # [ 89.752659] nix-daemon[848]: reaped child process 1222, status = succeeded lb # [ 89.748148] niks3-server[959]: 2026/09/22 22:21:42 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 89.771512] nix-grpc-daemon[863]: time=2026-09-22T22:21:42.410Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 89.773488] nix-grpc-daemon[863]: time=2026-09-22T22:21:42.410Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)" lb # [ 89.798610] niks3-server[959]: 2026/09/22 22:21:42 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls worker1 # [ 89.821847] nix-grpc-daemon[863]: time=2026-09-22T22:21:42.459Z level=INFO msg="Uploading 1 narinfos" lb # [ 89.814759] niks3-server[959]: 2026/09/22 22:21:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign lb # [ 89.822479] niks3-server[959]: 2026/09/22 22:21:42 INFO Signed narinfos id=25 count=1 lb # [ 89.827206] niks3-server[959]: 2026/09/22 22:21:42 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst worker1 # [ 89.846966] nix-grpc-daemon[863]: time=2026-09-22T22:21:42.486Z level=INFO msg="Upload complete. (281ms)" lb # [ 89.839572] niks3-server[959]: 2026/09/22 22:21:42 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete lb # [ 89.846218] niks3-server[959]: 2026/09/22 22:21:42 INFO Completed upload id=25 lb # [ 89.849636] niks3-server[959]: 2026/09/22 22:21:42 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo worker1 # [ 89.860914] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 89.873030] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 89.889537] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 89.897709] nix-daemon[848]: reaped child process 1217, status = succeeded worker1 # [ 89.914746] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) lb # [ 89.918510] envoy[895]: conn peer=[2001:db8:1::1]:35752 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3029 tx=790 ms=4313 lb # [ 89.923681] envoy[895]: conn peer=[2001:db8:1::1]:35762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1198 ms=5134 lb # [ 89.929227] envoy[895]: conn peer=[2001:db8:1::1]:53388 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3420 tx=1966 ms=1611 lb # [ 89.934277] envoy[895]: conn peer=[2001:db8:1::1]:53400 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=69 lb # [ 89.939172] envoy[895]: conn peer=[2001:db8:1::1]:53406 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3904 tx=2173 ms=2009 lb # [ 89.944396] envoy[895]: conn peer=[2001:db8:1::1]:53418 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=40 worker1 # [ 90.838568] nix-daemon[848]: reaped child process 1226, status = succeeded worker1 # [ 90.920663] nix-daemon[848]: accepted connection from pid 1242, user nix-grpc-daemon (trusted) worker1 # [ 90.945566] nix-daemon[848]: reaped child process 1245, status = succeeded lb # [ 90.942415] niks3-server[959]: 2026/09/22 22:21:43 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 90.968509] nix-grpc-daemon[863]: time=2026-09-22T22:21:43.605Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 90.971001] nix-grpc-daemon[863]: time=2026-09-22T22:21:43.606Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)" lb # [ 91.013088] niks3-server[959]: 2026/09/22 22:21:43 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls lb # [ 91.035169] niks3-server[959]: 2026/09/22 22:21:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign lb # [ 91.055989] niks3-server[959]: 2026/09/22 22:21:43 INFO Signed narinfos id=26 count=1 worker1 # [ 91.066159] nix-grpc-daemon[863]: time=2026-09-22T22:21:43.704Z level=INFO msg="Uploading 1 narinfos" lb # [ 91.067071] niks3-server[959]: 2026/09/22 22:21:43 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst lb # [ 91.076815] niks3-server[959]: 2026/09/22 22:21:43 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv lb # [ 91.092740] niks3-server[959]: 2026/09/22 22:21:43 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete lb # [ 91.101222] niks3-server[959]: 2026/09/22 22:21:43 INFO Completed upload id=26 worker1 # [ 91.113048] nix-grpc-daemon[863]: time=2026-09-22T22:21:43.751Z level=INFO msg="Upload complete. (382ms)" worker1 # [ 91.141828] nix-grpc-daemon[766]: ts=2026-09-22T22:21:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=1 worker1 # [ 91.172232] nix-daemon[848]: reaped child process 1224, status = succeeded lb # [ 91.733476] niks3-server[959]: 2026/09/22 22:21:43 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo worker2 # [ 91.955983] nix-grpc-daemon[769]: ts=2026-09-22T22:21:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 91.979955] nix-grpc-daemon[769]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'... worker2 # [ 92.057667] nix-grpc-daemon[769]: ts=2026-09-22T22:21:44Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 92.067662] nix-grpc-daemon[769]: ts=2026-09-22T22:21:44Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_out=232 client # [ 92.208758] nix-daemon[1147]: reaped child process 1149, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.36 seconds) client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.09 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.53 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.33 seconds) client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.13 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.21 seconds) worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 93.303328] nix-daemon[1147]: accepted connection from pid 1230, user root (trusted) client # this derivation will be built: client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 93.461661] nix-grpc-daemon[766]: ts=2026-09-22T22:21:46Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40012 worker2 # [ 93.466113] nix-grpc-daemon[769]: ts=2026-09-22T22:21:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53690 client # copying 1 paths... client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'... worker1 # [ 93.482990] nix-grpc-daemon[766]: ts=2026-09-22T22:21:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 93.500560] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 93.675180] nix-daemon[848]: accepted connection from pid 1254, user nix-grpc-daemon (trusted) worker1 # [ 93.698261] nix-daemon[848]: reaped child process 1257, status = succeeded lb # [ 93.693535] niks3-server[959]: 2026/09/22 22:21:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 93.714544] nix-grpc-daemon[863]: time=2026-09-22T22:21:46.353Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 93.716216] nix-grpc-daemon[863]: time=2026-09-22T22:21:46.353Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)" lb # [ 93.723843] niks3-server[959]: 2026/09/22 22:21:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign worker1 # [ 93.745600] nix-grpc-daemon[863]: time=2026-09-22T22:21:46.384Z level=INFO msg="Uploading 1 narinfos" lb # [ 93.734815] niks3-server[959]: 2026/09/22 22:21:46 INFO Signed narinfos id=27 count=1 lb # [ 93.747584] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls lb # [ 93.758080] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst worker1 # [ 93.780606] nix-grpc-daemon[863]: time=2026-09-22T22:21:46.419Z level=INFO msg="Upload complete. (263ms)" lb # [ 93.771748] niks3-server[959]: 2026/09/22 22:21:46 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete lb # [ 93.775767] niks3-server[959]: 2026/09/22 22:21:46 INFO Completed upload id=27 lb # [ 93.779935] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo worker1 # [ 93.792447] nix-grpc-daemon[766]: ts=2026-09-22T22:21:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=120 worker1 # [ 93.814627] nix-grpc-daemon[766]: ts=2026-09-22T22:21:46Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=0 worker1 # [ 93.819495] nix-grpc-daemon[766]: ts=2026-09-22T22:21:46Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=1 worker1 # [ 93.824795] nix-daemon[848]: reaped child process 1252, status = succeeded worker2 # [ 93.850840] nix-grpc-daemon[769]: ts=2026-09-22T22:21:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 93.867996] nix-grpc-daemon[769]: ts=2026-09-22T22:21:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53690 client # copying 2 paths to 'lb:50051'... client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'... worker2 # [ 93.887851] nix-grpc-daemon[769]: ts=2026-09-22T22:21:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'... worker2 # [ 93.910289] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 94.086599] nix-daemon[837]: accepted connection from pid 1073, user nix-grpc-daemon (trusted) worker2 # [ 94.108127] nix-daemon[837]: reaped child process 1076, status = succeeded lb # [ 94.108968] niks3-server[959]: 2026/09/22 22:21:46 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 94.118226] niks3-server[959]: 2026/09/22 22:21:46 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 94.136489] nix-grpc-daemon[865]: time=2026-09-22T22:21:46.777Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 94.138287] nix-grpc-daemon[865]: time=2026-09-22T22:21:46.777Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)" worker2 # [ 94.142325] nix-grpc-daemon[865]: time=2026-09-22T22:21:46.777Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)" lb # [ 94.160115] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst lb # [ 94.172181] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst lb # [ 94.186411] niks3-server[959]: 2026/09/22 22:21:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign worker2 # [ 94.199114] nix-grpc-daemon[865]: time=2026-09-22T22:21:46.840Z level=INFO msg="Uploading 3 narinfos" lb # [ 94.195920] niks3-server[959]: 2026/09/22 22:21:46 INFO Signed narinfos id=28 count=1 lb # [ 94.201636] niks3-server[959]: 2026/09/22 22:21:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign lb # [ 94.209150] niks3-server[959]: 2026/09/22 22:21:46 INFO Signed narinfos id=29 count=2 lb # [ 94.211035] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls lb # [ 94.219267] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls lb # [ 94.231965] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 94.236110] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 94.243575] niks3-server[959]: 2026/09/22 22:21:46 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 94.249356] niks3-server[959]: 2026/09/22 22:21:46 INFO Completed upload id=28 lb # [ 94.251115] niks3-server[959]: 2026/09/22 22:21:46 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete lb # [ 94.254688] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo worker2 # [ 94.260814] nix-grpc-daemon[865]: time=2026-09-22T22:21:46.903Z level=INFO msg="Upload complete. (325ms)" lb # [ 94.257826] niks3-server[959]: 2026/09/22 22:21:46 INFO Completed upload id=29 worker2 # [ 94.271543] nix-grpc-daemon[769]: ts=2026-09-22T22:21:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=2 nar_bytes_in=984 worker2 # [ 94.288312] nix-grpc-daemon[769]: ts=2026-09-22T22:21:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 94.302973] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 94.314232] nix-daemon[837]: reaped child process 1071, status = succeeded worker2 # [ 94.325264] nix-grpc-daemon[769]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'... client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051... client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker2 # [ 94.346546] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 95.608659] nix-daemon[837]: reaped child process 1081, status = succeeded worker2 # [ 95.722943] nix-daemon[837]: accepted connection from pid 1097, user nix-grpc-daemon (trusted) worker2 # [ 95.745822] nix-daemon[837]: reaped child process 1100, status = succeeded lb # [ 95.738402] niks3-server[959]: 2026/09/22 22:21:48 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 95.755927] nix-grpc-daemon[865]: time=2026-09-22T22:21:48.396Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 95.776847] niks3-server[959]: 2026/09/22 22:21:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign worker2 # [ 95.794556] nix-grpc-daemon[865]: time=2026-09-22T22:21:48.437Z level=INFO msg="Uploading 1 narinfos" lb # [ 95.791218] niks3-server[959]: 2026/09/22 22:21:48 INFO Signed narinfos id=30 count=1 lb # [ 95.799726] niks3-server[959]: 2026/09/22 22:21:48 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls lb # [ 95.809786] niks3-server[959]: 2026/09/22 22:21:48 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker2 # [ 95.828842] nix-grpc-daemon[865]: time=2026-09-22T22:21:48.471Z level=INFO msg="Upload complete. (267ms)" lb # [ 95.823930] niks3-server[959]: 2026/09/22 22:21:48 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 95.829234] niks3-server[959]: 2026/09/22 22:21:48 INFO Completed upload id=30 lb # [ 95.833339] niks3-server[959]: 2026/09/22 22:21:48 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo client # copying 1 paths... worker2 # [ 95.841916] nix-grpc-daemon[769]: ts=2026-09-22T22:21:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=1 client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'... worker2 # [ 95.864602] nix-grpc-daemon[769]: ts=2026-09-22T22:21:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 95.870543] nix-daemon[837]: reaped child process 1078, status = succeeded worker2 # [ 95.888493] nix-grpc-daemon[769]: ts=2026-09-22T22:21:48Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 95.895385] nix-grpc-daemon[769]: ts=2026-09-22T22:21:48Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_out=120 client # [ 96.028232] nix-daemon[1147]: reaped child process 1233, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.93 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 96.193540] nix-daemon[1147]: accepted connection from pid 1251, user root (trusted) client # this derivation will be built: client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 96.349758] nix-grpc-daemon[766]: ts=2026-09-22T22:21:48Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:40012 worker2 # [ 96.350305] nix-grpc-daemon[769]: ts=2026-09-22T22:21:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 96.362919] nix-grpc-daemon[865]: time=2026-09-22T22:21:49.005Z level=INFO msg="All 1 paths already cached" worker1 # [ 96.384823] nix-grpc-daemon[766]: ts=2026-09-22T22:21:49Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=0 worker1 # [ 96.392625] nix-grpc-daemon[766]: ts=2026-09-22T22:21:49Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 96.403421] nix-grpc-daemon[766]: ts=2026-09-22T22:21:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 96.415621] nix-grpc-daemon[766]: ts=2026-09-22T22:21:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40012 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'... worker1 # [ 96.438002] nix-grpc-daemon[766]: ts=2026-09-22T22:21:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 96.453735] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 96.475494] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5) worker1 # [ 96.517967] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 87 ms (attempt 2/5) worker1 # [ 96.607127] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 219 ms (attempt 3/5) worker1 # [ 96.828209] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 572 ms (attempt 4/5) worker1 # [ 97.403728] nix-grpc-daemon[766]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 97.410598] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 97.418710] nix-grpc-daemon[766]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'... worker1 # [ 97.593798] nix-daemon[848]: accepted connection from pid 1270, user nix-grpc-daemon (trusted) worker1 # [ 97.614351] nix-daemon[848]: reaped child process 1273, status = succeeded lb # [ 97.610196] niks3-server[959]: 2026/09/22 22:21:50 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 97.633334] nix-grpc-daemon[863]: time=2026-09-22T22:21:50.271Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 97.634977] nix-grpc-daemon[863]: time=2026-09-22T22:21:50.271Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)" lb # [ 97.648830] niks3-server[959]: 2026/09/22 22:21:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign worker1 # [ 97.665579] nix-grpc-daemon[863]: time=2026-09-22T22:21:50.304Z level=INFO msg="Uploading 1 narinfos" lb # [ 97.660903] niks3-server[959]: 2026/09/22 22:21:50 INFO Signed narinfos id=31 count=1 lb # [ 97.668358] niks3-server[959]: 2026/09/22 22:21:50 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst lb # [ 97.678619] niks3-server[959]: 2026/09/22 22:21:50 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls worker1 # [ 97.700333] nix-grpc-daemon[863]: time=2026-09-22T22:21:50.339Z level=INFO msg="Upload complete. (261ms)" lb # [ 97.690936] niks3-server[959]: 2026/09/22 22:21:50 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete lb # [ 97.697339] niks3-server[959]: 2026/09/22 22:21:50 INFO Completed upload id=31 lb # [ 97.704231] niks3-server[959]: 2026/09/22 22:21:50 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo worker1 # [ 97.711957] nix-grpc-daemon[766]: ts=2026-09-22T22:21:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 paths=1 nar_bytes_in=568 worker1 # [ 97.724480] nix-grpc-daemon[766]: ts=2026-09-22T22:21:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 97.737562] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) worker1 # [ 97.742903] nix-daemon[848]: reaped child process 1263, status = succeeded client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051... worker1 # [ 97.757898] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted) client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 99.426939] nix-daemon[848]: reaped child process 1277, status = succeeded worker1 # [ 99.548539] nix-daemon[848]: accepted connection from pid 1293, user nix-grpc-daemon (trusted) worker1 # [ 99.569477] nix-daemon[848]: reaped child process 1296, status = succeeded lb # [ 99.559632] niks3-server[959]: 2026/09/22 22:21:52 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 99.580723] nix-grpc-daemon[863]: time=2026-09-22T22:21:52.218Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 99.590685] niks3-server[959]: 2026/09/22 22:21:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign worker1 # [ 99.609541] nix-grpc-daemon[863]: time=2026-09-22T22:21:52.248Z level=INFO msg="Uploading 1 narinfos" lb # [ 99.605059] niks3-server[959]: 2026/09/22 22:21:52 INFO Signed narinfos id=32 count=1 lb # [ 99.615951] niks3-server[959]: 2026/09/22 22:21:52 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls lb # [ 99.625227] niks3-server[959]: 2026/09/22 22:21:52 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv lb # [ 99.637202] niks3-server[959]: 2026/09/22 22:21:52 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 99.643699] niks3-server[959]: 2026/09/22 22:21:52 INFO Completed upload id=32 worker1 # [ 99.652799] nix-grpc-daemon[863]: time=2026-09-22T22:21:52.292Z level=INFO msg="Upload complete. (282ms)" lb # [ 99.653186] niks3-server[959]: 2026/09/22 22:21:52 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo worker1 # [ 99.665280] nix-grpc-daemon[766]: ts=2026-09-22T22:21:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=1 worker1 # [ 99.688565] nix-daemon[848]: reaped child process 1275, status = succeeded client # copying 1 paths... worker2 # [ 99.709928] nix-grpc-daemon[769]: ts=2026-09-22T22:21:52Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 99.723582] nix-grpc-daemon[769]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'... client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'... worker1 # [ 99.771796] nix-grpc-daemon[766]: ts=2026-09-22T22:21:52Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 99.779537] nix-grpc-daemon[766]: ts=2026-09-22T22:21:52Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_out=120 client # [ 99.859806] nix-daemon[1147]: reaped child process 1254, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.83 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2 lb # [ 99.919961] envoy[895]: conn peer=[2001:db8:1::1]:53434 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4343 tx=2431 ms=2686 lb # [ 99.925028] envoy[895]: conn peer=[2001:db8:1::1]:40832 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=407 ms=48 lb # [ 99.929939] envoy[895]: conn peer=[2001:db8:1::1]:40836 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4262 tx=2073 ms=2502 lb # [ 99.935412] envoy[895]: conn peer=[2001:db8:1::1]:40844 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=74 lb # [ 99.941213] envoy[895]: conn peer=[2001:db8:1::1]:40854 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3757 tx=2122 ms=3448 lb # [ 99.946522] envoy[895]: conn peer=[2001:db8:1::1]:40864 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=27 client # [ 100.022388] nix-daemon[1147]: accepted connection from pid 1273, user root (trusted) client # this derivation will be built: client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker2 # [ 100.173441] nix-grpc-daemon[769]: ts=2026-09-22T22:21:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:53690 client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 100.181219] nix-grpc-daemon[769]: ts=2026-09-22T22:21:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 100.189055] nix-grpc-daemon[865]: time=2026-09-22T22:21:52.829Z level=INFO msg="All 1 paths already cached" worker1 # [ 100.204205] nix-grpc-daemon[766]: ts=2026-09-22T22:21:52Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=0 worker1 # [ 100.210901] nix-grpc-daemon[766]: ts=2026-09-22T22:21:52Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 100.210470] nix-grpc-daemon[769]: ts=2026-09-22T22:21:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 100.227074] nix-grpc-daemon[769]: ts=2026-09-22T22:21:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 100.239759] nix-grpc-daemon[865]: time=2026-09-22T22:21:52.882Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'... worker2 # [ 100.250854] nix-grpc-daemon[769]: ts=2026-09-22T22:21:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 100.266473] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 100.424414] nix-daemon[837]: accepted connection from pid 1105, user nix-grpc-daemon (trusted) worker2 # [ 100.448212] nix-daemon[837]: reaped child process 1108, status = succeeded lb # [ 100.440306] niks3-server[959]: 2026/09/22 22:21:53 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 100.457520] nix-grpc-daemon[865]: time=2026-09-22T22:21:53.099Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 100.459075] nix-grpc-daemon[865]: time=2026-09-22T22:21:53.099Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)" lb # [ 100.472212] niks3-server[959]: 2026/09/22 22:21:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign worker2 # [ 100.484228] nix-grpc-daemon[865]: time=2026-09-22T22:21:53.126Z level=INFO msg="Uploading 1 narinfos" lb # [ 100.484269] niks3-server[959]: 2026/09/22 22:21:53 INFO Signed narinfos id=33 count=1 lb # [ 100.487453] niks3-server[959]: 2026/09/22 22:21:53 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls lb # [ 100.504110] niks3-server[959]: 2026/09/22 22:21:53 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete worker2 # [ 100.516689] nix-grpc-daemon[865]: time=2026-09-22T22:21:53.159Z level=INFO msg="Upload complete. (233ms)" lb # [ 100.507975] niks3-server[959]: 2026/09/22 22:21:53 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst lb # [ 100.519735] niks3-server[959]: 2026/09/22 22:21:53 INFO Completed upload id=33 worker2 # [ 100.524150] nix-grpc-daemon[769]: ts=2026-09-22T22:21:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_in=568 lb # [ 100.525149] niks3-server[959]: 2026/09/22 22:21:53 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo worker2 # [ 100.536357] nix-grpc-daemon[769]: ts=2026-09-22T22:21:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 100.547290] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) worker2 # [ 100.552314] nix-daemon[837]: reaped child process 1103, status = succeeded client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051... client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker2 # [ 100.564988] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted) client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 101.636972] nix-daemon[837]: reaped child process 1112, status = succeeded worker2 # [ 101.742695] nix-daemon[837]: accepted connection from pid 1128, user nix-grpc-daemon (trusted) worker2 # [ 101.762589] nix-daemon[837]: reaped child process 1131, status = succeeded lb # [ 101.762259] niks3-server[959]: 2026/09/22 22:21:54 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 101.782767] nix-grpc-daemon[865]: time=2026-09-22T22:21:54.422Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 101.797194] niks3-server[959]: 2026/09/22 22:21:54 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls worker2 # [ 101.815330] nix-grpc-daemon[865]: time=2026-09-22T22:21:54.457Z level=INFO msg="Uploading 1 narinfos" lb # [ 101.809254] niks3-server[959]: 2026/09/22 22:21:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign lb # [ 101.818671] niks3-server[959]: 2026/09/22 22:21:54 INFO Signed narinfos id=34 count=1 lb # [ 101.824562] niks3-server[959]: 2026/09/22 22:21:54 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv lb # [ 101.832665] niks3-server[959]: 2026/09/22 22:21:54 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 101.839133] niks3-server[959]: 2026/09/22 22:21:54 INFO Completed upload id=34 worker2 # [ 101.844778] nix-grpc-daemon[865]: time=2026-09-22T22:21:54.487Z level=INFO msg="Upload complete. (270ms)" lb # [ 101.842209] niks3-server[959]: 2026/09/22 22:21:54 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo client # copying 1 paths... client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'... worker2 # [ 101.857188] nix-grpc-daemon[769]: ts=2026-09-22T22:21:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=1 worker2 # [ 101.876490] nix-grpc-daemon[769]: ts=2026-09-22T22:21:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:53690 worker2 # [ 101.882112] nix-daemon[837]: reaped child process 1110, status = succeeded worker1 # [ 101.897848] nix-grpc-daemon[766]: ts=2026-09-22T22:21:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 101.908307] nix-grpc-daemon[766]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'... worker1 # [ 101.919261] nix-grpc-daemon[766]: ts=2026-09-22T22:21:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_out=120 client # [ 102.040144] nix-daemon[1147]: reaped child process 1276, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.18 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 9.65 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.34 seconds) client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.29 seconds) client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds) worker2 # [ 103.063221] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 103.733965] nix-grpc-daemon[769]: ts=2026-09-22T22:21:56Z level=info event=signal_exit worker2 # [ 104.866425] nix-grpc-daemon[769]: ts=2026-09-22T22:21:57Z level=info event=scheduler_restarting worker1 # [ 104.870179] nix-grpc-daemon[766]: ts=2026-09-22T22:21:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:40002 worker1 # [ 104.875295] nix-grpc-daemon[766]: ts=2026-09-22T22:21:57Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 104.893161] nix-grpc-daemon[769]: ts=2026-09-22T22:21:57Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 104.913167] nix-daemon[837]: reaped child process 839, status = succeeded worker2 # [ 104.966372] nix-grpc-daemon[769]: E0000 00:00:1790115717.608652 814 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker2 # [ 104.973043] nix-grpc-daemon[769]: E0000 00:00:1790115717.608976 814 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 106.057340] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 106.062500] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 106.069437] systemd[1]: nix-grpc-daemon.service: Consumed 3.147s CPU time over 1min 25.923s wall clock time, 60.2M memory peak, 81.4K incoming IP traffic, 186.6K outgoing IP traffic. worker2 # [ 106.085039] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 106.090442] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 106.092964] systemd[1]: nix-grpc-daemon.socket: Consumed 661us CPU time over 1min 35.292s wall clock time, 256K memory peak, 93.7K incoming IP traffic, 84.1K outgoing IP traffic. lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2 worker1 # [ 106.410506] nix-grpc-daemon[766]: ts=2026-09-22T22:21:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40012 worker1 # [ 106.419536] nix-grpc-daemon[766]: ts=2026-09-22T22:21:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:40012 client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.27 seconds) worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds) worker2 # [ 106.562163] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 106.645660] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 107.056318] systemd[1]: Starting scheduler-token.service... worker2 # [ 107.146703] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 107.157018] systemd[1]: Finished scheduler-token.service. worker2 # [ 107.159002] systemd[1]: scheduler-token.service: Consumed 61ms CPU time over 95ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 107.166117] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 107.298583] nix-grpc-daemon[1155]: ts=2026-09-22T22:21:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 107.300927] nix-grpc-daemon[1155]: ts=2026-09-22T22:21:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 107.304787] nix-grpc-daemon[1155]: ts=2026-09-22T22:21:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 107.311937] nix-grpc-daemon[1155]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 107.323617] nix-grpc-daemon[1155]: E0000 00:00:1790115719.952161 1155 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 107.356102] nix-grpc-daemon[1155]: E0000 00:00:1790115719.954444 1155 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 107.358476] nix-grpc-daemon[1155]: ts=2026-09-22T22:21:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 107.366857] nix-grpc-daemon[1155]: E0000 00:00:1790115719.974366 1195 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 107.369821] nix-grpc-daemon[1155]: E0000 00:00:1790115719.989582 1195 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 107.372332] nix-grpc-daemon[1155]: E0000 00:00:1790115719.989741 1195 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 107.377490] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 107.380299] nix-grpc-daemon[1155]: E0000 00:00:1790115720.008681 1195 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 107.382854] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 107.387440] nix-grpc-daemon[766]: ts=2026-09-22T22:22:00Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:40002 worker1 # [ 107.393545] nix-grpc-daemon[766]: ts=2026-09-22T22:22:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 107.549854] nix-grpc-daemon[766]: ts=2026-09-22T22:22:00Z level=info event=signal_exit lb # [ 107.857441] niks3-server[959]: 2026/09/22 22:22:00 INFO lead: released remote=192.168.1.3:41082 worker2 # [ 108.862753] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:01Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 108.865351] nix-grpc-daemon[766]: ts=2026-09-22T22:22:01Z level=info event=scheduler_restarting worker1 # [ 108.879600] nix-grpc-daemon[766]: ts=2026-09-22T22:22:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:40002 worker1 # [ 108.884680] nix-grpc-daemon[766]: ts=2026-09-22T22:22:01Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 108.889801] nix-grpc-daemon[766]: ts=2026-09-22T22:22:01Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 108.894610] nix-grpc-daemon[766]: ts=2026-09-22T22:22:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:40002 worker1 # [ 108.902149] nix-grpc-daemon[766]: ts=2026-09-22T22:22:01Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 108.928672] nix-daemon[848]: reaped child process 850, status = succeeded worker1 # [ 108.986921] nix-grpc-daemon[766]: E0000 00:00:1790115721.625898 811 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true worker1 # [ 108.993878] nix-grpc-daemon[766]: E0000 00:00:1790115721.626229 811 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker2 # [ 109.063811] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 109.069367] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 109.104678] nix-grpc-daemon[766]: ts=2026-09-22T22:22:01Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 109.270420] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 109.275800] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 109.477008] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 109.482253] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 109.683610] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 109.689045] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 109.892156] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 109.896309] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 109.919171] envoy[895]: conn peer=[2001:db8:1::1]:40866 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3745 tx=2101 ms=1801 lb # [ 109.924424] envoy[895]: conn peer=[2001:db8:1::1]:33102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=83 lb # [ 109.929279] envoy[895]: conn peer=[2001:db8:1::4]:38654 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=54325 tx=3171 ms=84318 lb # [ 109.934447] envoy[895]: conn peer=[2001:db8:1::1]:33106 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=29 lb # [ 109.939166] envoy[895]: conn peer=[2001:db8:1::3]:52148 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=10708 tx=3202 ms=87683 worker1 # [ 109.982994] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 109.985818] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 109.990331] systemd[1]: nix-grpc-daemon.service: Consumed 4.840s CPU time over 1min 30.054s wall clock time, 70.6M memory peak, 116.4K incoming IP traffic, 230.7K outgoing IP traffic. worker1 # [ 109.998459] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 110.000478] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 110.002720] systemd[1]: nix-grpc-daemon.socket: Consumed 643us CPU time over 1min 39.241s wall clock time, 256K memory peak, 129.1K incoming IP traffic, 110.5K outgoing IP traffic. lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds) worker2 # [ 110.092833] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_connected addr=lb:50051 client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2 worker2 # [ 110.095124] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 110.296778] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 110.302194] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 110.329013] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 110.346816] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) worker2 # [ 110.374404] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5) worker2 # [ 110.408466] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 182 ms (attempt 2/5) worker2 # [ 110.503512] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 110.509242] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 110.593138] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 196 ms (attempt 3/5) worker2 # [ 110.710943] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 110.716682] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 110.792154] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 680 ms (attempt 4/5) worker2 # [ 110.919808] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 110.924256] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 111.121180] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 111.126776] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 111.327870] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 111.334122] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 111.477365] nix-grpc-daemon[1155]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 111.483188] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 111.511027] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 111.530204] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) worker2 # [ 111.535802] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 111.542673] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 111.580459] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=niks3_push_started pid=1215 worker2 # [ 111.726726] nix-grpc-daemon[1215]: time=2026-09-22T22:22:04.368Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 111.742887] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 111.745727] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 111.841336] nix-daemon[837]: accepted connection from pid 1228, user nix-grpc-daemon (trusted) worker2 # [ 111.861625] nix-daemon[837]: reaped child process 1231, status = succeeded lb # [ 111.861095] niks3-server[959]: 2026/09/22 22:22:04 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 111.868764] niks3-server[959]: 2026/09/22 22:22:04 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 111.884063] nix-grpc-daemon[1215]: time=2026-09-22T22:22:04.524Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 111.885635] nix-grpc-daemon[1215]: time=2026-09-22T22:22:04.524Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)" worker2 # [ 111.887373] nix-grpc-daemon[1215]: time=2026-09-22T22:22:04.524Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)" lb # [ 111.908992] niks3-server[959]: 2026/09/22 22:22:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign worker2 # [ 111.923436] nix-grpc-daemon[1215]: time=2026-09-22T22:22:04.564Z level=INFO msg="Uploading 3 narinfos" lb # [ 111.919200] niks3-server[959]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst lb # [ 111.927244] niks3-server[959]: 2026/09/22 22:22:04 INFO Signed narinfos id=35 count=1 lb # [ 111.932561] niks3-server[959]: 2026/09/22 22:22:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 111.934538] niks3-server[959]: 2026/09/22 22:22:04 INFO Signed narinfos id=36 count=2 lb # [ 111.935709] niks3-server[959]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls worker2 # [ 111.946532] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 111.949581] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 111.945850] niks3-server[959]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls lb # [ 111.960901] niks3-server[959]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo lb # [ 111.966544] niks3-server[959]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 111.973050] niks3-server[959]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 111.974762] niks3-server[959]: 2026/09/22 22:22:04 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 111.978719] niks3-server[959]: 2026/09/22 22:22:04 INFO Completed upload id=35 lb # [ 111.980807] niks3-server[959]: 2026/09/22 22:22:04 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete lb # [ 111.983779] niks3-server[959]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 111.986170] niks3-server[959]: 2026/09/22 22:22:04 INFO Completed upload id=36 worker2 # [ 111.992742] nix-grpc-daemon[1215]: time=2026-09-22T22:22:04.635Z level=INFO msg="Upload complete. (262ms)" worker2 # [ 112.004502] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 112.020480] nix-daemon[837]: reaped child process 1213, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.93 seconds) worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds) worker1 # [ 112.135309] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 112.145177] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 112.151233] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 112.156787] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 112.299852] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 112.302425] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 112.306819] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 112.312525] nix-grpc-daemon[1322]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 112.314024] nix-grpc-daemon[1322]: E0000 00:00:1790115724.951602 1322 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 112.317516] niks3-server[959]: 2026/09/22 22:22:04 INFO lead: acquired remote=192.168.1.3:48508 lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 112.360220] nix-grpc-daemon[1322]: E0000 00:00:1790115724.957852 1322 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 112.362776] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 112.357844] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 112.366123] nix-grpc-daemon[1322]: E0000 00:00:1790115724.965805 1364 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 112.369243] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:04Z level=info event=scheduler_take_over worker2 # [ 112.363437] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 112.370648] nix-grpc-daemon[1322]: E0000 00:00:1790115724.982875 1364 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 112.373007] nix-grpc-daemon[1322]: E0000 00:00:1790115724.983083 1364 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 112.375400] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 112.376253] nix-grpc-daemon[1322]: E0000 00:00:1790115725.004442 1364 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 112.385415] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 112.387440] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker2 # [ 112.564842] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 112.570315] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 112.588852] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 112.594586] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 112.771646] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 112.777203] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 112.795104] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 112.799280] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 112.978649] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 112.981609] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 113.000972] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 113.010670] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:32804 worker1 # [ 113.015766] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 113.183197] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 113.194019] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 113.195478] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:32804 worker1 # [ 113.197581] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.44 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb # [ 113.610111] niks3-server[959]: 2026/09/22 22:22:06 INFO Shutdown signal received, draining in-flight requests timeout=10s lb # [ 113.610655] systemd[1]: Stopping niks3 server... worker2 # [ 113.617068] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:06Z level=info event=scheduler_lock_lost status=200 lb # [ 113.616989] niks3-server[959]: 2026/09/22 22:22:06 INFO lead: released remote=192.168.1.3:48508 worker1 # [ 113.624777] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:06Z level=info event=scheduler_lock_lost status=200 lb # [ 113.638696] systemd[1]: niks3.service: Deactivated successfully. lb # [ 113.640708] systemd[1]: Stopped niks3 server. lb # [ 113.642308] systemd[1]: niks3.service: Consumed 1.004s CPU time over 1min 30.303s wall clock time, 28M memory peak, 98.2K incoming IP traffic, 141.6K outgoing IP traffic. lb # [ 113.653063] systemd[1]: Starting niks3 server... lb # [ 113.723174] niks3-server[1100]: 2026/09/22 22:22:06 goose: no migrations to run. current version: 20260920000000 lb # [ 113.728779] niks3-server[1100]: 2026/09/22 22:22:06 OK 1_commit_pending_closure.sql (7.18ms) lb # [ 113.732575] niks3-server[1100]: 2026/09/22 22:22:06 OK 2_object_stats_trigger.sql (3.43ms) lb # [ 113.733762] niks3-server[1100]: 2026/09/22 22:22:06 goose: up to current file version: 2 lb # [ 113.735086] niks3-server[1100]: 2026/09/22 22:22:06 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 113.747316] niks3-server[1100]: 2026/09/22 22:22:06 INFO Read proxy enabled — serving cache objects from S3 lb # [ 113.753245] niks3-server[1100]: 2026/09/22 22:22:06 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 113.759441] niks3-server[1100]: 2026/09/22 22:22:06 INFO systemd watchdog enabled interval=15s lb: waiting for unit niks3.service lb # [ 113.762123] niks3-server[1100]: 2026/09/22 22:22:06 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 113.767187] systemd[1]: Started niks3 server. lb: (finished: waiting for unit niks3.service, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.08 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2 worker1 # [ 114.300090] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:32814 worker1 # [ 114.316285] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted) client # don't know how to build these paths: client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker2 # [ 114.343841] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57962 worker1 # [ 114.388425] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:07Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=0 worker1 # [ 114.395875] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:07Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 114.445161] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32814 worker1 # [ 114.465746] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 1/5) worker1 # [ 114.489361] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 2/5) worker1 # [ 114.553731] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 242 ms (attempt 3/5) lb # [ 114.621227] niks3-server[1100]: 2026/09/22 22:22:07 INFO lead: acquired remote=192.168.1.3:48516 worker1 # [ 114.799279] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 521 ms (attempt 4/5) worker1 # [ 115.323023] nix-grpc-daemon[1322]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 115.331353] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 115.352150] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:32814 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'... worker1 # [ 115.369822] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32814 worker1 # [ 115.386233] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted) worker1 # [ 115.418414] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:08Z level=info event=niks3_push_started pid=1394 worker1 # [ 115.566246] nix-grpc-daemon[1394]: time=2026-09-22T22:22:08.205Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 115.716398] nix-daemon[848]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker1 # [ 115.738667] nix-daemon[848]: reaped child process 1410, status = succeeded lb # [ 115.732848] niks3-server[1100]: 2026/09/22 22:22:08 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 115.760084] nix-grpc-daemon[1394]: time=2026-09-22T22:22:08.398Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 115.762032] nix-grpc-daemon[1394]: time=2026-09-22T22:22:08.398Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)" lb # [ 115.783223] niks3-server[1100]: 2026/09/22 22:22:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign lb # [ 115.832100] niks3-server[1100]: 2026/09/22 22:22:08 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls lb # [ 115.841681] niks3-server[1100]: 2026/09/22 22:22:08 INFO Signed narinfos id=37 count=1 worker1 # [ 115.856211] nix-grpc-daemon[1394]: time=2026-09-22T22:22:08.493Z level=INFO msg="Uploading 1 narinfos" lb # [ 115.852968] niks3-server[1100]: 2026/09/22 22:22:08 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst lb # [ 115.861547] niks3-server[1100]: 2026/09/22 22:22:08 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete lb # [ 115.869151] niks3-server[1100]: 2026/09/22 22:22:08 INFO Completed upload id=37 worker1 # [ 115.879441] nix-grpc-daemon[1394]: time=2026-09-22T22:22:08.518Z level=INFO msg="Upload complete. (309ms)" lb # [ 115.871602] niks3-server[1100]: 2026/09/22 22:22:08 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo worker1 # [ 115.892727] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32814 duration_s=0 paths=1 nar_bytes_in=544 worker1 # [ 115.912625] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32814 worker1 # [ 115.921160] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted) worker1 # [ 115.927214] nix-daemon[848]: reaped child process 1392, status = succeeded client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051... worker1 # [ 115.939552] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted) client # farm-a> node-a: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker1 # [ 116.828728] nix-daemon[848]: reaped child process 1414, status = succeeded worker1 # [ 116.923460] nix-daemon[848]: accepted connection from pid 1430, user nix-grpc-daemon (trusted) worker1 # [ 116.943977] nix-daemon[848]: reaped child process 1433, status = succeeded lb # [ 116.938843] niks3-server[1100]: 2026/09/22 22:22:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 116.964511] nix-grpc-daemon[1394]: time=2026-09-22T22:22:09.601Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 116.967825] nix-grpc-daemon[1394]: time=2026-09-22T22:22:09.601Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)" lb # [ 116.980070] niks3-server[1100]: 2026/09/22 22:22:09 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv lb # [ 116.994927] niks3-server[1100]: 2026/09/22 22:22:09 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls lb # [ 117.004121] niks3-server[1100]: 2026/09/22 22:22:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign worker1 # [ 117.020711] nix-grpc-daemon[1394]: time=2026-09-22T22:22:09.659Z level=INFO msg="Uploading 1 narinfos" lb # [ 117.014800] niks3-server[1100]: 2026/09/22 22:22:09 INFO Signed narinfos id=38 count=1 lb # [ 117.019615] niks3-server[1100]: 2026/09/22 22:22:09 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst lb # [ 117.028203] niks3-server[1100]: 2026/09/22 22:22:09 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete lb # [ 117.033868] niks3-server[1100]: 2026/09/22 22:22:09 INFO Completed upload id=38 worker1 # [ 117.044130] nix-grpc-daemon[1394]: time=2026-09-22T22:22:09.683Z level=INFO msg="Upload complete. (290ms)" lb # [ 117.036380] niks3-server[1100]: 2026/09/22 22:22:09 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo worker1 # [ 117.069285] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32814 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 worker1 # [ 117.085182] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:09Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=0 worker1 # [ 117.088778] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:09Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 117.092785] nix-daemon[848]: reaped child process 1412, status = succeeded worker1 # [ 117.110672] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32814 worker1 # [ 117.126169] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:32814 worker1 # [ 117.143953] nix-grpc-daemon[1394]: time=2026-09-22T22:22:09.783Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'... worker1 # [ 117.156633] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32814 worker1 # [ 117.171285] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted) worker1 # [ 117.327579] nix-daemon[848]: accepted connection from pid 1439, user nix-grpc-daemon (trusted) worker1 # [ 117.350040] nix-daemon[848]: reaped child process 1442, status = succeeded lb # [ 117.345395] niks3-server[1100]: 2026/09/22 22:22:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 117.367588] nix-grpc-daemon[1394]: time=2026-09-22T22:22:10.006Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 117.369801] nix-grpc-daemon[1394]: time=2026-09-22T22:22:10.006Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)" lb # [ 117.380941] niks3-server[1100]: 2026/09/22 22:22:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign worker1 # [ 117.397837] nix-grpc-daemon[1394]: time=2026-09-22T22:22:10.036Z level=INFO msg="Uploading 1 narinfos" lb # [ 117.396740] niks3-server[1100]: 2026/09/22 22:22:10 INFO Signed narinfos id=39 count=1 lb # [ 117.399565] niks3-server[1100]: 2026/09/22 22:22:10 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls lb # [ 117.417180] niks3-server[1100]: 2026/09/22 22:22:10 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete worker1 # [ 117.434992] nix-grpc-daemon[1394]: time=2026-09-22T22:22:10.074Z level=INFO msg="Upload complete. (246ms)" lb # [ 117.426934] niks3-server[1100]: 2026/09/22 22:22:10 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst lb # [ 117.432936] niks3-server[1100]: 2026/09/22 22:22:10 INFO Completed upload id=39 worker1 # [ 117.444390] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32814 duration_s=0 paths=1 nar_bytes_in=656 lb # [ 117.437325] niks3-server[1100]: 2026/09/22 22:22:10 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo worker1 # [ 117.464964] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32814 worker1 # [ 117.472963] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted) worker1 # [ 117.479172] nix-daemon[848]: reaped child process 1437, status = succeeded client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051... worker1 # [ 117.493600] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted) client # farm-b> node-a: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker1 # [ 118.728238] nix-daemon[848]: reaped child process 1446, status = succeeded worker1 # [ 118.833421] nix-daemon[848]: accepted connection from pid 1462, user nix-grpc-daemon (trusted) worker1 # [ 118.853298] nix-daemon[848]: reaped child process 1465, status = succeeded lb # [ 118.851166] niks3-server[1100]: 2026/09/22 22:22:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 118.873861] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.512Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 118.875394] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.512Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)" lb # [ 118.896826] niks3-server[1100]: 2026/09/22 22:22:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign worker1 # [ 118.914662] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.553Z level=INFO msg="Uploading 1 narinfos" lb # [ 118.913230] niks3-server[1100]: 2026/09/22 22:22:11 INFO Signed narinfos id=40 count=1 lb # [ 118.929631] niks3-server[1100]: 2026/09/22 22:22:11 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst lb # [ 118.940154] niks3-server[1100]: 2026/09/22 22:22:11 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv lb # [ 118.952838] niks3-server[1100]: 2026/09/22 22:22:11 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls worker1 # [ 118.968296] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.607Z level=INFO msg="Upload complete. (259ms)" lb # [ 118.958996] niks3-server[1100]: 2026/09/22 22:22:11 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete lb # [ 118.963838] niks3-server[1100]: 2026/09/22 22:22:11 INFO Completed upload id=40 lb # [ 118.968746] niks3-server[1100]: 2026/09/22 22:22:11 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo worker1 # [ 118.981375] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32814 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 worker1 # [ 119.006013] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=0 worker1 # [ 119.013852] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 119.018656] nix-daemon[848]: reaped child process 1444, status = succeeded worker1 # [ 119.051466] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32814 worker1 # [ 119.067589] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:32814 worker1 # [ 119.083951] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.723Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'... worker1 # [ 119.097619] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32814 worker1 # [ 119.112533] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted) worker1 # [ 119.274497] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted) worker1 # [ 119.294098] nix-daemon[848]: reaped child process 1474, status = succeeded lb # [ 119.289921] niks3-server[1100]: 2026/09/22 22:22:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 119.313367] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.952Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 119.314950] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.952Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)" lb # [ 119.317370] niks3-server[1100]: 2026/09/22 22:22:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign worker1 # [ 119.330867] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.969Z level=INFO msg="Uploading 1 narinfos" lb # [ 119.322920] niks3-server[1100]: 2026/09/22 22:22:11 INFO Signed narinfos id=41 count=1 lb # [ 119.325965] niks3-server[1100]: 2026/09/22 22:22:11 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls lb # [ 119.327551] niks3-server[1100]: 2026/09/22 22:22:11 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst lb # [ 119.334038] niks3-server[1100]: 2026/09/22 22:22:11 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete lb # [ 119.339332] niks3-server[1100]: 2026/09/22 22:22:11 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo lb # [ 119.342707] niks3-server[1100]: 2026/09/22 22:22:11 INFO Completed upload id=41 worker1 # [ 119.352617] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.991Z level=INFO msg="Upload complete. (222ms)" worker1 # [ 119.360553] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32814 duration_s=0 paths=1 nar_bytes_in=784 worker1 # [ 119.371004] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32814 worker1 # [ 119.376922] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted) worker1 # [ 119.381506] nix-daemon[848]: reaped child process 1469, status = succeeded client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051... worker1 # [ 119.393824] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted) client # farm-top> node-a: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv lb # [ 119.919366] envoy[895]: conn peer=[2001:db8:1::1]:33112 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1010 tx=305 ms=1688 client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker1 # [ 121.062904] nix-daemon[848]: reaped child process 1478, status = succeeded worker1 # [ 121.180632] nix-daemon[848]: accepted connection from pid 1494, user nix-grpc-daemon (trusted) worker1 # [ 121.203181] nix-daemon[848]: reaped child process 1497, status = succeeded lb # [ 121.199423] niks3-server[1100]: 2026/09/22 22:22:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 121.220472] nix-grpc-daemon[1394]: time=2026-09-22T22:22:13.858Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 121.222301] nix-grpc-daemon[1394]: time=2026-09-22T22:22:13.859Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)" lb # [ 121.241795] niks3-server[1100]: 2026/09/22 22:22:13 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls lb # [ 121.256233] niks3-server[1100]: 2026/09/22 22:22:13 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst worker1 # [ 121.279703] nix-grpc-daemon[1394]: time=2026-09-22T22:22:13.918Z level=INFO msg="Uploading 1 narinfos" lb # [ 121.264523] niks3-server[1100]: 2026/09/22 22:22:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign lb # [ 121.274583] niks3-server[1100]: 2026/09/22 22:22:13 INFO Signed narinfos id=42 count=1 lb # [ 121.280357] niks3-server[1100]: 2026/09/22 22:22:13 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv lb # [ 121.291826] niks3-server[1100]: 2026/09/22 22:22:13 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 121.298996] niks3-server[1100]: 2026/09/22 22:22:13 INFO Completed upload id=42 worker1 # [ 121.308927] nix-grpc-daemon[1394]: time=2026-09-22T22:22:13.948Z level=INFO msg="Upload complete. (303ms)" lb # [ 121.302596] niks3-server[1100]: 2026/09/22 22:22:13 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo worker1 # [ 121.321137] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32814 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 worker1 # [ 121.338614] nix-daemon[848]: reaped child process 1476, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 7.30 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.19 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 8.06 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1 # [ 121.817962] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 122.332926] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:14Z level=info event=signal_exit lb # [ 122.629287] niks3-server[1100]: 2026/09/22 22:22:15 INFO lead: released remote=192.168.1.3:48516 worker2 # [ 123.636951] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:16Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 123.643257] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:16Z level=info event=scheduler_restarting worker1 # [ 123.650420] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:32804 worker1 # [ 123.656326] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:16Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 123.660866] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:16Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 123.666264] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:32804 worker1 # [ 123.672448] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:16Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 123.693846] nix-daemon[848]: reaped child process 1382, status = succeeded worker2 # [ 123.837046] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 123.842866] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 123.863634] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:16Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 123.950772] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 123.958696] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 123.965102] systemd[1]: nix-grpc-daemon.service: Consumed 1.291s CPU time over 11.809s wall clock time, 38M memory peak, 36.9K incoming IP traffic, 59K outgoing IP traffic. worker1 # [ 123.979653] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 123.986382] systemd[1]: Closed Nix worker-protocol over gRPC. worker1 # [ 123.990847] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 11.858s wall clock time, 256K memory peak, 32.1K incoming IP traffic, 25.4K outgoing IP traffic. worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker2 # [ 124.043351] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.045838] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 124.247138] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:16Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.252125] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 124.453426] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.458802] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 124.628311] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_take_over lb # [ 124.621140] niks3-server[1100]: 2026/09/22 22:22:17 INFO lead: acquired remote=192.168.1.4:37182 worker2 # [ 124.661763] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.665371] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 124.863285] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.868886] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 125.069408] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 125.073257] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.09 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds) worker2 # [ 125.273005] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 125.275176] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 125.476497] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:18Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 125.504204] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:18Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 125.510130] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:18Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:33102 worker2 # [ 125.515609] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:18Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2 worker2 # [ 126.506366] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:57962 client # don't know how to build these paths: client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker2 # [ 126.520506] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 126.560986] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=0 worker2 # [ 126.570942] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 126.618003] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 126.644531] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57962 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'... worker2 # [ 126.669725] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 126.694747] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) worker2 # [ 126.922138] nix-daemon[837]: accepted connection from pid 1260, user nix-grpc-daemon (trusted) worker2 # [ 126.944955] nix-daemon[837]: reaped child process 1263, status = succeeded lb # [ 126.952823] niks3-server[1100]: 2026/09/22 22:22:19 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 126.978656] nix-grpc-daemon[1215]: time=2026-09-22T22:22:19.619Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 126.980461] nix-grpc-daemon[1215]: time=2026-09-22T22:22:19.619Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)" lb # [ 127.033151] niks3-server[1100]: 2026/09/22 22:22:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign lb # [ 127.047794] niks3-server[1100]: 2026/09/22 22:22:19 INFO Signed narinfos id=43 count=1 worker2 # [ 127.056272] nix-grpc-daemon[1215]: time=2026-09-22T22:22:19.696Z level=INFO msg="Uploading 1 narinfos" lb # [ 127.073174] niks3-server[1100]: 2026/09/22 22:22:19 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst lb # [ 127.092938] niks3-server[1100]: 2026/09/22 22:22:19 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls lb # [ 127.106447] niks3-server[1100]: 2026/09/22 22:22:19 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete lb # [ 127.118801] niks3-server[1100]: 2026/09/22 22:22:19 INFO Completed upload id=43 worker2 # [ 127.125634] nix-grpc-daemon[1215]: time=2026-09-22T22:22:19.768Z level=INFO msg="Upload complete. (405ms)" lb # [ 127.122633] niks3-server[1100]: 2026/09/22 22:22:19 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo worker2 # [ 127.141101] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=0 paths=1 nar_bytes_in=520 worker2 # [ 127.156801] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 127.174738] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) worker2 # [ 127.184321] nix-daemon[837]: reaped child process 1258, status = succeeded client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051... client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker2 # [ 127.203533] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker2 # [ 128.860654] nix-daemon[837]: reaped child process 1267, status = succeeded worker2 # [ 128.982649] nix-daemon[837]: accepted connection from pid 1283, user nix-grpc-daemon (trusted) worker2 # [ 129.003400] nix-daemon[837]: reaped child process 1286, status = succeeded lb # [ 129.005697] niks3-server[1100]: 2026/09/22 22:22:21 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 129.024523] nix-grpc-daemon[1215]: time=2026-09-22T22:22:21.666Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 129.026329] nix-grpc-daemon[1215]: time=2026-09-22T22:22:21.666Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)" lb # [ 129.059076] niks3-server[1100]: 2026/09/22 22:22:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign worker2 # [ 129.075190] nix-grpc-daemon[1215]: time=2026-09-22T22:22:21.717Z level=INFO msg="Uploading 1 narinfos" lb # [ 129.077379] niks3-server[1100]: 2026/09/22 22:22:21 INFO Signed narinfos id=44 count=1 lb # [ 129.087229] niks3-server[1100]: 2026/09/22 22:22:21 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst lb # [ 129.097818] niks3-server[1100]: 2026/09/22 22:22:21 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv lb # [ 129.108331] niks3-server[1100]: 2026/09/22 22:22:21 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls lb # [ 129.111751] niks3-server[1100]: 2026/09/22 22:22:21 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete lb # [ 129.117522] niks3-server[1100]: 2026/09/22 22:22:21 INFO Completed upload id=44 worker2 # [ 129.123262] nix-grpc-daemon[1215]: time=2026-09-22T22:22:21.765Z level=INFO msg="Upload complete. (303ms)" lb # [ 129.119737] niks3-server[1100]: 2026/09/22 22:22:21 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo worker2 # [ 129.154961] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 worker2 # [ 129.171743] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:21Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=0 worker2 # [ 129.175270] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:21Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 129.179242] nix-daemon[837]: reaped child process 1265, status = succeeded worker2 # [ 129.200913] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 129.222038] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 129.241321] nix-grpc-daemon[1215]: time=2026-09-22T22:22:21.883Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'... worker2 # [ 129.255817] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 129.275001] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) worker2 # [ 129.443302] nix-daemon[837]: accepted connection from pid 1292, user nix-grpc-daemon (trusted) worker2 # [ 129.466254] nix-daemon[837]: reaped child process 1295, status = succeeded lb # [ 129.465020] niks3-server[1100]: 2026/09/22 22:22:22 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 129.481241] nix-grpc-daemon[1215]: time=2026-09-22T22:22:22.123Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 129.482854] nix-grpc-daemon[1215]: time=2026-09-22T22:22:22.123Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)" lb # [ 129.496963] niks3-server[1100]: 2026/09/22 22:22:22 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst worker2 # [ 129.514968] nix-grpc-daemon[1215]: time=2026-09-22T22:22:22.157Z level=INFO msg="Uploading 1 narinfos" lb # [ 129.513971] niks3-server[1100]: 2026/09/22 22:22:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign lb # [ 129.523146] niks3-server[1100]: 2026/09/22 22:22:22 INFO Signed narinfos id=45 count=1 lb # [ 129.528365] niks3-server[1100]: 2026/09/22 22:22:22 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls lb # [ 129.538588] niks3-server[1100]: 2026/09/22 22:22:22 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete lb # [ 129.544513] niks3-server[1100]: 2026/09/22 22:22:22 INFO Completed upload id=45 lb # [ 129.548419] niks3-server[1100]: 2026/09/22 22:22:22 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo worker2 # [ 129.551084] nix-grpc-daemon[1215]: time=2026-09-22T22:22:22.193Z level=INFO msg="Upload complete. (253ms)" worker2 # [ 129.560574] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=0 paths=1 nar_bytes_in=640 worker2 # [ 129.576860] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 129.584634] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) worker2 # [ 129.590211] nix-daemon[837]: reaped child process 1290, status = succeeded client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051... client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker2 # [ 129.604786] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) lb # [ 129.918710] envoy[895]: conn peer=[2001:db8:1::1]:49296 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9727 tx=4459 ms=7041 lb # [ 129.924519] envoy[895]: conn peer=[2001:db8:1::3]:46084 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1487 tx=609 ms=11476 client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker2 # [ 131.073682] nix-daemon[837]: reaped child process 1299, status = succeeded worker2 # [ 131.204123] nix-daemon[837]: accepted connection from pid 1315, user nix-grpc-daemon (trusted) worker2 # [ 131.223666] nix-daemon[837]: reaped child process 1318, status = succeeded lb # [ 131.230634] niks3-server[1100]: 2026/09/22 22:22:23 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 131.248709] nix-grpc-daemon[1215]: time=2026-09-22T22:22:23.889Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 131.251053] nix-grpc-daemon[1215]: time=2026-09-22T22:22:23.889Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)" lb # [ 131.274626] niks3-server[1100]: 2026/09/22 22:22:23 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv lb # [ 131.296458] niks3-server[1100]: 2026/09/22 22:22:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign worker2 # [ 131.308170] nix-grpc-daemon[1215]: time=2026-09-22T22:22:23.950Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.305403] niks3-server[1100]: 2026/09/22 22:22:23 INFO Signed narinfos id=46 count=1 lb # [ 131.314224] niks3-server[1100]: 2026/09/22 22:22:23 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst lb # [ 131.322703] niks3-server[1100]: 2026/09/22 22:22:23 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls lb # [ 131.332222] niks3-server[1100]: 2026/09/22 22:22:23 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete lb # [ 131.343683] niks3-server[1100]: 2026/09/22 22:22:23 INFO Completed upload id=46 worker2 # [ 131.348413] nix-grpc-daemon[1215]: time=2026-09-22T22:22:23.990Z level=INFO msg="Upload complete. (331ms)" lb # [ 131.348561] niks3-server[1100]: 2026/09/22 22:22:23 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo worker2 # [ 131.367902] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 worker2 # [ 131.397711] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:24Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=0 worker2 # [ 131.403968] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:24Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 131.408860] nix-daemon[837]: reaped child process 1297, status = succeeded worker2 # [ 131.430895] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 131.444852] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 131.459962] nix-grpc-daemon[1215]: time=2026-09-22T22:22:24.102Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'... worker2 # [ 131.469840] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 131.482495] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) worker2 # [ 131.622985] nix-daemon[837]: accepted connection from pid 1324, user nix-grpc-daemon (trusted) worker2 # [ 131.646031] nix-daemon[837]: reaped child process 1327, status = succeeded lb # [ 131.637180] niks3-server[1100]: 2026/09/22 22:22:24 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 131.657320] nix-grpc-daemon[1215]: time=2026-09-22T22:22:24.299Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 131.659105] nix-grpc-daemon[1215]: time=2026-09-22T22:22:24.299Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)" lb # [ 131.674955] niks3-server[1100]: 2026/09/22 22:22:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign worker2 # [ 131.689746] nix-grpc-daemon[1215]: time=2026-09-22T22:22:24.332Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.690932] niks3-server[1100]: 2026/09/22 22:22:24 INFO Signed narinfos id=47 count=1 lb # [ 131.698162] niks3-server[1100]: 2026/09/22 22:22:24 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst worker2 # [ 131.718643] nix-grpc-daemon[1215]: time=2026-09-22T22:22:24.361Z level=INFO msg="Upload complete. (223ms)" lb # [ 131.713667] niks3-server[1100]: 2026/09/22 22:22:24 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls worker2 # [ 131.726143] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=0 paths=1 nar_bytes_in=768 lb # [ 131.719562] niks3-server[1100]: 2026/09/22 22:22:24 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete lb # [ 131.727144] niks3-server[1100]: 2026/09/22 22:22:24 INFO Completed upload id=47 lb # [ 131.730937] niks3-server[1100]: 2026/09/22 22:22:24 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo worker2 # [ 131.737101] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 131.749346] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) worker2 # [ 131.754711] nix-daemon[837]: reaped child process 1322, status = succeeded worker2 # [ 131.766130] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051... client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker2 # [ 133.229456] nix-daemon[837]: reaped child process 1331, status = succeeded worker2 # [ 133.364063] nix-daemon[837]: accepted connection from pid 1347, user nix-grpc-daemon (trusted) worker2 # [ 133.385334] nix-daemon[837]: reaped child process 1350, status = succeeded lb # [ 133.386345] niks3-server[1100]: 2026/09/22 22:22:26 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 133.408632] nix-grpc-daemon[1215]: time=2026-09-22T22:22:26.049Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 133.411006] nix-grpc-daemon[1215]: time=2026-09-22T22:22:26.049Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)" lb # [ 133.436596] niks3-server[1100]: 2026/09/22 22:22:26 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst worker2 # [ 133.452317] nix-grpc-daemon[1215]: time=2026-09-22T22:22:26.094Z level=INFO msg="Uploading 1 narinfos" lb # [ 133.448647] niks3-server[1100]: 2026/09/22 22:22:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign lb # [ 133.451695] niks3-server[1100]: 2026/09/22 22:22:26 INFO Signed narinfos id=48 count=1 lb # [ 133.461081] niks3-server[1100]: 2026/09/22 22:22:26 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv lb # [ 133.470544] niks3-server[1100]: 2026/09/22 22:22:26 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete lb # [ 133.481316] niks3-server[1100]: 2026/09/22 22:22:26 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls worker2 # [ 133.491115] nix-grpc-daemon[1215]: time=2026-09-22T22:22:26.133Z level=INFO msg="Upload complete. (310ms)" lb # [ 133.489721] niks3-server[1100]: 2026/09/22 22:22:26 INFO Completed upload id=48 worker2 # [ 133.508646] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 worker2 # [ 133.526571] nix-daemon[837]: reaped child process 1329, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 7.37 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.20 seconds) worker1 # [ 133.842949] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 133.853413] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 133.999257] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:26Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 134.001993] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 134.006443] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 134.012045] nix-grpc-daemon[1515]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 134.028195] nix-grpc-daemon[1515]: E0000 00:00:1790115746.651217 1515 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 134.060325] nix-grpc-daemon[1515]: E0000 00:00:1790115746.655463 1515 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 134.062821] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:26Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 134.074469] nix-grpc-daemon[1515]: E0000 00:00:1790115746.665519 1557 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 134.077547] nix-grpc-daemon[1515]: E0000 00:00:1790115746.689486 1557 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 134.080046] nix-grpc-daemon[1515]: E0000 00:00:1790115746.689640 1557 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 134.082431] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 134.083250] nix-grpc-daemon[1515]: E0000 00:00:1790115746.713048 1557 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 134.091161] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:26Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 134.089423] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:26Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:33102 worker2 # [ 134.094839] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.79 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs client # [ 134.414878] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: a68250cdb6b54c778086b27cb8b40d58 client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds) worker2 # [ 134.609312] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:57962 client # [ 134.616591] nix[1400]: don't know how to build these paths: client # [ 134.617616] nix[1400]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker2 # [ 134.629788] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=0 worker2 # [ 134.636939] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 134.675408] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 134.689569] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:57962 client # [ 134.703895] nix[1400]: copying 1 paths to 'lb:50051'... worker2 # [ 134.703520] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 client # [ 134.706451] nix[1400]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'... worker2 # [ 134.718854] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) worker2 # [ 134.885961] nix-daemon[837]: accepted connection from pid 1363, user nix-grpc-daemon (trusted) worker2 # [ 134.907968] nix-daemon[837]: reaped child process 1366, status = succeeded lb # [ 134.903908] niks3-server[1100]: 2026/09/22 22:22:27 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 134.924975] nix-grpc-daemon[1215]: time=2026-09-22T22:22:27.566Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 134.926521] nix-grpc-daemon[1215]: time=2026-09-22T22:22:27.566Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)" lb # [ 134.939047] niks3-server[1100]: 2026/09/22 22:22:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign worker2 # [ 134.958258] nix-grpc-daemon[1215]: time=2026-09-22T22:22:27.600Z level=INFO msg="Uploading 1 narinfos" lb # [ 134.958732] niks3-server[1100]: 2026/09/22 22:22:27 INFO Signed narinfos id=49 count=1 lb # [ 134.972615] niks3-server[1100]: 2026/09/22 22:22:27 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst lb # [ 134.988289] niks3-server[1100]: 2026/09/22 22:22:27 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls lb # [ 134.993871] niks3-server[1100]: 2026/09/22 22:22:27 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete lb # [ 135.000946] niks3-server[1100]: 2026/09/22 22:22:27 INFO Completed upload id=49 worker2 # [ 135.007470] nix-grpc-daemon[1215]: time=2026-09-22T22:22:27.650Z level=INFO msg="Upload complete. (270ms)" lb # [ 135.005438] niks3-server[1100]: 2026/09/22 22:22:27 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo worker2 # [ 135.020244] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 135.032691] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 worker2 # [ 135.046867] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) worker2 # [ 135.053229] nix-daemon[837]: reaped child process 1361, status = succeeded client # [ 135.063900] nix[1400]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051... worker2 # [ 135.068210] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker1 # [ 137.273634] systemd-journald[270]: Received client request to rotate journal, rotating. worker1 # [ 137.302736] systemd-journald[270]: Vacuuming done, freed 0B of archived journals from /var/log/journal/283d1111edd04c2c9541ffd544a9bf18. worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds) worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker2 # [ 137.355778] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds) client: must succeed: sleep 5.0 worker2 # [ 138.334185] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:30Z level=info event=draining lb # [ 139.918924] envoy[895]: conn peer=[2001:db8:1::1]:51130 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9675 tx=4436 ms=7018 client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker2 # [ 142.978199] nix-daemon[837]: reaped child process 1370, status = succeeded worker2 # [ 143.058772] nix-daemon[837]: accepted connection from pid 1411, user nix-grpc-daemon (trusted) worker2 # [ 143.080900] nix-daemon[837]: reaped child process 1414, status = succeeded lb # [ 143.073498] niks3-server[1100]: 2026/09/22 22:22:35 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 143.092474] nix-grpc-daemon[1215]: time=2026-09-22T22:22:35.734Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 143.094183] nix-grpc-daemon[1215]: time=2026-09-22T22:22:35.734Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)" lb # [ 143.111468] niks3-server[1100]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls worker2 # [ 143.124682] nix-grpc-daemon[1215]: time=2026-09-22T22:22:35.767Z level=INFO msg="Uploading 1 narinfos" lb # [ 143.116451] niks3-server[1100]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 143.125746] niks3-server[1100]: 2026/09/22 22:22:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign lb # [ 143.131060] niks3-server[1100]: 2026/09/22 22:22:35 INFO Signed narinfos id=50 count=1 lb # [ 143.135370] niks3-server[1100]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker2 # [ 143.149447] nix-grpc-daemon[1215]: time=2026-09-22T22:22:35.792Z level=INFO msg="Upload complete. (263ms)" lb # [ 143.144243] niks3-server[1100]: 2026/09/22 22:22:35 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 143.147796] niks3-server[1100]: 2026/09/22 22:22:35 INFO Completed upload id=50 lb # [ 143.151546] niks3-server[1100]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo worker2 # [ 143.161001] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 worker2 # [ 143.175691] nix-daemon[837]: reaped child process 1368, status = succeeded client # [ 143.204431] systemd[1]: rs.service: Deactivated successfully. client # [ 143.209724] systemd[1]: rs.service: Consumed 193ms CPU time over 8.793s wall clock time, 41.2M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic. worker2 # [ 143.334545] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:35Z level=info event=signal_exit lb # [ 143.628273] niks3-server[1100]: 2026/09/22 22:22:36 INFO lead: released remote=192.168.1.4:37182 lb # [ 144.014776] niks3-server[1100]: 2026/09/22 22:22:36 INFO lead: acquired remote=192.168.1.3:43276 worker1 # [ 144.026407] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:36Z level=info event=scheduler_take_over worker1 # [ 144.637456] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 144.638667] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:37Z level=info event=scheduler_restarting worker2 # [ 144.644306] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:37Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 144.649463] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:33102 worker2 # [ 144.654796] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:37Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 144.659914] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:33102 worker2 # [ 144.664821] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:37Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 144.682479] nix-daemon[837]: reaped child process 1207, status = succeeded worker1 # [ 144.838534] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 144.843766] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 144.857179] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:37Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.19 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 145.043141] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 145.046650] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker1 # [ 145.247746] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 145.257736] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:48340 worker1 # [ 145.263222] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 145.396652] nix-grpc-daemon[1155]: E0000 00:00:1790115758.039029 1200 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker2 # [ 145.421820] nix-grpc-daemon[1155]: E0000 00:00:1790115758.064341 1200 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2 # [ 146.240297] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 146.246919] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 146.253214] systemd[1]: nix-grpc-daemon.service: Consumed 1.983s CPU time over 39.076s wall clock time, 39.7M memory peak, 62.8K incoming IP traffic, 142.3K outgoing IP traffic. worker2 # [ 146.271702] systemd[1]: Starting scheduler-token.service... worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker2 # [ 146.350578] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 146.355706] systemd[1]: Finished scheduler-token.service. worker2 # [ 146.360724] systemd[1]: scheduler-token.service: Consumed 48ms CPU time over 81ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 146.368908] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 146.477489] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 146.479878] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 146.483953] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 146.491284] nix-grpc-daemon[1424]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 146.504151] nix-grpc-daemon[1424]: E0000 00:00:1790115759.131527 1424 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 146.506575] nix-grpc-daemon[1424]: E0000 00:00:1790115759.133786 1424 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 146.536928] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 146.539209] nix-grpc-daemon[1424]: E0000 00:00:1790115759.143174 1463 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 146.542025] nix-grpc-daemon[1424]: E0000 00:00:1790115759.161047 1463 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 146.547436] nix-grpc-daemon[1424]: E0000 00:00:1790115759.161232 1463 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 146.552345] nix-grpc-daemon[1424]: E0000 00:00:1790115759.178351 1463 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 146.555049] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 146.555992] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:39Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 146.568978] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:39Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 146.573147] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:39Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:48340 worker1 # [ 146.578557] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.06 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid client # [ 147.474821] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: e73fbeb486b14434ab5c58dad2b75114 client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.10 seconds) worker1 # [ 147.691881] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:48342 worker1 # [ 147.706656] nix-daemon[848]: accepted connection from pid 1515, user nix-grpc-daemon (trusted) client # [ 147.724999] nix[1436]: don't know how to build these paths: client # [ 147.726895] nix[1436]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker1 # [ 147.745111] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:40Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker1 # [ 147.752527] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:40Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 147.790342] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48342 worker1 # [ 147.799816] nix-grpc-daemon[1515]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 1/5) worker1 # [ 147.809090] nix-grpc-daemon[1515]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 193 ms (attempt 2/5) worker1 # [ 148.010850] nix-grpc-daemon[1515]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 297 ms (attempt 3/5) worker1 # [ 148.310003] nix-grpc-daemon[1515]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 314 ms (attempt 4/5) worker1 # [ 148.626746] nix-grpc-daemon[1515]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 148.634695] nix-grpc-daemon[1515]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 148.654944] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48342 client # [ 148.667723] nix[1436]: copying 1 paths to 'lb:50051'... client # [ 148.673356] nix[1436]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'... worker1 # [ 148.678997] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48342 worker1 # [ 148.700266] nix-daemon[848]: accepted connection from pid 1515, user nix-grpc-daemon (trusted) worker1 # [ 148.759308] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:41Z level=info event=niks3_push_started pid=1644 worker1 # [ 148.894136] nix-grpc-daemon[1644]: time=2026-09-22T22:22:41.532Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 149.007477] nix-daemon[848]: accepted connection from pid 1657, user nix-grpc-daemon (trusted) worker1 # [ 149.029014] nix-daemon[848]: reaped child process 1660, status = succeeded lb # [ 149.023773] niks3-server[1100]: 2026/09/22 22:22:41 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 149.044061] nix-grpc-daemon[1644]: time=2026-09-22T22:22:41.683Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 149.045550] nix-grpc-daemon[1644]: time=2026-09-22T22:22:41.683Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)" lb # [ 149.067713] niks3-server[1100]: 2026/09/22 22:22:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign worker1 # [ 149.088815] nix-grpc-daemon[1644]: time=2026-09-22T22:22:41.727Z level=INFO msg="Uploading 1 narinfos" lb # [ 149.080592] niks3-server[1100]: 2026/09/22 22:22:41 INFO Signed narinfos id=51 count=1 lb # [ 149.090071] niks3-server[1100]: 2026/09/22 22:22:41 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls lb # [ 149.099867] niks3-server[1100]: 2026/09/22 22:22:41 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst lb # [ 149.116802] niks3-server[1100]: 2026/09/22 22:22:41 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete worker1 # [ 149.133726] nix-grpc-daemon[1644]: time=2026-09-22T22:22:41.773Z level=INFO msg="Upload complete. (236ms)" lb # [ 149.125112] niks3-server[1100]: 2026/09/22 22:22:41 INFO Completed upload id=51 lb # [ 149.130299] niks3-server[1100]: 2026/09/22 22:22:41 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo worker1 # [ 149.145260] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48342 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 149.152348] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48342 worker1 # [ 149.164568] nix-daemon[848]: accepted connection from pid 1515, user nix-grpc-daemon (trusted) worker1 # [ 149.169582] nix-daemon[848]: reaped child process 1641, status = succeeded worker1 # [ 149.181900] nix-daemon[848]: accepted connection from pid 1515, user nix-grpc-daemon (trusted) client # [ 149.176097] nix[1436]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... lb # [ 149.920851] envoy[895]: conn peer=[2001:db8:1::1]:59574 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1541 ms=8575 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker1 # [ 150.346636] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1515 (nix-grpc-daemon) on client request. worker1 # [ 150.348697] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 149.925932] envoy[895]: conn peer=[2001:db8:1::4]:53934 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=24909 tx=1639 ms=37466 lb # [ 150.349402] niks3-server[1100]: 2026/09/22 22:22:42 INFO lead: released remote=192.168.1.3:43276 worker1 # [ 150.367607] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL client # [ 150.369806] nix[1436]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker2 # [ 150.370329] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:43Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination" worker1 # [ 150.382918] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. client # [ 150.374997] nix[1436]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again worker1 # [ 150.394239] systemd[1]: nix-grpc-daemon.service: Consumed 513ms CPU time over 16.515s wall clock time, 34.6M memory peak, 15.9K incoming IP traffic, 26K outgoing IP traffic. worker1 # [ 150.404224] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker1 # [ 150.405751] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 150.456328] nix-daemon[848]: reaped child process 1662, status = succeeded worker1 # [ 150.460327] nix-daemon[848]: reaped child process 1630, status = succeeded worker1 # [ 150.466016] nix-daemon[1664]: unexpected Nix daemon error: error: interrupted by the user worker1 # [ 150.479706] nix-daemon[848]: reaped child process 1664, status = failed with exit code 1 worker1 # [ 150.523872] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 150.526267] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 150.529984] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 150.535247] nix-grpc-daemon[1696]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 150.537143] niks3-server[1100]: 2026/09/22 22:22:43 INFO lead: acquired remote=192.168.1.3:55768 worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.24 seconds) client: must succeed: sleep 5.0 worker1 # [ 150.552193] nix-grpc-daemon[1696]: E0000 00:00:1790115763.173886 1696 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 150.571091] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 150.580147] nix-grpc-daemon[1696]: E0000 00:00:1790115763.177441 1696 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 150.582415] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 150.586894] nix-grpc-daemon[1696]: E0000 00:00:1790115763.186023 1738 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 150.589651] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=scheduler_take_over worker1 # [ 150.590738] nix-grpc-daemon[1696]: E0000 00:00:1790115763.204366 1738 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 150.594133] nix-grpc-daemon[1696]: E0000 00:00:1790115763.204503 1738 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 150.599941] nix-grpc-daemon[1696]: E0000 00:00:1790115763.219374 1738 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 150.605339] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 150.607970] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 150.609439] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:48358 worker1 # [ 150.610947] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 150.612723] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 150.613939] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:48358 worker1 # [ 150.615581] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 150.883819] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker1 # [ 150.891006] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 150.946299] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39588 worker1 # [ 150.959761] nix-daemon[848]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 150.980229] nix-daemon[848]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 150.997098] nix-daemon[848]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) client # [ 150.991768] nix[1436]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker1 # [ 155.817430] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:48Z level=info event=niks3_push_started pid=1780 worker1 # [ 155.845870] nix-daemon[848]: reaped child process 1755, status = succeeded client: waiting for success: ! systemctl is-active mid worker1 # [ 155.944332] nix-grpc-daemon[1780]: time=2026-09-22T22:22:48.582Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 156.050706] nix-daemon[848]: accepted connection from pid 1793, user nix-grpc-daemon (trusted) worker1 # [ 156.069202] nix-daemon[848]: reaped child process 1796, status = succeeded lb # [ 156.072069] niks3-server[1100]: 2026/09/22 22:22:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 156.096080] nix-grpc-daemon[1780]: time=2026-09-22T22:22:48.733Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 156.097774] nix-grpc-daemon[1780]: time=2026-09-22T22:22:48.733Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)" lb # [ 156.143468] niks3-server[1100]: 2026/09/22 22:22:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign lb # [ 156.154112] niks3-server[1100]: 2026/09/22 22:22:48 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls lb # [ 156.165693] niks3-server[1100]: 2026/09/22 22:22:48 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv lb # [ 156.174511] niks3-server[1100]: 2026/09/22 22:22:48 INFO Signed narinfos id=52 count=1 worker1 # [ 156.182180] nix-grpc-daemon[1780]: time=2026-09-22T22:22:48.819Z level=INFO msg="Uploading 1 narinfos" lb # [ 156.185831] niks3-server[1100]: 2026/09/22 22:22:48 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst lb # [ 156.195507] niks3-server[1100]: 2026/09/22 22:22:48 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 156.201481] niks3-server[1100]: 2026/09/22 22:22:48 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo lb # [ 156.206453] niks3-server[1100]: 2026/09/22 22:22:48 INFO Completed upload id=52 worker1 # [ 156.216827] nix-grpc-daemon[1780]: time=2026-09-22T22:22:48.856Z level=INFO msg="Upload complete. (269ms)" worker1 # [ 156.232542] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39588 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 worker1 # [ 156.253709] nix-daemon[848]: reaped child process 1753, status = succeeded client # [ 156.260196] systemd[1]: mid.service: Deactivated successfully. client # [ 156.265225] systemd[1]: mid.service: Consumed 202ms CPU time over 8.786s wall clock time, 41.5M memory peak, 8.7K incoming IP traffic, 10.3K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.62 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr client # [ 157.070790] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: d8d6f2d69ff14e83ac676d0946853a13 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds) worker1 # [ 157.251449] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:39588 client # [ 157.256479] nix[1472]: don't know how to build these paths: client # [ 157.257415] nix[1472]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker1 # [ 157.271565] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:49Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=0 worker1 # [ 157.277794] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:49Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 157.331245] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 157.345876] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 157.374685] nix-grpc-daemon[1424]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 1/5) worker2 # [ 157.407408] nix-grpc-daemon[1424]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 2/5) worker2 # [ 157.420877] nix-grpc-daemon[1424]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 3/5) worker2 # [ 157.545343] nix-grpc-daemon[1424]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 135 ms (attempt 4/5) worker2 # [ 157.684105] nix-grpc-daemon[1424]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 157.689838] nix-grpc-daemon[1424]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 157.709266] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36886 client # [ 157.724436] nix[1472]: copying 1 paths to 'lb:50051'... worker2 # [ 157.725899] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 client # [ 157.726863] nix[1472]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'... worker2 # [ 157.742563] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 157.771579] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:50Z level=info event=niks3_push_started pid=1512 worker2 # [ 157.928845] nix-grpc-daemon[1512]: time=2026-09-22T22:22:50.570Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 158.043426] nix-daemon[837]: accepted connection from pid 1525, user nix-grpc-daemon (trusted) worker2 # [ 158.063745] nix-daemon[837]: reaped child process 1528, status = succeeded lb # [ 158.063797] niks3-server[1100]: 2026/09/22 22:22:50 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 158.083119] nix-grpc-daemon[1512]: time=2026-09-22T22:22:50.724Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 158.087340] nix-grpc-daemon[1512]: time=2026-09-22T22:22:50.724Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)" lb # [ 158.099439] niks3-server[1100]: 2026/09/22 22:22:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign lb # [ 158.106032] niks3-server[1100]: 2026/09/22 22:22:50 INFO Signed narinfos id=53 count=1 worker2 # [ 158.115437] nix-grpc-daemon[1512]: time=2026-09-22T22:22:50.757Z level=INFO msg="Uploading 1 narinfos" lb # [ 158.111697] niks3-server[1100]: 2026/09/22 22:22:50 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst lb # [ 158.117762] niks3-server[1100]: 2026/09/22 22:22:50 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls lb # [ 158.123664] niks3-server[1100]: 2026/09/22 22:22:50 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete worker2 # [ 158.134942] nix-grpc-daemon[1512]: time=2026-09-22T22:22:50.777Z level=INFO msg="Upload complete. (206ms)" lb # [ 158.130077] niks3-server[1100]: 2026/09/22 22:22:50 INFO Completed upload id=53 lb # [ 158.131994] niks3-server[1100]: 2026/09/22 22:22:50 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo worker2 # [ 158.144588] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 158.147794] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 158.158324] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 158.162179] nix-daemon[837]: reaped child process 1510, status = succeeded worker2 # [ 158.172140] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) client # [ 158.170757] nix[1472]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051... client: must succeed: systemctl kill -s INT intr client # [ 159.548940] systemd[1]: intr.service: Sent signal SIGINT to main process 1472 (nix) on client request. worker2 # [ 159.555663] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:52Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv client # [ 159.559461] nix[1472]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 159.574073] nix[1472]: error: interrupted by the user client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds) worker2 # [ 159.599341] nix-daemon[837]: reaped child process 1530, status = succeeded client # [ 159.599058] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE worker2 # [ 159.605398] nix-daemon[1532]: unexpected Nix daemon error: error: interrupted by the user client # [ 159.603779] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 159.606861] systemd[1]: intr.service: Consumed 173ms CPU time over 2.531s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic. worker2 # [ 159.609933] nix-daemon[837]: reaped child process 1532, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 2.65 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw client # [ 159.742945] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: 2232792ef1ea4f93928a67ab9bef37d8 client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds) lb # [ 159.919421] envoy[895]: conn peer=[2001:db8:1::3]:54920 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=953 tx=431 ms=16292 lb # [ 159.924659] envoy[895]: conn peer=[2001:db8:1::1]:59092 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3884 tx=1922 ms=8569 lb # [ 159.935125] envoy[895]: conn peer=[2001:db8:1::1]:50056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=829 ms=2335 client # [ 159.954938] nix[1493]: don't know how to build these paths: client # [ 159.956342] nix[1493]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker2 # [ 159.953308] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:36886 worker1 # [ 159.970091] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:52Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=0 worker1 # [ 159.970378] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:52Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 160.014401] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39588 worker1 # [ 160.029568] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 1/5) worker1 # [ 160.059606] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 2/5) worker1 # [ 160.069149] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 205 ms (attempt 3/5) worker1 # [ 160.276333] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 4/5) worker1 # [ 160.346459] nix-grpc-daemon[1696]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 160.353561] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 160.370577] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39588 client # [ 160.379552] nix[1493]: copying 1 paths to 'lb:50051'... client # [ 160.384513] nix[1493]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'... worker1 # [ 160.389077] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39588 worker1 # [ 160.405245] nix-daemon[848]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 160.571780] nix-daemon[848]: accepted connection from pid 1828, user nix-grpc-daemon (trusted) worker1 # [ 160.590248] nix-daemon[848]: reaped child process 1831, status = succeeded lb # [ 160.589464] niks3-server[1100]: 2026/09/22 22:22:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 160.609392] nix-grpc-daemon[1780]: time=2026-09-22T22:22:53.248Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 160.611882] nix-grpc-daemon[1780]: time=2026-09-22T22:22:53.250Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)" lb # [ 160.623202] niks3-server[1100]: 2026/09/22 22:22:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign worker1 # [ 160.642062] nix-grpc-daemon[1780]: time=2026-09-22T22:22:53.280Z level=INFO msg="Uploading 1 narinfos" lb # [ 160.638366] niks3-server[1100]: 2026/09/22 22:22:53 INFO Signed narinfos id=54 count=1 lb # [ 160.645772] niks3-server[1100]: 2026/09/22 22:22:53 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls lb # [ 160.657540] niks3-server[1100]: 2026/09/22 22:22:53 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst lb # [ 160.665366] niks3-server[1100]: 2026/09/22 22:22:53 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete lb # [ 160.673952] niks3-server[1100]: 2026/09/22 22:22:53 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo lb # [ 160.680324] niks3-server[1100]: 2026/09/22 22:22:53 INFO Completed upload id=54 worker1 # [ 160.689986] nix-grpc-daemon[1780]: time=2026-09-22T22:22:53.329Z level=INFO msg="Upload complete. (267ms)" worker1 # [ 160.700649] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39588 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 160.719140] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39588 worker1 # [ 160.731595] nix-daemon[848]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1 # [ 160.737869] nix-daemon[848]: reaped child process 1826, status = succeeded client # [ 160.742563] nix[1493]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051... worker1 # [ 160.748370] nix-daemon[848]: accepted connection from pid 1696, user nix-grpc-daemon (trusted) worker1: must succeed: systemctl reload nix-grpc-daemon.service worker1 # [ 162.359478] systemd[1]: Reloading Nix worker-protocol over gRPC... worker1 # [ 162.402385] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds) worker1: must succeed: pgrep -f 'read -t [9]0 x' worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2 worker1 # [ 162.547255] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:55Z level=info event=draining worker2 # [ 162.664618] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:36886 client # don't know how to build these paths: client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker2 # [ 162.674628] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36886 worker1 # [ 162.702178] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:55Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=0 worker1 # [ 162.704155] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:55Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 162.751844] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 162.767434] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36886 client # copying 1 paths to 'lb:50051'... worker2 # [ 162.783964] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'... worker2 # [ 162.808328] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 162.977539] nix-daemon[837]: accepted connection from pid 1579, user nix-grpc-daemon (trusted) worker2 # [ 162.999323] nix-daemon[837]: reaped child process 1582, status = succeeded lb # [ 162.999685] niks3-server[1100]: 2026/09/22 22:22:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 163.018091] nix-grpc-daemon[1512]: time=2026-09-22T22:22:55.659Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 163.019770] nix-grpc-daemon[1512]: time=2026-09-22T22:22:55.659Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)" lb # [ 163.027646] niks3-server[1100]: 2026/09/22 22:22:55 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls worker2 # [ 163.046875] nix-grpc-daemon[1512]: time=2026-09-22T22:22:55.688Z level=INFO msg="Uploading 1 narinfos" lb # [ 163.040173] niks3-server[1100]: 2026/09/22 22:22:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign lb # [ 163.051611] niks3-server[1100]: 2026/09/22 22:22:55 INFO Signed narinfos id=55 count=1 lb # [ 163.061206] niks3-server[1100]: 2026/09/22 22:22:55 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst lb # [ 163.075786] niks3-server[1100]: 2026/09/22 22:22:55 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete lb # [ 163.082471] niks3-server[1100]: 2026/09/22 22:22:55 INFO Completed upload id=55 worker2 # [ 163.086995] nix-grpc-daemon[1512]: time=2026-09-22T22:22:55.729Z level=INFO msg="Upload complete. (261ms)" lb # [ 163.084902] niks3-server[1100]: 2026/09/22 22:22:55 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo worker2 # [ 163.098465] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=0 paths=1 nar_bytes_in=528 worker2 # [ 163.116275] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 163.122864] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 163.128214] nix-daemon[837]: reaped child process 1577, status = succeeded client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051... worker2 # [ 163.141871] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker2 # [ 164.808944] nix-daemon[837]: reaped child process 1586, status = succeeded worker2 # [ 164.925917] nix-daemon[837]: accepted connection from pid 1602, user nix-grpc-daemon (trusted) worker2 # [ 164.944840] nix-daemon[837]: reaped child process 1605, status = succeeded lb # [ 164.949369] niks3-server[1100]: 2026/09/22 22:22:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 164.968399] nix-grpc-daemon[1512]: time=2026-09-22T22:22:57.608Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 164.970104] nix-grpc-daemon[1512]: time=2026-09-22T22:22:57.608Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)" lb # [ 164.995613] niks3-server[1100]: 2026/09/22 22:22:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign worker2 # [ 165.013957] nix-grpc-daemon[1512]: time=2026-09-22T22:22:57.656Z level=INFO msg="Uploading 1 narinfos" lb # [ 165.010451] niks3-server[1100]: 2026/09/22 22:22:57 INFO Signed narinfos id=56 count=1 lb # [ 165.027202] niks3-server[1100]: 2026/09/22 22:22:57 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv lb # [ 165.042180] niks3-server[1100]: 2026/09/22 22:22:57 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst worker2 # [ 165.055488] nix-grpc-daemon[1512]: time=2026-09-22T22:22:57.697Z level=INFO msg="Upload complete. (301ms)" lb # [ 165.052518] niks3-server[1100]: 2026/09/22 22:22:57 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 165.058462] niks3-server[1100]: 2026/09/22 22:22:57 INFO Completed upload id=56 lb # [ 165.061500] niks3-server[1100]: 2026/09/22 22:22:57 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls worker2 # [ 165.069502] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=1 worker1 # [ 165.079818] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:57Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=0 worker1 # [ 165.081815] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:57Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=1 lb # [ 165.070469] niks3-server[1100]: 2026/09/22 22:22:57 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo worker2 # [ 165.091736] nix-daemon[837]: reaped child process 1584, status = succeeded worker2 # [ 165.128532] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 165.145988] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 165.161584] nix-grpc-daemon[1512]: time=2026-09-22T22:22:57.804Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 165.173084] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'... worker2 # [ 165.202233] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 165.363767] nix-daemon[837]: accepted connection from pid 1609, user nix-grpc-daemon (trusted) worker2 # [ 165.386445] nix-daemon[837]: reaped child process 1612, status = succeeded lb # [ 165.378801] niks3-server[1100]: 2026/09/22 22:22:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 165.396107] nix-grpc-daemon[1512]: time=2026-09-22T22:22:58.038Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 165.397725] nix-grpc-daemon[1512]: time=2026-09-22T22:22:58.038Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)" lb # [ 165.414810] niks3-server[1100]: 2026/09/22 22:22:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign worker2 # [ 165.430414] nix-grpc-daemon[1512]: time=2026-09-22T22:22:58.072Z level=INFO msg="Uploading 1 narinfos" lb # [ 165.430798] niks3-server[1100]: 2026/09/22 22:22:58 INFO Signed narinfos id=57 count=1 lb # [ 165.439826] niks3-server[1100]: 2026/09/22 22:22:58 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst worker2 # [ 165.459911] nix-grpc-daemon[1512]: time=2026-09-22T22:22:58.102Z level=INFO msg="Upload complete. (240ms)" lb # [ 165.453877] niks3-server[1100]: 2026/09/22 22:22:58 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls lb # [ 165.460778] niks3-server[1100]: 2026/09/22 22:22:58 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete worker2 # [ 165.468361] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=0 paths=1 nar_bytes_in=640 lb # [ 165.467481] niks3-server[1100]: 2026/09/22 22:22:58 INFO Completed upload id=57 lb # [ 165.471831] niks3-server[1100]: 2026/09/22 22:22:58 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo worker2 # [ 165.480363] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 165.493207] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 165.498502] nix-daemon[837]: reaped child process 1607, status = succeeded client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051... worker2 # [ 165.512300] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker2 # [ 166.779317] nix-daemon[837]: reaped child process 1616, status = succeeded worker2 # [ 166.913102] nix-daemon[837]: accepted connection from pid 1632, user nix-grpc-daemon (trusted) worker2 # [ 166.939783] nix-daemon[837]: reaped child process 1635, status = succeeded lb # [ 166.933027] niks3-server[1100]: 2026/09/22 22:22:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 166.952341] nix-grpc-daemon[1512]: time=2026-09-22T22:22:59.593Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 166.954566] nix-grpc-daemon[1512]: time=2026-09-22T22:22:59.593Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)" lb # [ 166.975003] niks3-server[1100]: 2026/09/22 22:22:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign worker2 # [ 166.989241] nix-grpc-daemon[1512]: time=2026-09-22T22:22:59.631Z level=INFO msg="Uploading 1 narinfos" lb # [ 166.988955] niks3-server[1100]: 2026/09/22 22:22:59 INFO Signed narinfos id=58 count=1 lb # [ 166.991704] niks3-server[1100]: 2026/09/22 22:22:59 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst lb # [ 167.006027] niks3-server[1100]: 2026/09/22 22:22:59 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls lb # [ 167.019679] niks3-server[1100]: 2026/09/22 22:22:59 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker2 # [ 167.025675] nix-grpc-daemon[1512]: time=2026-09-22T22:22:59.668Z level=INFO msg="Upload complete. (301ms)" lb # [ 167.028531] niks3-server[1100]: 2026/09/22 22:22:59 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete lb # [ 167.033129] niks3-server[1100]: 2026/09/22 22:22:59 INFO Completed upload id=58 worker1 # [ 167.047758] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:59Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=0 lb # [ 167.037227] niks3-server[1100]: 2026/09/22 22:22:59 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo worker1 # [ 167.049730] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:59Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 167.041447] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=1 worker2 # [ 167.062730] nix-daemon[837]: reaped child process 1614, status = succeeded worker2 # [ 167.096501] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 167.119751] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 167.135458] nix-grpc-daemon[1512]: time=2026-09-22T22:22:59.776Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 167.147422] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'... worker2 # [ 167.171973] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 167.336448] nix-daemon[837]: accepted connection from pid 1639, user nix-grpc-daemon (trusted) worker2 # [ 167.356617] nix-daemon[837]: reaped child process 1642, status = succeeded lb # [ 167.355883] niks3-server[1100]: 2026/09/22 22:22:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 167.373166] nix-grpc-daemon[1512]: time=2026-09-22T22:23:00.015Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 167.375025] nix-grpc-daemon[1512]: time=2026-09-22T22:23:00.015Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)" lb # [ 167.390820] niks3-server[1100]: 2026/09/22 22:23:00 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls worker2 # [ 167.409066] nix-grpc-daemon[1512]: time=2026-09-22T22:23:00.051Z level=INFO msg="Uploading 1 narinfos" lb # [ 167.402966] niks3-server[1100]: 2026/09/22 22:23:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign lb # [ 167.413454] niks3-server[1100]: 2026/09/22 22:23:00 INFO Signed narinfos id=59 count=1 lb # [ 167.419138] niks3-server[1100]: 2026/09/22 22:23:00 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst lb # [ 167.431379] niks3-server[1100]: 2026/09/22 22:23:00 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete worker2 # [ 167.442118] nix-grpc-daemon[1512]: time=2026-09-22T22:23:00.084Z level=INFO msg="Upload complete. (251ms)" lb # [ 167.438914] niks3-server[1100]: 2026/09/22 22:23:00 INFO Completed upload id=59 lb # [ 167.441377] niks3-server[1100]: 2026/09/22 22:23:00 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo worker2 # [ 167.452385] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=0 paths=1 nar_bytes_in=768 worker2 # [ 167.472266] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 167.479465] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 167.485399] nix-daemon[837]: reaped child process 1637, status = succeeded client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051... worker2 # [ 167.497637] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker2 # [ 168.388645] nix-daemon[837]: reaped child process 1646, status = succeeded worker2 # [ 168.483761] nix-daemon[837]: accepted connection from pid 1662, user nix-grpc-daemon (trusted) worker2 # [ 168.508056] nix-daemon[837]: reaped child process 1665, status = succeeded lb # [ 168.501150] niks3-server[1100]: 2026/09/22 22:23:01 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 168.519387] nix-grpc-daemon[1512]: time=2026-09-22T22:23:01.160Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 168.521133] nix-grpc-daemon[1512]: time=2026-09-22T22:23:01.160Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)" lb # [ 168.544638] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls lb # [ 168.558435] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst worker2 # [ 168.572492] nix-grpc-daemon[1512]: time=2026-09-22T22:23:01.214Z level=INFO msg="Uploading 1 narinfos" lb # [ 168.570300] niks3-server[1100]: 2026/09/22 22:23:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign lb # [ 168.576998] niks3-server[1100]: 2026/09/22 22:23:01 INFO Signed narinfos id=60 count=1 lb # [ 168.578973] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv lb # [ 168.589808] niks3-server[1100]: 2026/09/22 22:23:01 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete lb # [ 168.592582] niks3-server[1100]: 2026/09/22 22:23:01 INFO Completed upload id=60 worker2 # [ 168.595709] nix-grpc-daemon[1512]: time=2026-09-22T22:23:01.238Z level=INFO msg="Upload complete. (283ms)" lb # [ 168.594371] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo worker2 # [ 168.609263] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=1 worker2 # [ 168.626096] nix-daemon[837]: reaped child process 1644, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.21 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.20 seconds) worker1: must succeed: pkill -f 'read -t [9]0 x' worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds) client: waiting for success: ! systemctl is-active sw worker1 # [ 168.984200] nix-daemon[848]: reaped child process 1835, status = succeeded worker1 # [ 169.087172] nix-daemon[848]: accepted connection from pid 1882, user nix-grpc-daemon (trusted) worker1 # [ 169.107661] nix-daemon[848]: reaped child process 1885, status = succeeded lb # [ 169.104180] niks3-server[1100]: 2026/09/22 22:23:01 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 169.124523] nix-grpc-daemon[1780]: time=2026-09-22T22:23:01.763Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 169.126938] nix-grpc-daemon[1780]: time=2026-09-22T22:23:01.765Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)" lb # [ 169.148403] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv lb # [ 169.162305] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst worker1 # [ 169.185231] nix-grpc-daemon[1780]: time=2026-09-22T22:23:01.824Z level=INFO msg="Uploading 1 narinfos" lb # [ 169.174996] niks3-server[1100]: 2026/09/22 22:23:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign lb # [ 169.185376] niks3-server[1100]: 2026/09/22 22:23:01 INFO Signed narinfos id=61 count=1 lb # [ 169.191145] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls lb # [ 169.201387] niks3-server[1100]: 2026/09/22 22:23:01 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete lb # [ 169.209642] niks3-server[1100]: 2026/09/22 22:23:01 INFO Completed upload id=61 worker1 # [ 169.216987] nix-grpc-daemon[1780]: time=2026-09-22T22:23:01.856Z level=INFO msg="Upload complete. (247ms)" lb # [ 169.212404] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo worker1 # [ 169.233274] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39588 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=1 worker1 # [ 169.252485] nix-daemon[848]: reaped child process 1833, status = succeeded client # [ 169.269489] systemd[1]: sw.service: Deactivated successfully. client # [ 169.274243] systemd[1]: sw.service: Consumed 196ms CPU time over 9.530s wall clock time, 41.4M memory peak, 7.6K incoming IP traffic, 8.5K outgoing IP traffic. worker1 # [ 169.548832] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:02Z level=info event=signal_exit lb # [ 169.919439] envoy[895]: conn peer=[2001:db8:1::1]:55582 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9608 tx=4381 ms=5964 lb # [ 169.924738] envoy[895]: conn peer=[2001:db8:1::1]:50070 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2995 tx=1558 ms=9293 client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds) worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service lb # [ 170.541469] niks3-server[1100]: 2026/09/22 22:23:03 INFO lead: released remote=192.168.1.3:55768 lb # [ 171.494177] niks3-server[1100]: 2026/09/22 22:23:04 INFO lead: acquired remote=192.168.1.4:52476 worker2 # [ 171.506738] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_take_over worker1 # [ 171.550756] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:04Z level=info event=scheduler_restarting worker2 # [ 171.547925] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 171.562598] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:48358 worker1 # [ 171.568262] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:04Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 171.572479] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:04Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 171.577087] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:48358 worker1 # [ 171.582192] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:04Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 171.606020] nix-daemon[848]: reaped child process 1751, status = succeeded worker2 # [ 171.749231] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 171.754662] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 171.779703] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:04Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 171.955738] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 171.961436] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 172.156473] nix-grpc-daemon[1696]: E0000 00:00:1790115784.795663 1742 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true worker1 # [ 172.164176] nix-grpc-daemon[1696]: E0000 00:00:1790115784.803180 1742 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 172.162654] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 172.167876] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 172.368886] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 172.398123] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:05Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 172.402699] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:50648 worker2 # [ 172.408107] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 172.582784] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 172.589277] systemd[1]: nix-grpc-daemon.service: Consumed 765ms CPU time over 22.209s wall clock time, 37.4M memory peak, 25.1K incoming IP traffic, 32.6K outgoing IP traffic. worker1 # [ 172.616458] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 172.751732] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 172.754144] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 172.757910] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 172.763247] nix-grpc-daemon[1899]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 172.781333] nix-grpc-daemon[1899]: E0000 00:00:1790115785.401887 1899 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 172.783713] nix-grpc-daemon[1899]: E0000 00:00:1790115785.405446 1899 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.74 seconds) lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 172.812511] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 172.814724] nix-grpc-daemon[1899]: E0000 00:00:1790115785.413313 1940 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 172.817526] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 172.818327] nix-grpc-daemon[1899]: E0000 00:00:1790115785.435336 1940 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 172.820591] nix-grpc-daemon[1899]: E0000 00:00:1790115785.435483 1940 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 172.822872] nix-grpc-daemon[1899]: E0000 00:00:1790115785.450940 1940 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 172.830865] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 172.831915] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:50648 worker2 # [ 172.836956] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.38 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker2 # [ 174.280684] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 174.288463] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 174.333857] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:06Z level=debug event=want client=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=0 worker2 # [ 174.339901] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:06Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 174.346016] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:06Z level=debug event=want client=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=0 worker2 # [ 174.351209] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:06Z level=debug event=want client=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=0 worker1 # [ 174.388960] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 174.403997] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 174.428373] nix-grpc-daemon[1899]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 1/5) worker1 # [ 174.436874] nix-grpc-daemon[1899]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 116 ms (attempt 2/5) worker1 # [ 174.554727] nix-grpc-daemon[1899]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 309 ms (attempt 3/5) worker1 # [ 174.866121] nix-grpc-daemon[1899]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 134 ms (attempt 4/5) worker1 # [ 175.002760] nix-grpc-daemon[1899]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 175.009820] nix-grpc-daemon[1899]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 175.027695] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 175.044109] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 175.058558] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 175.099195] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:07Z level=info event=niks3_push_started pid=1959 worker1 # [ 175.252788] nix-grpc-daemon[1959]: time=2026-09-22T22:23:07.891Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 175.361178] nix-daemon[848]: accepted connection from pid 1972, user nix-grpc-daemon (trusted) worker1 # [ 175.382004] nix-daemon[848]: reaped child process 1975, status = succeeded lb # [ 175.373824] niks3-server[1100]: 2026/09/22 22:23:08 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 175.392168] nix-grpc-daemon[1959]: time=2026-09-22T22:23:08.031Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 175.396922] nix-grpc-daemon[1959]: time=2026-09-22T22:23:08.033Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)" lb # [ 175.413411] niks3-server[1100]: 2026/09/22 22:23:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign worker1 # [ 175.439440] nix-grpc-daemon[1959]: time=2026-09-22T22:23:08.077Z level=INFO msg="Uploading 1 narinfos" lb # [ 175.430248] niks3-server[1100]: 2026/09/22 22:23:08 INFO Signed narinfos id=62 count=1 lb # [ 175.439990] niks3-server[1100]: 2026/09/22 22:23:08 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst lb # [ 175.454024] niks3-server[1100]: 2026/09/22 22:23:08 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete worker1 # [ 175.471292] nix-grpc-daemon[1959]: time=2026-09-22T22:23:08.110Z level=INFO msg="Upload complete. (215ms)" lb # [ 175.465441] niks3-server[1100]: 2026/09/22 22:23:08 INFO Completed upload id=62 lb # [ 175.471059] niks3-server[1100]: 2026/09/22 22:23:08 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo worker1 # [ 175.480399] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 175.477370] niks3-server[1100]: 2026/09/22 22:23:08 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls worker1 # [ 175.488513] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 175.499328] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 175.505120] nix-daemon[848]: reaped child process 1957, status = succeeded worker1 # [ 175.516915] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 176.370308] nix-daemon[848]: reaped child process 1979, status = succeeded worker1 # [ 176.508892] nix-daemon[848]: accepted connection from pid 1995, user nix-grpc-daemon (trusted) worker1 # [ 176.529725] nix-daemon[848]: reaped child process 1998, status = succeeded worker1 # [ 176.550453] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.187Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 176.553512] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.187Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)" lb # [ 176.523908] niks3-server[1100]: 2026/09/22 22:23:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 176.588988] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.227Z level=INFO msg="Uploading 1 narinfos" lb # [ 176.570100] niks3-server[1100]: 2026/09/22 22:23:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign lb # [ 176.585811] niks3-server[1100]: 2026/09/22 22:23:09 INFO Signed narinfos id=63 count=1 lb # [ 176.596318] niks3-server[1100]: 2026/09/22 22:23:09 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv lb # [ 176.606002] niks3-server[1100]: 2026/09/22 22:23:09 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls lb # [ 176.617117] niks3-server[1100]: 2026/09/22 22:23:09 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst worker1 # [ 176.626771] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.266Z level=INFO msg="Upload complete. (300ms)" lb # [ 176.625809] niks3-server[1100]: 2026/09/22 22:23:09 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete lb # [ 176.630482] niks3-server[1100]: 2026/09/22 22:23:09 INFO Completed upload id=63 worker2 # [ 176.635688] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:09Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=1 lb # [ 176.634184] niks3-server[1100]: 2026/09/22 22:23:09 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo worker1 # [ 176.641948] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=1 outputs=1 worker2 # [ 176.648735] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:09Z level=debug event=want client=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=0 worker1 # [ 176.661587] nix-daemon[848]: reaped child process 1977, status = succeeded worker1 # [ 176.686604] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 176.703068] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 176.717890] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 176.732767] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 176.881165] nix-daemon[848]: accepted connection from pid 2002, user nix-grpc-daemon (trusted) worker1 # [ 176.901992] nix-daemon[848]: reaped child process 2005, status = succeeded lb # [ 176.889920] niks3-server[1100]: 2026/09/22 22:23:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 176.909174] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.548Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 176.910685] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.548Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)" lb # [ 176.922282] niks3-server[1100]: 2026/09/22 22:23:09 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst lb # [ 176.936456] niks3-server[1100]: 2026/09/22 22:23:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign worker1 # [ 176.951616] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.590Z level=INFO msg="Uploading 1 narinfos" lb # [ 176.947379] niks3-server[1100]: 2026/09/22 22:23:09 INFO Signed narinfos id=64 count=1 lb # [ 176.953007] niks3-server[1100]: 2026/09/22 22:23:09 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls lb # [ 176.965221] niks3-server[1100]: 2026/09/22 22:23:09 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete lb # [ 176.972151] niks3-server[1100]: 2026/09/22 22:23:09 INFO Completed upload id=64 worker1 # [ 176.981349] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.620Z level=INFO msg="Upload complete. (234ms)" lb # [ 176.974902] niks3-server[1100]: 2026/09/22 22:23:09 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo worker1 # [ 176.992348] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 177.000654] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 177.013772] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 177.018984] nix-daemon[848]: reaped child process 2000, status = succeeded worker1 # [ 177.029521] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 178.116553] nix-daemon[848]: reaped child process 2009, status = succeeded worker1 # [ 178.210780] nix-daemon[848]: accepted connection from pid 2025, user nix-grpc-daemon (trusted) worker1 # [ 178.232115] nix-daemon[848]: reaped child process 2028, status = succeeded lb # [ 178.221642] niks3-server[1100]: 2026/09/22 22:23:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 178.241953] nix-grpc-daemon[1959]: time=2026-09-22T22:23:10.880Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 178.243382] nix-grpc-daemon[1959]: time=2026-09-22T22:23:10.880Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)" lb # [ 178.250990] niks3-server[1100]: 2026/09/22 22:23:10 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls lb # [ 178.275530] niks3-server[1100]: 2026/09/22 22:23:10 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst worker1 # [ 178.296260] nix-grpc-daemon[1959]: time=2026-09-22T22:23:10.935Z level=INFO msg="Uploading 1 narinfos" lb # [ 178.291879] niks3-server[1100]: 2026/09/22 22:23:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign lb # [ 178.302129] niks3-server[1100]: 2026/09/22 22:23:10 INFO Signed narinfos id=65 count=1 lb # [ 178.309090] niks3-server[1100]: 2026/09/22 22:23:10 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker1 # [ 178.329284] nix-grpc-daemon[1959]: time=2026-09-22T22:23:10.968Z level=INFO msg="Upload complete. (287ms)" lb # [ 178.319883] niks3-server[1100]: 2026/09/22 22:23:10 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete lb # [ 178.329168] niks3-server[1100]: 2026/09/22 22:23:10 INFO Completed upload id=65 worker2 # [ 178.334732] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:10Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=1 lb # [ 178.332772] niks3-server[1100]: 2026/09/22 22:23:10 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo worker1 # [ 178.341181] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=2 outputs=1 worker2 # [ 178.348560] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:10Z level=debug event=want client=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=0 worker1 # [ 178.364462] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 178.370715] nix-daemon[848]: reaped child process 2007, status = succeeded worker1 # [ 178.379157] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 178.393048] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 178.405844] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 178.552557] nix-daemon[848]: accepted connection from pid 2032, user nix-grpc-daemon (trusted) worker1 # [ 178.573469] nix-daemon[848]: reaped child process 2035, status = succeeded lb # [ 178.564700] niks3-server[1100]: 2026/09/22 22:23:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 178.584178] nix-grpc-daemon[1959]: time=2026-09-22T22:23:11.223Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 178.586238] nix-grpc-daemon[1959]: time=2026-09-22T22:23:11.223Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)" lb # [ 178.601046] niks3-server[1100]: 2026/09/22 22:23:11 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls worker1 # [ 178.620195] nix-grpc-daemon[1959]: time=2026-09-22T22:23:11.259Z level=INFO msg="Uploading 1 narinfos" lb # [ 178.613467] niks3-server[1100]: 2026/09/22 22:23:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign lb # [ 178.624698] niks3-server[1100]: 2026/09/22 22:23:11 INFO Signed narinfos id=66 count=1 lb # [ 178.627254] niks3-server[1100]: 2026/09/22 22:23:11 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst lb # [ 178.639894] niks3-server[1100]: 2026/09/22 22:23:11 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete lb # [ 178.646902] niks3-server[1100]: 2026/09/22 22:23:11 INFO Completed upload id=66 worker1 # [ 178.657000] nix-grpc-daemon[1959]: time=2026-09-22T22:23:11.296Z level=INFO msg="Upload complete. (237ms)" lb # [ 178.649968] niks3-server[1100]: 2026/09/22 22:23:11 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo worker1 # [ 178.667763] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 178.683718] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 178.690573] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 178.695650] nix-daemon[848]: reaped child process 2030, status = succeeded worker1 # [ 178.706497] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 179.605191] nix-daemon[848]: reaped child process 2039, status = succeeded worker1 # [ 179.674762] nix-daemon[848]: accepted connection from pid 2055, user nix-grpc-daemon (trusted) worker1 # [ 179.697661] nix-daemon[848]: reaped child process 2058, status = succeeded lb # [ 179.687113] niks3-server[1100]: 2026/09/22 22:23:12 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 179.706718] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.345Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 179.710297] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.348Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)" lb # [ 179.730500] niks3-server[1100]: 2026/09/22 22:23:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign worker1 # [ 179.748887] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.388Z level=INFO msg="Uploading 1 narinfos" lb # [ 179.745623] niks3-server[1100]: 2026/09/22 22:23:12 INFO Signed narinfos id=67 count=1 lb # [ 179.753213] niks3-server[1100]: 2026/09/22 22:23:12 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls lb # [ 179.761058] niks3-server[1100]: 2026/09/22 22:23:12 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 179.771448] niks3-server[1100]: 2026/09/22 22:23:12 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker1 # [ 179.785380] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.424Z level=INFO msg="Upload complete. (267ms)" lb # [ 179.776706] niks3-server[1100]: 2026/09/22 22:23:12 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete lb # [ 179.781557] niks3-server[1100]: 2026/09/22 22:23:12 INFO Completed upload id=67 lb # [ 179.784790] niks3-server[1100]: 2026/09/22 22:23:12 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo worker2 # [ 179.793289] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:12Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 179.797167] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=3 outputs=1 worker2 # [ 179.806804] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:12Z level=debug event=want client=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=0 worker1 # [ 179.819419] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 179.825700] nix-daemon[848]: reaped child process 2037, status = succeeded worker1 # [ 179.833653] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 179.846477] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.485Z level=INFO msg="All 1 paths already cached" worker1 # [ 179.855725] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 179.868878] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) lb # [ 179.918837] envoy[895]: conn peer=[2001:db8:1::3]:55530 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=911 tx=361 ms=21174 worker1 # [ 180.018239] nix-daemon[848]: accepted connection from pid 2062, user nix-grpc-daemon (trusted) worker1 # [ 180.036782] nix-daemon[848]: reaped child process 2065, status = succeeded lb # [ 180.031748] niks3-server[1100]: 2026/09/22 22:23:12 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 180.050905] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.689Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 180.052919] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.689Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)" lb # [ 180.062699] niks3-server[1100]: 2026/09/22 22:23:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign worker1 # [ 180.078640] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.717Z level=INFO msg="Uploading 1 narinfos" lb # [ 180.077886] niks3-server[1100]: 2026/09/22 22:23:12 INFO Signed narinfos id=68 count=1 lb # [ 180.085428] niks3-server[1100]: 2026/09/22 22:23:12 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls lb # [ 180.091875] niks3-server[1100]: 2026/09/22 22:23:12 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst lb # [ 180.100339] niks3-server[1100]: 2026/09/22 22:23:12 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete lb # [ 180.109432] niks3-server[1100]: 2026/09/22 22:23:12 INFO Completed upload id=68 lb # [ 180.111847] niks3-server[1100]: 2026/09/22 22:23:12 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo worker1 # [ 180.116663] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.755Z level=INFO msg="Upload complete. (234ms)" worker1 # [ 180.128511] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 180.144920] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 180.151571] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 180.156980] nix-daemon[848]: reaped child process 2060, status = succeeded worker1 # [ 180.169538] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 181.608856] nix-daemon[848]: reaped child process 2069, status = succeeded worker1 # [ 181.716432] nix-daemon[848]: accepted connection from pid 2085, user nix-grpc-daemon (trusted) worker1 # [ 181.738317] nix-daemon[848]: reaped child process 2088, status = succeeded lb # [ 181.727128] niks3-server[1100]: 2026/09/22 22:23:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 181.748431] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.386Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 181.749952] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.386Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)" lb # [ 181.767305] niks3-server[1100]: 2026/09/22 22:23:14 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker1 # [ 181.792145] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.431Z level=INFO msg="Uploading 1 narinfos" lb # [ 181.781870] niks3-server[1100]: 2026/09/22 22:23:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign lb # [ 181.794446] niks3-server[1100]: 2026/09/22 22:23:14 INFO Signed narinfos id=69 count=1 lb # [ 181.801777] niks3-server[1100]: 2026/09/22 22:23:14 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls lb # [ 181.810525] niks3-server[1100]: 2026/09/22 22:23:14 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst lb # [ 181.822784] niks3-server[1100]: 2026/09/22 22:23:14 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 181.827894] niks3-server[1100]: 2026/09/22 22:23:14 INFO Completed upload id=69 worker1 # [ 181.838070] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.477Z level=INFO msg="Upload complete. (258ms)" lb # [ 181.833206] niks3-server[1100]: 2026/09/22 22:23:14 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo worker2 # [ 181.847379] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:14Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 181.849141] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=4 outputs=1 worker2 # [ 181.861913] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:14Z level=debug event=want client=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=0 worker1 # [ 181.871306] nix-daemon[848]: reaped child process 2067, status = succeeded worker1 # [ 181.906486] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 181.922242] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 181.938024] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.577Z level=INFO msg="All 1 paths already cached" worker1 # [ 181.947950] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 181.962735] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 182.118654] nix-daemon[848]: accepted connection from pid 2092, user nix-grpc-daemon (trusted) worker1 # [ 182.140057] nix-daemon[848]: reaped child process 2095, status = succeeded lb # [ 182.131163] niks3-server[1100]: 2026/09/22 22:23:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 182.150397] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.789Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 182.151918] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.789Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)" lb # [ 182.161080] niks3-server[1100]: 2026/09/22 22:23:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign worker1 # [ 182.176618] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.815Z level=INFO msg="Uploading 1 narinfos" lb # [ 182.174834] niks3-server[1100]: 2026/09/22 22:23:14 INFO Signed narinfos id=70 count=1 lb # [ 182.183155] niks3-server[1100]: 2026/09/22 22:23:14 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls lb # [ 182.196340] niks3-server[1100]: 2026/09/22 22:23:14 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst worker1 # [ 182.213573] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.852Z level=INFO msg="Upload complete. (233ms)" lb # [ 182.204960] niks3-server[1100]: 2026/09/22 22:23:14 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete lb # [ 182.214300] niks3-server[1100]: 2026/09/22 22:23:14 INFO Completed upload id=70 worker1 # [ 182.223464] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 182.218421] niks3-server[1100]: 2026/09/22 22:23:14 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo worker1 # [ 182.240515] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 182.246392] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 182.251165] nix-daemon[848]: reaped child process 2090, status = succeeded worker1 # [ 182.261910] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 183.126664] nix-daemon[848]: reaped child process 2099, status = succeeded worker1 # [ 183.240367] nix-daemon[848]: accepted connection from pid 2115, user nix-grpc-daemon (trusted) worker1 # [ 183.262703] nix-daemon[848]: reaped child process 2118, status = succeeded lb # [ 183.255656] niks3-server[1100]: 2026/09/22 22:23:15 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 183.276498] nix-grpc-daemon[1959]: time=2026-09-22T22:23:15.914Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 183.278275] nix-grpc-daemon[1959]: time=2026-09-22T22:23:15.914Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)" lb # [ 183.293353] niks3-server[1100]: 2026/09/22 22:23:15 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv lb # [ 183.306708] niks3-server[1100]: 2026/09/22 22:23:15 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls worker1 # [ 183.326521] nix-grpc-daemon[1959]: time=2026-09-22T22:23:15.965Z level=INFO msg="Uploading 1 narinfos" lb # [ 183.318599] niks3-server[1100]: 2026/09/22 22:23:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign lb # [ 183.330134] niks3-server[1100]: 2026/09/22 22:23:15 INFO Signed narinfos id=71 count=1 lb # [ 183.335802] niks3-server[1100]: 2026/09/22 22:23:15 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst lb # [ 183.345642] niks3-server[1100]: 2026/09/22 22:23:15 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete worker1 # [ 183.361567] nix-grpc-daemon[1959]: time=2026-09-22T22:23:16.000Z level=INFO msg="Upload complete. (288ms)" lb # [ 183.353513] niks3-server[1100]: 2026/09/22 22:23:15 INFO Completed upload id=71 lb # [ 183.356477] niks3-server[1100]: 2026/09/22 22:23:15 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo worker2 # [ 183.369495] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:16Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 183.377200] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=5 outputs=1 worker2 # [ 183.383039] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:16Z level=debug event=want client=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=0 worker1 # [ 183.392696] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 183.406212] nix-daemon[848]: reaped child process 2097, status = succeeded worker1 # [ 183.411364] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 183.422073] nix-grpc-daemon[1959]: time=2026-09-22T22:23:16.061Z level=INFO msg="All 1 paths already cached" worker1 # [ 183.431340] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 183.446171] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 183.585404] nix-daemon[848]: accepted connection from pid 2122, user nix-grpc-daemon (trusted) worker1 # [ 183.607252] nix-daemon[848]: reaped child process 2125, status = succeeded lb # [ 183.602507] niks3-server[1100]: 2026/09/22 22:23:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 183.621115] nix-grpc-daemon[1959]: time=2026-09-22T22:23:16.260Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 183.622629] nix-grpc-daemon[1959]: time=2026-09-22T22:23:16.260Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)" worker1 # [ 183.641263] nix-grpc-daemon[1959]: time=2026-09-22T22:23:16.280Z level=INFO msg="Uploading 1 narinfos" lb # [ 183.631436] niks3-server[1100]: 2026/09/22 22:23:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign lb # [ 183.644157] niks3-server[1100]: 2026/09/22 22:23:16 INFO Signed narinfos id=72 count=1 lb # [ 183.652590] niks3-server[1100]: 2026/09/22 22:23:16 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls worker1 # [ 183.679357] nix-grpc-daemon[1959]: time=2026-09-22T22:23:16.318Z level=INFO msg="Upload complete. (221ms)" lb # [ 183.671206] niks3-server[1100]: 2026/09/22 22:23:16 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete worker1 # [ 183.687887] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 183.682615] niks3-server[1100]: 2026/09/22 22:23:16 INFO Completed upload id=72 lb # [ 183.687366] niks3-server[1100]: 2026/09/22 22:23:16 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst worker1 # [ 183.703198] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 lb # [ 183.693167] niks3-server[1100]: 2026/09/22 22:23:16 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo worker1 # [ 183.709496] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 183.714717] nix-daemon[848]: reaped child process 2120, status = succeeded worker1 # [ 183.725081] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 184.596263] nix-daemon[848]: reaped child process 2129, status = succeeded worker1 # [ 184.685475] nix-daemon[848]: accepted connection from pid 2145, user nix-grpc-daemon (trusted) worker1 # [ 184.706503] nix-daemon[848]: reaped child process 2148, status = succeeded lb # [ 184.702198] niks3-server[1100]: 2026/09/22 22:23:17 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 184.723162] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.361Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 184.725301] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.361Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)" lb # [ 184.742112] niks3-server[1100]: 2026/09/22 22:23:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign worker1 # [ 184.759047] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.398Z level=INFO msg="Uploading 1 narinfos" lb # [ 184.756215] niks3-server[1100]: 2026/09/22 22:23:17 INFO Signed narinfos id=73 count=1 lb # [ 184.758896] niks3-server[1100]: 2026/09/22 22:23:17 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv lb # [ 184.775120] niks3-server[1100]: 2026/09/22 22:23:17 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst lb # [ 184.784127] niks3-server[1100]: 2026/09/22 22:23:17 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 184.787923] niks3-server[1100]: 2026/09/22 22:23:17 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls lb # [ 184.792308] niks3-server[1100]: 2026/09/22 22:23:17 INFO Completed upload id=73 worker1 # [ 184.792290] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.431Z level=INFO msg="Upload complete. (256ms)" lb # [ 184.795609] niks3-server[1100]: 2026/09/22 22:23:17 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo worker2 # [ 184.800981] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:17Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 184.806150] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=6 outputs=1 worker2 # [ 184.814214] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:17Z level=debug event=want client=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=0 worker1 # [ 184.824450] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 184.831196] nix-daemon[848]: reaped child process 2127, status = succeeded worker1 # [ 184.839567] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 184.852256] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.491Z level=INFO msg="All 2 paths already cached" worker1 # [ 184.861805] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 184.873600] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 185.011655] nix-daemon[848]: accepted connection from pid 2152, user nix-grpc-daemon (trusted) worker1 # [ 185.031912] nix-daemon[848]: reaped child process 2155, status = succeeded lb # [ 185.025755] niks3-server[1100]: 2026/09/22 22:23:17 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 185.044174] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.683Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 185.046190] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.685Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)" lb # [ 185.058552] niks3-server[1100]: 2026/09/22 22:23:17 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls worker1 # [ 185.080389] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.719Z level=INFO msg="Uploading 1 narinfos" lb # [ 185.074040] niks3-server[1100]: 2026/09/22 22:23:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign lb # [ 185.084620] niks3-server[1100]: 2026/09/22 22:23:17 INFO Signed narinfos id=74 count=1 lb # [ 185.087137] niks3-server[1100]: 2026/09/22 22:23:17 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst lb # [ 185.101730] niks3-server[1100]: 2026/09/22 22:23:17 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete worker1 # [ 185.116171] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.755Z level=INFO msg="Upload complete. (229ms)" lb # [ 185.106198] niks3-server[1100]: 2026/09/22 22:23:17 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo lb # [ 185.110229] niks3-server[1100]: 2026/09/22 22:23:17 INFO Completed upload id=74 worker1 # [ 185.126490] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 185.142854] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 185.150023] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 185.155307] nix-daemon[848]: reaped child process 2150, status = succeeded worker1 # [ 185.168889] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 185.833572] nix-daemon[848]: reaped child process 2159, status = succeeded worker1 # [ 185.958903] nix-daemon[848]: accepted connection from pid 2175, user nix-grpc-daemon (trusted) worker1 # [ 185.977293] nix-daemon[848]: reaped child process 2178, status = succeeded lb # [ 185.973457] niks3-server[1100]: 2026/09/22 22:23:18 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 185.996487] nix-grpc-daemon[1959]: time=2026-09-22T22:23:18.632Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 185.998324] nix-grpc-daemon[1959]: time=2026-09-22T22:23:18.632Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)" lb # [ 186.016477] niks3-server[1100]: 2026/09/22 22:23:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign worker1 # [ 186.035290] nix-grpc-daemon[1959]: time=2026-09-22T22:23:18.674Z level=INFO msg="Uploading 1 narinfos" lb # [ 186.031983] niks3-server[1100]: 2026/09/22 22:23:18 INFO Signed narinfos id=75 count=1 lb # [ 186.039262] niks3-server[1100]: 2026/09/22 22:23:18 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls lb # [ 186.049113] niks3-server[1100]: 2026/09/22 22:23:18 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker1 # [ 186.069372] nix-grpc-daemon[1959]: time=2026-09-22T22:23:18.708Z level=INFO msg="Upload complete. (286ms)" lb # [ 186.060153] niks3-server[1100]: 2026/09/22 22:23:18 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst lb # [ 186.063756] niks3-server[1100]: 2026/09/22 22:23:18 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete lb # [ 186.068736] niks3-server[1100]: 2026/09/22 22:23:18 INFO Completed upload id=75 lb # [ 186.071802] niks3-server[1100]: 2026/09/22 22:23:18 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo worker2 # [ 186.076815] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:18Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 186.081194] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=7 outputs=1 worker1 # [ 186.103531] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 186.110084] nix-daemon[848]: reaped child process 2157, status = succeeded worker1 # [ 186.117793] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 186.129605] nix-grpc-daemon[1959]: time=2026-09-22T22:23:18.768Z level=INFO msg="All 2 paths already cached" worker1 # [ 186.139389] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 186.153011] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 186.299900] nix-daemon[848]: accepted connection from pid 2182, user nix-grpc-daemon (trusted) worker1 # [ 186.318368] nix-daemon[848]: reaped child process 2185, status = succeeded lb # [ 186.313005] niks3-server[1100]: 2026/09/22 22:23:18 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 186.332103] nix-grpc-daemon[1959]: time=2026-09-22T22:23:18.971Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 186.333567] nix-grpc-daemon[1959]: time=2026-09-22T22:23:18.971Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)" lb # [ 186.345864] niks3-server[1100]: 2026/09/22 22:23:18 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst lb # [ 186.362447] niks3-server[1100]: 2026/09/22 22:23:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign worker1 # [ 186.378070] nix-grpc-daemon[1959]: time=2026-09-22T22:23:19.017Z level=INFO msg="Uploading 1 narinfos" lb # [ 186.374560] niks3-server[1100]: 2026/09/22 22:23:19 INFO Signed narinfos id=76 count=1 lb # [ 186.381548] niks3-server[1100]: 2026/09/22 22:23:19 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls lb # [ 186.392945] niks3-server[1100]: 2026/09/22 22:23:19 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete lb # [ 186.399071] niks3-server[1100]: 2026/09/22 22:23:19 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo lb # [ 186.404831] niks3-server[1100]: 2026/09/22 22:23:19 INFO Completed upload id=76 worker1 # [ 186.414819] nix-grpc-daemon[1959]: time=2026-09-22T22:23:19.054Z level=INFO msg="Upload complete. (248ms)" worker1 # [ 186.425707] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 186.442952] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 186.453785] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 186.460408] nix-daemon[848]: reaped child process 2180, status = succeeded worker1 # [ 186.469565] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 187.378031] nix-daemon[848]: reaped child process 2189, status = succeeded worker1 # [ 187.447009] nix-daemon[848]: accepted connection from pid 2205, user nix-grpc-daemon (trusted) worker1 # [ 187.469780] nix-daemon[848]: reaped child process 2208, status = succeeded lb # [ 187.459248] niks3-server[1100]: 2026/09/22 22:23:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 187.479739] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.118Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 187.482684] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.120Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)" lb # [ 187.500947] niks3-server[1100]: 2026/09/22 22:23:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign worker1 # [ 187.519936] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.159Z level=INFO msg="Uploading 1 narinfos" lb # [ 187.514281] niks3-server[1100]: 2026/09/22 22:23:20 INFO Signed narinfos id=77 count=1 lb # [ 187.525144] niks3-server[1100]: 2026/09/22 22:23:20 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls lb # [ 187.536979] niks3-server[1100]: 2026/09/22 22:23:20 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv lb # [ 187.543351] niks3-server[1100]: 2026/09/22 22:23:20 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst worker1 # [ 187.558658] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.198Z level=INFO msg="Upload complete. (280ms)" lb # [ 187.559830] niks3-server[1100]: 2026/09/22 22:23:20 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete worker2 # [ 187.565441] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:20Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=1 lb # [ 187.566018] niks3-server[1100]: 2026/09/22 22:23:20 INFO Completed upload id=77 worker1 # [ 187.573221] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=8 outputs=1 lb # [ 187.569468] niks3-server[1100]: 2026/09/22 22:23:20 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo worker1 # [ 187.592325] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 187.599976] nix-daemon[848]: reaped child process 2187, status = succeeded worker1 # [ 187.629846] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 187.644893] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.284Z level=INFO msg="All 2 paths already cached" worker1 # [ 187.654983] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 187.672344] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 187.823465] nix-daemon[848]: accepted connection from pid 2212, user nix-grpc-daemon (trusted) worker1 # [ 187.843771] nix-daemon[848]: reaped child process 2215, status = succeeded lb # [ 187.834941] niks3-server[1100]: 2026/09/22 22:23:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 187.854528] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.493Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 187.856225] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.493Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)" lb # [ 187.865366] niks3-server[1100]: 2026/09/22 22:23:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign worker1 # [ 187.883549] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.522Z level=INFO msg="Uploading 1 narinfos" lb # [ 187.879993] niks3-server[1100]: 2026/09/22 22:23:20 INFO Signed narinfos id=78 count=1 lb # [ 187.882480] niks3-server[1100]: 2026/09/22 22:23:20 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls lb # [ 187.895886] niks3-server[1100]: 2026/09/22 22:23:20 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst worker1 # [ 187.911001] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.550Z level=INFO msg="Upload complete. (226ms)" lb # [ 187.901645] niks3-server[1100]: 2026/09/22 22:23:20 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete lb # [ 187.906794] niks3-server[1100]: 2026/09/22 22:23:20 INFO Completed upload id=78 lb # [ 187.911826] niks3-server[1100]: 2026/09/22 22:23:20 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo worker1 # [ 187.920646] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 187.928866] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 worker1 # [ 187.940098] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 187.945445] nix-daemon[848]: reaped child process 2210, status = succeeded worker1 # [ 187.958224] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker1 # [ 189.261805] nix-daemon[848]: reaped child process 2219, status = succeeded worker1 # [ 189.331184] nix-daemon[848]: accepted connection from pid 2235, user nix-grpc-daemon (trusted) worker1 # [ 189.351860] nix-daemon[848]: reaped child process 2238, status = succeeded lb # [ 189.340716] niks3-server[1100]: 2026/09/22 22:23:21 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 189.364407] nix-grpc-daemon[1959]: time=2026-09-22T22:23:22.000Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 189.365975] nix-grpc-daemon[1959]: time=2026-09-22T22:23:22.000Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)" worker1 # [ 189.408483] nix-grpc-daemon[1959]: time=2026-09-22T22:23:22.047Z level=INFO msg="Uploading 1 narinfos" lb # [ 189.390605] niks3-server[1100]: 2026/09/22 22:23:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign lb # [ 189.405085] niks3-server[1100]: 2026/09/22 22:23:22 INFO Signed narinfos id=79 count=1 lb # [ 189.407836] niks3-server[1100]: 2026/09/22 22:23:22 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst worker1 # [ 189.440438] nix-grpc-daemon[1959]: time=2026-09-22T22:23:22.079Z level=INFO msg="Upload complete. (271ms)" lb # [ 189.425720] niks3-server[1100]: 2026/09/22 22:23:22 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv lb # [ 189.438075] niks3-server[1100]: 2026/09/22 22:23:22 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls lb # [ 189.443151] niks3-server[1100]: 2026/09/22 22:23:22 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete lb # [ 189.448107] niks3-server[1100]: 2026/09/22 22:23:22 INFO Completed upload id=79 worker1 # [ 189.453166] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=9 outputs=1 lb # [ 189.451788] niks3-server[1100]: 2026/09/22 22:23:22 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo worker1 # [ 189.472292] nix-daemon[848]: reaped child process 2217, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.46 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 189.653711] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 912e5a58db124e7e8f771605eb9140c0 client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 189.851790] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:53642 client # [ 189.855549] nix[1572]: don't know how to build these paths: client # [ 189.856723] nix[1572]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv worker1 # [ 189.864754] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53642 worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client # [ 189.887475] nix[1572]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker worker2 # [ 189.889108] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:22Z level=debug event=want client=2 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=0 lb # [ 189.919761] envoy[895]: conn peer=[2001:db8:1::1]:50512 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27170 tx=12400 ms=15196 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code client # [ 219.893771] nix[1572]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip} client # [ 219.906816] nix[1572]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed lb # [ 219.919643] envoy[895]: conn peer=[2001:db8:1::1]:58762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=30070 client # [ 219.945500] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 219.953363] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 219.956863] systemd[1]: gpu.service: Consumed 191ms CPU time over 30.297s wall clock time, 35.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.10 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.30 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.21 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1 # [ 221.798881] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:54Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2 client # don't know how to build these paths: client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker2 # [ 222.906145] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:36886 worker1 # [ 222.914881] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:53642 worker2 # [ 222.937576] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=debug event=want client=3 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=0 worker2 # [ 222.944779] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 222.983700] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 223.001086] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36886 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'... worker2 # [ 223.018067] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 223.033252] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 223.197111] nix-daemon[837]: accepted connection from pid 1704, user nix-grpc-daemon (trusted) worker2 # [ 223.213387] nix-daemon[837]: reaped child process 1707, status = succeeded lb # [ 223.216588] niks3-server[1100]: 2026/09/22 22:23:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 223.236377] nix-grpc-daemon[1512]: time=2026-09-22T22:23:55.878Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 223.238164] nix-grpc-daemon[1512]: time=2026-09-22T22:23:55.878Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)" lb # [ 223.251433] niks3-server[1100]: 2026/09/22 22:23:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign worker2 # [ 223.267086] nix-grpc-daemon[1512]: time=2026-09-22T22:23:55.909Z level=INFO msg="Uploading 1 narinfos" lb # [ 223.267335] niks3-server[1100]: 2026/09/22 22:23:55 INFO Signed narinfos id=80 count=1 lb # [ 223.276912] niks3-server[1100]: 2026/09/22 22:23:55 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls lb # [ 223.288643] niks3-server[1100]: 2026/09/22 22:23:55 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst worker2 # [ 223.304813] nix-grpc-daemon[1512]: time=2026-09-22T22:23:55.947Z level=INFO msg="Upload complete. (254ms)" lb # [ 223.299650] niks3-server[1100]: 2026/09/22 22:23:55 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete lb # [ 223.309155] niks3-server[1100]: 2026/09/22 22:23:55 INFO Completed upload id=80 lb # [ 223.311356] niks3-server[1100]: 2026/09/22 22:23:55 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo worker2 # [ 223.315865] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 223.328540] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 223.340391] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 223.345814] nix-daemon[837]: reaped child process 1702, status = succeeded client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051... worker2 # [ 223.358872] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 224.421373] nix-daemon[837]: reaped child process 1711, status = succeeded worker2 # [ 224.547992] nix-daemon[837]: accepted connection from pid 1727, user nix-grpc-daemon (trusted) worker2 # [ 224.569196] nix-daemon[837]: reaped child process 1730, status = succeeded lb # [ 224.566861] niks3-server[1100]: 2026/09/22 22:23:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 224.587830] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.227Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 224.589855] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.227Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)" lb # [ 224.612073] niks3-server[1100]: 2026/09/22 22:23:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign worker2 # [ 224.626570] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.269Z level=INFO msg="Uploading 1 narinfos" lb # [ 224.625981] niks3-server[1100]: 2026/09/22 22:23:57 INFO Signed narinfos id=81 count=1 lb # [ 224.633382] niks3-server[1100]: 2026/09/22 22:23:57 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls lb # [ 224.641038] niks3-server[1100]: 2026/09/22 22:23:57 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv lb # [ 224.650366] niks3-server[1100]: 2026/09/22 22:23:57 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst worker2 # [ 224.661478] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.304Z level=INFO msg="Upload complete. (293ms)" lb # [ 224.656283] niks3-server[1100]: 2026/09/22 22:23:57 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete lb # [ 224.660573] niks3-server[1100]: 2026/09/22 22:23:57 INFO Completed upload id=81 lb # [ 224.663649] niks3-server[1100]: 2026/09/22 22:23:57 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo worker2 # [ 224.677639] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=10 outputs=1 worker2 # [ 224.700631] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=debug event=want client=3 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=0 worker2 # [ 224.706980] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 224.712403] nix-daemon[837]: reaped child process 1709, status = succeeded worker2 # [ 224.734617] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 224.750288] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 224.765108] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.407Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'... worker2 # [ 224.776416] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 224.790449] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 224.945171] nix-daemon[837]: accepted connection from pid 1736, user nix-grpc-daemon (trusted) worker2 # [ 224.964309] nix-daemon[837]: reaped child process 1739, status = succeeded lb # [ 224.962655] niks3-server[1100]: 2026/09/22 22:23:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 224.978207] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.620Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 224.980236] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.620Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)" lb # [ 224.993861] niks3-server[1100]: 2026/09/22 22:23:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign worker2 # [ 225.009062] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.651Z level=INFO msg="Uploading 1 narinfos" lb # [ 225.009294] niks3-server[1100]: 2026/09/22 22:23:57 INFO Signed narinfos id=82 count=1 lb # [ 225.015934] niks3-server[1100]: 2026/09/22 22:23:57 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst worker2 # [ 225.032988] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.675Z level=INFO msg="Upload complete. (226ms)" lb # [ 225.029109] niks3-server[1100]: 2026/09/22 22:23:57 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls worker2 # [ 225.040136] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=0 paths=1 nar_bytes_in=632 lb # [ 225.034032] niks3-server[1100]: 2026/09/22 22:23:57 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete lb # [ 225.039522] niks3-server[1100]: 2026/09/22 22:23:57 INFO Completed upload id=82 lb # [ 225.043981] niks3-server[1100]: 2026/09/22 22:23:57 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo worker2 # [ 225.053340] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 225.059687] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 225.064665] nix-daemon[837]: reaped child process 1734, status = succeeded client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051... worker2 # [ 225.077363] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 225.923831] nix-daemon[837]: reaped child process 1743, status = succeeded worker2 # [ 226.054992] nix-daemon[837]: accepted connection from pid 1759, user nix-grpc-daemon (trusted) worker2 # [ 226.073269] nix-daemon[837]: reaped child process 1762, status = succeeded lb # [ 226.073261] niks3-server[1100]: 2026/09/22 22:23:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 226.092410] nix-grpc-daemon[1512]: time=2026-09-22T22:23:58.731Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 226.093961] nix-grpc-daemon[1512]: time=2026-09-22T22:23:58.731Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)" lb # [ 226.118470] niks3-server[1100]: 2026/09/22 22:23:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign worker2 # [ 226.137597] nix-grpc-daemon[1512]: time=2026-09-22T22:23:58.780Z level=INFO msg="Uploading 1 narinfos" lb # [ 226.133089] niks3-server[1100]: 2026/09/22 22:23:58 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv lb # [ 226.142080] niks3-server[1100]: 2026/09/22 22:23:58 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls lb # [ 226.154116] niks3-server[1100]: 2026/09/22 22:23:58 INFO Signed narinfos id=83 count=1 worker2 # [ 226.165264] nix-grpc-daemon[1512]: time=2026-09-22T22:23:58.807Z level=INFO msg="Upload complete. (278ms)" lb # [ 226.159598] niks3-server[1100]: 2026/09/22 22:23:58 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst lb # [ 226.164743] niks3-server[1100]: 2026/09/22 22:23:58 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete lb # [ 226.172362] niks3-server[1100]: 2026/09/22 22:23:58 INFO Completed upload id=83 lb # [ 226.175330] niks3-server[1100]: 2026/09/22 22:23:58 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo worker2 # [ 226.180561] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=11 outputs=1 worker2 # [ 226.201092] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:58Z level=debug event=want client=3 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=0 worker2 # [ 226.205479] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:58Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 226.210005] nix-daemon[837]: reaped child process 1741, status = succeeded worker2 # [ 226.242572] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 226.257662] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 226.271921] nix-grpc-daemon[1512]: time=2026-09-22T22:23:58.914Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'... worker2 # [ 226.283494] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 226.297122] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 226.454254] nix-daemon[837]: accepted connection from pid 1768, user nix-grpc-daemon (trusted) worker2 # [ 226.474519] nix-daemon[837]: reaped child process 1771, status = succeeded lb # [ 226.466409] niks3-server[1100]: 2026/09/22 22:23:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 226.481778] nix-grpc-daemon[1512]: time=2026-09-22T22:23:59.124Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 226.483597] nix-grpc-daemon[1512]: time=2026-09-22T22:23:59.124Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)" lb # [ 226.493877] niks3-server[1100]: 2026/09/22 22:23:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign worker2 # [ 226.508640] nix-grpc-daemon[1512]: time=2026-09-22T22:23:59.151Z level=INFO msg="Uploading 1 narinfos" lb # [ 226.508736] niks3-server[1100]: 2026/09/22 22:23:59 INFO Signed narinfos id=84 count=1 lb # [ 226.517816] niks3-server[1100]: 2026/09/22 22:23:59 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls lb # [ 226.528343] niks3-server[1100]: 2026/09/22 22:23:59 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst worker2 # [ 226.543482] nix-grpc-daemon[1512]: time=2026-09-22T22:23:59.186Z level=INFO msg="Upload complete. (229ms)" lb # [ 226.540709] niks3-server[1100]: 2026/09/22 22:23:59 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete lb # [ 226.547608] niks3-server[1100]: 2026/09/22 22:23:59 INFO Completed upload id=84 lb # [ 226.550276] niks3-server[1100]: 2026/09/22 22:23:59 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo worker2 # [ 226.552298] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=0 paths=1 nar_bytes_in=760 worker2 # [ 226.567514] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 worker2 # [ 226.574861] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) worker2 # [ 226.579904] nix-daemon[837]: reaped child process 1766, status = succeeded client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051... client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker2 # [ 226.594873] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 228.103215] nix-daemon[837]: reaped child process 1775, status = succeeded worker2 # [ 228.173745] nix-daemon[837]: accepted connection from pid 1791, user nix-grpc-daemon (trusted) worker2 # [ 228.191555] nix-daemon[837]: reaped child process 1794, status = succeeded lb # [ 228.193568] niks3-server[1100]: 2026/09/22 22:24:00 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 228.210656] nix-grpc-daemon[1512]: time=2026-09-22T22:24:00.851Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 228.212474] nix-grpc-daemon[1512]: time=2026-09-22T22:24:00.852Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)" lb # [ 228.227625] niks3-server[1100]: 2026/09/22 22:24:00 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst lb # [ 228.247851] niks3-server[1100]: 2026/09/22 22:24:00 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker2 # [ 228.261338] nix-grpc-daemon[1512]: time=2026-09-22T22:24:00.903Z level=INFO msg="Uploading 1 narinfos" lb # [ 228.257513] niks3-server[1100]: 2026/09/22 22:24:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign lb # [ 228.265263] niks3-server[1100]: 2026/09/22 22:24:00 INFO Signed narinfos id=85 count=1 lb # [ 228.267788] niks3-server[1100]: 2026/09/22 22:24:00 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls lb # [ 228.279515] niks3-server[1100]: 2026/09/22 22:24:00 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete lb # [ 228.285173] niks3-server[1100]: 2026/09/22 22:24:00 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo worker2 # [ 228.290823] nix-grpc-daemon[1512]: time=2026-09-22T22:24:00.933Z level=INFO msg="Upload complete. (284ms)" lb # [ 228.289961] niks3-server[1100]: 2026/09/22 22:24:00 INFO Completed upload id=85 worker2 # [ 228.305199] nix-grpc-daemon[1424]: ts=2026-09-22T22:24:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=12 outputs=1 worker2 # [ 228.322587] nix-daemon[837]: reaped child process 1773, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.67 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 228.806103] nix-grpc-daemon[1899]: ts=2026-09-22T22:24:01Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 9.51 seconds) (finished: run the VM test script, in 230.84 seconds) lb # [ 229.920375] envoy[895]: conn peer=[2001:db8:1::1]:37740 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9545 tx=4358 ms=5413 test script finished in 231.11s cleanup kill QemuMachine (pid 49) client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) client # [2026-09-22T22:24:02Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-22T22:24:02Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-22T22:24:02Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 50) lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) lb # [2026-09-22T22:24:02Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-22T22:24:02Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-22T22:24:02Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker1 # [2026-09-22T22:24:03Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-22T22:24:03Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-22T22:24:03Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker2 # [2026-09-22T22:24:03Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-22T22:24:03Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-22T22:24:03Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.25 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh