tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm worker1 # Disk image does not exist, creating the virtualisation disk image... lb # Disk image does not exist, creating the virtualisation disk image... worker1 # Formatting '/build/vm-state-worker1/tmp.UvaNkIoGPg', fmt=raw size=1073741824 worker1: QEMU running (pid 51) lb # Formatting '/build/vm-state-lb/tmp.0UL2hI1yNH', fmt=raw size=1073741824 lb: QEMU running (pid 49) worker2 # Disk image does not exist, creating the virtualisation disk image... worker2: QEMU running (pid 52) lb # mke2fs 1.47.4 (6-Mar-2025) client # Disk image does not exist, creating the virtualisation disk image... client: QEMU running (pid 50) lb # Discarding device blocks: 0/262144 done client # Formatting '/build/vm-state-client/tmp.fadhmClmP3', fmt=raw size=1073741824 worker2 # Formatting '/build/vm-state-worker2/tmp.hhniQSML51', fmt=raw size=1073741824 worker1 # mke2fs 1.47.4 (6-Mar-2025) lb # Creating filesystem with 262144 4k blocks and 65536 inodes client # mke2fs 1.47.4 (6-Mar-2025) (finished: start all VMs, in 0.46 seconds) lb # Filesystem UUID: 32ce671e-284c-4bdb-b1b3-4bb2ba4325d8 client # Discarding device blocks: 0/262144 done client: waiting for unit network-addresses-eth1.service lb # Superblock backups stored on blocks: worker2 # mke2fs 1.47.4 (6-Mar-2025) worker1 # Discarding device blocks: 0/262144 done lb # 32768, 98304, 163840, 229376 client: waiting for the VM to finish booting client # Creating filesystem with 262144 4k blocks and 65536 inodes lb # worker2 # Discarding device blocks: 0/262144 done worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # Filesystem UUID: 1ca2ebb5-06c7-441d-ade7-bf2e4a4cf314 lb # Allocating group tables: 0/8 done client # Filesystem UUID: 1254f221-7786-43be-840c-ce3d9642ab10 worker2 # Filesystem UUID: e3dfeb75-f1f6-4868-8710-4a19bb11e720 worker1 # Superblock backups stored on blocks: lb # Writing inode tables: 0/8 done client # Superblock backups stored on blocks: worker2 # Superblock backups stored on blocks: worker1 # 32768, 98304, 163840, 229376 lb # Creating journal (8192 blocks): done client # 32768, 98304, 163840, 229376 worker2 # 32768, 98304, 163840, 229376 worker1 # lb # Writing superblocks and filesystem accounting information: 0/8 done client # worker2 # worker1 # Allocating group tables: 0/8 done lb # client # Allocating group tables: 0/8 done worker2 # Allocating group tables: 0/8 done worker1 # Writing inode tables: 0/8 done lb # Virtualisation disk image created. client # Writing inode tables: 0/8 done worker2 # Writing inode tables: 0/8 done worker1 # Creating journal (8192 blocks): done lb # Starting virtiofs daemons... client # Creating journal (8192 blocks): done worker2 # Creating journal (8192 blocks): done worker1 # Writing superblocks and filesystem accounting information: 0/8 done lb # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # Writing superblocks and filesystem accounting information: 0/8 done worker2 # Writing superblocks and filesystem accounting information: 0/8 done worker1 # lb # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # worker2 # worker1 # Virtualisation disk image created. lb # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # Virtualisation disk image created. worker2 # Virtualisation disk image created. worker1 # Starting virtiofs daemons... lb # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # Starting virtiofs daemons... worker2 # Starting virtiofs daemons... worker1 # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-21T14:43:34Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-21T14:43:34Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-21T14:43:34Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-21T14:43:34Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-21T14:43:34Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-21T14:43:34Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-21T14:43:34Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-21T14:43:34Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-21T14:43:34Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-21T14:43:34Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-21T14:43:34Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-21T14:43:34Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-21T14:43:34Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-21T14:43:34Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-21T14:43:34Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-21T14:43:34Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-21T14:43:34Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-21T14:43:34Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-21T14:43:34Z INFO virtiofsd] Client connected, servicing requests client # [2026-09-21T14:43:34Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-21T14:43:34Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-21T14:43:34Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-21T14:43:34Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-21T14:43:34Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-21T14:43:34Z INFO virtiofsd] Client connected, servicing requests lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 lb # [ 0.000000] KASLR enabled worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 lb # [ 0.000000] random: crng init done worker1 # [ 0.000000] KASLR enabled lb # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] random: crng init done lb # [ 0.000000] efi: UEFI not found. worker1 # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] efi: UEFI not found. lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] lb # [ 0.000000] Zone ranges: client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] KASLR enabled worker1 # [ 0.000000] Zone ranges: lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] random: crng init done lb # [ 0.000000] DMA32 empty client # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Normal empty client # [ 0.000000] efi: UEFI not found. worker1 # [ 0.000000] DMA32 empty lb # [ 0.000000] Device empty worker1 # [ 0.000000] Normal empty lb # [ 0.000000] Movable zone start for each node worker1 # [ 0.000000] Device empty client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT lb # [ 0.000000] Early memory node ranges worker1 # [ 0.000000] Movable zone start for each node client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] Early memory node ranges lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Zone ranges: lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] DMA32 empty lb # [ 0.000000] psci: probing for conduit method from DT. worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] Normal empty client # [ 0.000000] Device empty lb # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] Movable zone start for each node lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] Early memory node ranges lb # [ 0.000000] psci: Trusted OS migration not required worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] psci: Trusted OS migration not required worker1 # [ 0.000000] psci: SMC Calling Convention v1.1 client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 client # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker1 # [ 0.000000] Detected PIPT I-cache on CPU0 client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000000] psci: Trusted OS migration not required lb # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000000] psci: SMC Calling Convention v1.1 lb # [ 0.000000] CPU features: detected: Spectre-v4 worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface lb # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] CPU features: detected: Spectre-v4 worker1 # [ 0.000000] CPU features: detected: Spectre-BHB lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.000000] alternatives: applying boot alternatives worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] CPU features: detected: GICv3 CPU interface worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker2 # [ 0.000000] KASLR enabled client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gcbvv54lc1mkgavjm2x1hn2y3wpbcps9-nixos-system-lb-test/init regInfo=/nix/store/zdpr9k12xnmsfn00vay1jzchyfdyhwwc-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker2 # [ 0.000000] random: crng init done client # [ 0.000000] alternatives: applying boot alternatives worker2 # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6xkk768q3zcqgvlrnm0xb1lyjz91891g-nixos-system-worker1-test/init regInfo=/nix/store/3946mr0lkcyn2ygj7zagrllcql5yvyk5-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker2 # [ 0.000000] efi: UEFI not found. lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/zdpr9k12xnmsfn00vay1jzchyfdyhwwc-closure-info/registration", will be passed to user space. worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/3946mr0lkcyn2ygj7zagrllcql5yvyk5-closure-info/registration", will be passed to user space. lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] Zone ranges: client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6v5lkzqzv8y8aql79zri93a767rl7bwj-nixos-system-client-test/init regInfo=/nix/store/d6pwbdkbnvnr66k6s6bqylk0yc9mwdmr-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] DMA32 empty lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.000000] Normal empty lb # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.000000] Device empty worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker2 # [ 0.000000] Movable zone start for each node lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker1 # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/d6pwbdkbnvnr66k6s6bqylk0yc9mwdmr-closure-info/registration", will be passed to user space. worker2 # [ 0.000000] Early memory node ranges worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] lb # [ 0.000000] Fallback order for Node 0: 0 worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 lb # [ 0.000000] Policy zone: DMA client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] psci: probing for conduit method from DT. lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker1 # [ 0.000000] Policy zone: DMA client # [ 0.000000] software IO TLB: area num 1. lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs lb # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.000000] psci: Trusted OS migration not required worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] Fallback order for Node 0: 0 lb # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages lb # [ 0.000000] rcu: RCU event tracing is enabled. client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups client # [ 0.000000] Policy zone: DMA worker2 # [ 0.000000] Detected PIPT I-cache on CPU0 lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000000] rcu: Hierarchical RCU implementation. client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off lb # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) lb # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. lb # [ 0.000000] Tracing variant of Tasks RCU enabled. client # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.000000] CPU features: detected: Spectre-v4 worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.000000] Rude variant of Tasks RCU enabled. worker2 # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled. worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.000000] rcu: Hierarchical RCU implementation. worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.000000] rcu: RCU event tracing is enabled. lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.000000] Trampoline variant of Tasks RCU enabled. lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] Rude variant of Tasks RCU enabled. lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] GICv3: 0 Extended SPIs implemented client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/1z69h5i65q7pa63098rza1vngylyipj5-nixos-system-worker2-test/init regInfo=/nix/store/a7nf1gfcihx1v6ly6fqcjk8c321ypyrd-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/a7nf1gfcihx1v6ly6fqcjk8c321ypyrd-closure-info/registration", will be passed to user space. worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) client # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB client # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] software IO TLB: area num 1. client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker2 # [ 0.000000] Fallback order for Node 0: 0 worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] Policy zone: DMA worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000000] allocated 3145728 bytes of page_ext client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages lb # [ 0.000031] arm-pv: using stolen time PV worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns lb # [ 0.000476] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.000031] arm-pv: using stolen time PV worker2 # [ 0.000000] rcu: RCU event tracing is enabled. lb # [ 0.000670] Console: colour dummy device 80x25 worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). lb # [ 0.000678] printk: legacy console [tty0] enabled worker1 # [ 0.000544] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker1 # [ 0.000726] Console: colour dummy device 80x25 worker2 # [ 0.000000] Rude variant of Tasks RCU enabled. client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.000871] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.000734] printk: legacy console [tty0] enabled worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled. lb # [ 0.000878] pid_max: default: 32768 minimum: 301 client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns lb # [ 0.000977] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.000032] arm-pv: using stolen time PV worker1 # [ 0.000925] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) lb # [ 0.001120] landlock: Up and running. lb # [ 0.001123] Yama: becoming mindful. worker1 # [ 0.000932] pid_max: default: 32768 minimum: 301 worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.000502] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.001656] LSM support for eBPF active worker1 # [ 0.001006] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.000680] Console: colour dummy device 80x25 worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.001192] landlock: Up and running. lb # [ 0.001829] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.001195] Yama: becoming mindful. client # [ 0.000688] printk: legacy console [tty0] enabled worker1 # [ 0.001678] LSM support for eBPF active worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.001862] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.000899] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) lb # [ 0.003776] rcu: Hierarchical SRCU implementation. worker1 # [ 0.001809] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000906] pid_max: default: 32768 minimum: 301 lb # [ 0.003782] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.001841] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 client # [ 0.000980] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.005115] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.003724] rcu: Hierarchical SRCU implementation. client # [ 0.001137] landlock: Up and running. lb # [ 0.005214] EFI services will not be available. worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented client # [ 0.001140] Yama: becoming mindful. worker1 # [ 0.003729] rcu: Max phase no-delay instances is 1000. lb # [ 0.005311] smp: Bringing up secondary CPUs ... client # [ 0.001633] LSM support for eBPF active worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.005319] smp: Brought up 1 node, 1 CPU worker1 # [ 0.005056] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.005151] EFI services will not be available. lb # [ 0.005323] SMP: Total of 1 processors activated. client # [ 0.001752] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.005328] CPU: All CPU(s) started at EL1 worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker1 # [ 0.005256] smp: Bringing up secondary CPUs ... worker1 # [ 0.005265] smp: Brought up 1 node, 1 CPU client # [ 0.001785] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.005338] CPU features: detected: Branch Target Identification worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker1 # [ 0.005268] SMP: Total of 1 processors activated. client # [ 0.003576] rcu: Hierarchical SRCU implementation. worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] lb # [ 0.005343] CPU features: detected: ARMv8.4 Translation Table Level worker1 # [ 0.005274] CPU: All CPU(s) started at EL1 client # [ 0.003581] rcu: Max phase no-delay instances is 1000. client # [ 0.004843] fsl-mc MSI: its@8080000 domain created worker1 # [ 0.005287] CPU features: detected: Branch Target Identification worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449d0000 (indirect, esz 8, psz 64K, shr 1) lb # [ 0.005346] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.004935] EFI services will not be available. worker1 # [ 0.005292] CPU features: detected: ARMv8.4 Translation Table Level client # [ 0.005002] smp: Bringing up secondary CPUs ... lb # [ 0.005350] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449e0000 (flat, esz 8, psz 64K, shr 1) client # [ 0.005010] smp: Brought up 1 node, 1 CPU worker1 # [ 0.005295] CPU features: detected: Instruction cache invalidation not required for I/D coherence lb # [ 0.005354] CPU features: detected: Common not Private translations worker2 # [ 0.000000] GICv3: using LPI property table @0x00000000449f0000 client # [ 0.005013] SMP: Total of 1 processors activated. lb # [ 0.005357] CPU features: detected: CRC32 instructions client # [ 0.005018] CPU: All CPU(s) started at EL1 worker1 # [ 0.005299] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000 client # [ 0.005028] CPU features: detected: Branch Target Identification lb # [ 0.005359] CPU features: detected: Data cache clean to Point of Deep Persistence worker1 # [ 0.005303] CPU features: detected: Common not Private translations worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.005032] CPU features: detected: ARMv8.4 Translation Table Level worker1 # [ 0.005306] CPU features: detected: CRC32 instructions lb # [ 0.005363] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker1 # [ 0.005309] CPU features: detected: Data cache clean to Point of Deep Persistence client # [ 0.005035] CPU features: detected: Instruction cache invalidation not required for I/D coherence lb # [ 0.005366] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). lb # [ 0.005369] CPU features: detected: E0PD worker1 # [ 0.005313] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.005039] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.005371] CPU features: detected: Enhanced Counter Virtualization worker1 # [ 0.005316] CPU features: detected: Data independent timing control (DIT) client # [ 0.005042] CPU features: detected: Common not Private translations worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.005374] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker1 # [ 0.005319] CPU features: detected: E0PD client # [ 0.005046] CPU features: detected: CRC32 instructions worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns lb # [ 0.005378] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.005322] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.000029] arm-pv: using stolen time PV client # [ 0.005048] CPU features: detected: Data cache clean to Point of Deep Persistence lb # [ 0.005381] CPU features: detected: Fine Grained Traps worker1 # [ 0.005325] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.005069] CPU features: detected: Data cache clean to Point of Persistence lb # [ 0.005384] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.000419] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.005328] CPU features: detected: Enhanced Virtualization Traps client # [ 0.005072] CPU features: detected: Data independent timing control (DIT) worker2 # [ 0.000580] Console: colour dummy device 80x25 worker1 # [ 0.005331] CPU features: detected: Fine Grained Traps lb # [ 0.005389] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.000588] printk: legacy console [tty0] enabled client # [ 0.005076] CPU features: detected: E0PD lb # [ 0.005392] CPU features: detected: LSE atomic instructions worker1 # [ 0.005335] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.005078] CPU features: detected: Enhanced Counter Virtualization lb # [ 0.005394] CPU features: detected: Privileged Access Never worker1 # [ 0.005339] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.000785] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) lb # [ 0.005397] CPU features: detected: PMUv3 client # [ 0.005081] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker1 # [ 0.005342] CPU features: detected: LSE atomic instructions worker2 # [ 0.000792] pid_max: default: 32768 minimum: 301 lb # [ 0.005400] CPU features: detected: RAS Extension Support client # [ 0.005085] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.005345] CPU features: detected: Privileged Access Never worker2 # [ 0.000864] LSM: initializing lsm=capability,landlock,yama,bpf,ima lb # [ 0.005403] CPU features: detected: RASv1p1 Extension Support client # [ 0.005088] CPU features: detected: Fine Grained Traps worker1 # [ 0.005348] CPU features: detected: PMUv3 worker2 # [ 0.001008] landlock: Up and running. lb # [ 0.005405] CPU features: detected: Random Number Generator worker2 # [ 0.001011] Yama: becoming mindful. worker1 # [ 0.005350] CPU features: detected: RAS Extension Support client # [ 0.005091] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.001466] LSM support for eBPF active worker1 # [ 0.005353] CPU features: detected: RASv1p1 Extension Support client # [ 0.005095] CPU features: detected: RCpc load-acquire (LDAPR) lb # [ 0.005408] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.001620] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.005356] CPU features: detected: Random Number Generator client # [ 0.005098] CPU features: detected: LSE atomic instructions lb # [ 0.005411] CPU features: detected: Stage-2 Force Write-Back worker1 # [ 0.005358] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.001653] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.005101] CPU features: detected: Privileged Access Never lb # [ 0.005414] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.003597] rcu: Hierarchical SRCU implementation. client # [ 0.005104] CPU features: detected: PMUv3 worker1 # [ 0.005361] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.003602] rcu: Max phase no-delay instances is 1000. lb # [ 0.005419] CPU features: detected: Speculative Store Bypassing Safe (SSBS) client # [ 0.005106] CPU features: detected: RAS Extension Support worker1 # [ 0.005364] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.004840] fsl-mc MSI: its@8080000 domain created lb # [ 0.005459] alternatives: applying system-wide alternatives client # [ 0.005109] CPU features: detected: RASv1p1 Extension Support worker1 # [ 0.005369] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker2 # [ 0.004929] EFI services will not be available. client # [ 0.005112] CPU features: detected: Random Number Generator lb # [ 0.008461] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.005008] smp: Bringing up secondary CPUs ... worker1 # [ 0.005408] alternatives: applying system-wide alternatives worker2 # [ 0.005016] smp: Brought up 1 node, 1 CPU client # [ 0.005114] CPU features: detected: Speculation barrier (SB) worker1 # [ 0.008387] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.005019] SMP: Total of 1 processors activated. client # [ 0.005117] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.008613] Memory: 1405680K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved) worker2 # [ 0.005024] CPU: All CPU(s) started at EL1 lb # [ 0.009349] devtmpfs: initialized client # [ 0.005120] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.005034] CPU features: detected: Branch Target Identification worker1 # [ 0.008590] Memory: 1405600K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved) lb # [ 0.011231] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.005124] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker1 # [ 0.009318] devtmpfs: initialized worker2 # [ 0.005038] CPU features: detected: ARMv8.4 Translation Table Level client # [ 0.005163] alternatives: applying system-wide alternatives lb # [ 0.011254] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.011109] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.005042] CPU features: detected: Instruction cache invalidation not required for I/D coherence client # [ 0.008109] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.011501] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker1 # [ 0.011131] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.011506] 0 pages in range for non-PLT usage worker2 # [ 0.005046] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.011507] 508272 pages in range for PLT usage worker1 # [ 0.011344] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker2 # [ 0.005049] CPU features: detected: Common not Private translations client # [ 0.008322] Memory: 1405612K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved) lb # [ 0.011617] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.011348] 0 pages in range for non-PLT usage client # [ 0.009132] devtmpfs: initialized worker2 # [ 0.005053] CPU features: detected: CRC32 instructions lb # [ 0.012405] DMI not present or invalid. worker1 # [ 0.011349] 508272 pages in range for PLT usage client # [ 0.010872] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.015620] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.005056] CPU features: detected: Data cache clean to Point of Deep Persistence worker1 # [ 0.011453] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.012215] DMI not present or invalid. worker2 # [ 0.005059] CPU features: detected: Data cache clean to Point of Persistence lb # [ 0.017813] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.010899] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.015279] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.011107] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL worker2 # [ 0.005062] CPU features: detected: Data independent timing control (DIT) lb # [ 0.018007] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker1 # [ 0.017539] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations lb # [ 0.018215] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.005065] CPU features: detected: E0PD worker1 # [ 0.017733] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.018236] audit: initializing netlink subsys (disabled) worker2 # [ 0.005068] CPU features: detected: Enhanced Counter Virtualization worker1 # [ 0.017931] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.018820] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.005071] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker1 # [ 0.017951] audit: initializing netlink subsys (disabled) lb # [ 0.018821] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.005074] CPU features: detected: Enhanced Virtualization Traps worker1 # [ 0.018507] thermal_sys: Registered thermal governor 'fair_share' lb # [ 0.018825] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.005077] CPU features: detected: Fine Grained Traps worker1 # [ 0.018509] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.018828] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.018512] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.005081] CPU features: detected: Generic authentication (architected QARMA5 algorithm) lb # [ 0.018833] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.018515] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.005085] CPU features: detected: RCpc load-acquire (LDAPR) lb # [ 0.018857] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker2 # [ 0.005088] CPU features: detected: LSE atomic instructions lb # [ 0.018866] cpuidle: using governor ladder worker2 # [ 0.005091] CPU features: detected: Privileged Access Never lb # [ 0.018871] cpuidle: using governor menu worker2 # [ 0.005094] CPU features: detected: PMUv3 lb # [ 0.019059] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.005097] CPU features: detected: RAS Extension Support lb # [ 0.019075] ASID allocator initialised with 65536 entries client # [ 0.011111] 0 pages in range for non-PLT usage worker2 # [ 0.005099] CPU features: detected: RASv1p1 Extension Support lb # [ 0.020310] Serial: AMBA PL011 UART driver client # [ 0.011112] 508272 pages in range for PLT usage worker2 # [ 0.005102] CPU features: detected: Random Number Generator client # [ 0.011221] pinctrl core: initialized pinctrl subsystem lb # [ 0.025589] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 client # [ 0.012021] DMI not present or invalid. worker2 # [ 0.005105] CPU features: detected: Speculation barrier (SB) lb # [ 0.025742] printk: console [ttyAMA0] enabled client # [ 0.015108] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.005107] CPU features: detected: Stage-2 Force Write-Back client # [ 0.017324] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.005110] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.005115] CPU features: detected: Speculative Store Bypassing Safe (SSBS) client # [ 0.017509] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.005151] alternatives: applying system-wide alternatives client # [ 0.017710] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.008232] CPU features: detected: BBM Level 2 without TLB conflict abort client # [ 0.017731] audit: initializing netlink subsys (disabled) worker1 # [ 0.018520] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.018258] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.018544] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 client # [ 0.018260] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.008450] Memory: 1405764K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved) worker1 # [ 0.018553] cpuidle: using governor ladder worker2 # [ 0.009221] devtmpfs: initialized client # [ 0.018264] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.018558] cpuidle: using governor menu worker2 # [ 0.011021] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) client # [ 0.018267] thermal_sys: Registered thermal governor 'user_space' worker1 # [ 0.018746] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.141760] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.018764] ASID allocator initialised with 65536 entries client # [ 0.018273] thermal_sys: Registered thermal governor 'power_allocator' worker2 # [ 0.011043] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). lb # [ 0.141780] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker1 # [ 0.019959] Serial: AMBA PL011 UART driver client # [ 0.018308] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.141785] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker2 # [ 0.011235] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.018316] cpuidle: using governor ladder worker2 # [ 0.011239] 0 pages in range for non-PLT usage worker1 # [ 0.025199] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.141789] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.018322] cpuidle: using governor menu worker2 # [ 0.011241] 508272 pages in range for PLT usage worker1 # [ 0.025362] printk: console [ttyAMA0] enabled lb # [ 0.141794] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.011335] pinctrl core: initialized pinctrl subsystem client # [ 0.018503] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker2 # [ 0.012120] DMI not present or invalid. lb # [ 0.141798] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.018517] ASID allocator initialised with 65536 entries worker1 # [ 0.144865] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker2 # [ 0.015274] NET: Registered PF_NETLINK/PF_ROUTE protocol family client # [ 0.019748] Serial: AMBA PL011 UART driver lb # [ 0.141803] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.144886] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker2 # [ 0.017506] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations lb # [ 0.141807] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page client # [ 0.024976] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 worker1 # [ 0.144892] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages client # [ 0.025110] printk: console [ttyAMA0] enabled worker2 # [ 0.017717] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker1 # [ 0.144896] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker2 # [ 0.017919] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker1 # [ 0.144900] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.143765] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.149008] fbcon: Taking over console worker2 # [ 0.017945] audit: initializing netlink subsys (disabled) worker1 # [ 0.144904] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.143785] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.149023] ACPI: Interpreter disabled. worker2 # [ 0.018528] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.144909] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages client # [ 0.143790] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker2 # [ 0.018530] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.144913] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page client # [ 0.143794] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker2 # [ 0.018533] thermal_sys: Registered thermal governor 'step_wise' client # [ 0.143798] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker2 # [ 0.018536] thermal_sys: Registered thermal governor 'user_space' client # [ 0.143803] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker2 # [ 0.018541] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.143807] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.152198] fbcon: Taking over console worker2 # [ 0.018566] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.152211] ACPI: Interpreter disabled. client # [ 0.143811] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker2 # [ 0.018574] cpuidle: using governor ladder worker1 # [ 0.154071] iommu: Default domain type: Translated worker2 # [ 0.018580] cpuidle: using governor menu worker1 # [ 0.154080] iommu: DMA domain TLB invalidation policy: strict mode worker2 # [ 0.018769] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.155839] SCSI subsystem initialized worker2 # [ 0.018783] ASID allocator initialised with 65536 entries client # [ 0.151194] fbcon: Taking over console worker2 # [ 0.019982] Serial: AMBA PL011 UART driver client # [ 0.151209] ACPI: Interpreter disabled. client # [ 0.153091] iommu: Default domain type: Translated worker2 # [ 0.025247] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.158048] iommu: Default domain type: Translated worker2 # [ 0.025370] printk: console [ttyAMA0] enabled client # [ 0.153101] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.158059] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.158430] SCSI subsystem initialized client # [ 0.154794] SCSI subsystem initialized lb # [ 0.160382] usbcore: registered new interface driver usbfs lb # [ 0.160414] usbcore: registered new interface driver hub lb # [ 0.160433] usbcore: registered new device driver usb lb # [ 0.160727] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.160733] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.160743] PTP clock support registered lb # [ 0.160793] EDAC MC: Ver: 3.0.0 worker1 # [ 0.163530] usbcore: registered new interface driver usbfs lb # [ 0.165369] scmi_core: SCMI protocol bus registered worker1 # [ 0.163564] usbcore: registered new interface driver hub worker2 # [ 0.143083] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages worker1 # [ 0.163596] usbcore: registered new device driver usb lb # [ 0.166317] FPGA manager framework worker1 # [ 0.163873] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.143101] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.167247] vgaarb: loaded worker2 # [ 0.143106] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.163879] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.167851] clocksource: Switched to clocksource arch_sys_counter client # [ 0.162464] usbcore: registered new interface driver usbfs worker1 # [ 0.163888] PTP clock support registered worker2 # [ 0.143111] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.162497] usbcore: registered new interface driver hub worker1 # [ 0.163938] EDAC MC: Ver: 3.0.0 worker2 # [ 0.143115] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.162523] usbcore: registered new device driver usb worker1 # [ 0.168513] scmi_core: SCMI protocol bus registered client # [ 0.162806] pps_core: LinuxPPS API ver. 1 registered worker2 # [ 0.143120] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.169504] FPGA manager framework worker2 # [ 0.143124] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages client # [ 0.162812] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker1 # [ 0.170475] vgaarb: loaded client # [ 0.162822] PTP clock support registered worker2 # [ 0.143128] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page client # [ 0.162868] EDAC MC: Ver: 3.0.0 worker1 # [ 0.171111] clocksource: Switched to clocksource arch_sys_counter client # [ 0.167444] scmi_core: SCMI protocol bus registered worker1 # [ 0.172485] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.150907] fbcon: Taking over console client # [ 0.168387] FPGA manager framework worker1 # [ 0.172522] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.150924] ACPI: Interpreter disabled. client # [ 0.169360] vgaarb: loaded lb # [ 0.174193] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.174081] netfs: FS-Cache loaded lb # [ 0.174242] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.174194] pnp: PnP ACPI: disabled client # [ 0.169975] clocksource: Switched to clocksource arch_sys_counter lb # [ 0.175864] netfs: FS-Cache loaded lb # [ 0.175981] pnp: PnP ACPI: disabled client # [ 0.171339] VFS: Disk quotas dquot_6.6.0 client # [ 0.171370] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) client # [ 0.172941] netfs: FS-Cache loaded client # [ 0.173077] pnp: PnP ACPI: disabled lb # [ 0.179825] NET: Registered PF_INET protocol family worker2 # [ 0.159962] iommu: Default domain type: Translated worker2 # [ 0.159973] iommu: DMA domain TLB invalidation policy: strict mode worker1 # [ 0.180122] NET: Registered PF_INET protocol family worker2 # [ 0.160291] SCSI subsystem initialized lb # [ 0.180711] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.180395] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.162283] usbcore: registered new interface driver usbfs worker2 # [ 0.162315] usbcore: registered new interface driver hub worker2 # [ 0.162333] usbcore: registered new device driver usb client # [ 0.178969] NET: Registered PF_INET protocol family worker2 # [ 0.162606] pps_core: LinuxPPS API ver. 1 registered client # [ 0.179232] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.162612] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker2 # [ 0.162634] PTP clock support registered worker2 # [ 0.162682] EDAC MC: Ver: 3.0.0 worker2 # [ 0.167335] scmi_core: SCMI protocol bus registered worker2 # [ 0.168324] FPGA manager framework worker2 # [ 0.169314] vgaarb: loaded worker2 # [ 0.169930] clocksource: Switched to clocksource arch_sys_counter worker2 # [ 0.173619] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.173657] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.177459] netfs: FS-Cache loaded worker2 # [ 0.177573] pnp: PnP ACPI: disabled worker2 # [ 0.181521] NET: Registered PF_INET protocol family worker2 # [ 0.181776] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.208748] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.208790] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.208817] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.208891] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.208478] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.209036] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.208520] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker1 # [ 0.210317] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.209117] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.208545] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker1 # [ 0.210360] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.209153] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.208614] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.209190] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.210384] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.209293] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.208756] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.210453] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.209314] NET: Registered PF_XDP protocol family client # [ 0.208839] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.209332] PCI: CLS 0 bytes, default 64 worker1 # [ 0.210597] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.209579] Trying to unpack rootfs image as initramfs... client # [ 0.208875] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.210674] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) client # [ 0.208911] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.210708] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.209025] NET: Registered PF_UNIX/PF_LOCAL protocol family worker1 # [ 0.210745] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) client # [ 0.209087] NET: Registered PF_XDP protocol family client # [ 0.209105] PCI: CLS 0 bytes, default 64 worker1 # [ 0.210834] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.209359] Trying to unpack rootfs image as initramfs... worker1 # [ 0.210859] NET: Registered PF_XDP protocol family worker1 # [ 0.210876] PCI: CLS 0 bytes, default 64 worker1 # [ 0.211118] Trying to unpack rootfs image as initramfs... lb # [ 0.225343] kvm [1]: HYP mode not available client # [ 0.223520] kvm [1]: HYP mode not available worker1 # [ 0.228843] kvm [1]: HYP mode not available worker2 # [ 0.211045] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) worker2 # [ 0.211090] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.211119] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.211191] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.211334] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.211420] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.211455] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.211490] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.211605] NET: Registered PF_UNIX/PF_LOCAL protocol family worker2 # [ 0.211665] NET: Registered PF_XDP protocol family worker2 # [ 0.211682] PCI: CLS 0 bytes, default 64 worker2 # [ 0.211951] Trying to unpack rootfs image as initramfs... worker2 # [ 0.227798] kvm [1]: HYP mode not available lb # [ 0.316460] Initialise system trusted keyrings lb # [ 0.317200] workingset: timestamp_bits=42 max_order=19 bucket_order=0 lb # [ 0.318498] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.314515] Initialise system trusted keyrings lb # [ 0.319280] 9p: Installing v9fs 9p2000 file system support worker1 # [ 0.323649] Initialise system trusted keyrings worker1 # [ 0.324420] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.325730] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.322032] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.326511] 9p: Installing v9fs 9p2000 file system support client # [ 0.323422] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.324195] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.322471] Initialise system trusted keyrings worker2 # [ 0.323199] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker2 # [ 0.324513] squashfs: version 4.0 (2009/01/31) Phillip Lougher lb # [ 0.348278] Key type asymmetric registered worker2 # [ 0.325302] 9p: Installing v9fs 9p2000 file system support lb # [ 0.348307] Asymmetric key parser 'x509' registered lb # [ 0.348379] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.344872] Key type asymmetric registered lb # [ 0.350486] io scheduler mq-deadline registered client # [ 0.344890] Asymmetric key parser 'x509' registered lb # [ 0.350497] io scheduler kyber registered client # [ 0.344976] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.347134] io scheduler mq-deadline registered client # [ 0.347144] io scheduler kyber registered worker1 # [ 0.355258] Key type asymmetric registered worker1 # [ 0.355282] Asymmetric key parser 'x509' registered worker1 # [ 0.355364] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker1 # [ 0.357514] io scheduler mq-deadline registered worker1 # [ 0.357526] io scheduler kyber registered lb # [ 0.359984] pl061_gpio 9030000.pl061: PL061 GPIO chip registered client # [ 0.358115] pl061_gpio 9030000.pl061: PL061 GPIO chip registered lb # [ 0.361377] ledtrig-cpu: registered to indicate activity on CPUs lb # [ 0.361779] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: lb # [ 0.361796] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 lb # [ 0.361808] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.359501] ledtrig-cpu: registered to indicate activity on CPUs lb # [ 0.361817] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.359923] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker1 # [ 0.367256] pl061_gpio 9030000.pl061: PL061 GPIO chip registered lb # [ 0.361838] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.359940] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 lb # [ 0.361864] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.359952] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 lb # [ 0.361938] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.359961] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 lb # [ 0.361948] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.368636] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.359993] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits lb # [ 0.361955] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.369084] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: lb # [ 0.361960] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.360018] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] lb # [ 0.361966] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker1 # [ 0.369102] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.360094] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.360104] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.362019] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker1 # [ 0.369115] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.360110] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] lb # [ 0.362470] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.369124] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.360115] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] lb # [ 0.362654] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.354032] Key type asymmetric registered lb # [ 0.362671] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.369147] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.354060] Asymmetric key parser 'x509' registered client # [ 0.360120] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] lb # [ 0.362701] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.369173] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker2 # [ 0.354139] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.360182] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint lb # [ 0.362717] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.356221] io scheduler mq-deadline registered worker1 # [ 0.369274] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.360616] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.356232] io scheduler kyber registered lb # [ 0.363170] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.369284] pci_bus 0000:00: root bus resource [bus 00-ff] client # [ 0.360802] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.363352] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.369290] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.360818] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.363368] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.369296] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.360848] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.363397] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.360864] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.369301] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.361332] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.369356] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.361514] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.369798] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.361529] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.369983] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.361559] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.369999] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.370029] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.370044] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.370503] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.370685] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.366073] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker1 # [ 0.370700] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.370729] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.390154] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker2 # [ 0.367494] ledtrig-cpu: registered to indicate activity on CPUs worker1 # [ 0.390340] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker2 # [ 0.367900] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker1 # [ 0.390356] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.390386] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.367917] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker1 # [ 0.390847] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.367930] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 lb # [ 0.387915] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.391030] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.388122] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker2 # [ 0.367938] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker1 # [ 0.391045] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.388138] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.391075] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.367973] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits lb # [ 0.388169] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.391093] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.367998] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] lb # [ 0.388675] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.386061] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker2 # [ 0.368090] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 lb # [ 0.388859] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] client # [ 0.386262] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker1 # [ 0.398597] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.368100] pci_bus 0000:00: root bus resource [bus 00-ff] lb # [ 0.388875] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.386280] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.398786] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.368106] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] lb # [ 0.388904] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.386311] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.398815] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.388923] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker2 # [ 0.368111] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.386807] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.368116] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] lb # [ 0.389374] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.386991] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.389559] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.387007] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.368174] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint lb # [ 0.389589] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.387036] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.368623] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.387054] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] lb # [ 0.390068] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.368814] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.390276] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.368830] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.387505] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.368861] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.387692] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.390310] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.368878] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.387721] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.390740] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.390922] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker2 # [ 0.369341] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.388171] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.369527] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.388357] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.391168] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.369543] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.388386] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.391354] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.369573] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.391383] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.388770] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.388950] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.391834] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.389053] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint lb # [ 0.392034] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.389222] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.389247] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] lb # [ 0.392064] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.389412] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.389264] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.389442] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.392518] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.389294] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.392703] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.389898] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.389763] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint lb # [ 0.392733] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.390099] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.407382] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.390129] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.407591] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.393194] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint lb # [ 0.393469] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.407621] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.390593] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.393486] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.390788] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.408076] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.393515] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.390817] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.408262] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.393967] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.391292] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.408512] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.394156] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.391567] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.408698] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.394173] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.391585] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.408728] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.394202] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.391614] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.409179] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.394779] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned client # [ 0.392084] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.409376] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.394790] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.392266] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.409406] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.392283] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.394796] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.409863] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.392313] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.394842] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.410052] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.392885] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned lb # [ 0.394889] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker1 # [ 0.410081] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.392896] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned lb # [ 0.394934] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.410542] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint client # [ 0.392902] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.410833] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.394980] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned client # [ 0.392947] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.410851] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.395026] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker1 # [ 0.410881] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.392995] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.395071] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker1 # [ 0.411357] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.393040] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned lb # [ 0.395116] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker1 # [ 0.411542] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.393096] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker1 # [ 0.411557] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.402029] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.395162] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned client # [ 0.393142] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.402062] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.411588] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.395208] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned client # [ 0.393189] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.402094] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.412186] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned lb # [ 0.395277] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.402114] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.412195] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.393235] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned lb # [ 0.395322] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.402623] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.393280] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker1 # [ 0.412200] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.395344] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.402820] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.395365] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned client # [ 0.393327] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker1 # [ 0.412247] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker2 # [ 0.402850] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.395387] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned client # [ 0.393412] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker1 # [ 0.412293] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker2 # [ 0.403318] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint lb # [ 0.395415] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned client # [ 0.393473] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker1 # [ 0.412341] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker2 # [ 0.403511] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.395437] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned client # [ 0.393495] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker1 # [ 0.412393] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.403554] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.395461] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned client # [ 0.393517] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.412440] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.404013] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint lb # [ 0.395487] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned client # [ 0.393538] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.404199] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.395511] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.412486] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned client # [ 0.393559] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.395534] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.404459] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.393581] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.412532] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.404655] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.395558] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned client # [ 0.393602] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.412578] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.404685] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.395581] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned client # [ 0.393627] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.412626] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.395602] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.405147] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.393649] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.412701] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned lb # [ 0.395625] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.405341] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.393671] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned lb # [ 0.395647] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker1 # [ 0.412749] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.405372] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.393693] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned lb # [ 0.395670] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker1 # [ 0.412770] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned client # [ 0.393716] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker2 # [ 0.405832] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.395692] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.412792] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned client # [ 0.393737] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.406040] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.395716] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.412813] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned client # [ 0.393758] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.406072] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.395743] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.412835] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned client # [ 0.393779] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.395752] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker2 # [ 0.406540] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.412856] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned client # [ 0.393800] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker2 # [ 0.406830] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] lb # [ 0.395757] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] client # [ 0.393821] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker1 # [ 0.412877] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.406849] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.393843] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.412902] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.406879] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.393869] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.412924] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned client # [ 0.393878] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker2 # [ 0.407361] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.412946] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.407562] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] client # [ 0.393883] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker1 # [ 0.412968] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.407578] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.412990] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker2 # [ 0.407609] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.413011] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.408185] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned worker1 # [ 0.413032] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.408196] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.413052] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker1 # [ 0.413073] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker2 # [ 0.408201] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned worker1 # [ 0.413094] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.456652] pci 0000:00:07.0: enabling device (0000 -> 0002) worker2 # [ 0.408247] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.413115] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.408295] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker1 # [ 0.413141] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker1 # [ 0.413150] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] worker2 # [ 0.408344] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.413155] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] worker2 # [ 0.408393] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned client # [ 0.454776] pci 0000:00:07.0: enabling device (0000 -> 0002) worker1 # [ 0.413977] pci 0000:00:07.0: enabling device (0000 -> 0002) worker2 # [ 0.408440] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.408487] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.408535] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.408582] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.408630] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.408726] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.408774] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.408796] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.408818] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker2 # [ 0.408840] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.408862] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.408883] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.408905] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.408930] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.408953] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker2 # [ 0.408975] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.408998] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.409020] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker2 # [ 0.409042] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.409064] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned worker2 # [ 0.409085] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned worker2 # [ 0.409107] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker2 # [ 0.409128] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.409150] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker2 # [ 0.409177] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] worker2 # [ 0.409186] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.482635] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.409191] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] client # [ 0.480360] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker1 # [ 0.486269] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.466080] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.489284] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.491396] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) client # [ 0.487146] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.492543] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.489174] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker1 # [ 0.494873] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.501458] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.503518] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) client # [ 0.499329] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) client # [ 0.501470] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker1 # [ 0.504567] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.506952] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker2 # [ 0.487365] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker2 # [ 0.489576] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) client # [ 0.512874] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) lb # [ 0.515275] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker1 # [ 0.518237] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) lb # [ 0.517108] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) lb # [ 0.518900] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker2 # [ 0.500585] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) client # [ 0.515505] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.524196] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.526017] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) client # [ 0.517338] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) client # [ 0.519242] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) client # [ 0.521224] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) lb # [ 0.529073] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) lb # [ 0.531130] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.536236] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker2 # [ 0.510496] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.538608] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker2 # [ 0.514507] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker2 # [ 0.516578] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) client # [ 0.535529] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) lb # [ 0.542889] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker1 # [ 0.550109] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.526115] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker2 # [ 0.528676] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) client # [ 0.551238] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.557115] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled client # [ 0.553875] msm_serial: driver initialized lb # [ 0.559807] msm_serial: driver initialized worker1 # [ 0.560361] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled client # [ 0.554593] SuperH (H)SCI(F) driver initialized client # [ 0.554651] STM32 USART driver initialized lb # [ 0.560511] SuperH (H)SCI(F) driver initialized worker1 # [ 0.563104] msm_serial: driver initialized lb # [ 0.560572] STM32 USART driver initialized worker2 # [ 0.538484] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker2 # [ 0.541401] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.563793] SuperH (H)SCI(F) driver initialized worker1 # [ 0.563850] STM32 USART driver initialized worker2 # [ 0.553186] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker2 # [ 0.563188] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.590717] loop: module loaded worker2 # [ 0.566245] msm_serial: driver initialized worker2 # [ 0.566395] SuperH (H)SCI(F) driver initialized lb # [ 0.590914] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 0.586582] loop: module loaded worker2 # [ 0.566448] STM32 USART driver initialized client # [ 0.586771] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 0.591771] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) client # [ 0.587537] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker1 # [ 0.597714] loop: module loaded worker1 # [ 0.597920] virtio_blk virtio2: 1/0/0 default/read/poll queues worker1 # [ 0.598652] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 0.596464] megasas: 07.734.00.00-rc1 lb # [ 0.597140] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] client # [ 0.594634] megasas: 07.734.00.00-rc1 worker1 # [ 0.603744] megasas: 07.734.00.00-rc1 client # [ 0.595342] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker1 # [ 0.604460] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] client # [ 0.598567] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.598591] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.607113] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.600230] Using buffer write method client # [ 0.600288] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.617110] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.615175] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.616850] Using buffer write method lb # [ 0.617145] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.618813] Using buffer write method worker1 # [ 0.616934] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.618894] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] client # [ 0.614016] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.618772] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.614037] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.618795] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.615775] Using buffer write method client # [ 0.615803] Concatenating MTD devices: client # [ 0.615807] (0): "0.flash" worker2 # [ 0.600803] loop: module loaded client # [ 0.615811] (1): "0.flash" client # [ 0.615821] into device "0.flash" worker2 # [ 0.600983] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 0.601784] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 0.623899] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.623919] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.625865] Using buffer write method lb # [ 0.625892] Concatenating MTD devices: lb # [ 0.625896] (0): "0.flash" lb # [ 0.625899] (1): "0.flash" lb # [ 0.625911] into device "0.flash" worker1 # [ 0.632209] Using buffer write method worker1 # [ 0.632243] Concatenating MTD devices: worker1 # [ 0.632247] (0): "0.flash" worker1 # [ 0.632251] (1): "0.flash" worker1 # [ 0.632261] into device "0.flash" worker2 # [ 0.614572] megasas: 07.734.00.00-rc1 worker2 # [ 0.615278] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.617833] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.617861] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.619500] Using buffer write method worker2 # [ 0.619593] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.621602] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.621627] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.639944] Using buffer write method worker2 # [ 0.639980] Concatenating MTD devices: worker2 # [ 0.639984] (0): "0.flash" worker2 # [ 0.639988] (1): "0.flash" worker2 # [ 0.639997] into device "0.flash" worker1 # [ 0.877131] Freeing initrd memory: 26976K client # [ 0.874250] Freeing initrd memory: 26972K lb # [ 0.879691] Freeing initrd memory: 26948K worker1 # [ 0.883269] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.880253] tun: Universal TUN/TAP device driver, 1.6 lb # [ 0.885802] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 0.886849] thunder_xcv, ver 1.0 worker1 # [ 0.886890] thunder_bgx, ver 1.0 worker1 # [ 0.886912] nicpf, ver 1.0 client # [ 0.884055] thunder_xcv, ver 1.0 client # [ 0.884099] thunder_bgx, ver 1.0 client # [ 0.884121] nicpf, ver 1.0 client # [ 0.884683] e1000: Intel(R) PRO/1000 Network Driver lb # [ 0.889395] thunder_xcv, ver 1.0 client # [ 0.884690] e1000: Copyright (c) 1999-2006 Intel Corporation. lb # [ 0.889439] thunder_bgx, ver 1.0 worker1 # [ 0.888667] e1000: Intel(R) PRO/1000 Network Driver lb # [ 0.889463] nicpf, ver 1.0 client # [ 0.884714] e1000e: Intel(R) PRO/1000 Network Driver worker1 # [ 0.888678] e1000: Copyright (c) 1999-2006 Intel Corporation. lb # [ 0.890041] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.884721] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker1 # [ 0.888702] e1000e: Intel(R) PRO/1000 Network Driver lb # [ 0.890048] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.884751] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 0.888710] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. lb # [ 0.890073] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.884756] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 0.888739] igb: Intel(R) Gigabit Ethernet Network Driver lb # [ 0.890080] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 0.884779] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.888745] igb: Copyright (c) 2007-2014 Intel Corporation. lb # [ 0.890110] igb: Intel(R) Gigabit Ethernet Network Driver client # [ 0.884785] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 0.888783] igbvf: Intel(R) Gigabit Virtual Function Network Driver lb # [ 0.890116] igb: Copyright (c) 2007-2014 Intel Corporation. client # [ 0.884920] sky2: driver version 1.30 worker1 # [ 0.888789] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. lb # [ 0.890140] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.888941] sky2: driver version 1.30 lb # [ 0.890158] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 0.890497] usbcore: registered new interface driver usb-storage client # [ 0.893435] usbcore: registered new interface driver usb-storage lb # [ 0.890304] sky2: driver version 1.30 worker1 # [ 0.890608] usbcore: registered new interface driver usbserial_generic lb # [ 0.892534] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.893486] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.890622] usbserial: USB Serial support registered for generic client # [ 0.893500] usbserial: USB Serial support registered for generic lb # [ 0.892559] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.891844] ehci-pci 0000:00:07.0: EHCI Host Controller lb # [ 0.892827] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.891869] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 client # [ 0.896855] ehci-pci 0000:00:07.0: EHCI Host Controller worker1 # [ 0.892132] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 client # [ 0.896885] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 lb # [ 0.902037] usbcore: registered new interface driver usb-storage client # [ 0.897231] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 lb # [ 0.902128] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.902559] hv_vmbus: registering driver hyperv_keyboard lb # [ 0.902142] usbserial: USB Serial support registered for generic worker1 # [ 0.903175] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.899795] hv_vmbus: registering driver hyperv_keyboard worker2 # [ 0.881461] Freeing initrd memory: 26944K lb # [ 0.902797] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 0.903462] hub 1-0:1.0: USB hub found worker1 # [ 0.903481] hub 1-0:1.0: 6 ports detected client # [ 0.901325] rtc-pl031 9010000.pl031: registered as rtc0 lb # [ 0.906376] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 lb # [ 0.906677] hub 1-0:1.0: USB hub found worker1 # [ 0.906429] rtc-pl031 9010000.pl031: registered as rtc0 client # [ 0.901354] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T14:43:36 UTC (1790001816) lb # [ 0.906698] hub 1-0:1.0: 6 ports detected client # [ 0.901653] i2c_dev: i2c /dev entries driver worker1 # [ 0.906455] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T14:43:36 UTC (1790001816) worker1 # [ 0.906765] i2c_dev: i2c /dev entries driver lb # [ 0.909058] rtc-pl031 9010000.pl031: registered as rtc0 worker2 # [ 0.887519] tun: Universal TUN/TAP device driver, 1.6 lb # [ 0.909086] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T14:43:36 UTC (1790001816) lb # [ 0.909407] i2c_dev: i2c /dev entries driver client # [ 0.906966] sdhci: Secure Digital Host Controller Interface driver client # [ 0.906977] sdhci: Copyright(c) Pierre Ossman lb # [ 0.914763] sdhci: Secure Digital Host Controller Interface driver worker1 # [ 0.911995] sdhci: Secure Digital Host Controller Interface driver client # [ 0.907239] Synopsys Designware Multimedia Card Interface Driver lb # [ 0.914774] sdhci: Copyright(c) Pierre Ossman worker1 # [ 0.912005] sdhci: Copyright(c) Pierre Ossman client # [ 0.907618] sdhci-pltfm: SDHCI platform and OF driver helper lb # [ 0.915038] Synopsys Designware Multimedia Card Interface Driver worker1 # [ 0.912259] Synopsys Designware Multimedia Card Interface Driver worker2 # [ 0.891234] thunder_xcv, ver 1.0 client # [ 0.909116] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.891278] thunder_bgx, ver 1.0 lb # [ 0.915410] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.912622] sdhci-pltfm: SDHCI platform and OF driver helper worker2 # [ 0.891299] nicpf, ver 1.0 client # [ 0.909352] usbcore: registered new interface driver usbhid worker1 # [ 0.914129] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.891885] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.909359] usbhid: USB HID core driver worker1 # [ 0.914362] usbcore: registered new interface driver usbhid worker2 # [ 0.891893] e1000: Copyright (c) 1999-2006 Intel Corporation. worker1 # [ 0.914369] usbhid: USB HID core driver lb # [ 0.919624] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.891917] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.914381] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.914694] hub 1-0:1.0: USB hub found worker2 # [ 0.891925] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 0.914714] hub 1-0:1.0: 6 ports detected worker2 # [ 0.891956] igb: Intel(R) Gigabit Ethernet Network Driver lb # [ 0.920507] usbcore: registered new interface driver usbhid lb # [ 0.920517] usbhid: USB HID core driver worker2 # [ 0.891962] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 0.891985] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker1 # [ 0.919853] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available client # [ 0.917139] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.891991] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 0.921378] drop_monitor: Initializing network drop monitor service worker2 # [ 0.892127] sky2: driver version 1.30 lb # [ 0.922775] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 0.921514] NET: Registered PF_INET6 protocol family worker2 # [ 0.893675] usbcore: registered new interface driver usb-storage client # [ 0.919714] drop_monitor: Initializing network drop monitor service worker1 # [ 0.924450] Segment Routing with IPv6 worker2 # [ 0.893724] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.924472] In-situ OAM (IOAM) with IPv6 client # [ 0.919892] NET: Registered PF_INET6 protocol family lb # [ 0.925394] drop_monitor: Initializing network drop monitor service worker2 # [ 0.893738] usbserial: USB Serial support registered for generic worker1 # [ 0.924500] NET: Registered PF_PACKET protocol family lb # [ 0.925555] NET: Registered PF_INET6 protocol family worker2 # [ 0.894712] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.921784] Segment Routing with IPv6 client # [ 0.921802] In-situ OAM (IOAM) with IPv6 lb # [ 0.927482] Segment Routing with IPv6 worker2 # [ 0.894738] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 lb # [ 0.927501] In-situ OAM (IOAM) with IPv6 client # [ 0.921830] NET: Registered PF_PACKET protocol family worker1 # [ 0.926086] 9pnet: Installing 9P2000 support worker2 # [ 0.894982] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.926131] Key type dns_resolver registered lb # [ 0.927529] NET: Registered PF_PACKET protocol family worker2 # [ 0.906403] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker2 # [ 0.906697] hub 1-0:1.0: USB hub found worker2 # [ 0.906712] hub 1-0:1.0: 6 ports detected client # [ 0.923466] 9pnet: Installing 9P2000 support client # [ 0.923514] Key type dns_resolver registered lb # [ 0.929295] 9pnet: Installing 9P2000 support lb # [ 0.929342] Key type dns_resolver registered worker2 # [ 0.909196] hv_vmbus: registering driver hyperv_keyboard worker2 # [ 0.910854] rtc-pl031 9010000.pl031: registered as rtc0 worker2 # [ 0.910878] rtc-pl031 9010000.pl031: setting system clock to 2026-09-21T14:43:36 UTC (1790001816) worker2 # [ 0.911177] i2c_dev: i2c /dev entries driver worker1 # [ 0.932864] registered taskstats version 1 worker1 # [ 0.933013] Loading compiled-in X.509 certificates client # [ 0.930096] registered taskstats version 1 client # [ 0.930246] Loading compiled-in X.509 certificates lb # [ 0.935968] registered taskstats version 1 lb # [ 0.936117] Loading compiled-in X.509 certificates worker2 # [ 0.916502] sdhci: Secure Digital Host Controller Interface driver worker2 # [ 0.916512] sdhci: Copyright(c) Pierre Ossman worker2 # [ 0.916778] Synopsys Designware Multimedia Card Interface Driver worker2 # [ 0.917145] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.941457] Demotion targets for Node 0: null worker1 # [ 0.941556] Key type .fscrypt registered worker1 # [ 0.941562] Key type fscrypt-provisioning registered client # [ 0.938931] Demotion targets for Node 0: null worker1 # [ 0.941654] ima: No TPM chip found, activating TPM-bypass! worker2 # [ 0.921415] hid: raw HID events driver (C) Jiri Kosina client # [ 0.939036] Key type .fscrypt registered worker1 # [ 0.941673] ima: Allocated hash algorithm: sha1 lb # [ 0.944698] Demotion targets for Node 0: null worker2 # [ 0.921681] usbcore: registered new interface driver usbhid client # [ 0.939042] Key type fscrypt-provisioning registered lb # [ 0.944797] Key type .fscrypt registered worker1 # [ 0.941694] ima: No architecture policies found worker2 # [ 0.921687] usbhid: USB HID core driver client # [ 0.939139] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.944804] Key type fscrypt-provisioning registered client # [ 0.939158] ima: Allocated hash algorithm: sha1 lb # [ 0.944894] ima: No TPM chip found, activating TPM-bypass! worker1 # [ 0.945589] input: gpio-keys as /devices/platform/gpio-keys/input/input0 client # [ 0.939180] ima: No architecture policies found lb # [ 0.944913] ima: Allocated hash algorithm: sha1 lb # [ 0.944935] ima: No architecture policies found client # [ 0.942997] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.924538] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available lb # [ 0.948750] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.927153] drop_monitor: Initializing network drop monitor service worker2 # [ 0.927311] NET: Registered PF_INET6 protocol family worker2 # [ 0.929232] Segment Routing with IPv6 worker2 # [ 0.929251] In-situ OAM (IOAM) with IPv6 worker2 # [ 0.929280] NET: Registered PF_PACKET protocol family worker2 # [ 0.931124] 9pnet: Installing 9P2000 support worker2 # [ 0.931169] Key type dns_resolver registered worker2 # [ 0.937992] registered taskstats version 1 worker2 # [ 0.938150] Loading compiled-in X.509 certificates worker1 # [ 0.963811] clk: Disabling unused clocks client # [ 0.960834] clk: Disabling unused clocks worker1 # [ 0.963834] PM: genpd: Disabling unused power domains client # [ 0.960861] PM: genpd: Disabling unused power domains lb # [ 0.966954] clk: Disabling unused clocks lb # [ 0.966979] PM: genpd: Disabling unused power domains worker1 # [ 0.968021] Freeing unused kernel memory: 4736K worker1 # [ 0.968224] Run /init as init process worker2 # [ 0.946790] Demotion targets for Node 0: null worker2 # [ 0.946897] Key type .fscrypt registered client # [ 0.965076] Freeing unused kernel memory: 4736K worker2 # [ 0.946903] Key type fscrypt-provisioning registered client # [ 0.965276] Run /init as init process worker2 # [ 0.947002] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.971131] Freeing unused kernel memory: 4736K lb # [ 0.971343] Run /init as init process worker2 # [ 0.947021] ima: Allocated hash algorithm: sha1 worker2 # [ 0.947044] ima: No architecture policies found worker2 # [ 0.951188] input: gpio-keys as /devices/platform/gpio-keys/input/input0 client # [ 0.980174] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.986077] systemd[1]: Successfully made /usr/ read-only. worker1 # [ 0.985386] systemd[1]: Successfully made /usr/ read-only. worker2 # [ 0.969298] clk: Disabling unused clocks worker2 # [ 0.969321] PM: genpd: Disabling unused power domains worker2 # [ 0.973441] Freeing unused kernel memory: 4736K worker2 # [ 0.973639] Run /init as init process worker2 # [ 0.988164] systemd[1]: Successfully made /usr/ read-only. lb # [ 1.151918] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.151205] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.162044] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 1.154012] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.303664] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 lb # [ 1.304530] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 client # [ 1.314233] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 lb # [ 1.320889] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.320258] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 1.333102] systemd[1]: Detected virtualization qemu. worker1 # [ 1.332389] systemd[1]: Detected virtualization qemu. worker2 # [ 1.308260] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 lb # [ 1.335145] systemd[1]: Detected architecture arm64. worker1 # [ 1.334466] systemd[1]: Detected architecture arm64. client # [ 1.320348] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 1.337131] systemd[1]: Running in initrd. worker1 # [ 1.336453] systemd[1]: Running in initrd. client # [ 1.332805] systemd[1]: Detected virtualization qemu. client # [ 1.334939] systemd[1]: Detected architecture arm64. lb # [ 1.339761] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.339080] systemd[1]: Initializing machine ID from random generator. client # [ 1.336854] systemd[1]: Running in initrd. lb # [ 1.342912] systemd[1]: Hostname set to . worker1 # [ 1.341945] systemd[1]: Hostname set to . client # [ 1.339521] systemd[1]: Initializing machine ID from random generator. client # [ 1.342487] systemd[1]: Hostname set to . worker2 # [ 1.323001] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.335174] systemd[1]: Detected virtualization qemu. worker2 # [ 1.337214] systemd[1]: Detected architecture arm64. worker2 # [ 1.339256] systemd[1]: Running in initrd. worker2 # [ 1.341805] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.344788] systemd[1]: Hostname set to . lb # [ 1.388160] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker1 # [ 1.391290] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 client # [ 1.398295] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker2 # [ 1.394212] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 lb # [ 1.511924] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.511200] usb 1-2: new high-speed USB device number 3 using ehci-pci client # [ 1.518067] usb 1-2: new high-speed USB device number 3 using ehci-pci worker2 # [ 1.518022] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.590148] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.647191] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.655147] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.657443] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.658527] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.653678] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.659858] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.661164] systemd[1]: Reached target Path Units. worker1 # [ 1.661901] systemd[1]: Reached target Slice Units. worker1 # [ 1.662826] systemd[1]: Reached target Swaps. worker1 # [ 1.663886] systemd[1]: Reached target Timer Units. worker1 # [ 1.664065] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.664233] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.664366] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.664493] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.664595] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.664614] systemd[1]: Reached target Socket Units. worker2 # [ 1.648567] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.668329] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker1 # [ 1.672586] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.674007] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 1.676407] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 lb # [ 1.676131] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.676559] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.684839] systemd[1]: Mounting Kernel Configuration File System... client # [ 1.678717] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.684476] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.696580] systemd[1]: Starting Journal Service... worker2 # [ 1.678776] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 1.683828] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.718432] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.718540] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.738616] systemd-journald[72]: Collecting audit messages is disabled. worker1 # [ 1.747309] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.759374] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.765151] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.774224] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.769853] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.775466] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 1.771534] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.776830] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.771812] systemd[1]: Mounted Kernel Configuration File System. lb # [ 1.777877] systemd[1]: Reached target Path Units. lb # [ 1.778696] systemd[1]: Reached target Slice Units. lb # [ 1.779493] systemd[1]: Reached target Swaps. lb # [ 1.780295] systemd[1]: Reached target Timer Units. lb # [ 1.781274] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 1.782473] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.783544] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.762810] systemd[1]: Queued start job for default target Initrd Default Target. lb # [ 1.784535] systemd[1]: Listening on udev Control Socket. lb # [ 1.784659] systemd[1]: Listening on udev Kernel Socket. client # [ 1.781700] systemd[1]: Created slice Slice /system/modprobe. lb # [ 1.784682] systemd[1]: Reached target Socket Units. client # [ 1.783004] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 1.788908] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.784370] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.785585] systemd[1]: Reached target Path Units. lb # [ 1.790017] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 1.786487] systemd[1]: Reached target Slice Units. client # [ 1.787340] systemd[1]: Reached target Swaps. client # [ 1.788108] systemd[1]: Reached target Timer Units. client # [ 1.789168] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 1.772399] systemd[1]: Created slice Slice /system/modprobe. worker2 # [ 1.773592] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.790494] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.797435] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 1.775009] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.790639] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.776081] systemd[1]: Reached target Path Units. client # [ 1.790776] systemd[1]: Listening on udev Control Socket. worker2 # [ 1.776904] systemd[1]: Reached target Slice Units. client # [ 1.790903] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.777738] systemd[1]: Reached target Swaps. client # [ 1.790928] systemd[1]: Reached target Socket Units. worker2 # [ 1.778564] systemd[1]: Reached target Timer Units. client # [ 1.796960] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 1.779613] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 1.780839] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.798190] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 1.802408] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.782028] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.783008] systemd[1]: Listening on udev Control Socket. worker2 # [ 1.784090] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.784970] systemd[1]: Reached target Socket Units. lb # [ 1.808131] systemd[1]: Starting Journal Service... worker2 # [ 1.787466] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 1.788666] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 1.809481] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 1.811169] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.817187] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker1 # [ 1.817417] [drm] features: -virgl +edid -resource_blob -host_visible worker1 # [ 1.817427] [drm] features: -context_init worker1 # [ 1.818096] [drm] number of scanouts: 1 worker2 # [ 1.798094] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 1.818114] [drm] number of cap sets: 0 worker2 # [ 1.806260] systemd[1]: Starting Journal Service... lb # [ 1.836073] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.835792] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 1.836198] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.838423] systemd[1]: Starting Create Static Device Nodes in /dev... client # [ 1.838763] systemd[1]: Starting Journal Service... client # [ 1.841450] systemd[1]: Starting Load Kernel Modules... lb # [ 1.848059] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.842573] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.830918] systemd[1]: Starting Load Kernel Modules... worker2 # [ 1.831919] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 1.852217] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.851660] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker1 # [ 1.851677] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 lb # [ 1.855909] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.872008] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.872705] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.867641] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.868367] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 1.856922] systemd-journald[72]: Collecting audit messages is disabled. worker2 # [ 1.862132] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.864567] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.888096] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.891783] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.892002] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.892026] systemd[1]: Reached target Local File Systems. worker1 # [ 1.895902] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.878154] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.879640] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 1.903332] Console: switching to colour frame buffer device 160x50 worker1 # [ 1.904010] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device client # [ 1.894142] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 1.898024] systemd-journald[72]: Collecting audit messages is disabled. lb # [ 1.924514] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 1.927022] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.927385] systemd[1]: Finished Load Kernel Modules. worker1 # [ 1.930617] systemd[1]: Starting Apply Kernel Variables... client # [ 1.925222] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 1.926121] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 1.930047] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.920476] systemd-modules-load[74]: Inserted module 'dm_mod' worker1 # [ 1.921732] systemd-modules-load[74]: Module 'virtio_balloon' is built in worker1 # [ 1.922815] systemd-modules-load[74]: Module 'virtio_console' is built in worker1 # [ 1.923837] systemd-modules-load[74]: Inserted module 'virtio_gpu' client # [ 1.938079] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker1 # [ 1.944079] systemd[1]: Started Journal Service. worker2 # [ 1.918650] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 1.927306] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.937644] systemd-modules-load[74]: Module 'virtio_rng' is built in client # [ 1.947724] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 client # [ 1.947958] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 1.947969] [drm] features: -context_init client # [ 1.948694] [drm] number of scanouts: 1 client # [ 1.948711] [drm] number of cap sets: 0 lb # [ 1.966641] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.955237] systemd[1]: Starting Create System Files and Directories... worker2 # [ 1.948640] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 1.975899] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.970626] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic client # [ 1.970645] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 lb # [ 1.983684] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 lb # [ 1.985301] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 1.985670] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 1.958000] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.985701] systemd[1]: Reached target Local File Systems. worker2 # [ 1.964169] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 worker2 # [ 1.964454] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 1.964465] [drm] features: -context_init worker2 # [ 1.965182] [drm] number of scanouts: 1 worker2 # [ 1.965201] [drm] number of cap sets: 0 worker1 # [ 1.977109] systemd[1]: Finished Apply Kernel Variables. lb # [ 1.996367] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 2.004056] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 1.993354] Console: switching to colour frame buffer device 160x50 lb # [ 2.004074] [drm] features: -context_init lb # [ 2.004831] [drm] number of scanouts: 1 lb # [ 2.004849] [drm] number of cap sets: 0 worker2 # [ 1.982637] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 1.982859] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 1.982886] systemd[1]: Reached target Local File Systems. client # [ 2.001719] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.992380] systemd-udevd[79]: Using default interface naming scheme 'v261'. client # [ 2.002091] systemd[1]: Reached target Preparation for Local File Systems. client # [ 2.002114] systemd[1]: Reached target Local File Systems. worker1 # [ 1.996079] systemd[1]: Finished Create System Files and Directories. worker2 # [ 1.990313] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 2.012206] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic lb # [ 2.012222] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 client # [ 2.014454] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 1.998270] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker2 # [ 1.998285] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 client # [ 2.027905] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 2.021053] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.036199] Console: switching to colour frame buffer device 160x50 lb # [ 2.032918] systemd-modules-load[74]: Inserted module 'dm_mod' lb # [ 2.034938] systemd-modules-load[74]: Module 'virtio_balloon' is built in worker2 # [ 2.021462] Console: switching to colour frame buffer device 160x50 client # [ 2.042283] systemd[1]: Finished Load Kernel Modules. client # [ 2.045556] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.019611] systemd-modules-load[74]: Inserted module 'dm_mod' worker2 # [ 2.034793] systemd[1]: Started Journal Service. lb # [ 2.045499] systemd-modules-load[74]: Module 'virtio_console' is built in lb # [ 2.056027] systemd[1]: Started Journal Service. worker2 # [ 2.023000] systemd-modules-load[74]: Module 'virtio_balloon' is built in lb # [ 2.049070] systemd[1]: Starting Create System Files and Directories... worker2 # [ 2.046111] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker2 # [ 2.034449] systemd-modules-load[74]: Module 'virtio_console' is built in worker2 # [ 2.037756] systemd[1]: Starting Create System Files and Directories... lb # [ 2.072452] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker2 # [ 2.040840] systemd-modules-load[74]: Inserted module 'virtio_gpu' lb # [ 2.066607] systemd-modules-load[74]: Inserted module 'virtio_gpu' lb # [ 2.072237] systemd-modules-load[74]: Module 'virtio_rng' is built in worker2 # [ 2.052565] systemd-modules-load[74]: Module 'virtio_rng' is built in worker2 # [ 2.056413] systemd[1]: Finished Load Kernel Modules. client # [ 2.088319] systemd[1]: Started Journal Service. lb # [ 2.090351] systemd[1]: Finished Load Kernel Modules. client # [ 2.084323] systemd-modules-load[73]: Inserted module 'dm_mod' client # [ 2.085565] systemd-modules-load[73]: Module 'virtio_balloon' is built in worker1 # [ 2.092107] systemd[1]: Starting Virtual Console Setup... lb # [ 2.097281] systemd-udevd[78]: Using default interface naming scheme 'v261'. client # [ 2.092435] systemd-modules-load[73]: Module 'virtio_console' is built in worker2 # [ 2.079458] systemd-udevd[78]: Using default interface naming scheme 'v261'. client # [ 2.100412] systemd-modules-load[73]: Inserted module 'virtio_gpu' lb # [ 2.108940] systemd[1]: Starting Apply Kernel Variables... client # [ 2.102799] systemd-modules-load[73]: Module 'virtio_rng' is built in lb # [ 2.112820] systemd[1]: Finished Create System Files and Directories. worker2 # [ 2.092881] systemd[1]: Starting Apply Kernel Variables... client # [ 2.112595] systemd[1]: Starting Create System Files and Directories... client # [ 2.116979] systemd[1]: Finished Apply Kernel Variables. client # [ 2.118211] systemd-udevd[78]: Using default interface naming scheme 'v261'. worker2 # [ 2.110751] systemd[1]: Finished Apply Kernel Variables. client # [ 2.131373] systemd[1]: Finished Create System Files and Directories. lb # [ 2.141350] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.145374] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.128866] systemd[1]: Finished Create System Files and Directories. client # [ 2.150420] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 2.137322] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 2.169263] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.172074] systemd[1]: Finished Virtual Console Setup. lb # [ 2.204089] systemd[1]: Starting Virtual Console Setup... client # [ 2.207790] systemd[1]: Starting Virtual Console Setup... worker2 # [ 2.192567] systemd[1]: Starting Virtual Console Setup... lb # [ 2.256488] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.259756] systemd[1]: Finished Virtual Console Setup. client # [ 2.256430] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.259789] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.244667] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.247918] systemd[1]: Finished Virtual Console Setup. worker1 # [ 2.770647] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 2.771605] systemd[1]: Reached target System Initialization. worker1 # [ 2.772552] systemd[1]: Reached target Basic System. client # [ 2.860207] systemd[1]: Finished Coldplug All udev Devices. client # [ 2.862383] systemd[1]: Reached target System Initialization. client # [ 2.864099] systemd[1]: Reached target Basic System. worker2 # [ 2.860217] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 2.861140] systemd[1]: Reached target System Initialization. worker2 # [ 2.861954] systemd[1]: Reached target Basic System. lb # [ 2.891945] systemd[1]: Finished Coldplug All udev Devices. lb # [ 2.892921] systemd[1]: Reached target System Initialization. lb # [ 2.893738] systemd[1]: Reached target Basic System. worker1 # [ 2.930247] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 2.941609] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 2.958676] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 3.007513] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.032970] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 3.037240] systemd[1]: Reached target Initrd Root Device. client # [ 3.033954] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.041504] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... lb # [ 3.057850] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 3.059716] (udev-worker)[100]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 3.064267] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.073874] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 3.054510] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 3.080805] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 3.058445] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.087830] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 3.096151] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 3.098188] systemd[1]: Mounting /sysroot... client # [ 3.120950] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.127596] systemd[1]: Reached target Initrd Root Device. lb # [ 3.138473] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.131862] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... lb # [ 3.140821] systemd[1]: Reached target Initrd Root Device. worker1 # [ 3.149727] EXT4-fs (vda): mounted filesystem 1ca2ebb5-06c7-441d-ade7-bf2e4a4cf314 r/w with ordered data mode. Quota mode: none. lb # [ 3.144478] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 3.141646] systemd[1]: Mounted /sysroot. worker1 # [ 3.143297] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.147137] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 3.132826] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.134498] systemd[1]: Reached target Initrd Root Device. worker2 # [ 3.138763] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 3.171073] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload. worker1 # [ 3.176113] systemd[1]: Reload requested from client PID 119 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker1 # [ 3.181618] systemd[1]: Reloading... client # [ 3.182347] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks client # [ 3.189064] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. lb # [ 3.197662] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks lb # [ 3.203014] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.180449] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks client # [ 3.202684] systemd[1]: Mounting /sysroot... worker2 # [ 3.188180] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.190393] systemd[1]: Mounting /sysroot... lb # [ 3.216255] systemd[1]: Mounting /sysroot... client # [ 3.244091] EXT4-fs (vda): mounted filesystem 1254f221-7786-43be-840c-ce3d9642ab10 r/w with ordered data mode. Quota mode: none. client # [ 3.237133] systemd[1]: Mounted /sysroot. client # [ 3.239208] systemd[1]: Reached target Initrd Root File System. client # [ 3.240587] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.258033] EXT4-fs (vda): mounted filesystem 32ce671e-284c-4bdb-b1b3-4bb2ba4325d8 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.238139] EXT4-fs (vda): mounted filesystem e3dfeb75-f1f6-4868-8710-4a19bb11e720 r/w with ordered data mode. Quota mode: none. worker2 # [ 3.229942] systemd[1]: Mounted /sysroot. lb # [ 3.254630] systemd[1]: Mounted /sysroot. lb # [ 3.256412] systemd[1]: Reached target Initrd Root File System. worker2 # [ 3.231692] systemd[1]: Reached target Initrd Root File System. lb # [ 3.260254] systemd[1]: Starting Mountpoints Configured in the Real Root... worker2 # [ 3.235555] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 3.267740] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. client # [ 3.272115] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... client # [ 3.276482] systemd[1]: Reloading... lb # [ 3.287935] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 3.264516] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.293433] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... worker2 # [ 3.270348] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.297207] systemd[1]: Reloading... worker2 # [ 3.277082] systemd[1]: Reloading... worker1 # [ 3.389260] systemd[1]: Reloading finished in 209 ms. worker1 # [ 3.420354] systemd-sysroot-fstab-check[119]: Requesting initrd-fs.target/start/replace... worker1 # [ 3.425967] systemd-sysroot-fstab-check[119]: Requesting swap.target/start/replace... worker1 # [ 3.432787] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker1 # [ 3.434860] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.436170] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.484694] systemd[1]: Reloading finished in 211 ms. lb # [ 3.502007] systemd[1]: Reloading finished in 204 ms. worker2 # [ 3.482527] systemd[1]: Reloading finished in 209 ms. client # [ 3.515220] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... client # [ 3.519710] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... client # [ 3.527170] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 3.529672] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 3.531833] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.514435] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... lb # [ 3.539576] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.518234] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... lb # [ 3.543712] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... lb # [ 3.548998] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.525306] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.550758] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.551738] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.528165] systemd[1]: Finished Mountpoints Configured in the Real Root. worker2 # [ 3.530236] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.737117] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.756770] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.760103] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.786739] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.810258] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.865875] fuse: init (API version 7.45) client # [ 3.856975] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.864240] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.861768] systemd[1]: Mounted /sysroot/nix/.rw-store. worker1 # [ 3.862940] systemd[1]: Mounted /sysroot/run. lb # [ 3.868177] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.883473] virtiofs virtio6: discovered new tag: nix-store worker1 # [ 3.884269] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 3.853134] systemd[1]: Mounting /sysroot/nix/.ro-store... lb # [ 3.880270] systemd[1]: Mounting /sysroot/run... client # [ 3.875503] systemd[1]: Mounting /sysroot/nix/.rw-store... client # [ 3.879650] systemd[1]: Mounting /sysroot/run... lb # [ 3.888761] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.900842] virtiofs virtio7: discovered new tag: shared worker1 # [ 3.901595] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 3.871534] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.909834] virtiofs virtio8: discovered new tag: xchg worker2 # [ 3.876422] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.910578] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 3.900113] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.904190] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 3.910334] systemd[1]: Mounting /sysroot/tmp/xchg... worker2 # [ 3.898551] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.917575] systemd[1]: Mounting /sysroot/tmp/xchg... worker1 # [ 3.922849] systemd[1]: Mounted /sysroot/nix/.ro-store. worker1 # [ 3.926240] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 3.929985] systemd[1]: Mounted /sysroot/tmp/xchg. worker1 # [ 3.936629] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.939726] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 3.923917] systemd[1]: Mounting /sysroot/tmp/xchg... lb # [ 3.983525] fuse: init (API version 7.45) client # [ 3.986727] fuse: init (API version 7.45) worker2 # [ 3.962327] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 3.998683] virtiofs virtio6: discovered new tag: nix-store lb # [ 3.999498] virtiofs virtio6: virtio_fs_setup_dax: No cache capability client # [ 3.995503] virtiofs virtio6: discovered new tag: nix-store client # [ 3.996317] virtiofs virtio6: virtio_fs_setup_dax: No cache capability worker2 # [ 3.982460] fuse: init (API version 7.45) client # [ 3.987437] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 3.989495] systemd[1]: Mounted /sysroot/run. lb # [ 3.999143] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 4.000501] systemd[1]: Mounted /sysroot/run. lb # [ 4.014621] virtiofs virtio7: discovered new tag: shared lb # [ 4.015382] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 4.011678] virtiofs virtio7: discovered new tag: shared worker2 # [ 3.994186] virtiofs virtio6: discovered new tag: nix-store client # [ 4.012436] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker2 # [ 3.995017] virtiofs virtio6: virtio_fs_setup_dax: No cache capability client # [ 4.020172] virtiofs virtio8: discovered new tag: xchg lb # [ 4.025397] virtiofs virtio8: discovered new tag: xchg client # [ 4.020912] virtiofs virtio8: virtio_fs_setup_dax: No cache capability lb # [ 4.026166] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker2 # [ 4.013075] virtiofs virtio7: discovered new tag: shared worker2 # [ 4.013865] virtiofs virtio7: virtio_fs_setup_dax: No cache capability lb # [ 4.028492] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 4.024980] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 4.021527] virtiofs virtio8: discovered new tag: xchg client # [ 4.027902] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.037167] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 4.028495] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker2 # [ 4.017269] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 4.043186] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.045879] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 4.021187] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 4.041082] systemd[1]: Mounted /sysroot/nix/.ro-store. client # [ 4.043636] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 4.026842] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.030585] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 4.063662] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.040141] systemd[1]: Mounted /sysroot/run. lb # [ 4.067001] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 4.060153] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 4.061210] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 4.063848] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 4.047652] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.072883] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 4.049109] systemd[1]: Mounted /sysroot/tmp/xchg. worker2 # [ 4.057022] systemd[1]: Mounting /sysroot/nix/store... lb # [ 4.112079] systemd[1]: Mounted /sysroot/nix/store. lb # [ 4.114339] systemd[1]: Reached target Initrd File Systems. client # [ 4.107975] systemd[1]: Mounted /sysroot/nix/store. client # [ 4.108861] systemd[1]: Reached target Initrd File Systems. lb # [ 4.116208] systemd[1]: Starting Find NixOS closure... client # [ 4.112118] systemd[1]: Starting Find NixOS closure... lb # [ 4.121401] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.117720] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker2 # [ 4.105525] systemd[1]: Mounted /sysroot/nix/store. worker2 # [ 4.106437] systemd[1]: Reached target Initrd File Systems. worker2 # [ 4.108139] systemd[1]: Starting Find NixOS closure... worker2 # [ 4.115301] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... lb # [ 4.156785] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 4.159652] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. lb # [ 4.171012] systemd[1]: Finished Find NixOS closure. lb # [ 4.171871] systemd[1]: Reached target Initrd Default Target. client # [ 4.166554] systemd[1]: Finished Find NixOS closure. lb # [ 4.173873] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... client # [ 4.168890] systemd[1]: Reached target Initrd Default Target. worker2 # [ 4.155187] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 4.173163] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker2 # [ 4.170567] systemd[1]: Finished Find NixOS closure. worker2 # [ 4.172210] systemd[1]: Reached target Initrd Default Target. worker2 # [ 4.176966] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 4.206649] systemd[1]: Stopped target Initrd Default Target. lb # [ 4.208925] systemd[1]: Stopped target Basic System. client # [ 4.203509] systemd[1]: Stopped target Initrd Default Target. lb # [ 4.212895] systemd[1]: Stopped target Initrd Root Device. lb # [ 4.213763] systemd[1]: Stopped target Path Units. client # [ 4.208683] systemd[1]: Stopped target Basic System. lb # [ 4.215629] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 4.209807] systemd[1]: Stopped target Initrd Root Device. client # [ 4.211388] systemd[1]: Stopped target Path Units. lb # [ 4.220081] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 4.213301] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 4.216734] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. lb # [ 4.224770] systemd[1]: Stopped target Slice Units. lb # [ 4.226128] systemd[1]: Stopped target Socket Units. client # [ 4.221375] systemd[1]: Stopped target Slice Units. lb # [ 4.229205] systemd[1]: Stopped target System Initialization. client # [ 4.223255] systemd[1]: Stopped target Socket Units. lb # [ 4.232448] systemd[1]: Stopped target Swaps. lb # [ 4.233741] systemd[1]: Stopped target Timer Units. worker2 # [ 4.210687] systemd[1]: Stopped target Initrd Default Target. client # [ 4.228477] systemd[1]: Stopped target System Initialization. lb # [ 4.237234] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 4.212835] systemd[1]: Stopped target Basic System. client # [ 4.230592] systemd[1]: Stopped target Swaps. client # [ 4.231290] systemd[1]: Stopped target Timer Units. worker2 # [ 4.213804] systemd[1]: Stopped target Initrd Root Device. client # [ 4.232946] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 4.240321] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 4.216209] systemd[1]: Stopped target Path Units. lb # [ 4.242168] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 4.236634] systemd[1]: Closed D-Bus System Message Bus Socket. worker2 # [ 4.220133] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 4.238268] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. lb # [ 4.246481] systemd[1]: Stopped Find NixOS closure. worker2 # [ 4.222157] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. lb # [ 4.247668] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 4.243319] systemd[1]: Stopped Find NixOS closure. lb # [ 4.251332] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 4.228159] systemd[1]: Stopped target Slice Units. worker2 # [ 4.230059] systemd[1]: Stopped target Socket Units. worker2 # [ 4.230817] systemd[1]: Stopped target System Initialization. client # [ 4.249216] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 4.232649] systemd[1]: Stopped target Swaps. client # [ 4.250214] systemd[1]: systemd-sysctl.service: Deactivated successfully. client # [ 4.251181] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 4.237079] systemd[1]: Stopped target Timer Units. lb # [ 4.262519] systemd[1]: Stopped Apply Kernel Variables. lb # [ 4.263415] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 4.239348] systemd[1]: dbus.socket: Deactivated successfully. lb # [ 4.265177] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 4.240958] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 4.251978] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker2 # [ 4.244270] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 4.262771] systemd[1]: Stopped Load Kernel Modules. lb # [ 4.269655] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 4.247830] systemd[1]: Stopped Find NixOS closure. lb # [ 4.272452] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 4.268811] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 4.276239] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker2 # [ 4.251770] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 4.277292] systemd[1]: Stopped Create System Files and Directories. client # [ 4.270220] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 4.278783] systemd[1]: Stopped target Local File Systems. client # [ 4.271340] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 4.279594] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 4.280720] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 4.273765] systemd[1]: Stopped Create System Files and Directories. lb # [ 4.281720] systemd[1]: Stopped Coldplug All udev Devices. client # [ 4.274778] systemd[1]: Stopped target Local File Systems. worker2 # [ 4.257752] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 4.282507] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker2 # [ 4.258825] systemd[1]: Stopped Apply Kernel Variables. lb # [ 4.283518] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.259578] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 4.278349] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 4.283915] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 4.292226] systemd[1]: Stopped Virtual Console Setup. client # [ 4.285164] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 4.293044] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 4.285935] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 4.293972] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 4.286965] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.294900] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.270933] systemd[1]: Stopped Load Kernel Modules. client # [ 4.287993] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.295888] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 4.289370] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker2 # [ 4.271727] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. client # [ 4.290310] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. lb # [ 4.298315] systemd[1]: systemd-udevd.service: Deactivated successfully. client # [ 4.291228] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.299260] systemd[1]: Stopped Rule-based Manager for Device Events and Files. client # [ 4.292277] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 4.293110] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 4.300350] systemd[1]: systemd-udevd.service: Consumed 1.089s CPU time over 2.302s wall clock time, 17.6M memory peak. worker2 # [ 4.276341] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 4.294029] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 4.301741] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 4.277411] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 4.302716] systemd[1]: Closed udev Control Socket. client # [ 4.294985] systemd[1]: systemd-udevd.service: Consumed 1.104s CPU time over 2.276s wall clock time, 17.9M memory peak. lb # [ 4.303404] systemd[1]: Starting Cleanup udev Database... worker2 # [ 4.280176] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 4.281913] systemd[1]: Stopped target Local File Systems. worker2 # [ 4.282725] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 4.308226] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 4.283651] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 4.309397] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 4.304320] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 4.305378] systemd[1]: Closed udev Control Socket. lb # [ 4.312285] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 4.306369] systemd[1]: Starting Cleanup udev Database... worker2 # [ 4.289218] systemd[1]: Stopped Coldplug All udev Devices. client # [ 4.307192] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 4.290890] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 4.316235] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 4.317296] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 4.318248] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 4.312277] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 4.313172] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker2 # [ 4.295825] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.314301] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 4.297382] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.298392] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 4.316088] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 4.299312] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 4.300463] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.301462] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.302308] systemd[1]: systemd-udevd.service: Deactivated successfully. client # [ 4.320446] systemd[1]: Stopped Create List of Static Device Nodes. worker2 # [ 4.303216] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 4.304302] systemd[1]: systemd-udevd.service: Consumed 1.119s CPU time over 2.310s wall clock time, 17.9M memory peak. worker2 # [ 4.305698] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker2 # [ 4.306758] systemd[1]: Closed udev Control Socket. worker2 # [ 4.307465] systemd[1]: Starting Cleanup udev Database... worker1 # [ 4.333869] (udev-worker)[94]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker2 # [ 4.316317] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 4.317421] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 4.344390] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. client # [ 4.337887] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker2 # [ 4.320435] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 4.340283] (udev-worker)[94]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory client # [ 4.338981] systemd[1]: Finished Cleanup udev Database. lb # [ 4.346365] systemd[1]: Finished Cleanup udev Database. lb # [ 4.348403] systemd[1]: Reached target Switch Root. worker2 # [ 4.324170] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 4.325186] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 4.326161] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 4.344276] systemd[1]: Reached target Switch Root. lb # [ 4.352118] systemd[1]: Starting NixOS Activation... client # [ 4.345040] systemd[1]: Starting NixOS Activation... worker1 # [ 4.364120] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.365258] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.366722] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.349946] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 4.371209] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.352145] systemd[1]: Finished Cleanup udev Database. worker1 # [ 4.376586] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.356619] systemd[1]: Reached target Switch Root. worker2 # [ 4.357393] systemd[1]: Starting NixOS Activation... worker1 # [ 4.378076] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.386198] systemd[1]: Starting Virtual Console Setup... worker1 # [ 4.413465] systemd-vconsole-setup[156]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 4.416856] systemd[1]: Finished Virtual Console Setup. client # [ 4.421071] initrd-nixos-activation-start[168]: booting system configuration /nix/store/6v5lkzqzv8y8aql79zri93a767rl7bwj-nixos-system-client-test lb # [ 4.430507] initrd-nixos-activation-start[170]: booting system configuration /nix/store/gcbvv54lc1mkgavjm2x1hn2y3wpbcps9-nixos-system-lb-test client # [ 4.453461] initrd-nixos-activation-start[168]: running activation script... worker2 # [ 4.435350] initrd-nixos-activation-start[167]: booting system configuration /nix/store/1z69h5i65q7pa63098rza1vngylyipj5-nixos-system-worker2-test lb # [ 4.463349] initrd-nixos-activation-start[170]: running activation script... worker2 # [ 4.468803] initrd-nixos-activation-start[167]: running activation script... client # [ 4.672830] initrd-nixos-activation-start[191]: setting up /etc... lb # [ 4.698688] initrd-nixos-activation-start[193]: setting up /etc... worker2 # [ 4.704465] initrd-nixos-activation-start[190]: setting up /etc... worker1 # [ 4.736179] systemd[1]: Mounting /sysroot/nix/store... client # [ 4.786096] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 4.788983] systemd[1]: Finished NixOS Activation. client # [ 4.790105] systemd[1]: Starting Switch Root... worker1 # [ 4.792749] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.796263] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.800332] systemd[1]: Starting Find NixOS closure... lb # [ 4.813075] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 4.815848] systemd[1]: Finished NixOS Activation. lb # [ 4.817107] systemd[1]: Starting Switch Root... worker1 # [ 4.812379] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.811090] systemd[1]: Switching root. lb # [ 4.839314] systemd[1]: Switching root. worker1 # [ 4.840505] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker2 # [ 4.823367] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 4.844183] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker2 # [ 4.826334] systemd[1]: Finished NixOS Activation. worker2 # [ 4.827483] systemd[1]: Starting Switch Root... worker1 # [ 4.854591] systemd[1]: Finished Find NixOS closure. worker1 # [ 4.856198] systemd[1]: Reached target Initrd Default Target. worker1 # [ 4.858036] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker2 # [ 4.850890] systemd[1]: Switching root. worker1 # [ 4.880462] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 4.884498] systemd[1]: Stopped target Basic System. worker1 # [ 4.885385] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 4.886287] systemd[1]: Stopped target Path Units. worker1 # [ 4.887718] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 4.892257] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 4.893426] systemd[1]: Stopped target Slice Units. worker1 # [ 4.894199] systemd[1]: Stopped target Socket Units. worker1 # [ 4.895646] systemd[1]: Stopped target System Initialization. worker1 # [ 4.899308] systemd[1]: Stopped target Swaps. worker1 # [ 4.903254] systemd[1]: Stopped target Timer Units. worker1 # [ 4.905028] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 4.907518] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 4.912330] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 4.915976] systemd[1]: Stopped Find NixOS closure. worker1 # [ 4.922990] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 4.925973] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 4.927533] systemd[1]: Stopped Apply Kernel Variables. worker1 # [ 4.931324] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 4.934886] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 4.935639] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 4.940636] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker1 # [ 4.942021] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker1 # [ 4.943044] systemd[1]: Stopped Create System Files and Directories. worker1 # [ 4.945135] systemd[1]: Stopped target Local File Systems. worker1 # [ 4.947215] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 4.948406] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker1 # [ 4.949406] systemd[1]: Stopped Coldplug All udev Devices. worker1 # [ 4.950199] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 4.951219] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.952298] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.953023] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 4.953938] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker1 # [ 4.954840] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 4.955752] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 4.960233] systemd[1]: systemd-udevd.service: Consumed 1.455s CPU time over 3.048s wall clock time, 22.3M memory peak. worker1 # [ 4.961701] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 4.964273] systemd[1]: Closed udev Control Socket. worker1 # [ 4.968187] systemd[1]: Starting Cleanup udev Database... worker1 # [ 4.969018] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker1 # [ 4.970073] systemd[1]: Stopped Create Static Device Nodes in /dev. worker1 # [ 4.970952] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. worker1 # [ 4.976120] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker1 # [ 4.977126] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker1 # [ 4.978071] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 4.978943] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 4.984390] systemd[1]: Finished rw-sysroot-nix-store.service. client # [ 4.995863] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). lb # [ 5.017975] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker1 # [ 5.005159] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. worker1 # [ 5.006709] systemd[1]: Finished Cleanup udev Database. worker1 # [ 5.008803] systemd[1]: Reached target Switch Root. worker1 # [ 5.010961] systemd[1]: Starting NixOS Activation... worker2 # [ 5.028021] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker1 # [ 5.090825] initrd-nixos-activation-start[180]: booting system configuration /nix/store/6xkk768q3zcqgvlrnm0xb1lyjz91891g-nixos-system-worker1-test worker1 # [ 5.124448] initrd-nixos-activation-start[180]: running activation script... worker1 # [ 5.365543] initrd-nixos-activation-start[203]: setting up /etc... worker1 # [ 5.484596] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 5.487316] systemd[1]: Finished NixOS Activation. worker1 # [ 5.488518] systemd[1]: Starting Switch Root... worker1 # [ 5.511109] systemd[1]: Switching root. worker2 # [ 5.504403] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 5.517019] systemd[1]: Detected virtualization qemu. lb # [ 5.538498] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 5.520145] systemd[1]: Detected architecture arm64. client # [ 5.533629] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 5.546036] systemd[1]: Detected virtualization qemu. client # [ 5.541522] systemd[1]: Detected virtualization qemu. worker2 # [ 5.524115] systemd[1]: Detected first boot. lb # [ 5.548201] systemd[1]: Detected architecture arm64. client # [ 5.543737] systemd[1]: Detected architecture arm64. lb # [ 5.551274] systemd[1]: Detected first boot. client # [ 5.546967] systemd[1]: Detected first boot. worker2 # [ 5.530130] systemd[1]: Initializing machine ID from random generator. lb # [ 5.555267] systemd[1]: Initializing machine ID from random generator. client # [ 5.551229] systemd[1]: Initializing machine ID from random generator. worker1 # [ 5.696879] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). client # [ 5.839924] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 5.849315] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 5.834454] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.047773] systemd[1]: Applying preset policy. client # [ 6.051806] systemd[1]: Applying preset policy. worker2 # [ 6.056823] systemd[1]: Applying preset policy. worker1 # [ 6.209805] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 6.222698] systemd[1]: Detected virtualization qemu. worker1 # [ 6.225914] systemd[1]: Detected architecture arm64. worker1 # [ 6.229797] systemd[1]: Detected first boot. worker1 # [ 6.235549] systemd[1]: Initializing machine ID from random generator. lb # [ 6.282107] systemd[1]: Populated /etc with preset unit settings. client # [ 6.302510] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 6.287879] systemd[1]: Populated /etc with preset unit settings. lb # [ 6.514026] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 6.515374] systemd[1]: Stopped initrd-switch-root.service. lb # [ 6.520635] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. lb # [ 6.523315] systemd[1]: Created slice Slice /system/getty. lb # [ 6.525560] systemd[1]: Created slice User and Session Slice. client # [ 6.520703] systemd[1]: initrd-switch-root.service: Deactivated successfully. lb # [ 6.526919] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 6.505440] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 6.522523] systemd[1]: Stopped initrd-switch-root.service. lb # [ 6.529053] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 6.506886] systemd[1]: Stopped initrd-switch-root.service. lb # [ 6.530843] systemd[1]: Expecting device /dev/hvc0... lb # [ 6.532460] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 6.510414] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 6.526794] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. lb # [ 6.534048] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 6.513179] systemd[1]: Created slice Slice /system/getty. client # [ 6.530986] systemd[1]: Created slice Slice /system/getty. lb # [ 6.535701] systemd[1]: Stopped target initrd-fs.target. lb # [ 6.537263] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 6.515417] systemd[1]: Created slice User and Session Slice. client # [ 6.532832] systemd[1]: Created slice User and Session Slice. lb # [ 6.538870] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 6.516696] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 6.540647] systemd[1]: Reached target Virtual Machines and Containers. client # [ 6.535295] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. lb # [ 6.542413] systemd[1]: Reached target Path Units. client # [ 6.537653] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 6.543944] systemd[1]: Reached target Remote File Systems. worker2 # [ 6.518809] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 6.519176] systemd[1]: Expecting device /dev/hvc0... worker2 # [ 6.519460] systemd[1]: Expecting device /dev/ttyAMA0... client # [ 6.539862] systemd[1]: Expecting device /dev/hvc0... lb # [ 6.545527] systemd[1]: Reached target Slice Units. worker2 # [ 6.519729] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 6.519987] systemd[1]: Stopped target initrd-fs.target. lb # [ 6.547066] systemd[1]: Reached target Swaps. worker2 # [ 6.520235] systemd[1]: Stopped target initrd-root-fs.target. client # [ 6.541733] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 6.520479] systemd[1]: Stopped target initrd-switch-root.target. client # [ 6.543786] systemd[1]: Reached target Local Encrypted Volumes. worker2 # [ 6.520730] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 6.520984] systemd[1]: Reached target Path Units. lb # [ 6.550164] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 6.544880] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 6.521227] systemd[1]: Reached target Remote File Systems. worker2 # [ 6.521459] systemd[1]: Reached target Slice Units. worker2 # [ 6.521691] systemd[1]: Reached target Swaps. client # [ 6.546504] systemd[1]: Stopped target initrd-root-fs.target. lb # [ 6.553174] systemd[1]: Listening on Process Core Dump Socket. client # [ 6.548865] systemd[1]: Stopped target initrd-switch-root.target. worker2 # [ 6.531674] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 6.555440] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 6.550053] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 6.534788] systemd[1]: Listening on Process Core Dump Socket. lb # [ 6.557904] systemd[1]: Listening on Factory Reset Management. client # [ 6.552505] systemd[1]: Reached target Path Units. lb # [ 6.559144] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 6.537069] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 6.554377] systemd[1]: Reached target Remote File Systems. client # [ 6.556261] systemd[1]: Reached target Slice Units. worker2 # [ 6.539535] systemd[1]: Listening on Factory Reset Management. lb # [ 6.563405] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 6.540756] systemd[1]: Listening on Hostname Service Socket. client # [ 6.558131] systemd[1]: Reached target Swaps. worker1 # [ 6.561145] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.565737] systemd[1]: Listening on Journal Audit Socket. client # [ 6.561397] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 6.544955] systemd[1]: Starting Journal Log Access Socket... lb # [ 6.569279] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 6.564454] systemd[1]: Listening on Process Core Dump Socket. worker2 # [ 6.547342] systemd[1]: Listening on Journal Audit Socket. lb # [ 6.570863] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 6.566701] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 6.572333] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.550976] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 6.569060] systemd[1]: Listening on Factory Reset Management. worker2 # [ 6.552600] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. client # [ 6.570287] systemd[1]: Listening on Hostname Service Socket. lb # [ 6.575522] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 6.554661] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.556559] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 6.575013] systemd[1]: Starting Journal Log Access Socket... lb # [ 6.581177] systemd[1]: Listening on Disk Repartitioning Service Socket. client # [ 6.576678] systemd[1]: Listening on Journal Audit Socket. lb # [ 6.582569] systemd[1]: Listening on udev Control Socket. lb # [ 6.584124] systemd[1]: Listening on udev Varlink Socket. client # [ 6.579472] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 6.581032] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 6.562112] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 6.587446] systemd[1]: Mounting Huge Pages File System... worker2 # [ 6.562550] systemd[1]: Listening on udev Control Socket. worker2 # [ 6.562894] systemd[1]: Listening on udev Varlink Socket. client # [ 6.582740] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.567374] systemd[1]: Mounting Huge Pages File System... client # [ 6.585973] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki lb # [ 6.592230] systemd[1]: Mounting POSIX Message Queue File System... client # [ 6.591882] systemd[1]: Listening on Disk Repartitioning Service Socket. worker2 # [ 6.574112] systemd[1]: Mounting POSIX Message Queue File System... client # [ 6.593268] systemd[1]: Listening on udev Control Socket. client # [ 6.595403] systemd[1]: Listening on udev Varlink Socket. client # [ 6.599860] systemd[1]: Mounting Huge Pages File System... worker2 # [ 6.582199] systemd[1]: Mounting Kernel Debug File System... lb # [ 6.608010] systemd[1]: Mounting Kernel Debug File System... client # [ 6.607203] systemd[1]: Mounting POSIX Message Queue File System... worker2 # [ 6.593025] systemd[1]: Mounting Kernel Trace File System... lb # [ 6.616330] systemd[1]: Mounting Kernel Trace File System... client # [ 6.619363] systemd[1]: Mounting Kernel Debug File System... client # [ 6.622955] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 6.608416] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 6.611687] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 6.634769] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 6.636551] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 6.622831] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 6.624446] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm client # [ 6.642705] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 6.650435] systemd[1]: Mounting Kernel Configuration File System... client # [ 6.645113] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 6.630131] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 6.653138] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 6.631984] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 6.660527] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 6.662383] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 6.659737] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 6.647726] systemd[1]: Mounting FUSE Control File System... client # [ 6.666181] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker2 # [ 6.650143] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 6.670582] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 6.673867] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 6.684493] systemd[1]: Mounting FUSE Control File System... lb # [ 6.686800] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 6.690047] systemd[1]: Mounting FUSE Control File System... client # [ 6.694132] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker2 # [ 6.681156] systemd[1]: Starting Journal Service... lb # [ 6.716948] systemd[1]: Starting Journal Service... worker1 # [ 6.720339] systemd[1]: Applying preset policy. worker2 # [ 6.707406] systemd[1]: Starting Load Kernel Modules... client # [ 6.726252] systemd[1]: Starting Journal Service... lb # [ 6.735465] systemd[1]: Starting Load Kernel Modules... worker2 # [ 6.716854] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 6.721917] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 6.723420] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 6.742452] systemd[1]: Starting Load Kernel Modules... lb # [ 6.750782] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 6.731126] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 6.738322] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 6.740286] systemd[1]: Mounted Huge Pages File System. worker2 # [ 6.742413] systemd[1]: Mounted POSIX Message Queue File System. worker2 # [ 6.745600] systemd[1]: Mounted Kernel Debug File System. client # [ 6.762199] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 6.770842] systemd[1]: Starting Remount Root and Kernel File Systems... lb # [ 6.772351] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.751313] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 6.753387] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 6.755556] systemd[1]: Mounted FUSE Control File System. client # [ 6.775827] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 6.778239] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 6.782111] systemd-journald[264]: Collecting audit messages is enabled. lb # [ 6.779508] systemd[1]: Queued start job for default target Multi-User System. lb # [ 6.793779] systemd[1]: Starting Coldplug All udev Devices... lb # [ 6.784561] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 6.792068] systemd-journald[262]: Collecting audit messages is enabled. worker2 # [ 6.794883] systemd[1]: Finished Create List of Static Device Nodes. client # [ 6.799468] systemd[1]: Queued start job for default target Multi-User System. lb # [ 6.821136] systemd[1]: Started Journal Service. client # [ 6.805817] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 6.814697] systemd[1]: Listening on Journal Log Access Socket. client # [ 6.821809] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 6.803309] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 6.817781] systemd[1]: Mounted Huge Pages File System. lb # [ 6.818594] systemd[1]: Mounted POSIX Message Queue File System. client # [ 6.825339] systemd[1]: Started Journal Service. lb # [ 6.819427] systemd[1]: Mounted Kernel Debug File System. lb # [ 6.827857] systemd[1]: Mounted Kernel Trace File System. client # [ 6.821303] systemd[1]: Listening on Journal Log Access Socket. lb # [ 6.831869] systemd[1]: Finished Create List of Static Device Nodes. client # [ 6.825045] systemd[1]: Mounted Huge Pages File System. lb # [ 6.835132] systemd[1]: Mounted Kernel Configuration File System. client # [ 6.828327] systemd[1]: Mounted POSIX Message Queue File System. client # [ 6.829240] systemd[1]: Mounted Kernel Debug File System. client # [ 6.830123] systemd[1]: Mounted Kernel Trace File System. client # [ 6.842559] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 6.853904] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 6.856153] EXT4-fs (vda): re-mounted e3dfeb75-f1f6-4868-8710-4a19bb11e720. client # [ 6.863075] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 6.860427] systemd-journald[261]: Collecting audit messages is enabled. lb # [ 6.887881] EXT4-fs (vda): re-mounted 32ce671e-284c-4bdb-b1b3-4bb2ba4325d8. client # [ 6.871408] systemd-modules-load[263]: Module 'atkbd' is built in lb # [ 6.879452] systemd-modules-load[265]: Module 'atkbd' is built in lb # [ 6.882236] systemd-modules-load[265]: Module 'loop' is built in client # [ 6.875207] systemd-modules-load[263]: Module 'loop' is built in worker2 # [ 6.871037] systemd[1]: Finished Remount Root and Kernel File Systems. worker2 # [ 6.871738] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 6.872079] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 6.880426] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 6.885675] systemd[1]: Finished Load Kernel Modules. worker2 # [ 6.882894] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 6.895038] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 6.888386] systemd-oomd[265]: No swap; memory pressure usage will be degraded lb # [ 6.896313] systemd[1]: Finished Load Kernel Modules. worker2 # [ 6.885134] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 6.890053] systemd[1]: Finished Load Kernel Modules. lb # [ 6.899688] systemd[1]: Listening on Disk Image Download Service Socket. lb # [ 6.903476] systemd[1]: Starting Firewall... client # [ 6.899698] systemd[1]: Starting Apply Kernel Variables... client # [ 6.903211] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 6.902491] systemd[1]: Starting Firewall... lb # [ 6.915941] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 6.926063] EXT4-fs (vda): re-mounted 1254f221-7786-43be-840c-ce3d9642ab10. lb # [ 6.919279] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 6.925027] systemd-oomd[267]: No swap; memory pressure usage will be degraded lb # [ 6.926357] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 6.915522] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 6.919292] systemd[1]: Started Journal Service. worker2 # [ 6.909190] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 6.910460] systemd[1]: systemd-journald.service: Deactivated successfully. client # [ 6.930591] systemd[1]: Finished Remount Root and Kernel File Systems. client # [ 6.931623] systemd[1]: Listening on Disk Image Download Service Socket. lb # [ 6.946235] systemd[1]: Starting Apply Kernel Variables... client # [ 6.938971] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 6.947148] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 6.942069] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 6.943708] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 6.927287] systemd-modules-load[262]: Module 'atkbd' is built in lb # [ 6.952364] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. lb # [ 6.957330] systemd[1]: Mounted FUSE Control File System. worker2 # [ 6.937112] systemd-modules-load[262]: Module 'loop' is built in client # [ 6.955994] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 6.957268] systemd[1]: Mounted FUSE Control File System. worker1 # [ 6.976076] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 6.949335] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.034631] systemd-journald[262]: Received client request to flush runtime journal. worker2 # [ 7.013208] systemd-oomd[263]: No swap; memory pressure usage will be degraded lb # [ 7.040557] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 7.055070] systemd-journald[264]: Received client request to flush runtime journal. worker2 # [ 7.027466] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 7.031000] systemd[1]: Reached target First Boot Complete. worker2 # [ 7.039080] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 7.059725] systemd-journald[261]: Received client request to flush runtime journal. client # [ 7.071346] systemd[1]: Finished Load/Save OS Random Seed. client # [ 7.078558] systemd[1]: Reached target First Boot Complete. client # [ 7.081233] systemd[1]: Finished Flush Journal to Persistent Storage. lb # [ 7.092325] systemd[1]: Reached target First Boot Complete. lb # [ 7.095382] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 7.092981] systemd[1]: Finished Apply Kernel Variables. lb # [ 7.108796] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 7.118342] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 7.121517] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 7.113394] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 7.114652] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 7.115558] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 7.123621] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 7.207847] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker1 # [ 7.209553] systemd[1]: Stopped initrd-switch-root.service. worker1 # [ 7.212800] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker1 # [ 7.217067] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.219329] systemd[1]: Created slice User and Session Slice. worker1 # [ 7.220573] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 7.223326] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker1 # [ 7.225499] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.227356] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 7.229235] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 7.231240] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 7.233087] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 7.234987] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 7.236949] systemd[1]: Reached target Virtual Machines and Containers. worker1 # [ 7.238965] systemd[1]: Reached target Path Units. worker1 # [ 7.240797] systemd[1]: Reached target Remote File Systems. worker1 # [ 7.242694] systemd[1]: Reached target Slice Units. worker1 # [ 7.244626] systemd[1]: Reached target Swaps. worker1 # [ 7.247819] systemd[1]: Listening on Query the User Interactively for a Password. worker1 # [ 7.251014] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.253327] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 7.255738] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.256942] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 7.261230] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 7.263590] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.267257] systemd[1]: Listening on Console Output Muting Service Socket. worker1 # [ 7.269319] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 7.271141] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.272933] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 7.278840] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 7.280258] systemd[1]: Listening on udev Control Socket. worker1 # [ 7.281756] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 7.286017] systemd[1]: Mounting Huge Pages File System... worker1 # [ 7.291715] systemd[1]: Mounting POSIX Message Queue File System... client # [ 7.278316] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 7.280458] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 7.299974] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 7.274824] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 7.311651] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 7.276452] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 7.279449] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 7.323896] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 7.325332] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 7.342385] systemd[1]: Mounting Kernel Configuration File System... worker1 # [ 7.342752] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.343008] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.349287] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.367312] systemd[1]: Mounting FUSE Control File System... worker1 # [ 7.367708] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.398319] systemd[1]: Starting Journal Service... worker1 # [ 7.415102] systemd[1]: Starting Load Kernel Modules... lb # [ 7.412962] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 7.414290] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 7.431228] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 7.421718] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 7.438157] systemd[1]: Starting Remount Root and Kernel File Systems... worker1 # [ 7.439597] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.451513] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 7.455067] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 7.459011] systemd[1]: Mounted Huge Pages File System. worker1 # [ 7.462000] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 7.464925] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 7.467956] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 7.471493] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 7.471951] systemd[1]: Mounted FUSE Control File System. worker1 # [ 7.487483] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 7.492550] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 7.512736] systemd[1]: Mounting /run/wrappers... worker2 # [ 7.497905] systemd[1]: Mounting /run/wrappers... worker1 # [ 7.574100] systemd[1]: Finished Load Kernel Modules. worker1 # [ 7.579939] systemd[1]: Starting Firewall... lb # [ 7.576739] systemd[1]: Mounted /run/wrappers. lb # [ 7.577562] systemd[1]: Reached target Local File Systems. worker1 # [ 7.584360] systemd-journald[273]: Collecting audit messages is enabled. worker2 # [ 7.555213] systemd-udevd[300]: Using default interface naming scheme 'v261'. worker1 # [ 7.600302] EXT4-fs (vda): re-mounted 1ca2ebb5-06c7-441d-ade7-bf2e4a4cf314. worker2 # [ 7.569294] systemd[1]: Mounted /run/wrappers. worker1 # [ 7.603726] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 7.572380] systemd[1]: Reached target Local File Systems. worker2 # [ 7.573258] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 7.574198] systemd[1]: Starting register-nix-paths.service... lb # [ 7.600417] systemd[1]: Listening on Boot Loader Control Service Socket. lb # [ 7.601463] systemd[1]: Starting register-nix-paths.service... worker1 # [ 7.612119] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 7.612718] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 7.613008] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 7.612212] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 7.613152] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 7.625761] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 7.617125] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 7.592580] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 7.628441] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.600543] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 7.637247] systemd[1]: Started Journal Service. worker2 # [ 7.610472] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 7.632586] systemd[1]: Queued start job for default target Multi-User System. worker1 # [ 7.633739] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 7.640103] systemd[1]: Starting Create System Files and Directories... worker2 # [ 7.617582] systemd[1]: Starting Create System Files and Directories... worker1 # [ 7.642798] systemd-modules-load[274]: Module 'atkbd' is built in worker1 # [ 7.657373] systemd-modules-load[274]: Module 'loop' is built in client # [ 7.661035] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 7.664629] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.668267] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 7.670393] systemd[1]: Reached target Preparation for Local File Systems. client # [ 7.675271] systemd[1]: Mounting /run/wrappers... client # [ 7.677146] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 7.701120] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 7.719133] systemd[1]: Mounted /run/wrappers. client # [ 7.719919] systemd[1]: Reached target Local File Systems. client # [ 7.726632] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 7.730120] systemd[1]: Starting register-nix-paths.service... worker2 # [ 7.714249] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 7.736221] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 7.739282] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 7.741702] systemd-oomd[275]: No swap; memory pressure usage will be degraded client # [ 7.740289] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 7.744180] systemd[1]: Starting Save Transient machine-id to Disk... lb # [ 7.761511] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 7.759801] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 7.764539] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 7.765336] systemd[1]: Reached target First Boot Complete. worker1 # [ 7.766124] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker1 # [ 7.784088] systemd-journald[273]: Received client request to flush runtime journal. lb # [ 7.775999] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 7.769280] systemd[1]: Starting Create System Files and Directories... lb # [ 7.781677] systemd-udevd[300]: Using default interface naming scheme 'v261'. worker1 # [ 7.830113] systemd[1]: Finished Apply Kernel Variables. client # [ 7.837543] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 7.844801] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 7.825761] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 7.845253] systemd[1]: Finished Save Transient machine-id to Disk. worker2 # [ 7.833715] systemd[1]: Finished Create System Files and Directories. worker2 # [ 7.847150] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 7.861153] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 7.902770] systemd[1]: Finished Create System Files and Directories. client # [ 7.900446] systemd-udevd[292]: Using default interface naming scheme 'v261'. lb # [ 7.914528] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 7.920253] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 7.920528] systemd[1]: Finished Create System Files and Directories. client # [ 7.928301] systemd[1]: Starting Rebuild Journal Catalog... client # [ 7.934730] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 7.938538] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 7.944347] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 7.948569] systemd[1]: Starting Rule-based Manager for Device Events and Files... lb # [ 7.979530] systemd[1]: Started Rule-based Manager for Device Events and Files. worker2 # [ 7.984639] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 8.041471] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 8.056126] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 8.063546] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 8.067116] systemd-udevd[308]: Using default interface naming scheme 'v261'. client # [ 8.069537] systemd[1]: Starting Update is Completed... worker2 # [ 8.073458] systemd[1]: Finished Rebuild Journal Catalog. worker2 # [ 8.078294] systemd[1]: Starting Update is Completed... lb # [ 8.129322] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 8.132818] systemd[1]: Finished Rebuild Journal Catalog. client # [ 8.131494] systemd[1]: Finished Update is Completed. lb # [ 8.148095] systemd[1]: Starting Update is Completed... worker1 # [ 8.179930] systemd[1]: Started Rule-based Manager for Device Events and Files. worker1 # [ 8.199455] systemd[1]: Mounting /run/wrappers... worker2 # [ 8.190573] systemd[1]: Finished Update is Completed. lb # [ 8.252871] systemd[1]: Finished Update is Completed. worker1 # [ 8.345380] systemd[1]: Mounted /run/wrappers. worker1 # [ 8.346271] systemd[1]: Reached target Local File Systems. worker1 # [ 8.357364] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 8.373001] systemd[1]: Starting register-nix-paths.service... worker1 # [ 8.380301] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 8.381208] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 8.382477] systemd[1]: Starting Save Transient machine-id to Disk... worker1 # [ 8.383368] systemd[1]: Starting Create System Files and Directories... lb # [ 8.485326] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 8.481556] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker1 # [ 8.485946] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 8.590208] systemd[1]: Finished Create System Files and Directories. worker1 # [ 8.600726] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 8.609263] systemd[1]: Starting Record System Boot/Shutdown in UTMP... client # [ 8.608560] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 8.612210] systemd[1]: Finished Create SUID/SGID Wrappers. worker1 # [ 8.717302] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker2 # [ 8.711082] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 8.712630] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 8.744710] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker1 # [ 8.789440] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 8.792818] systemd[1]: Starting Update is Completed... lb # [ 8.890065] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 8.889217] systemd[1]: Finished Update is Completed. lb # [ 8.895037] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 8.953429] systemd[1]: Finished register-nix-paths.service. lb # [ 8.956966] systemd[1]: Reached target System Initialization. lb # [ 8.957869] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 8.958856] systemd[1]: Started niks3 garbage collection timer. lb # [ 8.959696] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 8.937676] systemd[1]: Finished Firewall. lb # [ 8.967443] systemd[1]: Reached target Timer Units. lb # [ 8.969133] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 8.974672] systemd[1]: Listening on niks3 server socket. lb # [ 8.975553] systemd[1]: Listening on Nix Daemon Socket. lb # [ 8.980333] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 8.981761] systemd[1]: Reached target Socket Units. lb # [ 8.982499] systemd[1]: Reached target Basic System. lb # [ 8.983213] systemd[1]: Starting Import lastlog data into lastlog2 database... lb # [ 8.991880] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.000986] systemd[1]: Starting Post-Boot Actions... lb # [ 9.016165] systemd[1]: Started Reset console on configuration changes. lb # [ 9.057804] systemd[1]: Starting resolvconf update... lb # [ 9.073838] systemd[1]: Started rustfs.service. lb # [ 9.090798] systemd[1]: Starting rustfs-bucket.service... lb # [ 9.155025] systemd[1]: Starting D-Bus System Message Bus... lb # [ 9.165587] nsncd[439]: Sep 21 14:43:44.765 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 9.174954] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 9.175901] systemd[1]: Finished Post-Boot Actions. worker2 # [ 9.175455] systemd[1]: Finished Coldplug All udev Devices. lb # [ 9.213799] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 9.214856] systemd[1]: Reached target User and Group Name Lookups. client # [ 9.214843] systemd[1]: Starting Virtual Console Setup... client # [ 9.237442] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 9.241725] systemd[1]: Stopped Virtual Console Setup. lb # [ 9.261765] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 9.258114] systemd[1]: Starting User Login Management... client # [ 9.251456] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 9.256751] systemd[1]: Starting Virtual Console Setup... client # [ 9.291721] systemd[1]: Finished register-nix-paths.service. worker1 # [ 9.299329] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 9.301142] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.356994] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 9.359820] systemd[1]: Finished register-nix-paths.service. worker2 # [ 9.363238] systemd[1]: Reached target System Initialization. worker2 # [ 9.364255] systemd[1]: Started Discard unused filesystem blocks once a week. worker2 # [ 9.371304] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 9.398695] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 9.383792] systemd[1]: Reached target Timer Units. worker2 # [ 9.392346] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 9.400464] systemd[1]: Listening on Nix Daemon Socket. client # [ 9.421355] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 9.408452] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 9.414456] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 9.416165] systemd[1]: Reached target Socket Units. worker2 # [ 9.424105] systemd[1]: Reached target Basic System. worker2 # [ 9.430398] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 9.442044] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 9.458674] systemd[1]: Starting Post-Boot Actions... worker2 # [ 9.462704] systemd[1]: Started Reset console on configuration changes. worker2 # [ 9.471035] systemd[1]: Starting resolvconf update... client # [ 9.492560] (udev-worker)[342]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 9.501814] dbus-broker-launch[450]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 9.479656] systemd[1]: Starting D-Bus System Message Bus... client # [ 9.497709] (udev-worker)[340]: Network interface NamePolicy= disabled on kernel command line. client # [ 9.500595] (udev-worker)[342]: Network interface NamePolicy= disabled on kernel command line. lb # [ 9.512759] dbus-broker-launch[450]: NSS returned no entry for 'systemd-timesync' lb # [ 9.513806] dbus-broker-launch[450]: Invalid user-name in /nix/store/4yiprls1sll47jid3xzq0m9nmi1qz2ww-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 9.511347] systemd[1]: Finished Post-Boot Actions. lb # [ 9.536561] systemd-logind[470]: New seat seat0. lb # [ 9.547271] systemd[1]: Started User Login Management. lb # [ 9.554707] systemd[1]: Starting linger-users.service... worker2 # [ 9.544612] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 9.549304] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 9.574539] systemd[1]: Finished Coldplug All udev Devices. lb # [ 9.581851] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 9.570371] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker2 # [ 9.560523] nsncd[481]: Sep 21 14:43:45.149 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 9.592006] systemd[1]: Finished Firewall. worker2 # [ 9.580387] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 9.584315] systemd[1]: Starting User Login Management... worker2 # [ 9.600633] systemd[1]: Finished Import lastlog data into lastlog2 database. lb # [ 9.632880] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 9.637388] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 9.638355] systemd[1]: Stopped target User and Group Name Lookups. lb # [ 9.639233] systemd[1]: Stopping User and Group Name Lookups... worker2 # [ 9.616877] dbus-broker-launch[485]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 9.628073] dbus-broker-launch[485]: NSS returned no entry for 'systemd-timesync' client # [ 9.648554] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 9.656221] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 9.631027] dbus-broker-launch[485]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 9.657271] systemd[1]: nscd.service: Deactivated successfully. lb # [ 9.658123] systemd[1]: Stopped Name Service Cache Daemon (nsncd). client # [ 9.653287] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 9.657816] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 9.658729] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 9.659968] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 9.675734] dbus-broker-launch[450]: Ready worker2 # [ 9.652739] systemd[1]: Started D-Bus System Message Bus. client # [ 9.671093] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 9.676582] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 9.675235] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 9.678035] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 9.694571] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.702536] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 9.705692] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 9.711592] systemd[1]: Finished linger-users.service. worker2 # [ 9.696527] dbus-broker-launch[485]: Ready worker2 # [ 9.752422] systemd-logind[493]: New seat seat0. worker2 # [ 9.757374] systemd[1]: Started User Login Management. worker2 # [ 9.766359] systemd[1]: Starting linger-users.service... worker2 # [ 9.773325] systemd[1]: Stopped target Host and Network Name Lookups. lb # [ 9.805092] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 9.784798] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 9.809243] nsncd[540]: Sep 21 14:43:45.409 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 9.790785] systemd[1]: Stopped target User and Group Name Lookups. client # [ 9.822268] mousedev: PS/2 mouse device common for all mice lb # [ 9.821012] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 9.822031] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 9.800184] systemd[1]: Stopping User and Group Name Lookups... worker2 # [ 9.801883] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 9.803227] systemd[1]: nscd.service: Deactivated successfully. worker2 # [ 9.805674] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 9.812797] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 9.846334] systemd[1]: Finished resolvconf update. lb # [ 9.858813] systemd[1]: Starting DHCP Client... worker1 # [ 9.880789] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 9.865225] systemd[1]: linger-users.service: Deactivated successfully. client # [ 9.886143] systemd-vconsole-setup[383]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 9.870784] systemd[1]: Finished linger-users.service. client # [ 9.892091] systemd[1]: Finished Virtual Console Setup. worker2 # [ 9.880295] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 9.883214] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 9.888780] nsncd[549]: Sep 21 14:43:45.482 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 9.891509] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 9.907586] systemd[1]: Finished resolvconf update. worker1 # [ 9.930311] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 9.913748] systemd[1]: Reached target Preparation for Network. worker2 # [ 9.921456] systemd[1]: Starting DHCP Client... worker2 # [ 9.924228] systemd[1]: Starting Extra networking commands.... worker1 # [ 9.965939] systemd[1]: Finished register-nix-paths.service. worker1 # [ 9.967880] systemd[1]: Reached target System Initialization. worker1 # [ 9.971828] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 9.974759] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 9.977981] systemd[1]: Reached target Timer Units. worker1 # [ 9.983274] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 9.990258] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 9.998498] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 10.007251] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 10.016190] systemd[1]: Reached target Socket Units. worker1 # [ 10.022537] systemd[1]: Reached target Basic System. worker1 # [ 10.035137] systemd[1]: Started backdoor.service. worker1 # [ 10.041899] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 10.053116] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.066407] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.073184] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.083623] systemd[1]: Starting resolvconf update... worker1 # [ 10.093720] systemd[1]: Starting D-Bus System Message Bus... lb # [ 10.124510] dhcpcd[585]: dhcpcd-10.3.2 starting lb # [ 10.141663] dhcpcd[599]: dev: loaded udev worker1 # [ 10.139579] systemd[1]: Finished Post-Boot Actions. lb # [ 10.145621] dhcpcd[599]: no valid interfaces found lb # [ 10.149299] dhcpcd[599]: no valid interfaces found lb # [ 10.150259] dhcpcd[599]: libudev: received NULL device lb # [ 10.151014] dhcpcd[599]: libudev: received NULL device worker2 # [ 10.133649] dhcpcd[582]: dhcpcd-10.3.2 starting worker2 # [ 10.140529] dhcpcd[624]: dev: loaded udev worker2 # [ 10.144483] dhcpcd[624]: no valid interfaces found worker2 # [ 10.148307] dhcpcd[624]: no valid interfaces found worker2 # [ 10.149422] dhcpcd[624]: libudev: received NULL device worker2 # [ 10.152358] dhcpcd[624]: libudev: received NULL device worker1 # [ 10.172919] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 10.175566] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 10.181470] systemd[1]: Reached target Host and Network Name Lookups. client # [ 10.179719] systemd[1]: Finished Create SUID/SGID Wrappers. worker1 # connecting to host... client # [ 10.184396] systemd[1]: Reached target System Initialization. client # [ 10.185250] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 10.186204] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 10.187123] systemd[1]: Reached target Timer Units. client # [ 10.187836] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 10.190070] nsncd[492]: Sep 21 14:43:45.782 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.174073] systemd[1]: Finished Extra networking commands.. worker2 # [ 10.176418] systemd[1]: Reached target Network. client # [ 10.197395] systemd[1]: Listening on Nix Daemon Socket. client # [ 10.199588] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker2 # [ 10.184122] systemd[1]: Starting Permit User Sessions... worker1 # [ 10.206640] systemd[1]: Reached target User and Group Name Lookups. client # [ 10.204559] systemd[1]: Reached target Socket Units. client # [ 10.205891] systemd[1]: Reached target Basic System. client # [ 10.208909] systemd[1]: Started backdoor.service. worker1 # [ 10.213531] systemd[1]: Starting User Login Management... client # [ 10.212268] systemd[1]: Starting Import lastlog data into lastlog2 database... client # [ 10.214417] systemd[1]: Starting nftables firewall... client # [ 10.230777] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 10.218030] systemd[1]: Finished Permit User Sessions. worker2 # [ 10.220202] systemd[1]: Started Getty on tty1. worker1 # [ 10.240943] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 10.221834] systemd[1]: Reached target Login Prompts. client # [ 10.247713] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.261513] dbus-broker-launch[496]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 10.276721] dbus-broker-launch[496]: NSS returned no entry for 'systemd-timesync' worker1 # [ 10.277784] dbus-broker-launch[496]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client # [ 10.290737] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.298285] systemd[1]: Started D-Bus System Message Bus. client # [ 10.306779] systemd[1]: Starting resolvconf update... worker1 # [ 10.342584] dbus-broker-launch[496]: Ready worker2 # [ 10.325005] systemd[1]: Starting Virtual Console Setup... lb # [ 10.359619] systemd[1]: Finished Firewall. lb # [ 10.361761] systemd[1]: Reached target Preparation for Network. lb # [ 10.366078] systemd[1]: Starting Extra networking commands.... worker2 # [ 10.345401] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 10.346603] systemd[1]: Stopped Virtual Console Setup. client # [ 10.364603] systemd[1]: Starting D-Bus System Message Bus... client # [ 10.367853] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.352717] systemd[1]: Starting Virtual Console Setup... client # [ 10.372839] nsncd[454]: Sep 21 14:43:45.976 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.388408] systemd-logind[515]: New seat seat0. client # connecting to host... worker1 # [ 10.391959] (udev-worker)[483]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.401351] systemd[1]: Started User Login Management. worker1 # [ 10.402160] systemd[1]: Starting linger-users.service... client # [ 10.400326] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.382692] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 10.410413] (udev-worker)[335]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 10.416489] systemd[1]: Starting Virtual Console Setup... worker1 # [ 10.420190] (udev-worker)[335]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.430747] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 10.431740] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 10.442518] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client: Guest shell says: b'Spawning backdoor root shell...\n' lb # [ 10.447547] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 10.448310] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 10.456624] systemd[1]: Stopping User and Group Name Lookups... lb # [ 10.462948] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 10.465502] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 10.478657] systemd[1]: Starting Virtual Console Setup... client # [ 10.466574] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.457483] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client: connected to guest root shell worker1 # [ 10.458318] systemd[1]: nscd.service: Deactivated successfully. client: (connecting took 10.96 seconds) worker1 # [ 10.459058] systemd[1]: Stopped Name Service Cache Daemon (nsncd). client: (finished: waiting for the VM to finish booting, in 10.96 seconds) worker1 # [ 10.459839] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 10.488237] systemd[1]: Starting User Login Management... worker1 # [ 10.503864] systemd[1]: linger-users.service: Deactivated successfully. client # [ 10.517765] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 10.506687] systemd[1]: Finished linger-users.service. worker1 # [ 10.539428] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.543503] nsncd[567]: Sep 21 14:43:46.147 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.553060] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.557557] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.567932] systemd[1]: Finished resolvconf update. worker1 # [ 10.572725] systemd[1]: Reached target Preparation for Network. worker1 # [ 10.573560] systemd[1]: Starting DHCP Client... worker1 # [ 10.585781] systemd[1]: Starting Extra networking commands.... worker2 # [ 10.574401] systemd-vconsole-setup[639]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 10.581628] systemd[1]: Finished Virtual Console Setup. client # [ 10.643971] dbus-broker-launch[460]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.667640] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 10.686922] systemd-logind[484]: New seat seat0. client # [ 10.692933] systemd-logind[484]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) client # [ 10.694199] systemd-logind[484]: Watching system buttons on /dev/input/event0 (gpio-keys) worker2 # [ 10.684344] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 10.690261] systemd[1]: Started backdoor.service. client # [ 10.711407] systemd[1]: Started User Login Management. client # [ 10.714663] dbus-broker-launch[460]: NSS returned no entry for 'systemd-timesync' client # [ 10.715695] dbus-broker-launch[460]: Invalid user-name in /nix/store/6wvgzhp3xj1v1xybf1ixpbqi5gif5fbv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" lb # [ 10.730432] systemd[1]: Finished Extra networking commands.. lb # [ 10.737665] systemd[1]: Reached target Network. client # [ 10.732668] systemd[1]: Starting linger-users.service... lb # [ 10.745904] systemd[1]: Starting PostgreSQL Server... lb # [ 10.753875] systemd[1]: Starting Permit User Sessions... client # [ 10.760270] systemd[1]: Started D-Bus System Message Bus. worker2 # connecting to host... worker1 # [ 10.790378] dhcpcd[601]: dhcpcd-10.3.2 starting client # [ 10.796799] systemd[1]: Stopped target Host and Network Name Lookups. worker2 # [ 10.794867] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 10.802385] dhcpcd[639]: dev: loaded udev client # [ 10.805252] systemd[1]: Stopping Host and Network Name Lookups... client # [ 10.806228] systemd[1]: Stopped target User and Group Name Lookups. client # [ 10.807120] systemd[1]: Stopping User and Group Name Lookups... client # [ 10.807910] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 10.791026] (udev-worker)[651]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 10.797373] (udev-worker)[650]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 10.798545] (udev-worker)[651]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.841911] 8021q: 802.1Q VLAN Support v1.8 client # [ 10.828203] systemd[1]: nscd.service: Deactivated successfully. client # [ 10.829102] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 10.842031] systemd[1]: Finished Permit User Sessions. client # [ 10.837780] dbus-broker-launch[460]: Ready client # [ 10.842548] systemd[1]: linger-users.service: Deactivated successfully. lb # [ 10.850089] systemd[1]: Started Getty on tty1. lb # [ 10.850864] systemd[1]: Reached target Login Prompts. worker1 # [ 10.847169] systemd-logind[515]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 10.847933] systemd[1]: Finished linger-users.service. worker1 # [ 10.855335] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 10.872758] systemd[1]: Starting Address configuration of eth1... client # [ 10.875801] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.912194] systemd[1]: Finished Extra networking commands.. worker2 # [ 10.905850] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 10.918126] systemd[1]: Reached target Network. worker1 # [ 10.921606] systemd[1]: Starting Permit User Sessions... worker1 # [ 10.949784] cfg80211: Loading compiled-in X.509 certificates for regulatory database client # [ 10.944196] systemd[1]: Finished resolvconf update. client # [ 10.946911] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 10.953524] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.950869] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 10.954772] nsncd[551]: Sep 21 14:43:46.565 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.951389] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 10.957455] systemd[1]: Reached target User and Group Name Lookups. client # [ 10.959408] systemd[1]: Starting DHCP Client... worker2 # [ 10.954147] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 10.954471] cfg80211: failed to load regulatory.db worker1 # [ 10.979231] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 10.979759] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' worker1 # [ 10.982474] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 10.982814] cfg80211: failed to load regulatory.db lb # [ 10.996649] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 11.007018] systemd[1]: Finished Permit User Sessions. lb # [ 11.012636] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker1 # [ 11.012269] systemd[1]: Started Getty on tty1. worker1 # [ 11.013032] systemd[1]: Reached target Login Prompts. lb # [ 11.021297] systemd[1]: Started backdoor.service. worker2 # [ 11.005949] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 11.010040] systemd[1]: Starting Address configuration of eth1... worker1 # [ 11.055850] 8021q: adding VLAN 0 to HW filter on device eth1 worker1 # [ 11.059702] network-addresses-eth1-start[651]: adding address 192.168.1.3/24... done worker1 # [ 11.076495] network-addresses-eth1-start[651]: adding address 2001:db8:1::3/64... done worker1 # [ 11.094141] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 11.083457] postgresql-pre-start[691]: The files belonging to this database system will be owned by user "postgres". worker1 # [ 11.082478] dhcpcd[639]: eth0: waiting for carrier lb # [ 11.088339] postgresql-pre-start[691]: This user must also own the server process. worker1 # [ 11.086392] dhcpcd[639]: eth0: waiting for carrier worker1 # [ 11.090728] dhcpcd[639]: eth0: carrier acquired lb # [ 11.095871] postgresql-pre-start[691]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 11.100498] postgresql-pre-start[691]: The default database encoding has accordingly been set to "UTF8". lb # [ 11.101757] postgresql-pre-start[691]: The default text search configuration will be set to "english". lb # [ 11.102994] postgresql-pre-start[691]: Data page checksums are enabled. lb # [ 11.103881] postgresql-pre-start[691]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok worker1 # [ 11.103949] systemd[1]: Finished Address configuration of eth1. worker1 # [ 11.109317] dhcpcd[639]: DUID 00:01:00:01:32:43:ff:22:52:54:00:12:34:56 worker1 # [ 11.110308] dhcpcd[639]: eth0: IAID 00:12:34:56 worker1 # [ 11.110931] dhcpcd[639]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 11.117968] postgresql-pre-start[691]: creating subdirectories ... ok worker2 # [ 11.106769] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 11.118888] postgresql-pre-start[691]: selecting dynamic shared memory implementation ... posix worker2 # [ 11.095712] dhcpcd[624]: eth0: waiting for carrier[ 11.111355] 8021q: adding VLAN 0 to HW filter on device eth1 worker2 # worker2 # [ 11.101640] dhcpcd[624]: eth0: carrier acquired lb # connecting to host... worker2 # [ 11.113821] dhcpcd[624]: DUID 00:01:00:01:32:43:ff:22:52:54:00:12:34:56 worker2 # [ 11.114790] dhcpcd[624]: eth0: IAID 00:12:34:56 worker2 # [ 11.115414] dhcpcd[624]: eth0: adding address fe80::5054:ff:fe12:3456 worker2 # [ 11.121586] network-addresses-eth1-start[669]: adding address 192.168.1.4/24... done client # [ 11.140618] dhcpcd[582]: dhcpcd-10.3.2 starting worker2 # [ 11.130514] network-addresses-eth1-start[669]: adding address 2001:db8:1::4/64... done client # [ 11.148678] dhcpcd[598]: dev: loaded udev worker2 # [ 11.144735] systemd[1]: Finished Address configuration of eth1. client # [ 11.202296] 8021q: 802.1Q VLAN Support v1.8 worker2 # [ 11.174343] systemd-logind[493]: Watching system buttons on /dev/input/event0 (gpio-keys) worker1 # [ 11.209315] mousedev: PS/2 mouse device common for all mice lb # [ 11.220636] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 11.225907] systemd-logind[515]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.259277] systemd-vconsole-setup[627]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 11.267380] systemd[1]: Finished Virtual Console Setup. lb # [ 11.271664] (udev-worker)[689]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 11.279751] (udev-worker)[685]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 11.270075] mousedev: PS/2 mouse device common for all mice lb # [ 11.282661] (udev-worker)[689]: Network interface NamePolicy= disabled on kernel command line. client # [ 11.318758] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 11.329146] postgresql-pre-start[691]: selecting default "max_connections" ... 100 client # [ 11.358336] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 11.358810] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 11.362828] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 11.363142] cfg80211: failed to load regulatory.db lb # [ 11.379205] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker2 # [ 11.377887] systemd-logind[493]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.431738] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 11.433266] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 11.436431] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 11.436751] cfg80211: failed to load regulatory.db client # [ 11.457141] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 11.445629] dhcpcd[598]: eth0: waiting for carrier client # [ 11.448288] dhcpcd[598]: libudev: received NULL device client # [ 11.449058] dhcpcd[598]: libudev: received NULL device client # [ 11.449926] dhcpcd[598]: eth0: carrier acquired client # [ 11.461779] systemd[1]: Finished nftables firewall. client # [ 11.466578] systemd[1]: Reached target Preparation for Network. client # [ 11.471318] dhcpcd[598]: DUID 00:01:00:01:32:43:ff:23:52:54:00:12:34:56 client # [ 11.475554] dhcpcd[598]: eth0: IAID 00:12:34:56 client # [ 11.477007] systemd[1]: Starting Address configuration of eth1... client # [ 11.477843] dhcpcd[598]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 11.482166] systemd[1]: Starting Extra networking commands.... lb # [ 11.505382] postgresql-pre-start[691]: selecting default "shared_buffers" ... 128MB lb # [ 11.548196] systemd-logind[470]: Watching system buttons on /dev/input/event0 (gpio-keys) lb # [ 11.556831] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 11.567025] systemd[1]: Starting Address configuration of eth1... client # [ 11.582947] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 11.592887] network-addresses-eth1-start[622]: adding address 192.168.1.1/24... done client # [ 11.620933] network-addresses-eth1-start[622]: adding address 2001:db8:1::1/64... done lb # [ 11.656132] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 11.648398] dhcpcd[599]: eth0: waiting for carrier lb # [ 11.649209] dhcpcd[599]: eth0: carrier acquired client # [ 11.646821] systemd[1]: Finished Address configuration of eth1. lb # [ 11.663462] dhcpcd[599]: DUID 00:01:00:01:32:43:ff:23:52:54:00:12:34:56 lb # [ 11.666152] dhcpcd[599]: eth0: IAID 00:12:34:56 lb # [ 11.666835] dhcpcd[599]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 11.687423] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 11.700402] network-addresses-eth1-start[718]: adding address 192.168.1.2/24... done lb # [ 11.716326] network-addresses-eth1-start[718]: adding address 2001:db8:1::2/64... done client # [ 11.713706] systemd[1]: Finished Extra networking commands.. client # [ 11.717549] systemd[1]: Reached target Network. client # [ 11.720869] systemd[1]: Starting Permit User Sessions... lb # [ 11.748177] systemd[1]: Finished Address configuration of eth1. client # [ 11.774897] systemd[1]: Finished Permit User Sessions. client # [ 11.782246] systemd[1]: Started Getty on tty1. client # [ 11.782994] systemd[1]: Reached target Login Prompts. lb # [ 11.809141] mousedev: PS/2 mouse device common for all mice lb # [ 11.799591] dhcpcd[599]: eth0: soliciting a DHCP lease lb # [ 11.804577] dhcpcd[599]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 11.812195] dhcpcd[599]: eth0: probing address 10.0.2.15/24 lb # [ 11.854538] systemd-logind[470]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker1 # [ 11.929619] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client: (finished: waiting for unit network-addresses-eth1.service, in 12.40 seconds) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } worker2 # [ 11.992600] dhcpcd[624]: eth0: soliciting a DHCP lease worker2 # [ 12.001676] dhcpcd[624]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 12.008312] dhcpcd[624]: eth0: probing address 10.0.2.15/24 client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.08 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) lb: (finished: waiting for unit network-addresses-eth1.service, in 0.07 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } client # [ 12.144795] dhcpcd[598]: eth0: soliciting a DHCP lease client # [ 12.152849] dhcpcd[598]: eth0: offered 10.0.2.15 from 10.0.2.2 lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.05 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) client # [ 12.160300] dhcpcd[598]: eth0: probing address 10.0.2.15/24 client # [ 12.217054] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 12.220970] systemd[1]: Stopped Virtual Console Setup. client # [ 12.221744] systemd[1]: Stopping Virtual Console Setup... worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.07 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } client # [ 12.225616] systemd[1]: Starting Virtual Console Setup... client # [ 12.249434] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 12.251133] systemd[1]: Stopped Virtual Console Setup. client # [ 12.259306] systemd[1]: Starting Virtual Console Setup... worker1 # [ 12.273235] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1 # [ 12.285028] systemd[1]: Starting Virtual Console Setup... worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.06 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) client # [ 12.295221] systemd-logind[484]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 12.304508] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 12.305646] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 12.306396] systemd[1]: Starting Virtual Console Setup... client # [ 12.312982] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } worker2 # [ 12.352931] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 12.354039] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 12.354772] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 12.368332] systemd[1]: Starting Virtual Console Setup... client # [ 12.388726] systemd-vconsole-setup[699]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 12.391992] systemd[1]: Finished Virtual Console Setup. worker2 # [ 12.381029] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 12.389455] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 12.390263] systemd[1]: Starting Virtual Console Setup... worker1 # [ 12.411416] systemd-logind[515]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 12.405492] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.08 seconds) lb: waiting for unit niks3.service worker1 # [ 12.466701] systemd-vconsole-setup[704]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 12.470691] systemd[1]: Finished Virtual Console Setup. worker1 # [ 12.495238] dhcpcd[639]: eth0: soliciting a DHCP lease worker1 # [ 12.500560] dhcpcd[639]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 12.485572] systemd-logind[493]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 12.508368] dhcpcd[639]: eth0: probing address 10.0.2.15/24 worker2 # [ 12.545973] systemd-vconsole-setup[700]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 12.550166] systemd[1]: Finished Virtual Console Setup. client # [ 12.820234] dhcpcd[598]: eth0: soliciting an IPv6 router client # [ 12.823775] dhcpcd[598]: eth0: Router Advertisement from fe80::2 client # [ 12.826533] dhcpcd[598]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 12.829354] dhcpcd[598]: eth0: adding route to fec0::/64 client # [ 12.831546] dhcpcd[598]: eth0: adding default route via fe80::2 lb # [ 13.119090] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 13.121980] systemd[1]: Stopped Virtual Console Setup. lb # [ 13.122776] systemd[1]: Stopping Virtual Console Setup... lb # [ 13.128190] systemd[1]: Starting Virtual Console Setup... lb # [ 13.161501] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 13.177223] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 13.180365] systemd[1]: Stopped Virtual Console Setup. lb # [ 13.187950] systemd[1]: Starting Virtual Console Setup... worker2 # [ 13.199351] dhcpcd[624]: eth0: soliciting an IPv6 router lb # [ 13.226329] systemd-logind[470]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2 # [ 13.202845] dhcpcd[624]: eth0: Router Advertisement from fe80::2 worker2 # [ 13.205695] dhcpcd[624]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 13.208477] dhcpcd[624]: eth0: adding route to fec0::/64 worker2 # [ 13.210675] dhcpcd[624]: eth0: adding default route via fe80::2 lb # [ 13.394516] dhcpcd[599]: eth0: soliciting an IPv6 router lb # [ 13.395362] dhcpcd[599]: eth0: Router Advertisement from fe80::2 lb # [ 13.396396] dhcpcd[599]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 13.397277] dhcpcd[599]: eth0: adding route to fec0::/64 lb # [ 13.397986] dhcpcd[599]: eth0: adding default route via fe80::2 lb # [ 13.445496] systemd-vconsole-setup[760]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 13.449680] systemd[1]: Finished Virtual Console Setup. lb # [ 13.633571] postgresql-pre-start[691]: selecting default time zone ... UTC worker1 # [ 13.629824] dhcpcd[639]: eth0: soliciting an IPv6 router lb # [ 13.638327] postgresql-pre-start[691]: creating configuration files ... ok worker1 # [ 13.633184] dhcpcd[639]: eth0: Router Advertisement from fe80::2 worker1 # [ 13.635704] dhcpcd[639]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 13.638794] dhcpcd[639]: eth0: adding route to fec0::/64 worker1 # [ 13.641187] dhcpcd[639]: eth0: adding default route via fe80::2 lb # [ 13.891045] postgresql-pre-start[691]: running bootstrap script ... ok lb # [ 14.378994] postgresql-pre-start[691]: performing post-bootstrap initialization ... ok lb # [ 14.535401] postgresql-pre-start[691]: syncing data to disk ... ok lb # [ 14.537532] postgresql-pre-start[691]: initdb: warning: enabling "trust" authentication for local connections lb # [ 14.538820] postgresql-pre-start[691]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 14.541027] postgresql-pre-start[691]: Success. You can now start the database server using: lb # [ 14.542204] postgresql-pre-start[691]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 14.632440] postgres[784]: [784] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 14.635241] postgres[784]: [784] LOG: listening on IPv6 address "::1", port 5432 lb # [ 14.636671] postgres[784]: [784] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 14.638393] postgres[784]: [784] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 14.649337] postgres[793]: [793] LOG: database system was shut down at 2026-09-21 14:43:49 GMT lb # [ 14.656471] postgres[784]: [784] LOG: database system is ready to accept connections lb # [ 14.661667] systemd[1]: Started PostgreSQL Server. lb # [ 14.668552] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 14.931619] postgresql-setup-start[804]: CREATE DATABASE lb # [ 14.969288] postgresql-setup-start[819]: CREATE ROLE lb # [ 14.985493] postgresql-setup-start[821]: ALTER DATABASE lb # [ 14.991023] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 14.993058] systemd[1]: Reached target PostgreSQL. lb # [ 16.301084] dhcpcd[599]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 16.304901] dhcpcd[599]: eth0: adding route to 10.0.2.0/24 lb # [ 16.307269] dhcpcd[599]: eth0: adding default route via 10.0.2.2 lb # [ 16.446024] systemd[1]: Started DHCP Client. lb # [ 16.449239] systemd[1]: Reached target Network is Online. lb # [ 16.454660] systemd[1]: Started Envoy reverse proxy. lb # [ 16.461477] systemd[1]: Started mock-oidc.service. lb # [ 16.529061] mock-oidc-server[889]: Mock OIDC Server running lb # [ 16.531706] mock-oidc-server[889]: OIDC Address: 192.168.1.2:8080 lb # [ 16.533523] mock-oidc-server[889]: Issue Address: 0.0.0.0:8081 lb # [ 16.534770] mock-oidc-server[889]: Issuer: http://192.168.1.2:8080/oidc lb # [ 16.536062] mock-oidc-server[889]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 16.538464] mock-oidc-server[889]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 16.541042] mock-oidc-server[889]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 16.725300] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120) lb # [ 16.727141] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 16.728821] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 16.731549] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 16.734338] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 16.736980] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 16.744186] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 16.746602] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 16.750065] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 16.753081] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 16.756404] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 16.759703] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 16.764057] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 16.765861] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 16.770349] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 16.772244] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 16.778044] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 16.787541] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 16.789529] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 16.793900] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 16.799414] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 16.803524] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 16.807240] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 16.810466] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 16.813502] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 16.827795] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 16.829882] envoy[888]: [2026-09-21 14:43:52.326][888][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 16.835598] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 16.838423] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 16.840167] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher lb # [ 16.842542] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 16.845104] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 16.847322] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 16.850127] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 16.857210] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 16.859853] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 16.862381] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 16.870099] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation lb # [ 16.872840] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 16.874674] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 16.877640] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 16.881913] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 16.883742] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder lb # [ 16.886492] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd lb # [ 16.891893] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router worker1 # [ 16.887805] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 16.895002] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 16.898755] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 16.900636] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 16.902732] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 16.904709] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 16.906223] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 16.907923] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 16.909564] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 16.912718] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 16.915001] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 16.916812] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 16.919531] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 16.921944] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 16.925702] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 16.927521] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 16.930561] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 16.935483] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 16.937875] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 16.939908] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 16.941803] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 16.944390] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 16.946861] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 16.949007] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 16.950789] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default lb # [ 16.952682] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 16.954580] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 16.962799] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 16.965163] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 16.968085] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 16.969767] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 16.971600] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 16.974849] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 16.976652] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 16.983566] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 16.985424] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 16.988062] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr lb # [ 16.989853] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 16.991844] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy lb # [ 17.007200] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 17.010171] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 17.012224] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 17.014057] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche lb # [ 17.015917] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 17.018026] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 17.019881] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 17.022985] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 17.024827] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 17.032000] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 17.043708] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 17.047148] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher worker1 # [ 17.065373] nix-grpc-daemon[713]: ts=2026-09-21T14:43:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 17.074064] nix-grpc-daemon[713]: ts=2026-09-21T14:43:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc lb # [ 17.050051] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router worker1 # [ 17.075963] nix-grpc-daemon[713]: ts=2026-09-21T14:43:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 17.089353] nix-grpc-daemon[713]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 17.088314] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 17.095859] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 worker1 # [ 17.092353] nix-grpc-daemon[713]: E0000 00:00:1790001832.697507 713 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 17.097912] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 17.100657] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter worker1 # [ 17.097500] nix-grpc-daemon[713]: E0000 00:00:1790001832.705812 713 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 17.102568] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 17.106221] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 17.109813] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 17.117670] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator worker1 # [ 17.119040] nix-grpc-daemon[713]: ts=2026-09-21T14:43:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket lb # [ 17.123740] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction worker1 # [ 17.123866] systemd[1]: Started Nix worker-protocol over gRPC. lb # [ 17.130358] envoy[888]: [2026-09-21 14:43:52.327][888][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection worker1 # [ 17.132150] nix-grpc-daemon[713]: E0000 00:00:1790001832.733965 754 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 17.138004] envoy[888]: [2026-09-21 14:43:52.352][888][info][main] [source/server/server.cc:531] HTTP header map info: lb # [ 17.139453] envoy[888]: [2026-09-21 14:43:52.352][888][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 17.157485] envoy[888]: [2026-09-21 14:43:52.352][888][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: worker1 # [ 17.156066] nix-grpc-daemon[713]: E0000 00:00:1790001832.753427 754 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 17.158477] nix-grpc-daemon[713]: E0000 00:00:1790001832.753949 754 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 17.159034] envoy[888]: [2026-09-21 14:43:52.352][888][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 17.168669] envoy[888]: [2026-09-21 14:43:52.352][888][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 17.170465] envoy[888]: [2026-09-21 14:43:52.401][888][info][main] [source/server/server.cc:968] runtime: {} lb # [ 17.171967] envoy[888]: [2026-09-21 14:43:52.405][888][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. worker1 # [ 17.167476] nix-grpc-daemon[713]: E0000 00:00:1790001832.775128 754 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 17.174052] envoy[888]: [2026-09-21 14:43:52.405][888][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 worker1 # [ 17.173266] dhcpcd[639]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 17.177796] envoy[888]: [2026-09-21 14:43:52.406][888][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 lb # [ 17.179432] envoy[888]: [2026-09-21 14:43:52.409][888][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 17.182138] envoy[888]: [2026-09-21 14:43:52.409][888][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 17.184399] envoy[888]: [2026-09-21 14:43:52.409][888][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) worker1 # [ 17.181436] dhcpcd[639]: eth0: adding route to 10.0.2.0/24 lb # [ 17.185958] envoy[888]: [2026-09-21 14:43:52.483][888][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) worker1 # [ 17.182252] nix-grpc-daemon[713]: ts=2026-09-21T14:43:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 17.183530] dhcpcd[639]: eth0: adding default route via 10.0.2.2 lb # [ 17.187458] envoy[888]: [2026-09-21 14:43:52.500][888][info][config] [source/server/configuration_impl.cc:164] loading stats configuration lb # [ 17.189717] envoy[888]: [2026-09-21 14:43:52.501][888][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 17.193417] envoy[888]: [2026-09-21 14:43:52.501][888][info][main] [source/server/server.cc:1077] starting main dispatch loop worker1 # [ 17.189625] nix-grpc-daemon[713]: ts=2026-09-21T14:43:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 17.194847] envoy[888]: [2026-09-21 14:43:52.732][888][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization lb # [ 17.196591] envoy[888]: [2026-09-21 14:43:52.732][888][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized lb # [ 17.198356] envoy[888]: [2026-09-21 14:43:52.732][888][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager lb # [ 17.200063] envoy[888]: [2026-09-21 14:43:52.758][888][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers worker1 # [ 17.264596] systemd[1]: Started DHCP Client. worker1 # [ 17.266419] systemd[1]: Reached target Multi-User System. worker1 # [ 17.267686] systemd[1]: Startup finished in 957ms (kernel) + 4.823s (initrd) + 11.486s (userspace) = 17.267s. worker1 # [ 17.390468] nix-grpc-daemon[713]: ts=2026-09-21T14:43:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 17.393058] nix-grpc-daemon[713]: ts=2026-09-21T14:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" client # [ 17.391182] dhcpcd[598]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 17.393868] dhcpcd[598]: eth0: adding route to 10.0.2.0/24 client # [ 17.395771] dhcpcd[598]: eth0: adding default route via 10.0.2.2 client # [ 17.534822] systemd[1]: Started DHCP Client. client # [ 17.537956] systemd[1]: Reached target Multi-User System. client # [ 17.539559] systemd[1]: Startup finished in 955ms (kernel) + 4.138s (initrd) + 12.445s (userspace) = 17.539s. worker2 # [ 17.533146] dhcpcd[624]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 17.536485] dhcpcd[624]: eth0: adding route to 10.0.2.0/24 worker2 # [ 17.538822] dhcpcd[624]: eth0: adding default route via 10.0.2.2 worker1 # [ 17.593872] nix-grpc-daemon[713]: ts=2026-09-21T14:43:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 17.597741] nix-grpc-daemon[713]: ts=2026-09-21T14:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 17.717570] systemd[1]: Started DHCP Client. worker2 # [ 17.720230] systemd[1]: Reached target Multi-User System. worker2 # [ 17.721044] systemd[1]: Reached target Network is Online. worker2 # [ 17.723568] systemd[1]: Starting scheduler-token.service... worker2 # [ 17.775512] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 17.779729] systemd[1]: Finished scheduler-token.service. worker1 # [ 17.798755] nix-grpc-daemon[713]: ts=2026-09-21T14:43:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 17.780772] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 53ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 17.783715] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 17.804241] nix-grpc-daemon[713]: ts=2026-09-21T14:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 17.895552] nix-grpc-daemon[770]: ts=2026-09-21T14:43:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 17.903565] nix-grpc-daemon[770]: ts=2026-09-21T14:43:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 17.905418] nix-grpc-daemon[770]: ts=2026-09-21T14:43:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 17.915636] nix-grpc-daemon[770]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 17.918613] nix-grpc-daemon[770]: E0000 00:00:1790001833.518409 770 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 17.923093] nix-grpc-daemon[770]: E0000 00:00:1790001833.525869 770 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 17.937813] nix-grpc-daemon[770]: ts=2026-09-21T14:43:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 17.941933] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 17.948435] systemd[1]: Startup finished in 964ms (kernel) + 4.118s (initrd) + 12.843s (userspace) = 17.926s. worker2 # [ 17.964122] nix-grpc-daemon[770]: E0000 00:00:1790001833.546157 811 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 17.967688] nix-grpc-daemon[770]: E0000 00:00:1790001833.570503 811 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 18.005081] nix-grpc-daemon[713]: ts=2026-09-21T14:43:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 17.986985] nix-grpc-daemon[770]: E0000 00:00:1790001833.573255 811 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 17.989937] nix-grpc-daemon[770]: E0000 00:00:1790001833.589014 811 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 18.010527] nix-grpc-daemon[713]: ts=2026-09-21T14:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 18.000273] nix-grpc-daemon[770]: ts=2026-09-21T14:43:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.001929] nix-grpc-daemon[770]: ts=2026-09-21T14:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 18.213098] nix-grpc-daemon[713]: ts=2026-09-21T14:43:53Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 18.217330] nix-grpc-daemon[713]: ts=2026-09-21T14:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 18.203731] nix-grpc-daemon[770]: ts=2026-09-21T14:43:53Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.208993] nix-grpc-daemon[770]: ts=2026-09-21T14:43:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 18.413538] nix-grpc-daemon[713]: ts=2026-09-21T14:43:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 18.417145] nix-grpc-daemon[713]: ts=2026-09-21T14:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 18.410302] nix-grpc-daemon[770]: ts=2026-09-21T14:43:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.415558] nix-grpc-daemon[770]: ts=2026-09-21T14:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 18.618019] nix-grpc-daemon[713]: ts=2026-09-21T14:43:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 18.622019] nix-grpc-daemon[713]: ts=2026-09-21T14:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 18.616762] nix-grpc-daemon[770]: ts=2026-09-21T14:43:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.621642] nix-grpc-daemon[770]: ts=2026-09-21T14:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 18.822723] nix-grpc-daemon[713]: ts=2026-09-21T14:43:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 18.825878] nix-grpc-daemon[713]: ts=2026-09-21T14:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 18.822874] nix-grpc-daemon[770]: ts=2026-09-21T14:43:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.828093] nix-grpc-daemon[770]: ts=2026-09-21T14:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.026874] nix-grpc-daemon[713]: ts=2026-09-21T14:43:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.032269] nix-grpc-daemon[713]: ts=2026-09-21T14:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.030786] nix-grpc-daemon[770]: ts=2026-09-21T14:43:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.035417] nix-grpc-daemon[770]: ts=2026-09-21T14:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.234902] nix-grpc-daemon[713]: ts=2026-09-21T14:43:54Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.239065] nix-grpc-daemon[713]: ts=2026-09-21T14:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.236506] nix-grpc-daemon[770]: ts=2026-09-21T14:43:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.241475] nix-grpc-daemon[770]: ts=2026-09-21T14:43:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.435492] nix-grpc-daemon[713]: ts=2026-09-21T14:43:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.441005] nix-grpc-daemon[713]: ts=2026-09-21T14:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.442688] nix-grpc-daemon[770]: ts=2026-09-21T14:43:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.447825] nix-grpc-daemon[770]: ts=2026-09-21T14:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.641980] nix-grpc-daemon[713]: ts=2026-09-21T14:43:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.647187] nix-grpc-daemon[713]: ts=2026-09-21T14:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.649131] nix-grpc-daemon[770]: ts=2026-09-21T14:43:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.654234] nix-grpc-daemon[770]: ts=2026-09-21T14:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.848108] nix-grpc-daemon[713]: ts=2026-09-21T14:43:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.853323] nix-grpc-daemon[713]: ts=2026-09-21T14:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.855594] nix-grpc-daemon[770]: ts=2026-09-21T14:43:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.860789] nix-grpc-daemon[770]: ts=2026-09-21T14:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.054430] nix-grpc-daemon[713]: ts=2026-09-21T14:43:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.059812] nix-grpc-daemon[713]: ts=2026-09-21T14:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.063510] nix-grpc-daemon[770]: ts=2026-09-21T14:43:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.067682] nix-grpc-daemon[770]: ts=2026-09-21T14:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.262410] nix-grpc-daemon[713]: ts=2026-09-21T14:43:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.266531] nix-grpc-daemon[713]: ts=2026-09-21T14:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.264391] nix-grpc-daemon[770]: ts=2026-09-21T14:43:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.269777] nix-grpc-daemon[770]: ts=2026-09-21T14:43:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.463264] nix-grpc-daemon[713]: ts=2026-09-21T14:43:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.468617] nix-grpc-daemon[713]: ts=2026-09-21T14:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.470910] nix-grpc-daemon[770]: ts=2026-09-21T14:43:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.476118] nix-grpc-daemon[770]: ts=2026-09-21T14:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.669440] nix-grpc-daemon[713]: ts=2026-09-21T14:43:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.674749] nix-grpc-daemon[713]: ts=2026-09-21T14:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.677241] nix-grpc-daemon[770]: ts=2026-09-21T14:43:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.682269] nix-grpc-daemon[770]: ts=2026-09-21T14:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.875672] nix-grpc-daemon[713]: ts=2026-09-21T14:43:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.880888] nix-grpc-daemon[713]: ts=2026-09-21T14:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.883329] nix-grpc-daemon[770]: ts=2026-09-21T14:43:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.888471] nix-grpc-daemon[770]: ts=2026-09-21T14:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.081785] nix-grpc-daemon[713]: ts=2026-09-21T14:43:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.087101] nix-grpc-daemon[713]: ts=2026-09-21T14:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.091388] nix-grpc-daemon[770]: ts=2026-09-21T14:43:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.095580] nix-grpc-daemon[770]: ts=2026-09-21T14:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.290488] nix-grpc-daemon[713]: ts=2026-09-21T14:43:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.295030] nix-grpc-daemon[713]: ts=2026-09-21T14:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.292107] nix-grpc-daemon[770]: ts=2026-09-21T14:43:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.297033] nix-grpc-daemon[770]: ts=2026-09-21T14:43:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.491036] nix-grpc-daemon[713]: ts=2026-09-21T14:43:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.496220] nix-grpc-daemon[713]: ts=2026-09-21T14:43:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.498137] nix-grpc-daemon[770]: ts=2026-09-21T14:43:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.503204] nix-grpc-daemon[770]: ts=2026-09-21T14:43:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.697143] nix-grpc-daemon[713]: ts=2026-09-21T14:43:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.702483] nix-grpc-daemon[713]: ts=2026-09-21T14:43:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.704410] nix-grpc-daemon[770]: ts=2026-09-21T14:43:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.709385] nix-grpc-daemon[770]: ts=2026-09-21T14:43:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.903298] nix-grpc-daemon[713]: ts=2026-09-21T14:43:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.908336] nix-grpc-daemon[713]: ts=2026-09-21T14:43:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.910847] nix-grpc-daemon[770]: ts=2026-09-21T14:43:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.915780] nix-grpc-daemon[770]: ts=2026-09-21T14:43:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.109298] nix-grpc-daemon[713]: ts=2026-09-21T14:43:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.114357] nix-grpc-daemon[713]: ts=2026-09-21T14:43:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.118845] nix-grpc-daemon[770]: ts=2026-09-21T14:43:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.123066] nix-grpc-daemon[770]: ts=2026-09-21T14:43:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.316617] nix-grpc-daemon[713]: ts=2026-09-21T14:43:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.321346] nix-grpc-daemon[713]: ts=2026-09-21T14:43:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.319671] nix-grpc-daemon[770]: ts=2026-09-21T14:43:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.324896] nix-grpc-daemon[770]: ts=2026-09-21T14:43:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.522189] nix-grpc-daemon[713]: ts=2026-09-21T14:43:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.527448] nix-grpc-daemon[713]: ts=2026-09-21T14:43:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.526318] nix-grpc-daemon[770]: ts=2026-09-21T14:43:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.531689] nix-grpc-daemon[770]: ts=2026-09-21T14:43:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.728465] nix-grpc-daemon[713]: ts=2026-09-21T14:43:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.733697] nix-grpc-daemon[713]: ts=2026-09-21T14:43:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.733023] nix-grpc-daemon[770]: ts=2026-09-21T14:43:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.738226] nix-grpc-daemon[770]: ts=2026-09-21T14:43:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.934713] nix-grpc-daemon[713]: ts=2026-09-21T14:43:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.939572] nix-grpc-daemon[713]: ts=2026-09-21T14:43:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.939527] nix-grpc-daemon[770]: ts=2026-09-21T14:43:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.944908] nix-grpc-daemon[770]: ts=2026-09-21T14:43:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.140285] nix-grpc-daemon[713]: ts=2026-09-21T14:43:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.145299] nix-grpc-daemon[713]: ts=2026-09-21T14:43:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.148255] nix-grpc-daemon[770]: ts=2026-09-21T14:43:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.152354] nix-grpc-daemon[770]: ts=2026-09-21T14:43:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.348103] nix-grpc-daemon[713]: ts=2026-09-21T14:43:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.352158] nix-grpc-daemon[713]: ts=2026-09-21T14:43:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.348772] nix-grpc-daemon[770]: ts=2026-09-21T14:43:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.353719] nix-grpc-daemon[770]: ts=2026-09-21T14:43:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.548729] nix-grpc-daemon[713]: ts=2026-09-21T14:43:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.553853] nix-grpc-daemon[713]: ts=2026-09-21T14:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.554852] nix-grpc-daemon[770]: ts=2026-09-21T14:43:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.559861] nix-grpc-daemon[770]: ts=2026-09-21T14:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.754846] nix-grpc-daemon[713]: ts=2026-09-21T14:43:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.760075] nix-grpc-daemon[713]: ts=2026-09-21T14:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.760986] nix-grpc-daemon[770]: ts=2026-09-21T14:43:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.766028] nix-grpc-daemon[770]: ts=2026-09-21T14:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 23.890037] rustfs-bucket-start[968]: mb s3://farm lb # [ 23.897129] systemd[1]: Finished rustfs-bucket.service. lb # [ 23.905135] systemd[1]: Starting niks3 server... worker1 # [ 23.961056] nix-grpc-daemon[713]: ts=2026-09-21T14:43:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.966281] nix-grpc-daemon[713]: ts=2026-09-21T14:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.966982] nix-grpc-daemon[770]: ts=2026-09-21T14:43:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.972166] nix-grpc-daemon[770]: ts=2026-09-21T14:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 24.029171] postgres[985]: [985] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 24.030429] postgres[985]: [985] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC lb # [ 24.056352] niks3-server[979]: 2026/09/21 14:43:59 OK 20241026095416_initial_model.sql (15.26ms) lb # [ 24.060128] niks3-server[979]: 2026/09/21 14:43:59 OK 20251210153512_drop_unused_gin_index.sql (4.11ms) lb # [ 24.065508] niks3-server[979]: 2026/09/21 14:43:59 OK 20251218171726_add_pins.sql (5.33ms) lb # [ 24.070039] niks3-server[979]: 2026/09/21 14:43:59 OK 20260628120000_add_object_size_and_stats.sql (4.49ms) lb # [ 24.074802] niks3-server[979]: 2026/09/21 14:43:59 OK 20260905000000_add_claims.sql (4.73ms) lb # [ 24.078436] niks3-server[979]: 2026/09/21 14:43:59 OK 20260920000000_drop_claims.sql (3.53ms) lb # [ 24.079632] niks3-server[979]: 2026/09/21 14:43:59 goose: successfully migrated database to version: 20260920000000 lb # [ 24.083689] niks3-server[979]: 2026/09/21 14:43:59 OK 1_commit_pending_closure.sql (5.22ms) lb # [ 24.086445] niks3-server[979]: 2026/09/21 14:43:59 OK 2_object_stats_trigger.sql (2.62ms) lb # [ 24.088392] niks3-server[979]: 2026/09/21 14:43:59 goose: up to current file version: 2 lb # [ 24.090231] niks3-server[979]: 2026/09/21 14:43:59 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 24.111847] niks3-server[979]: 2026/09/21 14:43:59 INFO Read proxy enabled — serving cache objects from S3 lb # [ 24.115921] niks3-server[979]: 2026/09/21 14:43:59 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 24.120498] niks3-server[979]: 2026/09/21 14:43:59 INFO systemd watchdog enabled interval=15s lb # [ 24.123046] niks3-server[979]: 2026/09/21 14:43:59 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 24.125012] systemd[1]: Started niks3 server. lb # [ 24.125996] systemd[1]: Reached target Multi-User System. lb # [ 24.126979] systemd[1]: Startup finished in 964ms (kernel) + 4.145s (initrd) + 19.013s (userspace) = 24.122s. worker1 # [ 24.167317] nix-grpc-daemon[713]: ts=2026-09-21T14:43:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.172627] nix-grpc-daemon[713]: ts=2026-09-21T14:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.174897] nix-grpc-daemon[770]: ts=2026-09-21T14:43:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.179039] nix-grpc-daemon[770]: ts=2026-09-21T14:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.375138] nix-grpc-daemon[713]: ts=2026-09-21T14:43:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.379001] nix-grpc-daemon[713]: ts=2026-09-21T14:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.375409] nix-grpc-daemon[770]: ts=2026-09-21T14:43:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.380754] nix-grpc-daemon[770]: ts=2026-09-21T14:43:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.576223] nix-grpc-daemon[713]: ts=2026-09-21T14:44:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.581457] nix-grpc-daemon[713]: ts=2026-09-21T14:44:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.581881] nix-grpc-daemon[770]: ts=2026-09-21T14:44:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.586905] nix-grpc-daemon[770]: ts=2026-09-21T14:44:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.782665] nix-grpc-daemon[713]: ts=2026-09-21T14:44:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.787914] nix-grpc-daemon[713]: ts=2026-09-21T14:44:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.788224] nix-grpc-daemon[770]: ts=2026-09-21T14:44:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.793353] nix-grpc-daemon[770]: ts=2026-09-21T14:44:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: waiting for unit niks3.service, in 12.51 seconds) lb: waiting for TCP port 5751 on localhost worker1 # [ 24.988914] nix-grpc-daemon[713]: ts=2026-09-21T14:44:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.994084] nix-grpc-daemon[713]: ts=2026-09-21T14:44:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.994299] nix-grpc-daemon[770]: ts=2026-09-21T14:44:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.999370] nix-grpc-daemon[770]: ts=2026-09-21T14:44:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.07 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.05 seconds) lb: waiting for TCP port 50051 on localhost lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! lb # [ 25.115994] envoy[888]: conn peer=[::1]:60432 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=4 lb: (finished: waiting for TCP port 50051 on localhost, in 0.03 seconds) worker1 # [ 25.194712] nix-grpc-daemon[713]: ts=2026-09-21T14:44:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.198254] nix-grpc-daemon[713]: ts=2026-09-21T14:44:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.203877] nix-grpc-daemon[770]: ts=2026-09-21T14:44:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.212486] nix-grpc-daemon[770]: ts=2026-09-21T14:44:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.263395] systemd[1]: Starting scheduler-token.service... worker2 # [ 25.337945] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 25.342563] systemd[1]: Finished scheduler-token.service. worker2 # [ 25.346523] systemd[1]: scheduler-token.service: Consumed 44ms CPU time over 79ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 25.401098] nix-grpc-daemon[713]: ts=2026-09-21T14:44:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.405049] nix-grpc-daemon[713]: ts=2026-09-21T14:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.402707] nix-grpc-daemon[770]: ts=2026-09-21T14:44:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.407599] nix-grpc-daemon[770]: ts=2026-09-21T14:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.08 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1 # [ 25.601602] nix-grpc-daemon[713]: ts=2026-09-21T14:44:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.606759] nix-grpc-daemon[713]: ts=2026-09-21T14:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.608886] nix-grpc-daemon[770]: ts=2026-09-21T14:44:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.613881] nix-grpc-daemon[770]: ts=2026-09-21T14:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.807717] nix-grpc-daemon[713]: ts=2026-09-21T14:44:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.813033] nix-grpc-daemon[713]: ts=2026-09-21T14:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.814920] nix-grpc-daemon[770]: ts=2026-09-21T14:44:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.819921] nix-grpc-daemon[770]: ts=2026-09-21T14:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.013863] nix-grpc-daemon[713]: ts=2026-09-21T14:44:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.019148] nix-grpc-daemon[713]: ts=2026-09-21T14:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.021201] nix-grpc-daemon[770]: ts=2026-09-21T14:44:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.026246] nix-grpc-daemon[770]: ts=2026-09-21T14:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.219906] nix-grpc-daemon[713]: ts=2026-09-21T14:44:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.225041] nix-grpc-daemon[713]: ts=2026-09-21T14:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.229124] nix-grpc-daemon[770]: ts=2026-09-21T14:44:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.233206] nix-grpc-daemon[770]: ts=2026-09-21T14:44:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.427619] nix-grpc-daemon[713]: ts=2026-09-21T14:44:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.431795] nix-grpc-daemon[713]: ts=2026-09-21T14:44:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.429894] nix-grpc-daemon[770]: ts=2026-09-21T14:44:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.434934] nix-grpc-daemon[770]: ts=2026-09-21T14:44:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 26.628813] nix-grpc-daemon[713]: ts=2026-09-21T14:44:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.634235] nix-grpc-daemon[713]: ts=2026-09-21T14:44:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.636056] nix-grpc-daemon[770]: ts=2026-09-21T14:44:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.641065] nix-grpc-daemon[770]: ts=2026-09-21T14:44:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.834930] nix-grpc-daemon[713]: ts=2026-09-21T14:44:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.840100] nix-grpc-daemon[713]: ts=2026-09-21T14:44:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.842032] nix-grpc-daemon[770]: ts=2026-09-21T14:44:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.847183] nix-grpc-daemon[770]: ts=2026-09-21T14:44:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.041148] nix-grpc-daemon[713]: ts=2026-09-21T14:44:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.046177] nix-grpc-daemon[713]: ts=2026-09-21T14:44:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.047902] nix-grpc-daemon[770]: ts=2026-09-21T14:44:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.052912] nix-grpc-daemon[770]: ts=2026-09-21T14:44:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.246871] nix-grpc-daemon[713]: ts=2026-09-21T14:44:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.251980] nix-grpc-daemon[713]: ts=2026-09-21T14:44:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.255598] nix-grpc-daemon[770]: ts=2026-09-21T14:44:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.259745] nix-grpc-daemon[770]: ts=2026-09-21T14:44:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.454749] nix-grpc-daemon[713]: ts=2026-09-21T14:44:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.458911] nix-grpc-daemon[713]: ts=2026-09-21T14:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.456157] nix-grpc-daemon[770]: ts=2026-09-21T14:44:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.461169] nix-grpc-daemon[770]: ts=2026-09-21T14:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 27.655272] nix-grpc-daemon[713]: ts=2026-09-21T14:44:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.666731] nix-grpc-daemon[713]: ts=2026-09-21T14:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.662255] nix-grpc-daemon[770]: ts=2026-09-21T14:44:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.671502] nix-grpc-daemon[770]: ts=2026-09-21T14:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.864563] nix-grpc-daemon[713]: ts=2026-09-21T14:44:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.869668] nix-grpc-daemon[713]: ts=2026-09-21T14:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.869327] nix-grpc-daemon[770]: ts=2026-09-21T14:44:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.874613] nix-grpc-daemon[770]: ts=2026-09-21T14:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.070479] nix-grpc-daemon[713]: ts=2026-09-21T14:44:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.075750] nix-grpc-daemon[713]: ts=2026-09-21T14:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.075701] nix-grpc-daemon[770]: ts=2026-09-21T14:44:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.080944] nix-grpc-daemon[770]: ts=2026-09-21T14:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.276664] nix-grpc-daemon[713]: ts=2026-09-21T14:44:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.281939] nix-grpc-daemon[713]: ts=2026-09-21T14:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.283663] nix-grpc-daemon[770]: ts=2026-09-21T14:44:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.287861] nix-grpc-daemon[770]: ts=2026-09-21T14:44:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.485006] nix-grpc-daemon[713]: ts=2026-09-21T14:44:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.489218] nix-grpc-daemon[713]: ts=2026-09-21T14:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.484557] nix-grpc-daemon[770]: ts=2026-09-21T14:44:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.489542] nix-grpc-daemon[770]: ts=2026-09-21T14:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 28.685810] nix-grpc-daemon[713]: ts=2026-09-21T14:44:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.691232] nix-grpc-daemon[713]: ts=2026-09-21T14:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 28.690692] nix-grpc-daemon[770]: ts=2026-09-21T14:44:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.695748] nix-grpc-daemon[770]: ts=2026-09-21T14:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.892130] nix-grpc-daemon[713]: ts=2026-09-21T14:44:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.897528] nix-grpc-daemon[713]: ts=2026-09-21T14:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.896986] nix-grpc-daemon[770]: ts=2026-09-21T14:44:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.902049] nix-grpc-daemon[770]: ts=2026-09-21T14:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.098554] nix-grpc-daemon[713]: ts=2026-09-21T14:44:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.104152] nix-grpc-daemon[713]: ts=2026-09-21T14:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.102913] nix-grpc-daemon[770]: ts=2026-09-21T14:44:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.108079] nix-grpc-daemon[770]: ts=2026-09-21T14:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.304714] nix-grpc-daemon[713]: ts=2026-09-21T14:44:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.309943] nix-grpc-daemon[713]: ts=2026-09-21T14:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.310904] nix-grpc-daemon[770]: ts=2026-09-21T14:44:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.315015] nix-grpc-daemon[770]: ts=2026-09-21T14:44:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.513063] nix-grpc-daemon[713]: ts=2026-09-21T14:44:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.517215] nix-grpc-daemon[713]: ts=2026-09-21T14:44:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.511465] nix-grpc-daemon[770]: ts=2026-09-21T14:44:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.516645] nix-grpc-daemon[770]: ts=2026-09-21T14:44:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 29.713773] nix-grpc-daemon[713]: ts=2026-09-21T14:44:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.719194] nix-grpc-daemon[713]: ts=2026-09-21T14:44:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.717874] nix-grpc-daemon[770]: ts=2026-09-21T14:44:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.722979] nix-grpc-daemon[770]: ts=2026-09-21T14:44:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 29.920248] nix-grpc-daemon[713]: ts=2026-09-21T14:44:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.925363] nix-grpc-daemon[713]: ts=2026-09-21T14:44:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.924119] nix-grpc-daemon[770]: ts=2026-09-21T14:44:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.929157] nix-grpc-daemon[770]: ts=2026-09-21T14:44:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.126275] nix-grpc-daemon[713]: ts=2026-09-21T14:44:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.131341] nix-grpc-daemon[713]: ts=2026-09-21T14:44:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.130202] nix-grpc-daemon[770]: ts=2026-09-21T14:44:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.135243] nix-grpc-daemon[770]: ts=2026-09-21T14:44:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.332288] nix-grpc-daemon[713]: ts=2026-09-21T14:44:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.337442] nix-grpc-daemon[713]: ts=2026-09-21T14:44:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.338158] nix-grpc-daemon[770]: ts=2026-09-21T14:44:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.342272] nix-grpc-daemon[770]: ts=2026-09-21T14:44:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.540392] nix-grpc-daemon[713]: ts=2026-09-21T14:44:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.544467] nix-grpc-daemon[713]: ts=2026-09-21T14:44:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.538933] nix-grpc-daemon[770]: ts=2026-09-21T14:44:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.544079] nix-grpc-daemon[770]: ts=2026-09-21T14:44:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.741125] nix-grpc-daemon[713]: ts=2026-09-21T14:44:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.746391] nix-grpc-daemon[713]: ts=2026-09-21T14:44:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 30.745075] nix-grpc-daemon[770]: ts=2026-09-21T14:44:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.750057] nix-grpc-daemon[770]: ts=2026-09-21T14:44:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker1 # [ 30.948399] nix-grpc-daemon[713]: ts=2026-09-21T14:44:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.955701] nix-grpc-daemon[713]: ts=2026-09-21T14:44:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.951416] nix-grpc-daemon[770]: ts=2026-09-21T14:44:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.963253] nix-grpc-daemon[770]: ts=2026-09-21T14:44:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.153902] nix-grpc-daemon[713]: ts=2026-09-21T14:44:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.159193] nix-grpc-daemon[713]: ts=2026-09-21T14:44:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.159524] nix-grpc-daemon[770]: ts=2026-09-21T14:44:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.164719] nix-grpc-daemon[770]: ts=2026-09-21T14:44:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.359909] nix-grpc-daemon[713]: ts=2026-09-21T14:44:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.365051] nix-grpc-daemon[713]: ts=2026-09-21T14:44:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.367964] nix-grpc-daemon[770]: ts=2026-09-21T14:44:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.371968] nix-grpc-daemon[770]: ts=2026-09-21T14:44:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.567814] nix-grpc-daemon[713]: ts=2026-09-21T14:44:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.572119] nix-grpc-daemon[713]: ts=2026-09-21T14:44:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.568264] nix-grpc-daemon[770]: ts=2026-09-21T14:44:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.573185] nix-grpc-daemon[770]: ts=2026-09-21T14:44:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.768477] nix-grpc-daemon[713]: ts=2026-09-21T14:44:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.773502] nix-grpc-daemon[713]: ts=2026-09-21T14:44:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.774382] nix-grpc-daemon[770]: ts=2026-09-21T14:44:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.779405] nix-grpc-daemon[770]: ts=2026-09-21T14:44:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 31.974425] nix-grpc-daemon[713]: ts=2026-09-21T14:44:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.979633] nix-grpc-daemon[713]: ts=2026-09-21T14:44:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.980291] nix-grpc-daemon[770]: ts=2026-09-21T14:44:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.985289] nix-grpc-daemon[770]: ts=2026-09-21T14:44:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.180438] nix-grpc-daemon[713]: ts=2026-09-21T14:44:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.185604] nix-grpc-daemon[713]: ts=2026-09-21T14:44:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.186339] nix-grpc-daemon[770]: ts=2026-09-21T14:44:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.191210] nix-grpc-daemon[770]: ts=2026-09-21T14:44:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.387348] nix-grpc-daemon[713]: ts=2026-09-21T14:44:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.393274] nix-grpc-daemon[713]: ts=2026-09-21T14:44:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.395506] nix-grpc-daemon[770]: ts=2026-09-21T14:44:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.399923] nix-grpc-daemon[770]: ts=2026-09-21T14:44:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.593077] nix-grpc-daemon[713]: ts=2026-09-21T14:44:08Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.597350] nix-grpc-daemon[713]: ts=2026-09-21T14:44:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.597111] nix-grpc-daemon[770]: ts=2026-09-21T14:44:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.602780] nix-grpc-daemon[770]: ts=2026-09-21T14:44:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.794158] nix-grpc-daemon[713]: ts=2026-09-21T14:44:08Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.799888] nix-grpc-daemon[713]: ts=2026-09-21T14:44:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.804428] nix-grpc-daemon[770]: ts=2026-09-21T14:44:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.809999] nix-grpc-daemon[770]: ts=2026-09-21T14:44:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 33.001301] nix-grpc-daemon[713]: ts=2026-09-21T14:44:08Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.007104] nix-grpc-daemon[713]: ts=2026-09-21T14:44:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.12 seconds) worker2 # [ 33.011660] nix-grpc-daemon[770]: ts=2026-09-21T14:44:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.017985] nix-grpc-daemon[770]: ts=2026-09-21T14:44:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.208130] nix-grpc-daemon[713]: ts=2026-09-21T14:44:08Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.213618] nix-grpc-daemon[713]: ts=2026-09-21T14:44:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.219524] nix-grpc-daemon[770]: ts=2026-09-21T14:44:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.226013] nix-grpc-daemon[770]: ts=2026-09-21T14:44:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.414705] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.420937] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.430809] nix-grpc-daemon[770]: ts=2026-09-21T14:44:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.435199] nix-grpc-daemon[770]: ts=2026-09-21T14:44:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.624119] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.630485] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.631635] nix-grpc-daemon[770]: ts=2026-09-21T14:44:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.637949] nix-grpc-daemon[770]: ts=2026-09-21T14:44:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.831450] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.837569] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.839175] nix-grpc-daemon[770]: ts=2026-09-21T14:44:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.845482] nix-grpc-daemon[770]: ts=2026-09-21T14:44:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 34.038614] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.044392] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.047067] nix-grpc-daemon[770]: ts=2026-09-21T14:44:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.052907] nix-grpc-daemon[770]: ts=2026-09-21T14:44:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.123223] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=scheduler_take_over lb # [ 34.129281] niks3-server[979]: 2026/09/21 14:44:09 INFO lead: acquired remote=192.168.1.3:49934 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.12 seconds) worker1 # [ 34.245736] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.272785] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:52918 worker2 # [ 34.254688] nix-grpc-daemon[770]: ts=2026-09-21T14:44:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.290473] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 34.311211] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 34.319751] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:52918 worker1 # [ 34.332277] nix-grpc-daemon[713]: ts=2026-09-21T14:44:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 10.72 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker2 # [ 36.494316] nix-grpc-daemon[770]: ts=2026-09-21T14:44:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 36.523595] systemd[1]: Started Nix Daemon. worker2 # [ 36.731572] nix-daemon[838]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 36.756175] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5) worker2 # [ 36.816207] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 176 ms (attempt 2/5) worker2 # [ 36.995929] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 31 ms (attempt 3/5) worker2 # [ 37.029876] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 653 ms (attempt 4/5) worker2 # [ 37.698577] nix-grpc-daemon[770]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 37.706731] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 37.777300] nix-grpc-daemon[713]: ts=2026-09-21T14:44:13Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 37.791245] systemd[1]: Started Nix Daemon. worker1 # [ 37.978456] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 38.001207] nix-grpc-daemon[713]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5) worker1 # [ 38.039808] nix-grpc-daemon[713]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 129 ms (attempt 2/5) worker1 # [ 38.172115] nix-grpc-daemon[713]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 251 ms (attempt 3/5) worker1 # [ 38.433337] nix-grpc-daemon[713]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 738 ms (attempt 4/5) worker1 # [ 39.175285] nix-grpc-daemon[713]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 39.180589] nix-grpc-daemon[713]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 39.195817] nix-grpc-daemon[713]: ts=2026-09-21T14:44:14Z level=debug event=path_info_miss path=6v5lkzqzv8y8aql79zri93a767rl7bwj-nixos-system-client-test error="error: path '/nix/store/6v5lkzqzv8y8aql79zri93a767rl7bwj-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 39.190115] nix-grpc-daemon[770]: ts=2026-09-21T14:44:14Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 39.203045] nix-grpc-daemon[770]: ts=2026-09-21T14:44:14Z level=debug event=path_info_miss path=6v5lkzqzv8y8aql79zri93a767rl7bwj-nixos-system-client-test error="error: path '/nix/store/6v5lkzqzv8y8aql79zri93a767rl7bwj-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds) client: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds) worker1 # [ 39.551857] nix-grpc-daemon[713]: ts=2026-09-21T14:44:15Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:52934 worker1 # [ 39.568956] nix-grpc-daemon[713]: ts=2026-09-21T14:44:15Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:52934 worker1 # [ 39.580965] nix-grpc-daemon[713]: ts=2026-09-21T14:44:15Z level=debug event=path_info_miss path=6v5lkzqzv8y8aql79zri93a767rl7bwj-nixos-system-client-test error="error: path '/nix/store/6v5lkzqzv8y8aql79zri93a767rl7bwj-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 39.576292] nix-grpc-daemon[770]: ts=2026-09-21T14:44:15Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 39.584159] nix-grpc-daemon[770]: ts=2026-09-21T14:44:15Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:45396 worker2 # [ 39.592183] nix-grpc-daemon[770]: ts=2026-09-21T14:44:15Z level=debug event=path_info_miss path=6v5lkzqzv8y8aql79zri93a767rl7bwj-nixos-system-client-test error="error: path '/nix/store/6v5lkzqzv8y8aql79zri93a767rl7bwj-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds) worker1 # [ 39.836237] nix-grpc-daemon[713]: ts=2026-09-21T14:44:15Z level=info event=denied method=QueryMissing cn=- role=none worker2 # [ 39.990298] nix-grpc-daemon[770]: ts=2026-09-21T14:44:15Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.86 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 40.161113] nix-grpc-daemon[770]: ts=2026-09-21T14:44:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 40.170186] nix-grpc-daemon[770]: ts=2026-09-21T14:44:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45396 worker1 # [ 40.227878] nix-grpc-daemon[713]: ts=2026-09-21T14:44:15Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=0 worker1 # [ 40.233203] nix-grpc-daemon[713]: ts=2026-09-21T14:44:15Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 40.277667] nix-grpc-daemon[713]: ts=2026-09-21T14:44:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 40.295301] nix-grpc-daemon[713]: ts=2026-09-21T14:44:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52934 client # copying 1 paths to 'lb:50051'... worker1 # [ 40.311627] nix-grpc-daemon[713]: ts=2026-09-21T14:44:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'... worker1 # [ 40.333658] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 40.376810] nix-grpc-daemon[713]: ts=2026-09-21T14:44:15Z level=info event=niks3_push_started pid=866 worker1 # [ 40.557630] nix-grpc-daemon[866]: time=2026-09-21T14:44:16.165Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 40.669138] nix-daemon[852]: accepted connection from pid 878, user nix-grpc-daemon (trusted) worker1 # [ 40.691651] nix-daemon[852]: reaped child process 881, status = succeeded worker1 # [ 40.786912] nix-daemon[852]: accepted connection from pid 883, user nix-grpc-daemon (trusted) worker1 # [ 40.805453] nix-daemon[852]: reaped child process 886, status = succeeded lb # [ 40.809425] niks3-server[979]: 2026/09/21 14:44:16 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 40.829683] nix-grpc-daemon[866]: time=2026-09-21T14:44:16.437Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 40.831154] nix-grpc-daemon[866]: time=2026-09-21T14:44:16.437Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)" lb # [ 40.889158] niks3-server[979]: 2026/09/21 14:44:16 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst lb # [ 40.903342] niks3-server[979]: 2026/09/21 14:44:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign worker1 # [ 40.906859] nix-grpc-daemon[866]: time=2026-09-21T14:44:16.514Z level=INFO msg="Uploading 1 narinfos" lb # [ 40.916644] niks3-server[979]: 2026/09/21 14:44:16 INFO Signed narinfos id=1 count=1 lb # [ 40.931562] niks3-server[979]: 2026/09/21 14:44:16 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls lb # [ 40.940152] niks3-server[979]: 2026/09/21 14:44:16 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 40.948406] niks3-server[979]: 2026/09/21 14:44:16 INFO Completed upload id=1 worker1 # [ 40.945938] nix-grpc-daemon[866]: time=2026-09-21T14:44:16.553Z level=INFO msg="Upload complete. (383ms)" lb # [ 40.956390] niks3-server[979]: 2026/09/21 14:44:16 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo worker1 # [ 40.957865] nix-grpc-daemon[713]: ts=2026-09-21T14:44:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=0 paths=1 nar_bytes_in=504 worker1 # [ 40.983140] nix-grpc-daemon[713]: ts=2026-09-21T14:44:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 40.991133] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 41.001076] nix-daemon[852]: reaped child process 864, status = succeeded client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051... client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker1 # [ 41.020326] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker1 # [ 42.065798] nix-daemon[852]: reaped child process 890, status = succeeded worker1 # [ 42.168960] nix-daemon[852]: accepted connection from pid 907, user nix-grpc-daemon (trusted) worker1 # [ 42.189175] nix-daemon[852]: reaped child process 910, status = succeeded lb # [ 42.198373] niks3-server[979]: 2026/09/21 14:44:17 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 42.207313] nix-grpc-daemon[866]: time=2026-09-21T14:44:17.813Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 42.210098] nix-grpc-daemon[866]: time=2026-09-21T14:44:17.813Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)" lb # [ 42.227449] niks3-server[979]: 2026/09/21 14:44:17 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv lb # [ 42.251857] niks3-server[979]: 2026/09/21 14:44:17 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst worker1 # [ 42.276112] nix-grpc-daemon[866]: time=2026-09-21T14:44:17.883Z level=INFO msg="Uploading 1 narinfos" lb # [ 42.273358] niks3-server[979]: 2026/09/21 14:44:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign lb # [ 42.284317] niks3-server[979]: 2026/09/21 14:44:17 INFO Signed narinfos id=2 count=1 lb # [ 42.287362] niks3-server[979]: 2026/09/21 14:44:17 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls lb # [ 42.304144] niks3-server[979]: 2026/09/21 14:44:17 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete lb # [ 42.315349] niks3-server[979]: 2026/09/21 14:44:17 INFO Completed upload id=2 worker1 # [ 42.311545] nix-grpc-daemon[866]: time=2026-09-21T14:44:17.920Z level=INFO msg="Upload complete. (267ms)" lb # [ 42.319803] niks3-server[979]: 2026/09/21 14:44:17 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo worker1 # [ 42.337089] nix-grpc-daemon[713]: ts=2026-09-21T14:44:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=1 worker1 # [ 42.348338] nix-grpc-daemon[713]: ts=2026-09-21T14:44:17Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=0 worker1 # [ 42.354688] nix-grpc-daemon[713]: ts=2026-09-21T14:44:17Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 42.358868] nix-daemon[852]: reaped child process 888, status = succeeded worker1 # [ 42.376597] nix-grpc-daemon[713]: ts=2026-09-21T14:44:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 42.392413] nix-grpc-daemon[713]: ts=2026-09-21T14:44:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 42.406515] nix-grpc-daemon[866]: time=2026-09-21T14:44:18.015Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 42.416118] nix-grpc-daemon[713]: ts=2026-09-21T14:44:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'... worker1 # [ 42.438337] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 42.599702] nix-daemon[852]: accepted connection from pid 916, user nix-grpc-daemon (trusted) worker1 # [ 42.619574] nix-daemon[852]: reaped child process 919, status = succeeded worker1 # [ 42.724669] nix-daemon[852]: accepted connection from pid 921, user nix-grpc-daemon (trusted) worker1 # [ 42.741131] nix-daemon[852]: reaped child process 924, status = succeeded lb # [ 42.750460] niks3-server[979]: 2026/09/21 14:44:18 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 42.755691] nix-grpc-daemon[866]: time=2026-09-21T14:44:18.363Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 42.757397] nix-grpc-daemon[866]: time=2026-09-21T14:44:18.363Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)" lb # [ 42.783143] niks3-server[979]: 2026/09/21 14:44:18 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst worker1 # [ 42.789320] nix-grpc-daemon[866]: time=2026-09-21T14:44:18.397Z level=INFO msg="Uploading 1 narinfos" lb # [ 42.798642] niks3-server[979]: 2026/09/21 14:44:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign lb # [ 42.809281] niks3-server[979]: 2026/09/21 14:44:18 INFO Signed narinfos id=3 count=1 lb # [ 42.811683] niks3-server[979]: 2026/09/21 14:44:18 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls lb # [ 42.823704] niks3-server[979]: 2026/09/21 14:44:18 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 42.830452] niks3-server[979]: 2026/09/21 14:44:18 INFO Completed upload id=3 worker1 # [ 42.826491] nix-grpc-daemon[866]: time=2026-09-21T14:44:18.435Z level=INFO msg="Upload complete. (371ms)" lb # [ 42.832630] niks3-server[979]: 2026/09/21 14:44:18 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo worker1 # [ 42.836282] nix-grpc-daemon[713]: ts=2026-09-21T14:44:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 42.851248] nix-grpc-daemon[713]: ts=2026-09-21T14:44:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 42.860563] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 42.863701] nix-daemon[852]: reaped child process 914, status = succeeded client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051... worker1 # [ 42.874780] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker1 # [ 44.390837] nix-daemon[852]: reaped child process 928, status = succeeded worker1 # [ 44.455034] nix-daemon[852]: accepted connection from pid 944, user nix-grpc-daemon (trusted) worker1 # [ 44.480002] nix-daemon[852]: reaped child process 947, status = succeeded lb # [ 44.494574] niks3-server[979]: 2026/09/21 14:44:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 44.501408] nix-grpc-daemon[866]: time=2026-09-21T14:44:20.108Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 44.503219] nix-grpc-daemon[866]: time=2026-09-21T14:44:20.109Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)" lb # [ 44.529932] niks3-server[979]: 2026/09/21 14:44:20 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst lb # [ 44.546006] niks3-server[979]: 2026/09/21 14:44:20 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv lb # [ 44.553987] niks3-server[979]: 2026/09/21 14:44:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign worker1 # [ 44.550710] nix-grpc-daemon[866]: time=2026-09-21T14:44:20.158Z level=INFO msg="Uploading 1 narinfos" lb # [ 44.563364] niks3-server[979]: 2026/09/21 14:44:20 INFO Signed narinfos id=4 count=1 worker1 # [ 44.576323] nix-grpc-daemon[866]: time=2026-09-21T14:44:20.184Z level=INFO msg="Upload complete. (288ms)" lb # [ 44.573355] niks3-server[979]: 2026/09/21 14:44:20 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete lb # [ 44.588865] niks3-server[979]: 2026/09/21 14:44:20 INFO Completed upload id=4 lb # [ 44.590998] niks3-server[979]: 2026/09/21 14:44:20 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls lb # [ 44.596775] niks3-server[979]: 2026/09/21 14:44:20 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo worker1 # [ 44.594251] nix-grpc-daemon[713]: ts=2026-09-21T14:44:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=1 worker1 # [ 44.615036] nix-grpc-daemon[713]: ts=2026-09-21T14:44:20Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=0 worker1 # [ 44.620658] nix-grpc-daemon[713]: ts=2026-09-21T14:44:20Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 44.626177] nix-daemon[852]: reaped child process 926, status = succeeded worker1 # [ 44.646103] nix-grpc-daemon[713]: ts=2026-09-21T14:44:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 44.660971] nix-grpc-daemon[713]: ts=2026-09-21T14:44:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 44.675194] nix-grpc-daemon[866]: time=2026-09-21T14:44:20.283Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker1 # [ 44.685302] nix-grpc-daemon[713]: ts=2026-09-21T14:44:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'... worker1 # [ 44.702748] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 44.856739] nix-daemon[852]: accepted connection from pid 954, user nix-grpc-daemon (trusted) worker1 # [ 44.875479] nix-daemon[852]: reaped child process 957, status = succeeded worker1 # [ 44.978449] nix-daemon[852]: accepted connection from pid 959, user nix-grpc-daemon (trusted) worker1 # [ 44.994764] nix-daemon[852]: reaped child process 962, status = succeeded lb # [ 45.005102] niks3-server[979]: 2026/09/21 14:44:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 45.011137] nix-grpc-daemon[866]: time=2026-09-21T14:44:20.619Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 45.012676] nix-grpc-daemon[866]: time=2026-09-21T14:44:20.619Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)" lb # [ 45.039295] niks3-server[979]: 2026/09/21 14:44:20 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst worker1 # [ 45.047967] nix-grpc-daemon[866]: time=2026-09-21T14:44:20.656Z level=INFO msg="Uploading 1 narinfos" lb # [ 45.054586] niks3-server[979]: 2026/09/21 14:44:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign lb # [ 45.064711] niks3-server[979]: 2026/09/21 14:44:20 INFO Signed narinfos id=5 count=1 lb # [ 45.069998] niks3-server[979]: 2026/09/21 14:44:20 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls worker1 # [ 45.077203] nix-grpc-daemon[866]: time=2026-09-21T14:44:20.685Z level=INFO msg="Upload complete. (356ms)" lb # [ 45.081386] niks3-server[979]: 2026/09/21 14:44:20 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete lb # [ 45.086415] niks3-server[979]: 2026/09/21 14:44:20 INFO Completed upload id=5 worker1 # [ 45.084221] nix-grpc-daemon[713]: ts=2026-09-21T14:44:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 45.090404] niks3-server[979]: 2026/09/21 14:44:20 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo worker1 # [ 45.097486] nix-grpc-daemon[713]: ts=2026-09-21T14:44:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 45.103540] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 45.108832] nix-daemon[852]: reaped child process 952, status = succeeded client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051... client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 45.123538] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker1 # [ 46.008956] nix-daemon[852]: reaped child process 966, status = succeeded worker1 # [ 46.110598] nix-daemon[852]: accepted connection from pid 982, user nix-grpc-daemon (trusted) worker1 # [ 46.131298] nix-daemon[852]: reaped child process 985, status = succeeded lb # [ 46.139890] niks3-server[979]: 2026/09/21 14:44:21 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 46.149936] nix-grpc-daemon[866]: time=2026-09-21T14:44:21.756Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 46.151846] nix-grpc-daemon[866]: time=2026-09-21T14:44:21.756Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)" lb # [ 46.182314] niks3-server[979]: 2026/09/21 14:44:21 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 46.195779] nix-grpc-daemon[866]: time=2026-09-21T14:44:21.803Z level=INFO msg="Uploading 1 narinfos" lb # [ 46.200096] niks3-server[979]: 2026/09/21 14:44:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign lb # [ 46.203086] niks3-server[979]: 2026/09/21 14:44:21 INFO Signed narinfos id=6 count=1 lb # [ 46.211961] niks3-server[979]: 2026/09/21 14:44:21 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst lb # [ 46.218688] niks3-server[979]: 2026/09/21 14:44:21 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls lb # [ 46.225352] niks3-server[979]: 2026/09/21 14:44:21 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete lb # [ 46.229013] niks3-server[979]: 2026/09/21 14:44:21 INFO Completed upload id=6 worker1 # [ 46.224897] nix-grpc-daemon[866]: time=2026-09-21T14:44:21.833Z level=INFO msg="Upload complete. (290ms)" lb # [ 46.231800] niks3-server[979]: 2026/09/21 14:44:21 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo worker1 # [ 46.237241] nix-grpc-daemon[713]: ts=2026-09-21T14:44:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=1 worker1 # [ 46.256240] nix-daemon[852]: reaped child process 964, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 6.25 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.18 seconds) client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client # copying 3 paths... client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.77 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top lb # [ 46.884880] envoy[888]: conn peer=[2001:db8:1::1]:50630 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=2751 lb # [ 46.889833] envoy[888]: conn peer=[2001:db8:1::1]:50634 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=74 lb # [ 46.894372] envoy[888]: conn peer=[2001:db8:1::1]:50646 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=15 lb # [ 46.899239] envoy[888]: conn peer=[2001:db8:1::1]:50656 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=13 lb # [ 46.903926] envoy[888]: conn peer=[2001:db8:1::1]:50672 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9562 tx=4074 ms=6071 worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds) worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 47.468627] nix-grpc-daemon[770]: ts=2026-09-21T14:44:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45396 worker1 # [ 47.493341] nix-grpc-daemon[713]: ts=2026-09-21T14:44:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 47.506771] nix-grpc-daemon[866]: time=2026-09-21T14:44:23.114Z level=INFO msg="All 2 paths already cached" worker1 # [ 47.523526] nix-grpc-daemon[713]: ts=2026-09-21T14:44:23Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.34 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.18 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 1.25 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2 client # don't know how to build these paths: client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 48.413614] nix-grpc-daemon[770]: ts=2026-09-21T14:44:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45396 worker1 # [ 48.463317] nix-grpc-daemon[713]: ts=2026-09-21T14:44:24Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=0 worker1 # [ 48.470433] nix-grpc-daemon[713]: ts=2026-09-21T14:44:24Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 48.510074] nix-grpc-daemon[713]: ts=2026-09-21T14:44:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 48.525311] nix-grpc-daemon[713]: ts=2026-09-21T14:44:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52934 client # copying 1 paths to 'lb:50051'... worker1 # [ 48.541485] nix-grpc-daemon[866]: time=2026-09-21T14:44:24.148Z level=INFO msg="All 2 paths already cached" client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'... worker1 # [ 48.546761] nix-grpc-daemon[713]: ts=2026-09-21T14:44:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 48.561833] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 48.722817] nix-daemon[852]: accepted connection from pid 1014, user nix-grpc-daemon (trusted) worker1 # [ 48.744441] nix-daemon[852]: reaped child process 1017, status = succeeded worker1 # [ 48.840951] nix-daemon[852]: accepted connection from pid 1019, user nix-grpc-daemon (trusted) worker1 # [ 48.858164] nix-daemon[852]: reaped child process 1022, status = succeeded lb # [ 48.870624] niks3-server[979]: 2026/09/21 14:44:24 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 48.876361] nix-grpc-daemon[866]: time=2026-09-21T14:44:24.484Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 48.877860] nix-grpc-daemon[866]: time=2026-09-21T14:44:24.484Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)" lb # [ 48.903240] niks3-server[979]: 2026/09/21 14:44:24 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst worker1 # [ 48.910041] nix-grpc-daemon[866]: time=2026-09-21T14:44:24.518Z level=INFO msg="Uploading 1 narinfos" lb # [ 48.919005] niks3-server[979]: 2026/09/21 14:44:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign lb # [ 48.928372] niks3-server[979]: 2026/09/21 14:44:24 INFO Signed narinfos id=7 count=1 lb # [ 48.937058] niks3-server[979]: 2026/09/21 14:44:24 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls worker1 # [ 48.937075] nix-grpc-daemon[866]: time=2026-09-21T14:44:24.545Z level=INFO msg="Upload complete. (358ms)" lb # [ 48.946010] niks3-server[979]: 2026/09/21 14:44:24 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete worker1 # [ 48.944132] nix-grpc-daemon[713]: ts=2026-09-21T14:44:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 48.953421] niks3-server[979]: 2026/09/21 14:44:24 INFO Completed upload id=7 lb # [ 48.955947] niks3-server[979]: 2026/09/21 14:44:24 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo worker1 # [ 48.955943] nix-grpc-daemon[713]: ts=2026-09-21T14:44:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 48.961934] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 48.966462] nix-daemon[852]: reaped child process 1012, status = succeeded client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051... client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker1 # [ 48.979781] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker2 # [ 48.421549] nix-grpc-daemon[770]: ts=2026-09-21T14:44:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45396 client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker1 # [ 50.280528] nix-daemon[852]: reaped child process 1026, status = succeeded worker1 # [ 50.356391] nix-daemon[852]: accepted connection from pid 1043, user nix-grpc-daemon (trusted) worker1 # [ 50.375765] nix-daemon[852]: reaped child process 1046, status = succeeded lb # [ 50.387387] niks3-server[979]: 2026/09/21 14:44:25 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 50.395416] nix-grpc-daemon[866]: time=2026-09-21T14:44:26.001Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 50.397239] nix-grpc-daemon[866]: time=2026-09-21T14:44:26.001Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)" lb # [ 50.419506] niks3-server[979]: 2026/09/21 14:44:26 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker1 # [ 50.434955] nix-grpc-daemon[866]: time=2026-09-21T14:44:26.043Z level=INFO msg="Uploading 1 narinfos" lb # [ 50.439537] niks3-server[979]: 2026/09/21 14:44:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign lb # [ 50.450690] niks3-server[979]: 2026/09/21 14:44:26 INFO Signed narinfos id=8 count=1 lb # [ 50.455533] niks3-server[979]: 2026/09/21 14:44:26 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst lb # [ 50.461303] niks3-server[979]: 2026/09/21 14:44:26 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls lb # [ 50.471733] niks3-server[979]: 2026/09/21 14:44:26 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete lb # [ 50.478769] niks3-server[979]: 2026/09/21 14:44:26 INFO Completed upload id=8 worker1 # [ 50.473733] nix-grpc-daemon[866]: time=2026-09-21T14:44:26.082Z level=INFO msg="Upload complete. (281ms)" lb # [ 50.481945] niks3-server[979]: 2026/09/21 14:44:26 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo worker1 # [ 50.489158] nix-grpc-daemon[713]: ts=2026-09-21T14:44:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 worker1 # [ 50.504873] nix-daemon[852]: reaped child process 1024, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.30 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.83 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 50.945983] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: c7f676b3c6224775932eacc5964babd4 client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.09 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 51.133646] nix[928]: don't know how to build these paths: client # [ 51.134851] nix[928]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 51.135323] nix-grpc-daemon[713]: ts=2026-09-21T14:44:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 51.148250] nix-grpc-daemon[713]: ts=2026-09-21T14:44:26Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker1 # [ 51.154111] nix-grpc-daemon[713]: ts=2026-09-21T14:44:26Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 51.138545] nix-grpc-daemon[770]: ts=2026-09-21T14:44:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 51.155746] nix-grpc-daemon[770]: ts=2026-09-21T14:44:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45396 client # [ 51.187259] nix[928]: copying 1 paths to 'lb:50051'... worker2 # [ 51.170882] nix-grpc-daemon[770]: ts=2026-09-21T14:44:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45396 client # [ 51.189886] nix[928]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'... worker2 # [ 51.185286] nix-daemon[838]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 51.212124] nix-grpc-daemon[770]: ts=2026-09-21T14:44:26Z level=info event=niks3_push_started pid=866 worker2 # [ 51.379362] nix-grpc-daemon[866]: time=2026-09-21T14:44:26.981Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 51.488478] nix-daemon[838]: accepted connection from pid 879, user nix-grpc-daemon (trusted) worker2 # [ 51.509257] nix-daemon[838]: reaped child process 882, status = succeeded worker2 # [ 51.615281] nix-daemon[838]: accepted connection from pid 884, user nix-grpc-daemon (trusted) worker2 # [ 51.631413] nix-daemon[838]: reaped child process 887, status = succeeded lb # [ 51.665966] niks3-server[979]: 2026/09/21 14:44:27 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 51.650028] nix-grpc-daemon[866]: time=2026-09-21T14:44:27.251Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 51.651510] nix-grpc-daemon[866]: time=2026-09-21T14:44:27.251Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)" lb # [ 51.705242] niks3-server[979]: 2026/09/21 14:44:27 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst worker2 # [ 51.692070] nix-grpc-daemon[866]: time=2026-09-21T14:44:27.294Z level=INFO msg="Uploading 1 narinfos" lb # [ 51.719873] niks3-server[979]: 2026/09/21 14:44:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign lb # [ 51.729511] niks3-server[979]: 2026/09/21 14:44:27 INFO Signed narinfos id=9 count=1 lb # [ 51.738408] niks3-server[979]: 2026/09/21 14:44:27 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls worker2 # [ 51.719817] nix-grpc-daemon[866]: time=2026-09-21T14:44:27.322Z level=INFO msg="Upload complete. (336ms)" lb # [ 51.744617] niks3-server[979]: 2026/09/21 14:44:27 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete lb # [ 51.750325] niks3-server[979]: 2026/09/21 14:44:27 INFO Completed upload id=9 worker2 # [ 51.728257] nix-grpc-daemon[770]: ts=2026-09-21T14:44:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45396 duration_s=0 paths=1 nar_bytes_in=432 lb # [ 51.756195] niks3-server[979]: 2026/09/21 14:44:27 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo worker2 # [ 51.739371] nix-grpc-daemon[770]: ts=2026-09-21T14:44:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 51.746056] nix-daemon[838]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 51.750691] nix-daemon[838]: reaped child process 864, status = succeeded client # [ 51.778352] nix[928]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker2 # [ 51.763894] nix-daemon[838]: accepted connection from pid 770, user nix-grpc-daemon (trusted) client # [ 52.042858] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: eabf0abde3d9437884e2d14c419fb5c1 client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds) client # [ 52.290282] nix[944]: don't know how to build these paths: worker1 # [ 52.289298] nix-grpc-daemon[713]: ts=2026-09-21T14:44:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:52934 client # [ 52.291439] nix[944]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker1 # [ 52.302625] nix-grpc-daemon[713]: ts=2026-09-21T14:44:27Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker2 # [ 52.288436] nix-grpc-daemon[770]: ts=2026-09-21T14:44:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45396 client # [ 52.309641] nix[944]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.05 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker2 # [ 55.595822] nix-daemon[838]: reaped child process 891, status = succeeded worker2 # [ 55.687582] nix-daemon[838]: accepted connection from pid 921, user nix-grpc-daemon (trusted) worker2 # [ 55.708871] nix-daemon[838]: reaped child process 924, status = succeeded lb # [ 55.737275] niks3-server[979]: 2026/09/21 14:44:31 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 55.724415] nix-grpc-daemon[866]: time=2026-09-21T14:44:31.325Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 55.725952] nix-grpc-daemon[866]: time=2026-09-21T14:44:31.325Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)" lb # [ 55.782998] niks3-server[979]: 2026/09/21 14:44:31 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv lb # [ 55.794817] niks3-server[979]: 2026/09/21 14:44:31 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst worker2 # [ 55.780723] nix-grpc-daemon[866]: time=2026-09-21T14:44:31.382Z level=INFO msg="Uploading 1 narinfos" lb # [ 55.809510] niks3-server[979]: 2026/09/21 14:44:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign lb # [ 55.815992] niks3-server[979]: 2026/09/21 14:44:31 INFO Signed narinfos id=10 count=1 lb # [ 55.820868] niks3-server[979]: 2026/09/21 14:44:31 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls worker2 # [ 55.807265] nix-grpc-daemon[866]: time=2026-09-21T14:44:31.410Z level=INFO msg="Upload complete. (231ms)" lb # [ 55.832973] niks3-server[979]: 2026/09/21 14:44:31 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete lb # [ 55.842222] niks3-server[979]: 2026/09/21 14:44:31 INFO Completed upload id=10 worker2 # [ 55.816907] nix-grpc-daemon[770]: ts=2026-09-21T14:44:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45396 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=1 lb # [ 55.848961] niks3-server[979]: 2026/09/21 14:44:31 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo worker2 # [ 55.833010] nix-grpc-daemon[770]: ts=2026-09-21T14:44:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45396 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=1 worker2 # [ 55.838984] nix-daemon[838]: reaped child process 889, status = succeeded client # [ 55.870514] systemd[1]: dedup2.service: Deactivated successfully. client # [ 55.875316] systemd[1]: dedup2.service: Consumed 219ms CPU time over 3.829s wall clock time, 33.3M memory peak, 5.8K incoming IP traffic, 5.7K outgoing IP traffic. client # [ 55.923531] systemd[1]: dedup1.service: Deactivated successfully. client # [ 55.926870] systemd[1]: dedup1.service: Consumed 169ms CPU time over 4.982s wall clock time, 43.5M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true lb # [ 56.885494] envoy[888]: conn peer=[2001:db8:1::1]:37458 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=56 lb # [ 56.892421] envoy[888]: conn peer=[2001:db8:1::1]:37462 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1783 ms=2064 lb # [ 56.897296] envoy[888]: conn peer=[2001:db8:1::1]:54232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1085 ms=3569 lb # [ 56.902984] envoy[888]: conn peer=[2001:db8:1::1]:54218 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1453 ms=4764 worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: two clients wanting the same drv build it once, in 6.15 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log client # [ 57.204482] systemd[1]: Started Nix Daemon. client # [ 57.320544] nix-daemon[986]: accepted connection from pid 983, user root (trusted) worker2 # [ 57.453182] nix-grpc-daemon[770]: ts=2026-09-21T14:44:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 57.459444] nix-grpc-daemon[770]: ts=2026-09-21T14:44:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45396 worker1 # [ 57.501263] nix-grpc-daemon[713]: ts=2026-09-21T14:44:33Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=0 worker1 # [ 57.501575] nix-grpc-daemon[713]: ts=2026-09-21T14:44:33Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 57.544216] nix-grpc-daemon[713]: ts=2026-09-21T14:44:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 57.560430] nix-grpc-daemon[713]: ts=2026-09-21T14:44:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 57.575495] nix-grpc-daemon[713]: ts=2026-09-21T14:44:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 57.590444] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 57.758956] nix-daemon[852]: accepted connection from pid 1082, user nix-grpc-daemon (trusted) worker1 # [ 57.782349] nix-daemon[852]: reaped child process 1085, status = succeeded worker1 # [ 57.877527] nix-daemon[852]: accepted connection from pid 1087, user nix-grpc-daemon (trusted) worker1 # [ 57.894160] nix-daemon[852]: reaped child process 1090, status = succeeded lb # [ 57.905712] niks3-server[979]: 2026/09/21 14:44:33 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 57.911784] nix-grpc-daemon[866]: time=2026-09-21T14:44:33.519Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 57.913410] nix-grpc-daemon[866]: time=2026-09-21T14:44:33.519Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)" lb # [ 57.945427] niks3-server[979]: 2026/09/21 14:44:33 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst worker1 # [ 57.953863] nix-grpc-daemon[866]: time=2026-09-21T14:44:33.562Z level=INFO msg="Uploading 1 narinfos" lb # [ 57.959286] niks3-server[979]: 2026/09/21 14:44:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign lb # [ 57.969276] niks3-server[979]: 2026/09/21 14:44:33 INFO Signed narinfos id=11 count=1 lb # [ 57.974505] niks3-server[979]: 2026/09/21 14:44:33 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls lb # [ 57.980559] niks3-server[979]: 2026/09/21 14:44:33 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete lb # [ 57.989389] niks3-server[979]: 2026/09/21 14:44:33 INFO Completed upload id=11 worker1 # [ 57.984654] nix-grpc-daemon[866]: time=2026-09-21T14:44:33.593Z level=INFO msg="Upload complete. (377ms)" lb # [ 57.991860] niks3-server[979]: 2026/09/21 14:44:33 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo worker1 # [ 57.994167] nix-grpc-daemon[713]: ts=2026-09-21T14:44:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 58.012450] nix-grpc-daemon[713]: ts=2026-09-21T14:44:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 58.022521] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 58.028859] nix-daemon[852]: reaped child process 1080, status = succeeded worker1 # [ 58.042298] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 58.940386] nix-daemon[852]: reaped child process 1094, status = succeeded worker1 # [ 59.007241] nix-daemon[852]: accepted connection from pid 1110, user nix-grpc-daemon (trusted) worker1 # [ 59.025047] nix-daemon[852]: reaped child process 1113, status = succeeded lb # [ 59.038398] niks3-server[979]: 2026/09/21 14:44:34 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 59.044438] nix-grpc-daemon[866]: time=2026-09-21T14:44:34.649Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 59.046063] nix-grpc-daemon[866]: time=2026-09-21T14:44:34.649Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)" lb # [ 59.072151] niks3-server[979]: 2026/09/21 14:44:34 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv lb # [ 59.086861] niks3-server[979]: 2026/09/21 14:44:34 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst worker1 # [ 59.097899] nix-grpc-daemon[866]: time=2026-09-21T14:44:34.706Z level=INFO msg="Uploading 1 narinfos" lb # [ 59.101698] niks3-server[979]: 2026/09/21 14:44:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign lb # [ 59.114058] niks3-server[979]: 2026/09/21 14:44:34 INFO Signed narinfos id=12 count=1 lb # [ 59.121530] niks3-server[979]: 2026/09/21 14:44:34 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls lb # [ 59.133545] niks3-server[979]: 2026/09/21 14:44:34 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete lb # [ 59.140722] niks3-server[979]: 2026/09/21 14:44:34 INFO Completed upload id=12 worker1 # [ 59.135775] nix-grpc-daemon[866]: time=2026-09-21T14:44:34.744Z level=INFO msg="Upload complete. (279ms)" lb # [ 59.144065] niks3-server[979]: 2026/09/21 14:44:34 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo worker1 # [ 59.147656] nix-grpc-daemon[713]: ts=2026-09-21T14:44:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=1 worker1 # [ 59.166729] nix-daemon[852]: reaped child process 1092, status = succeeded worker1 # [ 59.192820] nix-grpc-daemon[713]: ts=2026-09-21T14:44:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:52934 worker2 # [ 59.195947] nix-grpc-daemon[770]: ts=2026-09-21T14:44:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 59.206113] nix-grpc-daemon[770]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'... worker2 # [ 59.224307] nix-grpc-daemon[770]: ts=2026-09-21T14:44:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45396 duration_s=0 paths=1 nar_bytes_out=128 worker1 # [ 59.533771] nix-grpc-daemon[713]: ts=2026-09-21T14:44:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:52934 worker2 # [ 59.518046] nix-grpc-daemon[770]: ts=2026-09-21T14:44:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 59.528745] nix-grpc-daemon[866]: time=2026-09-21T14:44:35.130Z level=INFO msg="All 1 paths already cached" worker1 # [ 59.567175] nix-grpc-daemon[713]: ts=2026-09-21T14:44:35Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=0 worker1 # [ 59.574135] nix-grpc-daemon[713]: ts=2026-09-21T14:44:35Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 59.582922] nix-grpc-daemon[713]: ts=2026-09-21T14:44:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 59.594307] nix-grpc-daemon[713]: ts=2026-09-21T14:44:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 59.608769] nix-grpc-daemon[866]: time=2026-09-21T14:44:35.217Z level=INFO msg="All 1 paths already cached" worker1 # [ 59.617880] nix-grpc-daemon[713]: ts=2026-09-21T14:44:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 59.630778] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 59.786344] nix-daemon[852]: accepted connection from pid 1119, user nix-grpc-daemon (trusted) worker1 # [ 59.809365] nix-daemon[852]: reaped child process 1122, status = succeeded worker1 # [ 59.911720] nix-daemon[852]: accepted connection from pid 1124, user nix-grpc-daemon (trusted) worker1 # [ 59.931315] nix-daemon[852]: reaped child process 1127, status = succeeded lb # [ 59.939250] niks3-server[979]: 2026/09/21 14:44:35 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 59.944898] nix-grpc-daemon[866]: time=2026-09-21T14:44:35.553Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 59.946400] nix-grpc-daemon[866]: time=2026-09-21T14:44:35.553Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)" lb # [ 59.967591] niks3-server[979]: 2026/09/21 14:44:35 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst lb # [ 59.982697] niks3-server[979]: 2026/09/21 14:44:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign worker1 # [ 59.986099] nix-grpc-daemon[866]: time=2026-09-21T14:44:35.594Z level=INFO msg="Uploading 1 narinfos" lb # [ 59.994694] niks3-server[979]: 2026/09/21 14:44:35 INFO Signed narinfos id=13 count=1 lb # [ 60.006306] niks3-server[979]: 2026/09/21 14:44:35 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete lb # [ 60.022163] niks3-server[979]: 2026/09/21 14:44:35 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls lb # [ 60.032350] niks3-server[979]: 2026/09/21 14:44:35 INFO Completed upload id=13 worker1 # [ 60.023773] nix-grpc-daemon[866]: time=2026-09-21T14:44:35.632Z level=INFO msg="Upload complete. (377ms)" worker1 # [ 60.032501] nix-grpc-daemon[713]: ts=2026-09-21T14:44:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 60.051863] nix-grpc-daemon[713]: ts=2026-09-21T14:44:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 lb # [ 60.057227] niks3-server[979]: 2026/09/21 14:44:35 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo worker1 # [ 60.060444] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 60.068793] nix-daemon[852]: reaped child process 1117, status = succeeded worker1 # [ 60.082614] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 60.907422] nix-daemon[852]: reaped child process 1131, status = succeeded worker1 # [ 61.016497] nix-daemon[852]: accepted connection from pid 1147, user nix-grpc-daemon (trusted) worker1 # [ 61.038261] nix-daemon[852]: reaped child process 1150, status = succeeded lb # [ 61.040191] niks3-server[979]: 2026/09/21 14:44:36 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 61.051524] nix-grpc-daemon[866]: time=2026-09-21T14:44:36.658Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 61.053092] nix-grpc-daemon[866]: time=2026-09-21T14:44:36.658Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)" lb # [ 61.094179] niks3-server[979]: 2026/09/21 14:44:36 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker1 # [ 61.100920] nix-grpc-daemon[866]: time=2026-09-21T14:44:36.709Z level=INFO msg="Uploading 1 narinfos" lb # [ 61.106287] niks3-server[979]: 2026/09/21 14:44:36 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst lb # [ 61.116550] niks3-server[979]: 2026/09/21 14:44:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign lb # [ 61.119870] niks3-server[979]: 2026/09/21 14:44:36 INFO Signed narinfos id=14 count=1 lb # [ 61.134322] niks3-server[979]: 2026/09/21 14:44:36 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls lb # [ 61.139181] niks3-server[979]: 2026/09/21 14:44:36 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 61.143812] niks3-server[979]: 2026/09/21 14:44:36 INFO Completed upload id=14 worker1 # [ 61.139845] nix-grpc-daemon[866]: time=2026-09-21T14:44:36.748Z level=INFO msg="Upload complete. (247ms)" lb # [ 61.146803] niks3-server[979]: 2026/09/21 14:44:36 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo worker1 # [ 61.153169] nix-grpc-daemon[713]: ts=2026-09-21T14:44:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=1 worker1 # [ 61.172460] nix-grpc-daemon[713]: ts=2026-09-21T14:44:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 61.179871] nix-daemon[852]: reaped child process 1129, status = succeeded worker2 # [ 61.169944] nix-grpc-daemon[770]: ts=2026-09-21T14:44:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 61.182958] nix-grpc-daemon[770]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'... worker2 # [ 61.202461] nix-grpc-daemon[770]: ts=2026-09-21T14:44:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45396 duration_s=0 paths=1 nar_bytes_out=176 worker1 # [ 61.506687] nix-grpc-daemon[713]: ts=2026-09-21T14:44:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:52934 worker2 # [ 61.489981] nix-grpc-daemon[770]: ts=2026-09-21T14:44:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 61.500540] nix-grpc-daemon[866]: time=2026-09-21T14:44:37.102Z level=INFO msg="All 2 paths already cached" worker1 # [ 61.538513] nix-grpc-daemon[713]: ts=2026-09-21T14:44:37Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=0 worker1 # [ 61.545834] nix-grpc-daemon[713]: ts=2026-09-21T14:44:37Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 61.554670] nix-grpc-daemon[713]: ts=2026-09-21T14:44:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 61.565961] nix-grpc-daemon[713]: ts=2026-09-21T14:44:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 61.580260] nix-grpc-daemon[866]: time=2026-09-21T14:44:37.188Z level=INFO msg="All 2 paths already cached" worker1 # [ 61.589794] nix-grpc-daemon[713]: ts=2026-09-21T14:44:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 61.604632] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 61.754330] nix-daemon[852]: accepted connection from pid 1156, user nix-grpc-daemon (trusted) worker1 # [ 61.776734] nix-daemon[852]: reaped child process 1159, status = succeeded worker1 # [ 61.878113] nix-daemon[852]: accepted connection from pid 1161, user nix-grpc-daemon (trusted) worker1 # [ 61.898141] nix-daemon[852]: reaped child process 1164, status = succeeded lb # [ 61.905058] niks3-server[979]: 2026/09/21 14:44:37 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 61.910100] nix-grpc-daemon[866]: time=2026-09-21T14:44:37.518Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 61.911554] nix-grpc-daemon[866]: time=2026-09-21T14:44:37.518Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)" lb # [ 61.932503] niks3-server[979]: 2026/09/21 14:44:37 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst worker1 # [ 61.949956] nix-grpc-daemon[866]: time=2026-09-21T14:44:37.558Z level=INFO msg="Uploading 1 narinfos" lb # [ 61.946816] niks3-server[979]: 2026/09/21 14:44:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign lb # [ 61.961088] niks3-server[979]: 2026/09/21 14:44:37 INFO Signed narinfos id=15 count=1 lb # [ 61.963784] niks3-server[979]: 2026/09/21 14:44:37 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls lb # [ 61.977581] niks3-server[979]: 2026/09/21 14:44:37 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete worker1 # [ 61.981341] nix-grpc-daemon[866]: time=2026-09-21T14:44:37.589Z level=INFO msg="Upload complete. (360ms)" lb # [ 61.987241] niks3-server[979]: 2026/09/21 14:44:37 INFO Completed upload id=15 lb # [ 61.990758] niks3-server[979]: 2026/09/21 14:44:37 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo worker1 # [ 61.988265] nix-grpc-daemon[713]: ts=2026-09-21T14:44:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 62.002940] nix-grpc-daemon[713]: ts=2026-09-21T14:44:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 62.009390] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 62.014629] nix-daemon[852]: reaped child process 1154, status = succeeded worker1 # [ 62.025271] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 63.262839] nix-daemon[852]: reaped child process 1168, status = succeeded worker1 # [ 63.357512] nix-daemon[852]: accepted connection from pid 1184, user nix-grpc-daemon (trusted) worker1 # [ 63.374422] nix-daemon[852]: reaped child process 1187, status = succeeded lb # [ 63.386756] niks3-server[979]: 2026/09/21 14:44:38 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 63.392403] nix-grpc-daemon[866]: time=2026-09-21T14:44:39.000Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 63.393883] nix-grpc-daemon[866]: time=2026-09-21T14:44:39.000Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)" lb # [ 63.435944] niks3-server[979]: 2026/09/21 14:44:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign lb # [ 63.445432] niks3-server[979]: 2026/09/21 14:44:39 INFO Signed narinfos id=16 count=1 worker1 # [ 63.441866] nix-grpc-daemon[866]: time=2026-09-21T14:44:39.050Z level=INFO msg="Uploading 1 narinfos" lb # [ 63.455779] niks3-server[979]: 2026/09/21 14:44:39 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 63.533952] niks3-server[979]: 2026/09/21 14:44:39 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst lb # [ 63.548555] niks3-server[979]: 2026/09/21 14:44:39 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv lb # [ 63.558060] niks3-server[979]: 2026/09/21 14:44:39 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls worker1 # [ 63.560981] nix-grpc-daemon[866]: time=2026-09-21T14:44:39.169Z level=INFO msg="Upload complete. (324ms)" lb # [ 63.570227] niks3-server[979]: 2026/09/21 14:44:39 INFO Completed upload id=16 lb # [ 63.575949] niks3-server[979]: 2026/09/21 14:44:39 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo worker1 # [ 63.577268] nix-grpc-daemon[713]: ts=2026-09-21T14:44:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=1 worker1 # [ 63.595137] nix-grpc-daemon[713]: ts=2026-09-21T14:44:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 63.600903] nix-daemon[852]: reaped child process 1166, status = succeeded worker1 # [ 63.614924] nix-grpc-daemon[713]: ts=2026-09-21T14:44:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 63.620261] nix-grpc-daemon[713]: ts=2026-09-21T14:44:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=0 paths=1 nar_bytes_out=232 client # [ 63.738428] nix-daemon[986]: reaped child process 988, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.73 seconds) client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.85 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.29 seconds) client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds) worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 64.695621] nix-daemon[986]: accepted connection from pid 1069, user root (trusted) client # this derivation will be built: client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 64.854286] nix-grpc-daemon[713]: ts=2026-09-21T14:44:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:52934 worker2 # [ 64.840365] nix-grpc-daemon[770]: ts=2026-09-21T14:44:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45396 client # copying 1 paths... client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'... worker1 # [ 64.871924] nix-grpc-daemon[713]: ts=2026-09-21T14:44:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 64.886924] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 65.050872] nix-daemon[852]: accepted connection from pid 1196, user nix-grpc-daemon (trusted) worker1 # [ 65.073614] nix-daemon[852]: reaped child process 1199, status = succeeded lb # [ 65.081249] niks3-server[979]: 2026/09/21 14:44:40 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 65.086716] nix-grpc-daemon[866]: time=2026-09-21T14:44:40.694Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 65.088469] nix-grpc-daemon[866]: time=2026-09-21T14:44:40.694Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)" lb # [ 65.108828] niks3-server[979]: 2026/09/21 14:44:40 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst lb # [ 65.124683] niks3-server[979]: 2026/09/21 14:44:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign worker1 # [ 65.128942] nix-grpc-daemon[866]: time=2026-09-21T14:44:40.737Z level=INFO msg="Uploading 1 narinfos" lb # [ 65.137441] niks3-server[979]: 2026/09/21 14:44:40 INFO Signed narinfos id=17 count=1 lb # [ 65.145995] niks3-server[979]: 2026/09/21 14:44:40 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete lb # [ 65.263977] niks3-server[979]: 2026/09/21 14:44:40 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls lb # [ 65.275490] niks3-server[979]: 2026/09/21 14:44:40 INFO Completed upload id=17 worker1 # [ 65.271705] nix-grpc-daemon[866]: time=2026-09-21T14:44:40.880Z level=INFO msg="Upload complete. (369ms)" lb # [ 65.284172] niks3-server[979]: 2026/09/21 14:44:40 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo worker1 # [ 65.284426] nix-grpc-daemon[713]: ts=2026-09-21T14:44:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=0 paths=1 nar_bytes_in=120 worker1 # [ 65.304337] nix-grpc-daemon[713]: ts=2026-09-21T14:44:40Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=0 worker1 # [ 65.311065] nix-grpc-daemon[713]: ts=2026-09-21T14:44:40Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=1 worker1 # [ 65.318123] nix-daemon[852]: reaped child process 1194, status = succeeded worker2 # [ 65.323846] nix-grpc-daemon[770]: ts=2026-09-21T14:44:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 65.339552] nix-grpc-daemon[770]: ts=2026-09-21T14:44:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45396 client # copying 2 paths to 'lb:50051'... worker2 # [ 65.358200] nix-grpc-daemon[770]: ts=2026-09-21T14:44:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45396 client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'... client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'... worker2 # [ 65.374512] nix-daemon[838]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 65.549194] nix-daemon[838]: accepted connection from pid 942, user nix-grpc-daemon (trusted) worker2 # [ 65.568997] nix-daemon[838]: reaped child process 945, status = succeeded worker2 # [ 65.673306] nix-daemon[838]: accepted connection from pid 947, user nix-grpc-daemon (trusted) worker2 # [ 65.691050] nix-daemon[838]: reaped child process 950, status = succeeded lb # [ 65.721001] niks3-server[979]: 2026/09/21 14:44:41 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 65.730827] niks3-server[979]: 2026/09/21 14:44:41 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 65.716117] nix-grpc-daemon[866]: time=2026-09-21T14:44:41.318Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 65.717589] nix-grpc-daemon[866]: time=2026-09-21T14:44:41.318Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)" worker2 # [ 65.719248] nix-grpc-daemon[866]: time=2026-09-21T14:44:41.318Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)" lb # [ 65.763555] niks3-server[979]: 2026/09/21 14:44:41 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst lb # [ 65.785036] niks3-server[979]: 2026/09/21 14:44:41 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls lb # [ 65.793668] niks3-server[979]: 2026/09/21 14:44:41 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst lb # [ 65.802361] niks3-server[979]: 2026/09/21 14:44:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign worker2 # [ 65.785897] nix-grpc-daemon[866]: time=2026-09-21T14:44:41.387Z level=INFO msg="Uploading 3 narinfos" lb # [ 65.813635] niks3-server[979]: 2026/09/21 14:44:41 INFO Signed narinfos id=18 count=1 lb # [ 65.819211] niks3-server[979]: 2026/09/21 14:44:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign lb # [ 65.827378] niks3-server[979]: 2026/09/21 14:44:41 INFO Signed narinfos id=19 count=2 lb # [ 65.832730] niks3-server[979]: 2026/09/21 14:44:41 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls lb # [ 65.852524] niks3-server[979]: 2026/09/21 14:44:41 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 65.855214] niks3-server[979]: 2026/09/21 14:44:41 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete lb # [ 65.861950] niks3-server[979]: 2026/09/21 14:44:41 INFO Completed upload id=18 lb # [ 65.863568] niks3-server[979]: 2026/09/21 14:44:41 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete lb # [ 65.867534] niks3-server[979]: 2026/09/21 14:44:41 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo lb # [ 65.871272] niks3-server[979]: 2026/09/21 14:44:41 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 65.874448] niks3-server[979]: 2026/09/21 14:44:41 INFO Completed upload id=19 worker2 # [ 65.850264] nix-grpc-daemon[866]: time=2026-09-21T14:44:41.453Z level=INFO msg="Upload complete. (455ms)" worker2 # [ 65.862273] nix-grpc-daemon[770]: ts=2026-09-21T14:44:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45396 duration_s=0 paths=2 nar_bytes_in=984 worker2 # [ 65.881963] nix-grpc-daemon[770]: ts=2026-09-21T14:44:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 65.893208] nix-daemon[838]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 65.904364] nix-daemon[838]: reaped child process 940, status = succeeded worker2 # [ 65.917140] nix-grpc-daemon[770]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'... client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051... worker2 # [ 65.933877] nix-daemon[838]: accepted connection from pid 770, user nix-grpc-daemon (trusted) client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 66.769417] nix-daemon[838]: reaped child process 955, status = succeeded lb # [ 66.886591] envoy[888]: conn peer=[2001:db8:1::1]:54244 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3403 tx=1717 ms=1832 lb # [ 66.891534] envoy[888]: conn peer=[2001:db8:1::1]:54246 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=334 ms=94 worker2 # [ 66.872594] nix-daemon[838]: accepted connection from pid 971, user nix-grpc-daemon (trusted) lb # [ 66.896276] envoy[888]: conn peer=[2001:db8:1::1]:54250 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3888 tx=1932 ms=1747 lb # [ 66.901290] envoy[888]: conn peer=[2001:db8:1::1]:51782 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=97 lb # [ 66.907709] envoy[888]: conn peer=[2001:db8:1::1]:51784 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4339 tx=2165 ms=2166 worker2 # [ 66.889427] nix-daemon[838]: reaped child process 974, status = succeeded lb # [ 66.912556] envoy[888]: conn peer=[2001:db8:1::1]:51796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=64 lb # [ 66.922083] niks3-server[979]: 2026/09/21 14:44:42 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 66.905094] nix-grpc-daemon[866]: time=2026-09-21T14:44:42.507Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 66.948585] niks3-server[979]: 2026/09/21 14:44:42 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls lb # [ 66.959872] niks3-server[979]: 2026/09/21 14:44:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign worker2 # [ 66.941209] nix-grpc-daemon[866]: time=2026-09-21T14:44:42.543Z level=INFO msg="Uploading 1 narinfos" lb # [ 66.970277] niks3-server[979]: 2026/09/21 14:44:42 INFO Signed narinfos id=20 count=1 lb # [ 66.975393] niks3-server[979]: 2026/09/21 14:44:42 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv lb # [ 66.984470] niks3-server[979]: 2026/09/21 14:44:42 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete lb # [ 66.991774] niks3-server[979]: 2026/09/21 14:44:42 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo lb # [ 66.996612] niks3-server[979]: 2026/09/21 14:44:42 INFO Completed upload id=20 worker2 # [ 66.972302] nix-grpc-daemon[866]: time=2026-09-21T14:44:42.575Z level=INFO msg="Upload complete. (222ms)" client # copying 1 paths... worker2 # [ 66.984322] nix-grpc-daemon[770]: ts=2026-09-21T14:44:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45396 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=1 client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'... worker2 # [ 66.999157] nix-grpc-daemon[770]: ts=2026-09-21T14:44:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 67.003316] nix-daemon[838]: reaped child process 952, status = succeeded worker2 # [ 67.020229] nix-grpc-daemon[770]: ts=2026-09-21T14:44:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 67.025040] nix-grpc-daemon[770]: ts=2026-09-21T14:44:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45396 duration_s=0 paths=1 nar_bytes_out=120 client # [ 67.163573] nix-daemon[986]: reaped child process 1072, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.67 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 67.343182] nix-daemon[986]: accepted connection from pid 1091, user root (trusted) client # this derivation will be built: client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker2 # [ 67.474226] nix-grpc-daemon[770]: ts=2026-09-21T14:44:43Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:45396 client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 67.481754] nix-grpc-daemon[770]: ts=2026-09-21T14:44:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 67.488387] nix-grpc-daemon[866]: time=2026-09-21T14:44:43.090Z level=INFO msg="All 1 paths already cached" worker1 # [ 67.521338] nix-grpc-daemon[713]: ts=2026-09-21T14:44:43Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=0 worker1 # [ 67.528579] nix-grpc-daemon[713]: ts=2026-09-21T14:44:43Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 67.538280] nix-grpc-daemon[713]: ts=2026-09-21T14:44:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 67.549296] nix-grpc-daemon[713]: ts=2026-09-21T14:44:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:52934 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'... worker1 # [ 67.571090] nix-grpc-daemon[713]: ts=2026-09-21T14:44:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 67.585899] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 67.606411] nix-grpc-daemon[713]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2 ms (attempt 1/5) worker1 # [ 67.616113] nix-grpc-daemon[713]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 161 ms (attempt 2/5) worker1 # [ 67.786152] nix-grpc-daemon[713]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 243 ms (attempt 3/5) worker1 # [ 68.031241] nix-grpc-daemon[713]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 4/5) worker1 # [ 68.202928] nix-grpc-daemon[713]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 68.211899] nix-grpc-daemon[713]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 68.220563] nix-grpc-daemon[713]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'... worker1 # [ 68.395891] nix-daemon[852]: accepted connection from pid 1212, user nix-grpc-daemon (trusted) worker1 # [ 68.414689] nix-daemon[852]: reaped child process 1215, status = succeeded worker1 # [ 68.522431] nix-daemon[852]: accepted connection from pid 1217, user nix-grpc-daemon (trusted) worker1 # [ 68.541204] nix-daemon[852]: reaped child process 1220, status = succeeded lb # [ 68.550738] niks3-server[979]: 2026/09/21 14:44:44 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 68.558120] nix-grpc-daemon[866]: time=2026-09-21T14:44:44.166Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 68.559645] nix-grpc-daemon[866]: time=2026-09-21T14:44:44.166Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)" lb # [ 68.580616] niks3-server[979]: 2026/09/21 14:44:44 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst worker1 # [ 68.588605] nix-grpc-daemon[866]: time=2026-09-21T14:44:44.197Z level=INFO msg="Uploading 1 narinfos" lb # [ 68.594214] niks3-server[979]: 2026/09/21 14:44:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign lb # [ 68.605309] niks3-server[979]: 2026/09/21 14:44:44 INFO Signed narinfos id=21 count=1 lb # [ 68.613570] niks3-server[979]: 2026/09/21 14:44:44 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls worker1 # [ 68.621657] nix-grpc-daemon[866]: time=2026-09-21T14:44:44.230Z level=INFO msg="Upload complete. (388ms)" lb # [ 68.626443] niks3-server[979]: 2026/09/21 14:44:44 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete lb # [ 68.636881] niks3-server[979]: 2026/09/21 14:44:44 INFO Completed upload id=21 worker1 # [ 68.629058] nix-grpc-daemon[713]: ts=2026-09-21T14:44:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=1 paths=1 nar_bytes_in=568 lb # [ 68.640506] niks3-server[979]: 2026/09/21 14:44:44 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo worker1 # [ 68.639614] nix-grpc-daemon[713]: ts=2026-09-21T14:44:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 68.654505] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) worker1 # [ 68.657299] nix-daemon[852]: reaped child process 1205, status = succeeded client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051... worker1 # [ 68.669760] nix-daemon[852]: accepted connection from pid 713, user nix-grpc-daemon (trusted) client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 70.340108] nix-daemon[852]: reaped child process 1224, status = succeeded worker1 # [ 70.440821] nix-daemon[852]: accepted connection from pid 1240, user nix-grpc-daemon (trusted) worker1 # [ 70.459698] nix-daemon[852]: reaped child process 1243, status = succeeded lb # [ 70.469567] niks3-server[979]: 2026/09/21 14:44:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 70.475773] nix-grpc-daemon[866]: time=2026-09-21T14:44:46.082Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 70.500794] niks3-server[979]: 2026/09/21 14:44:46 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls worker1 # [ 70.508070] nix-grpc-daemon[866]: time=2026-09-21T14:44:46.116Z level=INFO msg="Uploading 1 narinfos" lb # [ 70.515113] niks3-server[979]: 2026/09/21 14:44:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign lb # [ 70.525726] niks3-server[979]: 2026/09/21 14:44:46 INFO Signed narinfos id=22 count=1 lb # [ 70.532692] niks3-server[979]: 2026/09/21 14:44:46 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 70.540277] nix-grpc-daemon[866]: time=2026-09-21T14:44:46.148Z level=INFO msg="Upload complete. (259ms)" lb # [ 70.544903] niks3-server[979]: 2026/09/21 14:44:46 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete lb # [ 70.551635] niks3-server[979]: 2026/09/21 14:44:46 INFO Completed upload id=22 client # copying 1 paths... lb # [ 70.555763] niks3-server[979]: 2026/09/21 14:44:46 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo worker1 # [ 70.553069] nix-grpc-daemon[713]: ts=2026-09-21T14:44:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52934 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=1 client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'... worker1 # [ 70.571787] nix-grpc-daemon[713]: ts=2026-09-21T14:44:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 70.576920] nix-daemon[852]: reaped child process 1222, status = succeeded worker2 # [ 70.570948] nix-grpc-daemon[770]: ts=2026-09-21T14:44:46Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 70.584367] nix-grpc-daemon[770]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'... worker2 # [ 70.600931] nix-grpc-daemon[770]: ts=2026-09-21T14:44:46Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45396 duration_s=0 paths=1 nar_bytes_out=120 client # [ 70.747640] nix-daemon[986]: reaped child process 1094, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.58 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 70.922718] nix-daemon[986]: accepted connection from pid 1113, user root (trusted) client # this derivation will be built: client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker1 # [ 71.075029] nix-grpc-daemon[713]: ts=2026-09-21T14:44:46Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:52934 client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 71.058192] nix-grpc-daemon[770]: ts=2026-09-21T14:44:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 71.069105] nix-grpc-daemon[866]: time=2026-09-21T14:44:46.671Z level=INFO msg="All 1 paths already cached" worker1 # [ 71.106315] nix-grpc-daemon[713]: ts=2026-09-21T14:44:46Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=0 worker2 # [ 71.088743] nix-grpc-daemon[770]: ts=2026-09-21T14:44:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45396 worker1 # [ 71.112754] nix-grpc-daemon[713]: ts=2026-09-21T14:44:46Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 71.103262] nix-grpc-daemon[770]: ts=2026-09-21T14:44:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 71.117684] nix-grpc-daemon[866]: time=2026-09-21T14:44:46.720Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'... worker2 # [ 71.128538] nix-grpc-daemon[770]: ts=2026-09-21T14:44:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 71.142821] nix-daemon[838]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 71.307181] nix-daemon[838]: accepted connection from pid 979, user nix-grpc-daemon (trusted) worker2 # [ 71.338718] nix-daemon[838]: reaped child process 982, status = succeeded worker2 # [ 71.436539] nix-daemon[838]: accepted connection from pid 984, user nix-grpc-daemon (trusted) worker2 # [ 71.457299] nix-daemon[838]: reaped child process 987, status = succeeded lb # [ 71.487583] niks3-server[979]: 2026/09/21 14:44:47 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 71.473135] nix-grpc-daemon[866]: time=2026-09-21T14:44:47.075Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 71.474598] nix-grpc-daemon[866]: time=2026-09-21T14:44:47.075Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)" lb # [ 71.520808] niks3-server[979]: 2026/09/21 14:44:47 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst lb # [ 71.537954] niks3-server[979]: 2026/09/21 14:44:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign worker2 # [ 71.519243] nix-grpc-daemon[866]: time=2026-09-21T14:44:47.121Z level=INFO msg="Uploading 1 narinfos" lb # [ 71.546453] niks3-server[979]: 2026/09/21 14:44:47 INFO Signed narinfos id=23 count=1 lb # [ 71.559149] niks3-server[979]: 2026/09/21 14:44:47 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls lb # [ 71.570484] niks3-server[979]: 2026/09/21 14:44:47 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete worker2 # [ 71.550585] nix-grpc-daemon[866]: time=2026-09-21T14:44:47.153Z level=INFO msg="Upload complete. (390ms)" lb # [ 71.581756] niks3-server[979]: 2026/09/21 14:44:47 INFO Completed upload id=23 lb # [ 71.584941] niks3-server[979]: 2026/09/21 14:44:47 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo worker2 # [ 71.560642] nix-grpc-daemon[770]: ts=2026-09-21T14:44:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45396 duration_s=0 paths=1 nar_bytes_in=568 worker2 # [ 71.574073] nix-grpc-daemon[770]: ts=2026-09-21T14:44:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 71.590233] nix-daemon[838]: accepted connection from pid 770, user nix-grpc-daemon (trusted) worker2 # [ 71.594131] nix-daemon[838]: reaped child process 977, status = succeeded client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051... worker2 # [ 71.608948] nix-daemon[838]: accepted connection from pid 770, user nix-grpc-daemon (trusted) client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 72.319044] nix-daemon[838]: reaped child process 991, status = succeeded worker2 # [ 72.406424] nix-daemon[838]: accepted connection from pid 1007, user nix-grpc-daemon (trusted) worker2 # [ 72.428253] nix-daemon[838]: reaped child process 1010, status = succeeded lb # [ 72.450092] niks3-server[979]: 2026/09/21 14:44:48 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 72.440458] nix-grpc-daemon[866]: time=2026-09-21T14:44:48.040Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 72.485469] niks3-server[979]: 2026/09/21 14:44:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign worker2 # [ 72.467267] nix-grpc-daemon[866]: time=2026-09-21T14:44:48.069Z level=INFO msg="Uploading 1 narinfos" lb # [ 72.501695] niks3-server[979]: 2026/09/21 14:44:48 INFO Signed narinfos id=24 count=1 lb # [ 72.509535] niks3-server[979]: 2026/09/21 14:44:48 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls lb # [ 72.525119] niks3-server[979]: 2026/09/21 14:44:48 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete lb # [ 72.531266] niks3-server[979]: 2026/09/21 14:44:48 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv lb # [ 72.537176] niks3-server[979]: 2026/09/21 14:44:48 INFO Completed upload id=24 worker2 # [ 72.513405] nix-grpc-daemon[866]: time=2026-09-21T14:44:48.116Z level=INFO msg="Upload complete. (292ms)" lb # [ 72.544277] niks3-server[979]: 2026/09/21 14:44:48 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo worker2 # [ 72.524317] nix-grpc-daemon[770]: ts=2026-09-21T14:44:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45396 duration_s=0 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=1 worker2 # [ 72.539632] nix-daemon[838]: reaped child process 989, status = succeeded client # copying 1 paths... worker2 # [ 72.578235] nix-grpc-daemon[770]: ts=2026-09-21T14:44:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45396 client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'... worker2 # [ 72.628278] nix-grpc-daemon[770]: ts=2026-09-21T14:44:48Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45396 worker2 # [ 72.633843] nix-grpc-daemon[770]: ts=2026-09-21T14:44:48Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45396 duration_s=0 paths=1 nar_bytes_out=120 client # [ 72.848763] nix-daemon[986]: reaped child process 1116, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.11 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 9.00 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.57 seconds) client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds) client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.11 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds) worker2 # [ 74.103353] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 74.958159] nix-grpc-daemon[770]: ts=2026-09-21T14:44:50Z level=info event=signal_exit worker1 # [ 76.127462] nix-grpc-daemon[713]: ts=2026-09-21T14:44:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:52918 worker1 # [ 76.127952] nix-grpc-daemon[713]: ts=2026-09-21T14:44:51Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 76.107912] nix-grpc-daemon[770]: ts=2026-09-21T14:44:51Z level=info event=scheduler_restarting worker2 # [ 76.136496] nix-grpc-daemon[770]: ts=2026-09-21T14:44:51Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 76.167074] nix-daemon[838]: reaped child process 840, status = succeeded lb # [ 76.886431] envoy[888]: conn peer=[2001:db8:1::1]:51808 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4234 tx=1852 ms=2247 lb # [ 76.891803] envoy[888]: conn peer=[2001:db8:1::1]:51818 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=64 lb # [ 76.897481] envoy[888]: conn peer=[2001:db8:1::1]:51826 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3741 tx=1900 ms=3188 lb # [ 76.902952] envoy[888]: conn peer=[2001:db8:1::1]:51828 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=93 lb # [ 76.907389] envoy[888]: conn peer=[2001:db8:1::1]:38668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=77 lb # [ 76.912663] envoy[888]: conn peer=[2001:db8:1::1]:38662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3725 tx=1884 ms=1636 lb # [ 76.917205] envoy[888]: conn peer=[2001:db8:1::4]:60712 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=61818 tx=3056 ms=58133 worker2 # [ 77.057012] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 77.060842] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 77.069517] systemd[1]: nix-grpc-daemon.service: Consumed 1.819s CPU time over 59.271s wall clock time, 53.9M memory peak, 51.2K incoming IP traffic, 146.9K outgoing IP traffic. worker2 # [ 77.084573] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 77.091918] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 77.093924] systemd[1]: nix-grpc-daemon.socket: Consumed 520us CPU time over 1min 7.725s wall clock time, 256K memory peak, 58.2K incoming IP traffic, 51.6K outgoing IP traffic. lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2 worker1 # [ 77.418922] nix-grpc-daemon[713]: ts=2026-09-21T14:44:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:52934 worker1 # [ 77.427668] nix-grpc-daemon[713]: ts=2026-09-21T14:44:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:52934 client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.26 seconds) worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds) worker2 # [ 77.538426] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 77.631237] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 78.098821] systemd[1]: Starting scheduler-token.service... worker1 # [ 78.139268] nix-grpc-daemon[713]: ts=2026-09-21T14:44:53Z level=info event=signal_exit worker2 # [ 78.194464] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 78.204984] systemd[1]: Finished scheduler-token.service. worker2 # [ 78.206958] systemd[1]: scheduler-token.service: Consumed 62ms CPU time over 100ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 78.214941] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 78.349223] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 78.351420] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 78.355399] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 78.361053] nix-grpc-daemon[1034]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 78.362469] nix-grpc-daemon[1034]: E0000 00:00:1790001893.963478 1034 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 78.404137] nix-grpc-daemon[1034]: E0000 00:00:1790001893.969710 1034 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 78.406406] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 78.407980] nix-grpc-daemon[1034]: E0000 00:00:1790001893.981178 1075 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 78.419070] nix-grpc-daemon[1034]: E0000 00:00:1790001893.999885 1075 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 78.421893] nix-grpc-daemon[1034]: E0000 00:00:1790001894.000038 1075 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 78.424433] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 78.425260] nix-grpc-daemon[1034]: E0000 00:00:1790001894.021233 1075 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 78.432715] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 78.434289] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 78.634527] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 78.639567] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 78.840685] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 78.845781] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 79.046791] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 79.051904] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 79.128450] niks3-server[979]: 2026/09/21 14:44:54 INFO lead: released remote=192.168.1.3:49934 worker2 # [ 79.253031] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:54Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 79.258023] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 79.460306] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 79.465310] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 79.666289] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 79.671418] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 79.872661] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 79.877725] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 80.078833] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 80.083952] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 80.125255] nix-grpc-daemon[713]: ts=2026-09-21T14:44:55Z level=info event=scheduler_restarting worker1 # [ 80.130121] nix-grpc-daemon[713]: ts=2026-09-21T14:44:55Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 80.135669] nix-grpc-daemon[713]: ts=2026-09-21T14:44:55Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:52918 worker1 # [ 80.140884] nix-grpc-daemon[713]: ts=2026-09-21T14:44:55Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 80.174036] nix-daemon[852]: reaped child process 854, status = succeeded worker1 # [ 80.236504] nix-grpc-daemon[713]: E0000 00:00:1790001895.844796 758 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true worker1 # [ 80.257455] nix-grpc-daemon[713]: E0000 00:00:1790001895.865880 758 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker2 # [ 80.285134] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 80.290183] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 80.349069] nix-grpc-daemon[713]: ts=2026-09-21T14:44:55Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 80.492857] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 80.496961] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 80.693590] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 80.698432] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 80.899396] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 80.904597] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 81.024315] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 81.029114] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 81.035011] systemd[1]: nix-grpc-daemon.service: Consumed 3.946s CPU time over 1min 4.138s wall clock time, 68.2M memory peak, 102.5K incoming IP traffic, 211.7K outgoing IP traffic. worker1 # [ 81.053550] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 81.066123] systemd[1]: Closed Nix worker-protocol over gRPC. lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 81.069484] systemd[1]: nix-grpc-daemon.socket: Consumed 544us CPU time over 1min 11.082s wall clock time, 256K memory peak, 95.8K incoming IP traffic, 78.5K outgoing IP traffic. worker2 # [ 81.105867] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 81.111009] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2 worker2 # [ 81.312069] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 81.317120] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 81.340152] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 81.355787] nix-daemon[838]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) worker2 # [ 81.384354] nix-grpc-daemon[1034]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 69 ms (attempt 1/5) worker2 # [ 81.455057] nix-grpc-daemon[1034]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 147 ms (attempt 2/5) worker2 # [ 81.519931] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 81.524155] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 81.604072] nix-grpc-daemon[1034]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 123 ms (attempt 3/5) worker2 # [ 81.720867] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 81.725862] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 81.733113] nix-grpc-daemon[1034]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 500 ms (attempt 4/5) worker2 # [ 81.926969] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 81.932194] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 82.133234] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 82.138278] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 82.243517] nix-grpc-daemon[1034]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 82.249603] nix-grpc-daemon[1034]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 82.273074] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39022 client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 82.290097] nix-daemon[838]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) worker2 # [ 82.324714] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:57Z level=info event=niks3_push_started pid=1094 worker2 # [ 82.348449] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 82.351131] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 82.489196] nix-grpc-daemon[1094]: time=2026-09-21T14:44:58.091Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 82.544774] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 82.547681] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 82.600452] nix-daemon[838]: accepted connection from pid 1106, user nix-grpc-daemon (trusted) worker2 # [ 82.618472] nix-daemon[838]: reaped child process 1109, status = succeeded lb # [ 82.649367] niks3-server[979]: 2026/09/21 14:44:58 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 82.660624] niks3-server[979]: 2026/09/21 14:44:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 82.646228] nix-grpc-daemon[1094]: time=2026-09-21T14:44:58.247Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 82.647682] nix-grpc-daemon[1094]: time=2026-09-21T14:44:58.247Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)" worker2 # [ 82.650555] nix-grpc-daemon[1094]: time=2026-09-21T14:44:58.247Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)" lb # [ 82.714016] niks3-server[979]: 2026/09/21 14:44:58 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 82.731024] niks3-server[979]: 2026/09/21 14:44:58 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst lb # [ 82.741151] niks3-server[979]: 2026/09/21 14:44:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign worker2 # [ 82.724155] nix-grpc-daemon[1094]: time=2026-09-21T14:44:58.326Z level=INFO msg="Uploading 3 narinfos" lb # [ 82.750239] niks3-server[979]: 2026/09/21 14:44:58 INFO Signed narinfos id=25 count=1 lb # [ 82.756096] niks3-server[979]: 2026/09/21 14:44:58 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls lb # [ 82.759439] niks3-server[979]: 2026/09/21 14:44:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign lb # [ 82.769286] niks3-server[979]: 2026/09/21 14:44:58 INFO Signed narinfos id=26 count=2 lb # [ 82.771281] niks3-server[979]: 2026/09/21 14:44:58 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls worker2 # [ 82.748107] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 82.753132] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 82.790236] niks3-server[979]: 2026/09/21 14:44:58 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo lb # [ 82.798444] niks3-server[979]: 2026/09/21 14:44:58 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete lb # [ 82.806078] niks3-server[979]: 2026/09/21 14:44:58 INFO Completed upload id=25 lb # [ 82.808570] niks3-server[979]: 2026/09/21 14:44:58 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete worker2 # [ 82.783554] nix-grpc-daemon[1094]: time=2026-09-21T14:44:58.386Z level=INFO msg="Upload complete. (290ms)" lb # [ 82.810964] niks3-server[979]: 2026/09/21 14:44:58 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 82.815817] niks3-server[979]: 2026/09/21 14:44:58 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 82.819650] niks3-server[979]: 2026/09/21 14:44:58 INFO Completed upload id=26 worker2 # [ 82.796117] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39022 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 82.810643] nix-daemon[838]: reaped child process 1092, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.71 seconds) worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds) worker1 # [ 82.951103] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 82.960972] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 82.954210] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 82.959411] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 83.107173] nix-grpc-daemon[1268]: ts=2026-09-21T14:44:58Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 83.112731] nix-grpc-daemon[1268]: ts=2026-09-21T14:44:58Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 83.114617] nix-grpc-daemon[1268]: ts=2026-09-21T14:44:58Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 83.135509] nix-grpc-daemon[1268]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 83.142893] niks3-server[979]: 2026/09/21 14:44:58 INFO lead: acquired remote=192.168.1.3:43654 lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 83.156126] nix-grpc-daemon[1268]: E0000 00:00:1790001898.727148 1268 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 83.175005] nix-grpc-daemon[1268]: E0000 00:00:1790001898.729737 1268 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 83.177839] nix-grpc-daemon[1268]: ts=2026-09-21T14:44:58Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 83.179647] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 83.180551] nix-grpc-daemon[1268]: ts=2026-09-21T14:44:58Z level=info event=scheduler_take_over worker1 # [ 83.181739] nix-grpc-daemon[1268]: E0000 00:00:1790001898.747163 1307 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 83.160509] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 83.184221] nix-grpc-daemon[1268]: E0000 00:00:1790001898.768272 1307 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 83.190824] nix-grpc-daemon[1268]: E0000 00:00:1790001898.768435 1307 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 83.197523] nix-grpc-daemon[1268]: E0000 00:00:1790001898.782794 1307 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 83.199792] nix-grpc-daemon[1268]: ts=2026-09-21T14:44:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 83.201159] nix-grpc-daemon[1268]: ts=2026-09-21T14:44:58Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:42912 worker1 # [ 83.202660] nix-grpc-daemon[1268]: ts=2026-09-21T14:44:58Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 83.203859] nix-grpc-daemon[1268]: ts=2026-09-21T14:44:58Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:42912 worker1 # [ 83.205623] nix-grpc-daemon[1268]: ts=2026-09-21T14:44:58Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 83.207316] nix-grpc-daemon[1268]: ts=2026-09-21T14:44:58Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.07 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.34 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb # [ 83.340766] niks3-server[979]: 2026/09/21 14:44:58 INFO Shutdown signal received, draining in-flight requests timeout=10s worker1 # [ 83.340603] nix-grpc-daemon[1268]: ts=2026-09-21T14:44:58Z level=info event=scheduler_lock_lost status=200 worker2 # [ 83.320189] nix-grpc-daemon[1034]: ts=2026-09-21T14:44:58Z level=info event=scheduler_lock_lost status=200 lb # [ 83.350076] niks3-server[979]: 2026/09/21 14:44:58 INFO lead: released remote=192.168.1.3:43654 lb # [ 83.351617] systemd[1]: Stopping niks3 server... lb # [ 86.884489] envoy[888]: conn peer=[2001:db8:1::1]:38678 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=31 lb # [ 86.884799] envoy[888]: conn peer=[2001:db8:1::3]:33518 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13257 tx=3626 ms=63149 lb # [ 86.884940] envoy[888]: conn peer=[2001:db8:1::1]:50472 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1008 tx=305 ms=1469 lb # [ 88.076466] systemd[1]: niks3.service: Deactivated successfully. lb # [ 88.081817] systemd[1]: Stopped niks3 server. lb # [ 88.086893] systemd[1]: niks3.service: Consumed 493ms CPU time over 1min 4.174s wall clock time, 25.5M memory peak, 68.2K incoming IP traffic, 98.9K outgoing IP traffic. lb # [ 88.110756] systemd[1]: Starting niks3 server... lb # [ 88.220503] niks3-server[1112]: 2026/09/21 14:45:03 goose: no migrations to run. current version: 20260920000000 lb # [ 88.227788] niks3-server[1112]: 2026/09/21 14:45:03 OK 1_commit_pending_closure.sql (7.75ms) lb # [ 88.232144] niks3-server[1112]: 2026/09/21 14:45:03 OK 2_object_stats_trigger.sql (3.94ms) lb # [ 88.233466] niks3-server[1112]: 2026/09/21 14:45:03 goose: up to current file version: 2 lb # [ 88.235294] niks3-server[1112]: 2026/09/21 14:45:03 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 88.247445] niks3-server[1112]: 2026/09/21 14:45:03 INFO Read proxy enabled — serving cache objects from S3 lb # [ 88.254556] niks3-server[1112]: 2026/09/21 14:45:03 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 88.258234] niks3-server[1112]: 2026/09/21 14:45:03 INFO systemd watchdog enabled interval=15s lb: waiting for unit niks3.service lb # [ 88.259793] niks3-server[1112]: 2026/09/21 14:45:03 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 88.262329] niks3-server[1112]: 2026/09/21 14:45:03 INFO lead: acquired remote=192.168.1.3:43666 lb # [ 88.263547] systemd[1]: Started niks3 server. lb: (finished: waiting for unit niks3.service, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2 worker1 # [ 88.749036] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:42928 worker1 # [ 88.764271] nix-daemon[852]: accepted connection from pid 1268, user nix-grpc-daemon (trusted) client # don't know how to build these paths: client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker1 # [ 88.788731] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42928 worker1 # [ 88.805462] nix-grpc-daemon[1268]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 1/5) worker1 # [ 88.836503] nix-grpc-daemon[1268]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 2/5) worker1 # [ 88.893680] nix-grpc-daemon[1268]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 167 ms (attempt 3/5) worker1 # [ 89.062757] nix-grpc-daemon[1268]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 219 ms (attempt 4/5) worker1 # [ 89.286617] nix-grpc-daemon[1268]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 89.292346] nix-grpc-daemon[1268]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 89.337464] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:04Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=0 worker1 # [ 89.344113] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:04Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 89.384206] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42928 worker1 # [ 89.401951] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42928 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'... worker1 # [ 89.416164] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42928 worker1 # [ 89.432383] nix-daemon[852]: accepted connection from pid 1268, user nix-grpc-daemon (trusted) worker1 # [ 89.460631] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:05Z level=info event=niks3_push_started pid=1340 worker1 # [ 89.612940] nix-grpc-daemon[1340]: time=2026-09-21T14:45:05.220Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 89.718480] nix-daemon[852]: accepted connection from pid 1353, user nix-grpc-daemon (trusted) worker1 # [ 89.736670] nix-daemon[852]: reaped child process 1356, status = succeeded worker1 # [ 89.836776] nix-daemon[852]: accepted connection from pid 1358, user nix-grpc-daemon (trusted) worker1 # [ 89.853097] nix-daemon[852]: reaped child process 1361, status = succeeded lb # [ 89.863537] niks3-server[1112]: 2026/09/21 14:45:05 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 89.874938] nix-grpc-daemon[1340]: time=2026-09-21T14:45:05.482Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 89.876730] nix-grpc-daemon[1340]: time=2026-09-21T14:45:05.482Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)" lb # [ 89.923243] niks3-server[1112]: 2026/09/21 14:45:05 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst lb # [ 89.937978] niks3-server[1112]: 2026/09/21 14:45:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign worker1 # [ 89.941234] nix-grpc-daemon[1340]: time=2026-09-21T14:45:05.549Z level=INFO msg="Uploading 1 narinfos" lb # [ 89.949653] niks3-server[1112]: 2026/09/21 14:45:05 INFO Signed narinfos id=27 count=1 lb # [ 89.955672] niks3-server[1112]: 2026/09/21 14:45:05 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls lb # [ 89.969255] niks3-server[1112]: 2026/09/21 14:45:05 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete lb # [ 89.982548] niks3-server[1112]: 2026/09/21 14:45:05 INFO Completed upload id=27 worker1 # [ 89.978925] nix-grpc-daemon[1340]: time=2026-09-21T14:45:05.587Z level=INFO msg="Upload complete. (366ms)" worker1 # [ 89.996258] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42928 duration_s=0 paths=1 nar_bytes_in=544 worker1 # [ 90.010427] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42928 worker1 # [ 90.020643] nix-grpc-daemon[1340]: time=2026-09-21T14:45:05.599Z level=WARN msg="Failed to register uploaded object" key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo error="server returned 404: object is not pending upload\n" worker1 # [ 90.032329] nix-daemon[852]: accepted connection from pid 1268, user nix-grpc-daemon (trusted) worker1 # [ 90.038605] nix-daemon[852]: reaped child process 1338, status = succeeded client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051... client # farm-a> node-a: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker1 # [ 90.045869] nix-daemon[852]: accepted connection from pid 1268, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker1 # [ 90.685514] nix-daemon[852]: reaped child process 1365, status = succeeded worker1 # [ 90.832831] nix-daemon[852]: accepted connection from pid 1381, user nix-grpc-daemon (trusted) worker1 # [ 90.851724] nix-daemon[852]: reaped child process 1384, status = succeeded lb # [ 90.861406] niks3-server[1112]: 2026/09/21 14:45:06 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 90.872654] nix-grpc-daemon[1340]: time=2026-09-21T14:45:06.479Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 90.878661] nix-grpc-daemon[1340]: time=2026-09-21T14:45:06.479Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)" lb # [ 90.924686] niks3-server[1112]: 2026/09/21 14:45:06 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker1 # [ 90.931828] nix-grpc-daemon[1340]: time=2026-09-21T14:45:06.538Z level=INFO msg="Uploading 1 narinfos" lb # [ 90.937146] niks3-server[1112]: 2026/09/21 14:45:06 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst lb # [ 90.945801] niks3-server[1112]: 2026/09/21 14:45:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign lb # [ 90.955228] niks3-server[1112]: 2026/09/21 14:45:06 INFO Signed narinfos id=28 count=1 lb # [ 90.960544] niks3-server[1112]: 2026/09/21 14:45:06 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls worker1 # [ 90.955877] nix-grpc-daemon[1340]: time=2026-09-21T14:45:06.564Z level=INFO msg="Upload complete. (298ms)" lb # [ 90.963788] niks3-server[1112]: 2026/09/21 14:45:06 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 90.969261] niks3-server[1112]: 2026/09/21 14:45:06 INFO Completed upload id=28 lb # [ 90.972191] niks3-server[1112]: 2026/09/21 14:45:06 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo worker1 # [ 90.976663] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42928 duration_s=0 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 worker1 # [ 90.995275] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:06Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=0 worker1 # [ 90.999990] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:06Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 91.004879] nix-daemon[852]: reaped child process 1363, status = succeeded worker1 # [ 91.020515] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42928 worker1 # [ 91.035196] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42928 worker1 # [ 91.050404] nix-grpc-daemon[1340]: time=2026-09-21T14:45:06.658Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'... worker1 # [ 91.060597] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42928 worker1 # [ 91.074480] nix-daemon[852]: accepted connection from pid 1268, user nix-grpc-daemon (trusted) worker1 # [ 91.225905] nix-daemon[852]: accepted connection from pid 1390, user nix-grpc-daemon (trusted) worker1 # [ 91.245918] nix-daemon[852]: reaped child process 1393, status = succeeded worker1 # [ 91.346228] nix-daemon[852]: accepted connection from pid 1395, user nix-grpc-daemon (trusted) worker1 # [ 91.362960] nix-daemon[852]: reaped child process 1398, status = succeeded lb # [ 91.373973] niks3-server[1112]: 2026/09/21 14:45:06 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 91.380094] nix-grpc-daemon[1340]: time=2026-09-21T14:45:06.988Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 91.381545] nix-grpc-daemon[1340]: time=2026-09-21T14:45:06.988Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)" lb # [ 91.406012] niks3-server[1112]: 2026/09/21 14:45:07 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst worker1 # [ 91.415349] nix-grpc-daemon[1340]: time=2026-09-21T14:45:07.023Z level=INFO msg="Uploading 1 narinfos" lb # [ 91.420616] niks3-server[1112]: 2026/09/21 14:45:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign lb # [ 91.434195] niks3-server[1112]: 2026/09/21 14:45:07 INFO Signed narinfos id=29 count=1 lb # [ 91.439593] niks3-server[1112]: 2026/09/21 14:45:07 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls lb # [ 91.451252] niks3-server[1112]: 2026/09/21 14:45:07 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete lb # [ 91.457184] niks3-server[1112]: 2026/09/21 14:45:07 INFO Completed upload id=29 worker1 # [ 91.453360] nix-grpc-daemon[1340]: time=2026-09-21T14:45:07.061Z level=INFO msg="Upload complete. (363ms)" lb # [ 91.462699] niks3-server[1112]: 2026/09/21 14:45:07 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo worker1 # [ 91.460157] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42928 duration_s=0 paths=1 nar_bytes_in=656 worker1 # [ 91.474147] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42928 worker1 # [ 91.480425] nix-daemon[852]: accepted connection from pid 1268, user nix-grpc-daemon (trusted) worker1 # [ 91.485259] nix-daemon[852]: reaped child process 1388, status = succeeded client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051... client # farm-b> node-a: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker1 # [ 91.498679] nix-daemon[852]: accepted connection from pid 1268, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker1 # [ 92.549175] nix-daemon[852]: reaped child process 1402, status = succeeded worker1 # [ 92.647370] nix-daemon[852]: accepted connection from pid 1418, user nix-grpc-daemon (trusted) worker1 # [ 92.663558] nix-daemon[852]: reaped child process 1421, status = succeeded lb # [ 92.675448] niks3-server[1112]: 2026/09/21 14:45:08 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 92.681774] nix-grpc-daemon[1340]: time=2026-09-21T14:45:08.289Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 92.683248] nix-grpc-daemon[1340]: time=2026-09-21T14:45:08.289Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)" lb # [ 92.718760] niks3-server[1112]: 2026/09/21 14:45:08 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst worker1 # [ 92.729465] nix-grpc-daemon[1340]: time=2026-09-21T14:45:08.337Z level=INFO msg="Uploading 1 narinfos" lb # [ 92.735350] niks3-server[1112]: 2026/09/21 14:45:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign lb # [ 92.748574] niks3-server[1112]: 2026/09/21 14:45:08 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv lb # [ 92.751595] niks3-server[1112]: 2026/09/21 14:45:08 INFO Signed narinfos id=30 count=1 worker1 # [ 92.760116] nix-grpc-daemon[1340]: time=2026-09-21T14:45:08.368Z level=INFO msg="Upload complete. (251ms)" lb # [ 92.766622] niks3-server[1112]: 2026/09/21 14:45:08 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 92.775965] niks3-server[1112]: 2026/09/21 14:45:08 INFO Completed upload id=30 lb # [ 92.777720] niks3-server[1112]: 2026/09/21 14:45:08 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo worker1 # [ 92.776936] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42928 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 lb # [ 92.784773] niks3-server[1112]: 2026/09/21 14:45:08 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls worker1 # [ 92.793443] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:08Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=0 worker1 # [ 92.798086] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:08Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 92.802853] nix-daemon[852]: reaped child process 1400, status = succeeded worker1 # [ 92.820233] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42928 worker1 # [ 92.833893] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:42928 worker1 # [ 92.848833] nix-grpc-daemon[1340]: time=2026-09-21T14:45:08.457Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'... worker1 # [ 92.858538] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42928 worker1 # [ 92.872418] nix-daemon[852]: accepted connection from pid 1268, user nix-grpc-daemon (trusted) worker1 # [ 93.017381] nix-daemon[852]: accepted connection from pid 1427, user nix-grpc-daemon (trusted) worker1 # [ 93.034450] nix-daemon[852]: reaped child process 1430, status = succeeded worker1 # [ 93.138100] nix-daemon[852]: accepted connection from pid 1432, user nix-grpc-daemon (trusted) worker1 # [ 93.154096] nix-daemon[852]: reaped child process 1435, status = succeeded lb # [ 93.163812] niks3-server[1112]: 2026/09/21 14:45:08 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 93.173335] nix-grpc-daemon[1340]: time=2026-09-21T14:45:08.781Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 93.174785] nix-grpc-daemon[1340]: time=2026-09-21T14:45:08.781Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)" lb # [ 93.199243] niks3-server[1112]: 2026/09/21 14:45:08 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst worker1 # [ 93.208805] nix-grpc-daemon[1340]: time=2026-09-21T14:45:08.817Z level=INFO msg="Uploading 1 narinfos" lb # [ 93.216570] niks3-server[1112]: 2026/09/21 14:45:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign lb # [ 93.227333] niks3-server[1112]: 2026/09/21 14:45:08 INFO Signed narinfos id=31 count=1 lb # [ 93.234265] niks3-server[1112]: 2026/09/21 14:45:08 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls worker1 # [ 93.242008] nix-grpc-daemon[1340]: time=2026-09-21T14:45:08.850Z level=INFO msg="Upload complete. (355ms)" lb # [ 93.246798] niks3-server[1112]: 2026/09/21 14:45:08 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete worker1 # [ 93.248242] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42928 duration_s=0 paths=1 nar_bytes_in=784 lb # [ 93.256317] niks3-server[1112]: 2026/09/21 14:45:08 INFO Completed upload id=31 lb # [ 93.259004] niks3-server[1112]: 2026/09/21 14:45:08 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo worker1 # [ 93.262524] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42928 worker1 # [ 93.269652] nix-daemon[852]: accepted connection from pid 1268, user nix-grpc-daemon (trusted) worker1 # [ 93.274943] nix-daemon[852]: reaped child process 1425, status = succeeded client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051... worker1 # [ 93.286661] nix-daemon[852]: accepted connection from pid 1268, user nix-grpc-daemon (trusted) client # farm-top> node-a: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker1 # [ 94.368123] nix-daemon[852]: reaped child process 1439, status = succeeded worker1 # [ 94.469240] nix-daemon[852]: accepted connection from pid 1455, user nix-grpc-daemon (trusted) worker1 # [ 94.489294] nix-daemon[852]: reaped child process 1458, status = succeeded lb # [ 94.499853] niks3-server[1112]: 2026/09/21 14:45:10 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 94.507424] nix-grpc-daemon[1340]: time=2026-09-21T14:45:10.114Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 94.509380] nix-grpc-daemon[1340]: time=2026-09-21T14:45:10.114Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)" lb # [ 94.537525] niks3-server[1112]: 2026/09/21 14:45:10 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv lb # [ 94.549065] niks3-server[1112]: 2026/09/21 14:45:10 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst worker1 # [ 94.564984] nix-grpc-daemon[1340]: time=2026-09-21T14:45:10.173Z level=INFO msg="Uploading 1 narinfos" lb # [ 94.562430] niks3-server[1112]: 2026/09/21 14:45:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign lb # [ 94.573290] niks3-server[1112]: 2026/09/21 14:45:10 INFO Signed narinfos id=32 count=1 lb # [ 94.579707] niks3-server[1112]: 2026/09/21 14:45:10 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls lb # [ 94.593406] niks3-server[1112]: 2026/09/21 14:45:10 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 94.600487] niks3-server[1112]: 2026/09/21 14:45:10 INFO Completed upload id=32 worker1 # [ 94.596056] nix-grpc-daemon[1340]: time=2026-09-21T14:45:10.204Z level=INFO msg="Upload complete. (297ms)" lb # [ 94.603526] niks3-server[1112]: 2026/09/21 14:45:10 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo worker1 # [ 94.609346] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42928 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 worker1 # [ 94.629507] nix-daemon[852]: reaped child process 1437, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.13 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 11.62 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1 # [ 95.074234] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 95.133324] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:10Z level=info event=signal_exit lb # [ 95.264714] niks3-server[1112]: 2026/09/21 14:45:10 INFO lead: released remote=192.168.1.3:43666 worker2 # [ 96.242969] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:11Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 96.267000] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:11Z level=info event=scheduler_restarting worker1 # [ 96.271876] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:11Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 96.276672] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:42912 worker1 # [ 96.281955] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:11Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 96.287233] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:11Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:42912 worker1 # [ 96.296507] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:11Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 96.313766] nix-daemon[852]: reaped child process 1330, status = succeeded worker2 # [ 96.442535] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 96.447961] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 96.482950] nix-grpc-daemon[1268]: ts=2026-09-21T14:45:12Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 96.649188] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 96.654315] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 96.855456] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 96.861062] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 96.888576] envoy[888]: conn peer=[2001:db8:1::1]:50488 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9710 tx=4086 ms=5880 lb # [ 96.888819] envoy[888]: conn peer=[2001:db8:1::3]:36964 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1065 tx=425 ms=13286 worker2 # [ 97.062102] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 97.067178] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 97.178549] nix-grpc-daemon[1268]: E0000 00:00:1790001912.786818 1313 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true worker1 # [ 97.178955] nix-grpc-daemon[1268]: E0000 00:00:1790001912.787041 1313 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2 # [ 97.270004] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 97.274177] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 97.470780] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 97.475977] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 97.676971] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 97.682193] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 97.883104] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 97.888455] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 98.089599] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 98.094659] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 98.227951] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:13Z level=info event=scheduler_take_over lb # [ 98.253166] niks3-server[1112]: 2026/09/21 14:45:13 INFO lead: acquired remote=192.168.1.4:59422 worker2 # [ 98.297496] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 98.301581] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 98.416696] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 98.421359] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 98.428399] systemd[1]: nix-grpc-daemon.service: Consumed 1.484s CPU time over 15.459s wall clock time, 38M memory peak, 36.1K incoming IP traffic, 56.4K outgoing IP traffic. worker1 # [ 98.442606] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 98.454956] systemd[1]: Closed Nix worker-protocol over gRPC. worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker1 # [ 98.458970] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 15.508s wall clock time, 256K memory peak, 36.2K incoming IP traffic, 26.5K outgoing IP traffic. worker2 # [ 98.498124] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:14Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 98.520319] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:14Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 98.523119] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:14Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:50952 worker2 # [ 98.533194] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:14Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.10 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2 worker2 # [ 98.785730] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:39022 client # don't know how to build these paths: client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker2 # [ 98.795736] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 98.823533] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:14Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=0 worker2 # [ 98.830165] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:14Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 98.874198] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 98.889462] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39022 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'... worker2 # [ 98.905224] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 98.920001] nix-daemon[838]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) worker2 # [ 99.078651] nix-daemon[838]: accepted connection from pid 1129, user nix-grpc-daemon (trusted) worker2 # [ 99.097273] nix-daemon[838]: reaped child process 1132, status = succeeded worker2 # [ 99.204156] nix-daemon[838]: accepted connection from pid 1134, user nix-grpc-daemon (trusted) worker2 # [ 99.219665] nix-daemon[838]: reaped child process 1137, status = succeeded lb # [ 99.250321] niks3-server[1112]: 2026/09/21 14:45:14 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 99.238639] nix-grpc-daemon[1094]: time=2026-09-21T14:45:14.841Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 99.240162] nix-grpc-daemon[1094]: time=2026-09-21T14:45:14.841Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)" lb # [ 99.288773] niks3-server[1112]: 2026/09/21 14:45:14 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst worker2 # [ 99.275445] nix-grpc-daemon[1094]: time=2026-09-21T14:45:14.878Z level=INFO msg="Uploading 1 narinfos" lb # [ 99.302748] niks3-server[1112]: 2026/09/21 14:45:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign lb # [ 99.314940] niks3-server[1112]: 2026/09/21 14:45:14 INFO Signed narinfos id=33 count=1 lb # [ 99.321368] niks3-server[1112]: 2026/09/21 14:45:14 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls worker2 # [ 99.309895] nix-grpc-daemon[1094]: time=2026-09-21T14:45:14.912Z level=INFO msg="Upload complete. (372ms)" lb # [ 99.334502] niks3-server[1112]: 2026/09/21 14:45:14 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete lb # [ 99.343800] niks3-server[1112]: 2026/09/21 14:45:14 INFO Completed upload id=33 worker2 # [ 99.319765] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39022 duration_s=0 paths=1 nar_bytes_in=520 lb # [ 99.350174] niks3-server[1112]: 2026/09/21 14:45:14 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo worker2 # [ 99.333910] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 99.342475] nix-daemon[838]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) worker2 # [ 99.350215] nix-daemon[838]: reaped child process 1127, status = succeeded client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051... worker2 # [ 99.357758] nix-daemon[838]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker2 # [ 100.632900] nix-daemon[838]: reaped child process 1141, status = succeeded worker2 # [ 100.738911] nix-daemon[838]: accepted connection from pid 1157, user nix-grpc-daemon (trusted) worker2 # [ 100.755897] nix-daemon[838]: reaped child process 1160, status = succeeded lb # [ 100.789573] niks3-server[1112]: 2026/09/21 14:45:16 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 100.776427] nix-grpc-daemon[1094]: time=2026-09-21T14:45:16.377Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 100.778269] nix-grpc-daemon[1094]: time=2026-09-21T14:45:16.377Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)" lb # [ 100.826792] niks3-server[1112]: 2026/09/21 14:45:16 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv lb # [ 100.840575] niks3-server[1112]: 2026/09/21 14:45:16 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst worker2 # [ 100.830120] nix-grpc-daemon[1094]: time=2026-09-21T14:45:16.432Z level=INFO msg="Uploading 1 narinfos" lb # [ 100.857046] niks3-server[1112]: 2026/09/21 14:45:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign lb # [ 100.866804] niks3-server[1112]: 2026/09/21 14:45:16 INFO Signed narinfos id=34 count=1 lb # [ 100.872227] niks3-server[1112]: 2026/09/21 14:45:16 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls lb # [ 100.877955] niks3-server[1112]: 2026/09/21 14:45:16 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete lb # [ 100.888057] niks3-server[1112]: 2026/09/21 14:45:16 INFO Completed upload id=34 worker2 # [ 100.862747] nix-grpc-daemon[1094]: time=2026-09-21T14:45:16.465Z level=INFO msg="Upload complete. (291ms)" lb # [ 100.891421] niks3-server[1112]: 2026/09/21 14:45:16 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo worker2 # [ 100.884850] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39022 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 worker2 # [ 100.900332] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:16Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=0 worker2 # [ 100.905531] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:16Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 100.909673] nix-daemon[838]: reaped child process 1139, status = succeeded worker2 # [ 100.927702] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 100.941779] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 100.956893] nix-grpc-daemon[1094]: time=2026-09-21T14:45:16.559Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'... worker2 # [ 100.966654] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 100.981151] nix-daemon[838]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) worker2 # [ 101.133127] nix-daemon[838]: accepted connection from pid 1166, user nix-grpc-daemon (trusted) worker2 # [ 101.152320] nix-daemon[838]: reaped child process 1169, status = succeeded worker2 # [ 101.252560] nix-daemon[838]: accepted connection from pid 1171, user nix-grpc-daemon (trusted) worker2 # [ 101.271675] nix-daemon[838]: reaped child process 1174, status = succeeded lb # [ 101.295174] niks3-server[1112]: 2026/09/21 14:45:16 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 101.281025] nix-grpc-daemon[1094]: time=2026-09-21T14:45:16.883Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 101.282485] nix-grpc-daemon[1094]: time=2026-09-21T14:45:16.883Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)" lb # [ 101.328031] niks3-server[1112]: 2026/09/21 14:45:16 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst worker2 # [ 101.316923] nix-grpc-daemon[1094]: time=2026-09-21T14:45:16.919Z level=INFO msg="Uploading 1 narinfos" lb # [ 101.345081] niks3-server[1112]: 2026/09/21 14:45:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign lb # [ 101.355365] niks3-server[1112]: 2026/09/21 14:45:16 INFO Signed narinfos id=35 count=1 lb # [ 101.361062] niks3-server[1112]: 2026/09/21 14:45:16 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls lb # [ 101.370488] niks3-server[1112]: 2026/09/21 14:45:16 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 101.376357] niks3-server[1112]: 2026/09/21 14:45:16 INFO Completed upload id=35 worker2 # [ 101.351613] nix-grpc-daemon[1094]: time=2026-09-21T14:45:16.954Z level=INFO msg="Upload complete. (355ms)" lb # [ 101.379487] niks3-server[1112]: 2026/09/21 14:45:16 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo worker2 # [ 101.360273] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39022 duration_s=0 paths=1 nar_bytes_in=640 worker2 # [ 101.373108] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 101.382020] nix-daemon[838]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) worker2 # [ 101.391672] nix-daemon[838]: reaped child process 1164, status = succeeded client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051... client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker2 # [ 101.399946] nix-daemon[838]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker2 # [ 102.859204] nix-daemon[838]: reaped child process 1178, status = succeeded worker2 # [ 102.964052] nix-daemon[838]: accepted connection from pid 1194, user nix-grpc-daemon (trusted) worker2 # [ 102.981800] nix-daemon[838]: reaped child process 1197, status = succeeded lb # [ 103.014035] niks3-server[1112]: 2026/09/21 14:45:18 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 103.003805] nix-grpc-daemon[1094]: time=2026-09-21T14:45:18.605Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 103.005774] nix-grpc-daemon[1094]: time=2026-09-21T14:45:18.605Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)" lb # [ 103.051845] niks3-server[1112]: 2026/09/21 14:45:18 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst lb # [ 103.070279] niks3-server[1112]: 2026/09/21 14:45:18 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker2 # [ 103.055142] nix-grpc-daemon[1094]: time=2026-09-21T14:45:18.657Z level=INFO msg="Uploading 1 narinfos" lb # [ 103.080776] niks3-server[1112]: 2026/09/21 14:45:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 103.089770] niks3-server[1112]: 2026/09/21 14:45:18 INFO Signed narinfos id=36 count=1 lb # [ 103.091807] niks3-server[1112]: 2026/09/21 14:45:18 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls lb # [ 103.102750] niks3-server[1112]: 2026/09/21 14:45:18 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete lb # [ 103.108959] niks3-server[1112]: 2026/09/21 14:45:18 INFO Completed upload id=36 worker2 # [ 103.084046] nix-grpc-daemon[1094]: time=2026-09-21T14:45:18.686Z level=INFO msg="Upload complete. (273ms)" lb # [ 103.111031] niks3-server[1112]: 2026/09/21 14:45:18 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo worker2 # [ 103.101197] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39022 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 worker2 # [ 103.116377] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:18Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=0 worker2 # [ 103.122674] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:18Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 103.127156] nix-daemon[838]: reaped child process 1176, status = succeeded worker2 # [ 103.144121] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 103.159094] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 103.173320] nix-grpc-daemon[1094]: time=2026-09-21T14:45:18.776Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'... worker2 # [ 103.182968] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 103.196114] nix-daemon[838]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) worker2 # [ 103.348854] nix-daemon[838]: accepted connection from pid 1203, user nix-grpc-daemon (trusted) worker2 # [ 103.367913] nix-daemon[838]: reaped child process 1206, status = succeeded worker2 # [ 103.469329] nix-daemon[838]: accepted connection from pid 1208, user nix-grpc-daemon (trusted) worker2 # [ 103.488117] nix-daemon[838]: reaped child process 1211, status = succeeded lb # [ 103.511477] niks3-server[1112]: 2026/09/21 14:45:19 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 103.497079] nix-grpc-daemon[1094]: time=2026-09-21T14:45:19.099Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 103.498527] nix-grpc-daemon[1094]: time=2026-09-21T14:45:19.099Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)" lb # [ 103.539218] niks3-server[1112]: 2026/09/21 14:45:19 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst lb # [ 103.554277] niks3-server[1112]: 2026/09/21 14:45:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign worker2 # [ 103.536222] nix-grpc-daemon[1094]: time=2026-09-21T14:45:19.138Z level=INFO msg="Uploading 1 narinfos" lb # [ 103.566366] niks3-server[1112]: 2026/09/21 14:45:19 INFO Signed narinfos id=37 count=1 lb # [ 103.572501] niks3-server[1112]: 2026/09/21 14:45:19 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls lb # [ 103.581671] niks3-server[1112]: 2026/09/21 14:45:19 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete worker2 # [ 103.562390] nix-grpc-daemon[1094]: time=2026-09-21T14:45:19.165Z level=INFO msg="Upload complete. (351ms)" lb # [ 103.590234] niks3-server[1112]: 2026/09/21 14:45:19 INFO Completed upload id=37 lb # [ 103.593924] niks3-server[1112]: 2026/09/21 14:45:19 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo worker2 # [ 103.572231] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39022 duration_s=0 paths=1 nar_bytes_in=768 worker2 # [ 103.583025] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 103.590405] nix-daemon[838]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) worker2 # [ 103.595357] nix-daemon[838]: reaped child process 1201, status = succeeded client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051... worker2 # [ 103.609635] nix-daemon[838]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker2 # [ 104.916904] nix-daemon[838]: reaped child process 1215, status = succeeded worker2 # [ 104.986638] nix-daemon[838]: accepted connection from pid 1231, user nix-grpc-daemon (trusted) worker2 # [ 105.006726] nix-daemon[838]: reaped child process 1234, status = succeeded lb # [ 105.037842] niks3-server[1112]: 2026/09/21 14:45:20 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 105.022736] nix-grpc-daemon[1094]: time=2026-09-21T14:45:20.623Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 105.024599] nix-grpc-daemon[1094]: time=2026-09-21T14:45:20.624Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)" lb # [ 105.075001] niks3-server[1112]: 2026/09/21 14:45:20 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker2 # [ 105.065043] nix-grpc-daemon[1094]: time=2026-09-21T14:45:20.667Z level=INFO msg="Uploading 1 narinfos" lb # [ 105.089154] niks3-server[1112]: 2026/09/21 14:45:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign lb # [ 105.100358] niks3-server[1112]: 2026/09/21 14:45:20 INFO Signed narinfos id=38 count=1 lb # [ 105.107693] niks3-server[1112]: 2026/09/21 14:45:20 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst lb # [ 105.120903] niks3-server[1112]: 2026/09/21 14:45:20 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls lb # [ 105.132939] niks3-server[1112]: 2026/09/21 14:45:20 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete worker2 # [ 105.114917] nix-grpc-daemon[1094]: time=2026-09-21T14:45:20.717Z level=INFO msg="Upload complete. (295ms)" lb # [ 105.141185] niks3-server[1112]: 2026/09/21 14:45:20 INFO Completed upload id=38 lb # [ 105.148266] niks3-server[1112]: 2026/09/21 14:45:20 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo worker2 # [ 105.129129] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39022 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 worker2 # [ 105.145890] nix-daemon[838]: reaped child process 1213, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 6.59 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.17 seconds) worker1 # [ 105.431050] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 105.443146] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 105.579760] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:21Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 105.582231] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:21Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 105.586002] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:21Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 105.591266] nix-grpc-daemon[1476]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 105.604071] nix-grpc-daemon[1476]: E0000 00:00:1790001921.199047 1476 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 105.632160] nix-grpc-daemon[1476]: E0000 00:00:1790001921.202578 1476 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 105.634437] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:21Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 105.641680] nix-grpc-daemon[1476]: E0000 00:00:1790001921.211242 1518 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 105.644576] nix-grpc-daemon[1476]: E0000 00:00:1790001921.229620 1518 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 105.647006] nix-grpc-daemon[1476]: E0000 00:00:1790001921.229835 1518 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 105.649462] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 105.650369] nix-grpc-daemon[1476]: E0000 00:00:1790001921.249413 1518 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 105.659535] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:21Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 105.642940] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:21Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:50952 worker2 # [ 105.648549] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:21Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.11 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs client # [ 106.007883] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: d76edc67bf7e450abc6182afa9062073 client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds) client # [ 106.222686] nix[1239]: don't know how to build these paths: client # [ 106.223721] nix[1239]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker2 # [ 106.204947] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 106.218028] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:21Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=0 worker2 # [ 106.225205] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:21Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 106.264860] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 106.282535] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:39022 client # [ 106.315241] nix[1239]: copying 1 paths to 'lb:50051'... worker2 # [ 106.298798] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39022 client # [ 106.317917] nix[1239]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'... worker2 # [ 106.313555] nix-daemon[838]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) worker2 # [ 106.475295] nix-daemon[838]: accepted connection from pid 1248, user nix-grpc-daemon (trusted) worker2 # [ 106.496264] nix-daemon[838]: reaped child process 1251, status = succeeded worker2 # [ 106.604615] nix-daemon[838]: accepted connection from pid 1253, user nix-grpc-daemon (trusted) worker2 # [ 106.625659] nix-daemon[838]: reaped child process 1256, status = succeeded lb # [ 106.649923] niks3-server[1112]: 2026/09/21 14:45:22 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 106.635045] nix-grpc-daemon[1094]: time=2026-09-21T14:45:22.237Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 106.636638] nix-grpc-daemon[1094]: time=2026-09-21T14:45:22.237Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)" lb # [ 106.678325] niks3-server[1112]: 2026/09/21 14:45:22 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst lb # [ 106.692471] niks3-server[1112]: 2026/09/21 14:45:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign worker2 # [ 106.673662] nix-grpc-daemon[1094]: time=2026-09-21T14:45:22.276Z level=INFO msg="Uploading 1 narinfos" lb # [ 106.701803] niks3-server[1112]: 2026/09/21 14:45:22 INFO Signed narinfos id=39 count=1 lb # [ 106.711491] niks3-server[1112]: 2026/09/21 14:45:22 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls worker2 # [ 106.699587] nix-grpc-daemon[1094]: time=2026-09-21T14:45:22.302Z level=INFO msg="Upload complete. (369ms)" lb # [ 106.724262] niks3-server[1112]: 2026/09/21 14:45:22 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete lb # [ 106.733330] niks3-server[1112]: 2026/09/21 14:45:22 INFO Completed upload id=39 worker2 # [ 106.708354] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39022 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 106.738692] niks3-server[1112]: 2026/09/21 14:45:22 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo worker2 # [ 106.716571] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39022 worker2 # [ 106.725792] nix-daemon[838]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) worker2 # [ 106.730260] nix-daemon[838]: reaped child process 1246, status = succeeded client # [ 106.756578] nix[1239]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051... worker2 # [ 106.741723] nix-daemon[838]: accepted connection from pid 1034, user nix-grpc-daemon (trusted) lb # [ 106.887901] envoy[888]: conn peer=[2001:db8:1::1]:35426 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9705 tx=4096 ms=6354 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker1 # [ 108.845778] systemd-journald[273]: Received client request to rotate journal, rotating. worker1 # [ 108.866073] systemd-journald[273]: Vacuuming done, freed 0B of archived journals from /var/log/journal/5cb0941880ae47a1834ad4fcf5bbc469. worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds) worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker2 # [ 108.899640] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds) client: must succeed: sleep 5.0 worker2 # [ 109.381996] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:24Z level=info event=draining client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker2 # [ 114.450913] nix-daemon[838]: reaped child process 1260, status = succeeded worker2 # [ 114.523788] nix-daemon[838]: accepted connection from pid 1301, user nix-grpc-daemon (trusted) worker2 # [ 114.542267] nix-daemon[838]: reaped child process 1304, status = succeeded lb # [ 114.572795] niks3-server[1112]: 2026/09/21 14:45:30 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 114.560059] nix-grpc-daemon[1094]: time=2026-09-21T14:45:30.160Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 114.561760] nix-grpc-daemon[1094]: time=2026-09-21T14:45:30.160Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)" lb # [ 114.612701] niks3-server[1112]: 2026/09/21 14:45:30 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 114.625142] niks3-server[1112]: 2026/09/21 14:45:30 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker2 # [ 114.608145] nix-grpc-daemon[1094]: time=2026-09-21T14:45:30.210Z level=INFO msg="Uploading 1 narinfos" lb # [ 114.638297] niks3-server[1112]: 2026/09/21 14:45:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign lb # [ 114.646324] niks3-server[1112]: 2026/09/21 14:45:30 INFO Signed narinfos id=40 count=1 lb # [ 114.654901] niks3-server[1112]: 2026/09/21 14:45:30 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls lb # [ 114.659483] niks3-server[1112]: 2026/09/21 14:45:30 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete lb # [ 114.666286] niks3-server[1112]: 2026/09/21 14:45:30 INFO Completed upload id=40 lb # [ 114.670795] niks3-server[1112]: 2026/09/21 14:45:30 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo worker2 # [ 114.642029] nix-grpc-daemon[1094]: time=2026-09-21T14:45:30.244Z level=INFO msg="Upload complete. (286ms)" worker2 # [ 114.657467] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39022 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 worker2 # [ 114.674865] nix-daemon[838]: reaped child process 1258, status = succeeded client # [ 114.705896] systemd[1]: rs.service: Deactivated successfully. client # [ 114.711039] systemd[1]: rs.service: Consumed 202ms CPU time over 8.699s wall clock time, 41.2M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic. worker2 # [ 115.385110] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:30Z level=info event=signal_exit lb # [ 116.258034] niks3-server[1112]: 2026/09/21 14:45:31 INFO lead: released remote=192.168.1.4:59422 lb # [ 116.890321] envoy[888]: conn peer=[2001:db8:1::1]:56808 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1430 ms=8464 worker1 # [ 117.255750] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:32Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 117.238986] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:32Z level=info event=scheduler_restarting worker2 # [ 117.239438] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:32Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 117.239566] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:50952 worker2 # [ 117.239695] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:32Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 117.239807] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:50952 worker2 # [ 117.239913] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:32Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 117.287263] nix-daemon[838]: reaped child process 1086, status = succeeded worker1 # [ 117.455835] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 117.466286] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 117.455754] nix-grpc-daemon[1034]: ts=2026-09-21T14:45:33Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 117.464418] nix-grpc-daemon[1034]: E0000 00:00:1790001933.066902 1079 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker2 # [ 117.470936] nix-grpc-daemon[1034]: E0000 00:00:1790001933.067351 1079 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.27 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 117.663601] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 117.669126] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 117.782294] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 117.791157] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 117.796842] systemd[1]: nix-grpc-daemon.service: Consumed 2.157s CPU time over 39.571s wall clock time, 39.1M memory peak, 61.3K incoming IP traffic, 136.2K outgoing IP traffic. worker2 # [ 117.811169] systemd[1]: Starting scheduler-token.service... worker1 # [ 117.869909] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 117.875171] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 117.888488] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 117.893197] systemd[1]: Finished scheduler-token.service. worker2 # [ 117.895007] systemd[1]: scheduler-token.service: Consumed 47ms CPU time over 81ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 117.916382] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 118.028433] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:33Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 118.030671] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:33Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 118.034414] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:33Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 118.040076] nix-grpc-daemon[1314]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 118.041519] nix-grpc-daemon[1314]: E0000 00:00:1790001933.641775 1314 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 118.077112] niks3-server[1112]: 2026/09/21 14:45:33 INFO lead: acquired remote=192.168.1.4:59958 worker1 # [ 118.076257] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 118.081745] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 118.081497] nix-grpc-daemon[1314]: E0000 00:00:1790001933.647936 1314 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 118.084233] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:33Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 118.086001] nix-grpc-daemon[1314]: E0000 00:00:1790001933.653853 1354 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 118.088288] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:33Z level=info event=scheduler_take_over worker2 # [ 118.092135] nix-grpc-daemon[1314]: E0000 00:00:1790001933.669153 1354 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 118.094518] nix-grpc-daemon[1314]: E0000 00:00:1790001933.669292 1354 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 118.097200] nix-grpc-daemon[1314]: E0000 00:00:1790001933.683539 1354 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 118.101716] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 118.102605] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 118.103862] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 118.284471] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:33Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 118.288621] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 118.302999] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:33Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 118.307962] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:33Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 118.484830] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 118.490615] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 118.509057] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 118.514229] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 118.690608] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:34Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 118.695880] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 118.715264] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 118.720739] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:34Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 118.896886] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 118.882149] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:34Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:34934 worker2 # [ 118.887395] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:34Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 118.921716] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 118.935132] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:34Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 118.939655] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:34Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:34934 worker2 # [ 118.945098] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:34Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.97 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid client # [ 119.980979] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: c50a24620b6f47c29c720228249ab9b6 client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds) worker2 # [ 120.173742] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34946 worker2 # [ 120.186791] nix-daemon[838]: accepted connection from pid 1314, user nix-grpc-daemon (trusted) client # [ 120.221429] nix[1275]: don't know how to build these paths: client # [ 120.222935] nix[1275]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker2 # [ 120.216707] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:35Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker2 # [ 120.223458] nix-grpc-daemon[1314]: ts=2026-09-21T14:45:35Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 120.303920] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38582 worker1 # [ 120.317873] nix-daemon[852]: accepted connection from pid 1476, user nix-grpc-daemon (trusted) worker1 # [ 120.341427] nix-grpc-daemon[1476]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5) worker1 # [ 120.357841] nix-grpc-daemon[1476]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 197 ms (attempt 2/5) worker1 # [ 120.556948] nix-grpc-daemon[1476]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 335 ms (attempt 3/5) worker1 # [ 120.893913] nix-grpc-daemon[1476]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 277 ms (attempt 4/5) worker1 # [ 121.175600] nix-grpc-daemon[1476]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 121.184466] nix-grpc-daemon[1476]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 121.202711] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:38582 client # [ 121.211308] nix[1275]: copying 1 paths to 'lb:50051'... worker1 # [ 121.214318] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38582 client # [ 121.211702] nix[1275]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'... worker1 # [ 121.232102] nix-daemon[852]: accepted connection from pid 1476, user nix-grpc-daemon (trusted) worker1 # [ 121.274088] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:36Z level=info event=niks3_push_started pid=1600 worker1 # [ 121.400715] nix-grpc-daemon[1600]: time=2026-09-21T14:45:37.008Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 121.509412] nix-daemon[852]: accepted connection from pid 1613, user nix-grpc-daemon (trusted) worker1 # [ 121.525725] nix-daemon[852]: reaped child process 1616, status = succeeded worker1 # [ 121.631263] nix-daemon[852]: accepted connection from pid 1618, user nix-grpc-daemon (trusted) worker1 # [ 121.647485] nix-daemon[852]: reaped child process 1621, status = succeeded lb # [ 121.658437] niks3-server[1112]: 2026/09/21 14:45:37 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 121.664689] nix-grpc-daemon[1600]: time=2026-09-21T14:45:37.272Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 121.666156] nix-grpc-daemon[1600]: time=2026-09-21T14:45:37.272Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)" lb # [ 121.694053] niks3-server[1112]: 2026/09/21 14:45:37 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst lb # [ 121.718450] niks3-server[1112]: 2026/09/21 14:45:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign worker1 # [ 121.720046] nix-grpc-daemon[1600]: time=2026-09-21T14:45:37.328Z level=INFO msg="Uploading 1 narinfos" lb # [ 121.727390] niks3-server[1112]: 2026/09/21 14:45:37 INFO Signed narinfos id=41 count=1 lb # [ 121.734895] niks3-server[1112]: 2026/09/21 14:45:37 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls lb # [ 121.747123] niks3-server[1112]: 2026/09/21 14:45:37 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete worker1 # [ 121.750607] nix-grpc-daemon[1600]: time=2026-09-21T14:45:37.359Z level=INFO msg="Upload complete. (346ms)" lb # [ 121.758167] niks3-server[1112]: 2026/09/21 14:45:37 INFO Completed upload id=41 worker1 # [ 121.761380] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38582 duration_s=0 paths=1 nar_bytes_in=424 worker1 # [ 121.778624] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38582 worker1 # [ 121.788441] nix-grpc-daemon[1600]: time=2026-09-21T14:45:37.381Z level=WARN msg="Failed to register uploaded object" key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo error="server returned 404: object is not pending upload\n" worker1 # [ 121.800296] nix-daemon[852]: accepted connection from pid 1476, user nix-grpc-daemon (trusted) worker1 # [ 121.802842] nix-daemon[852]: reaped child process 1598, status = succeeded client # [ 121.806651] nix[1275]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... worker1 # [ 121.812402] nix-daemon[852]: accepted connection from pid 1476, user nix-grpc-daemon (trusted) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker2 # [ 123.920994] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1314 (nix-grpc-daemon) on client request. worker2 # [ 123.927522] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 123.962597] niks3-server[1112]: 2026/09/21 14:45:39 INFO lead: released remote=192.168.1.4:59958 worker1 # [ 123.964472] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:39Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))" client # [ 123.962678] nix[1275]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker2 # [ 123.955019] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker2 # [ 123.975517] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. worker2 # [ 123.984237] systemd[1]: nix-grpc-daemon.service: Consumed 223ms CPU time over 6.044s wall clock time, 14.4M memory peak, 10K incoming IP traffic, 14.8K outgoing IP traffic. worker2 # [ 123.986559] systemd[1]: Starting scheduler-token.service... worker2 # [ 123.991201] nix-daemon[838]: reaped child process 1373, status = succeeded worker2 # [ 124.033684] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 124.037928] systemd[1]: Finished scheduler-token.service. worker2 # [ 124.039169] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 63ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 124.047046] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker2 # [ 124.052236] systemd[1]: Starting scheduler-token.service... worker2 # [ 124.098924] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 124.102043] systemd[1]: Finished scheduler-token.service. worker2 # [ 124.103034] systemd[1]: scheduler-token.service: Consumed 27ms CPU time over 54ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 124.109498] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 124.167413] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.216749] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 124.218590] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 124.222249] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 124.228195] nix-grpc-daemon[1407]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 124.266367] niks3-server[1112]: 2026/09/21 14:45:39 INFO lead: acquired remote=192.168.1.4:42352 worker2 # [ 124.260244] nix-grpc-daemon[1407]: E0000 00:00:1790001939.829575 1407 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 124.262613] nix-grpc-daemon[1407]: E0000 00:00:1790001939.834524 1407 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.41 seconds) client: must succeed: sleep 5.0 worker2 # [ 124.280441] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 124.282676] nix-grpc-daemon[1407]: E0000 00:00:1790001939.842961 1447 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 124.285319] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:39Z level=info event=scheduler_take_over worker2 # [ 124.286423] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:39Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:45410 worker2 # [ 124.290611] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=1 worker2 # [ 124.294904] nix-grpc-daemon[1407]: E0000 00:00:1790001939.862840 1447 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 124.297601] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 124.298870] nix-grpc-daemon[1407]: E0000 00:00:1790001939.868031 1447 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 124.301644] nix-grpc-daemon[1407]: E0000 00:00:1790001939.882697 1447 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 124.305685] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:39Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 124.306950] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:39Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 124.308230] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:39Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:45410 worker2 # [ 124.309887] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb # [ 126.889794] envoy[888]: conn peer=[2001:db8:1::4]:59488 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=27833 tx=1725 ms=39016 lb # [ 126.895066] envoy[888]: conn peer=[2001:db8:1::4]:44096 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4102 tx=385 ms=5858 client: (finished: must succeed: sleep 5.0, in 5.05 seconds) client: waiting for success: ! systemctl is-active mid worker1 # [ 129.675218] nix-daemon[852]: reaped child process 1625, status = succeeded worker1 # [ 129.789654] nix-daemon[852]: accepted connection from pid 1662, user nix-grpc-daemon (trusted) worker1 # [ 129.808670] nix-daemon[852]: reaped child process 1665, status = succeeded lb # [ 129.819455] niks3-server[1112]: 2026/09/21 14:45:45 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 129.827736] nix-grpc-daemon[1600]: time=2026-09-21T14:45:45.433Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 129.830617] nix-grpc-daemon[1600]: time=2026-09-21T14:45:45.433Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)" lb # [ 129.850327] niks3-server[1112]: 2026/09/21 14:45:45 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv lb # [ 129.885294] niks3-server[1112]: 2026/09/21 14:45:45 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst lb # [ 129.893638] niks3-server[1112]: 2026/09/21 14:45:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign worker1 # [ 129.894774] nix-grpc-daemon[1600]: time=2026-09-21T14:45:45.501Z level=INFO msg="Uploading 1 narinfos" lb # [ 129.901257] niks3-server[1112]: 2026/09/21 14:45:45 INFO Signed narinfos id=42 count=1 lb # [ 129.903294] niks3-server[1112]: 2026/09/21 14:45:45 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls lb # [ 129.914574] niks3-server[1112]: 2026/09/21 14:45:45 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 129.921394] niks3-server[1112]: 2026/09/21 14:45:45 INFO Completed upload id=42 worker1 # [ 129.916743] nix-grpc-daemon[1600]: time=2026-09-21T14:45:45.525Z level=INFO msg="Upload complete. (288ms)" worker1 # [ 129.929484] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38582 duration_s=8 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 worker1 # [ 129.947520] nix-grpc-daemon[1600]: time=2026-09-21T14:45:45.549Z level=WARN msg="Failed to register uploaded object" key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo error="server returned 404: object is not pending upload\n" worker1 # [ 129.954997] nix-daemon[852]: reaped child process 1623, status = succeeded client # [ 129.957765] systemd[1]: mid.service: Deactivated successfully. client # [ 129.962411] systemd[1]: mid.service: Consumed 197ms CPU time over 9.978s wall clock time, 41.4M memory peak, 7.9K incoming IP traffic, 8.9K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.84 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr client # [ 130.795526] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: 7f818f0441d94b4382af702434224812 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds) worker2 # [ 130.954433] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 130.969688] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) client # [ 131.003028] nix[1311]: don't know how to build these paths: client # [ 131.004543] nix[1311]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker2 # [ 130.995081] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:46Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=0 worker2 # [ 131.001966] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:46Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 131.047668] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 131.062994] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 31 ms (attempt 1/5) worker2 # [ 131.102817] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 2/5) worker2 # [ 131.166225] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 365 ms (attempt 3/5) worker2 # [ 131.540886] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 614 ms (attempt 4/5) worker2 # [ 132.158379] nix-grpc-daemon[1407]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 132.165567] nix-grpc-daemon[1407]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 132.182567] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33404 client # [ 132.214143] nix[1311]: copying 1 paths to 'lb:50051'... worker2 # [ 132.200096] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33404 client # [ 132.220182] nix[1311]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'... worker2 # [ 132.217367] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 132.252096] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:47Z level=info event=niks3_push_started pid=1485 worker2 # [ 132.397160] nix-grpc-daemon[1485]: time=2026-09-21T14:45:47.999Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 132.501654] nix-daemon[838]: accepted connection from pid 1497, user nix-grpc-daemon (trusted) worker2 # [ 132.517940] nix-daemon[838]: reaped child process 1500, status = succeeded worker2 # [ 132.619876] nix-daemon[838]: accepted connection from pid 1502, user nix-grpc-daemon (trusted) worker2 # [ 132.635959] nix-daemon[838]: reaped child process 1505, status = succeeded lb # [ 132.669566] niks3-server[1112]: 2026/09/21 14:45:48 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 132.657639] nix-grpc-daemon[1485]: time=2026-09-21T14:45:48.259Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 132.659294] nix-grpc-daemon[1485]: time=2026-09-21T14:45:48.259Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)" lb # [ 132.717717] niks3-server[1112]: 2026/09/21 14:45:48 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst worker2 # [ 132.703482] nix-grpc-daemon[1485]: time=2026-09-21T14:45:48.306Z level=INFO msg="Uploading 1 narinfos" lb # [ 132.731891] niks3-server[1112]: 2026/09/21 14:45:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign lb # [ 132.740576] niks3-server[1112]: 2026/09/21 14:45:48 INFO Signed narinfos id=43 count=1 lb # [ 132.748276] niks3-server[1112]: 2026/09/21 14:45:48 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls lb # [ 132.751159] niks3-server[1112]: 2026/09/21 14:45:48 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete lb # [ 132.757359] niks3-server[1112]: 2026/09/21 14:45:48 INFO Completed upload id=43 worker2 # [ 132.731766] nix-grpc-daemon[1485]: time=2026-09-21T14:45:48.334Z level=INFO msg="Upload complete. (331ms)" lb # [ 132.759603] niks3-server[1112]: 2026/09/21 14:45:48 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo worker2 # [ 132.740324] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33404 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 132.755609] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 132.764231] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 132.768904] nix-daemon[838]: reaped child process 1483, status = succeeded client # [ 132.796811] nix[1311]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051... worker2 # [ 132.782765] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) client: must succeed: systemctl kill -s INT intr worker2 # [ 134.407268] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:50Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv client # [ 134.421304] systemd[1]: intr.service: Sent signal SIGINT to main process 1311 (nix) on client request. client # [ 134.430381] nix[1311]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 134.444417] nix[1311]: error: interrupted by the user client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds) client # [ 134.474981] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 134.480795] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 134.483334] systemd[1]: intr.service: Consumed 168ms CPU time over 3.683s wall clock time, 41.4M memory peak, 6.5K incoming IP traffic, 8.3K outgoing IP traffic. worker2 # [ 134.607607] nix-daemon[838]: reaped child process 1507, status = succeeded worker2 # [ 134.611535] nix-daemon[1509]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 134.617858] nix-daemon[838]: reaped child process 1509, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 4.97 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw client # [ 135.792740] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: e4f2857809b04789b1751e6a7f9766ce client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds) client # [ 135.993914] nix[1332]: don't know how to build these paths: client # [ 135.994867] nix[1332]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker1 # [ 135.996463] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:38582 worker2 # [ 135.989378] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:51Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=0 worker2 # [ 135.994996] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:51Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 136.056122] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38582 worker1 # [ 136.072442] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:38582 client # [ 136.081608] nix[1332]: copying 1 paths to 'lb:50051'... client # [ 136.087251] nix[1332]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'... worker1 # [ 136.089975] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38582 worker1 # [ 136.105069] nix-daemon[852]: accepted connection from pid 1476, user nix-grpc-daemon (trusted) worker1 # [ 136.264632] nix-daemon[852]: accepted connection from pid 1697, user nix-grpc-daemon (trusted) worker1 # [ 136.287800] nix-daemon[852]: reaped child process 1700, status = succeeded worker1 # [ 136.383926] nix-daemon[852]: accepted connection from pid 1702, user nix-grpc-daemon (trusted) worker1 # [ 136.400372] nix-daemon[852]: reaped child process 1705, status = succeeded lb # [ 136.411944] niks3-server[1112]: 2026/09/21 14:45:52 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 136.421170] nix-grpc-daemon[1600]: time=2026-09-21T14:45:52.029Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 136.422594] nix-grpc-daemon[1600]: time=2026-09-21T14:45:52.029Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)" lb # [ 136.443663] niks3-server[1112]: 2026/09/21 14:45:52 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst worker1 # [ 136.454038] nix-grpc-daemon[1600]: time=2026-09-21T14:45:52.062Z level=INFO msg="Uploading 1 narinfos" lb # [ 136.457688] niks3-server[1112]: 2026/09/21 14:45:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign lb # [ 136.470631] niks3-server[1112]: 2026/09/21 14:45:52 INFO Signed narinfos id=44 count=1 lb # [ 136.477822] niks3-server[1112]: 2026/09/21 14:45:52 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls lb # [ 136.490127] niks3-server[1112]: 2026/09/21 14:45:52 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete worker1 # [ 136.492798] nix-grpc-daemon[1600]: time=2026-09-21T14:45:52.101Z level=INFO msg="Upload complete. (370ms)" lb # [ 136.500079] niks3-server[1112]: 2026/09/21 14:45:52 INFO Completed upload id=44 lb # [ 136.503807] niks3-server[1112]: 2026/09/21 14:45:52 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo worker1 # [ 136.500272] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38582 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 136.514771] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38582 worker1 # [ 136.523925] nix-daemon[852]: accepted connection from pid 1476, user nix-grpc-daemon (trusted) worker1 # [ 136.532323] nix-daemon[852]: reaped child process 1695, status = succeeded client # [ 136.534317] nix[1332]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051... worker1 # [ 136.540188] nix-daemon[852]: accepted connection from pid 1476, user nix-grpc-daemon (trusted) lb # [ 136.889725] envoy[888]: conn peer=[2001:db8:1::1]:32912 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3078 tx=1518 ms=9753 lb # [ 136.894601] envoy[888]: conn peer=[2001:db8:1::1]:52740 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=853 ms=3485 worker1: must succeed: systemctl reload nix-grpc-daemon.service worker1 # [ 138.388333] systemd[1]: Reloading Nix worker-protocol over gRPC... worker1 # [ 138.420499] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds) worker1: must succeed: pgrep -f 'read -t [9]0 x' worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2 worker1 # [ 138.607944] nix-grpc-daemon[1476]: ts=2026-09-21T14:45:54Z level=info event=draining client # don't know how to build these paths: client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker2 # [ 138.677008] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 138.684818] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 138.709313] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:54Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=0 worker2 # [ 138.716085] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:54Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 138.761118] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 138.776453] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33404 client # copying 1 paths to 'lb:50051'... worker2 # [ 138.792421] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33404 client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'... worker2 # [ 138.822058] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 138.990277] nix-daemon[838]: accepted connection from pid 1564, user nix-grpc-daemon (trusted) worker2 # [ 139.009890] nix-daemon[838]: reaped child process 1567, status = succeeded worker2 # [ 139.107583] nix-daemon[838]: accepted connection from pid 1569, user nix-grpc-daemon (trusted) worker2 # [ 139.122961] nix-daemon[838]: reaped child process 1572, status = succeeded lb # [ 139.157806] niks3-server[1112]: 2026/09/21 14:45:54 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 139.142482] nix-grpc-daemon[1485]: time=2026-09-21T14:45:54.744Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 139.143997] nix-grpc-daemon[1485]: time=2026-09-21T14:45:54.744Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)" lb # [ 139.185547] niks3-server[1112]: 2026/09/21 14:45:54 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst lb # [ 139.198377] niks3-server[1112]: 2026/09/21 14:45:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign worker2 # [ 139.181373] nix-grpc-daemon[1485]: time=2026-09-21T14:45:54.784Z level=INFO msg="Uploading 1 narinfos" lb # [ 139.212550] niks3-server[1112]: 2026/09/21 14:45:54 INFO Signed narinfos id=45 count=1 lb # [ 139.215299] niks3-server[1112]: 2026/09/21 14:45:54 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls lb # [ 139.228676] niks3-server[1112]: 2026/09/21 14:45:54 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete lb # [ 139.236256] niks3-server[1112]: 2026/09/21 14:45:54 INFO Completed upload id=45 worker2 # [ 139.211176] nix-grpc-daemon[1485]: time=2026-09-21T14:45:54.814Z level=INFO msg="Upload complete. (370ms)" lb # [ 139.238867] niks3-server[1112]: 2026/09/21 14:45:54 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo worker2 # [ 139.220276] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33404 duration_s=0 paths=1 nar_bytes_in=528 worker2 # [ 139.234946] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 139.244634] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 139.247510] nix-daemon[838]: reaped child process 1562, status = succeeded client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051... worker2 # [ 139.258737] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker2 # [ 140.160950] nix-daemon[838]: reaped child process 1576, status = succeeded worker2 # [ 140.230001] nix-daemon[838]: accepted connection from pid 1592, user nix-grpc-daemon (trusted) worker2 # [ 140.250504] nix-daemon[838]: reaped child process 1595, status = succeeded lb # [ 140.274604] niks3-server[1112]: 2026/09/21 14:45:55 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 140.260494] nix-grpc-daemon[1485]: time=2026-09-21T14:45:55.862Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 140.262042] nix-grpc-daemon[1485]: time=2026-09-21T14:45:55.862Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)" lb # [ 140.319753] niks3-server[1112]: 2026/09/21 14:45:55 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst lb # [ 140.332859] niks3-server[1112]: 2026/09/21 14:45:55 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker2 # [ 140.319463] nix-grpc-daemon[1485]: time=2026-09-21T14:45:55.921Z level=INFO msg="Uploading 1 narinfos" lb # [ 140.343974] niks3-server[1112]: 2026/09/21 14:45:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign lb # [ 140.351254] niks3-server[1112]: 2026/09/21 14:45:55 INFO Signed narinfos id=46 count=1 lb # [ 140.356381] niks3-server[1112]: 2026/09/21 14:45:55 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls lb # [ 140.364373] niks3-server[1112]: 2026/09/21 14:45:55 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete worker2 # [ 140.343621] nix-grpc-daemon[1485]: time=2026-09-21T14:45:55.946Z level=INFO msg="Upload complete. (272ms)" lb # [ 140.370633] niks3-server[1112]: 2026/09/21 14:45:55 INFO Completed upload id=46 lb # [ 140.373165] niks3-server[1112]: 2026/09/21 14:45:55 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo worker2 # [ 140.361542] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=1 worker2 # [ 140.375852] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:55Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=0 worker2 # [ 140.379989] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:55Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 140.383675] nix-daemon[838]: reaped child process 1574, status = succeeded worker2 # [ 140.413424] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 140.428489] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33404 client # copying 1 paths to 'lb:50051'... worker2 # [ 140.444759] nix-grpc-daemon[1485]: time=2026-09-21T14:45:56.046Z level=INFO msg="All 1 paths already cached" worker2 # [ 140.450397] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33404 client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'... worker2 # [ 140.472978] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 140.625833] nix-daemon[838]: accepted connection from pid 1601, user nix-grpc-daemon (trusted) worker2 # [ 140.645274] nix-daemon[838]: reaped child process 1604, status = succeeded worker2 # [ 140.748118] nix-daemon[838]: accepted connection from pid 1606, user nix-grpc-daemon (trusted) worker2 # [ 140.765231] nix-daemon[838]: reaped child process 1609, status = succeeded lb # [ 140.795240] niks3-server[1112]: 2026/09/21 14:45:56 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 140.779904] nix-grpc-daemon[1485]: time=2026-09-21T14:45:56.382Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 140.781614] nix-grpc-daemon[1485]: time=2026-09-21T14:45:56.382Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)" lb # [ 140.821878] niks3-server[1112]: 2026/09/21 14:45:56 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst worker2 # [ 140.810520] nix-grpc-daemon[1485]: time=2026-09-21T14:45:56.413Z level=INFO msg="Uploading 1 narinfos" lb # [ 140.836570] niks3-server[1112]: 2026/09/21 14:45:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign lb # [ 140.849365] niks3-server[1112]: 2026/09/21 14:45:56 INFO Signed narinfos id=47 count=1 lb # [ 140.854604] niks3-server[1112]: 2026/09/21 14:45:56 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls worker2 # [ 140.840906] nix-grpc-daemon[1485]: time=2026-09-21T14:45:56.443Z level=INFO msg="Upload complete. (351ms)" lb # [ 140.866722] niks3-server[1112]: 2026/09/21 14:45:56 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete lb # [ 140.871947] niks3-server[1112]: 2026/09/21 14:45:56 INFO Completed upload id=47 worker2 # [ 140.848139] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33404 duration_s=0 paths=1 nar_bytes_in=640 lb # [ 140.874931] niks3-server[1112]: 2026/09/21 14:45:56 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo worker2 # [ 140.860312] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 140.869503] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 140.874149] nix-daemon[838]: reaped child process 1599, status = succeeded client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051... client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker2 # [ 140.887392] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker2 # [ 142.151984] nix-daemon[838]: reaped child process 1613, status = succeeded worker2 # [ 142.267361] nix-daemon[838]: accepted connection from pid 1629, user nix-grpc-daemon (trusted) worker2 # [ 142.285618] nix-daemon[838]: reaped child process 1632, status = succeeded lb # [ 142.318818] niks3-server[1112]: 2026/09/21 14:45:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 142.302912] nix-grpc-daemon[1485]: time=2026-09-21T14:45:57.904Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 142.304778] nix-grpc-daemon[1485]: time=2026-09-21T14:45:57.904Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)" lb # [ 142.352518] niks3-server[1112]: 2026/09/21 14:45:57 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv lb # [ 142.374946] niks3-server[1112]: 2026/09/21 14:45:57 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst worker2 # [ 142.360198] nix-grpc-daemon[1485]: time=2026-09-21T14:45:57.962Z level=INFO msg="Uploading 1 narinfos" lb # [ 142.385752] niks3-server[1112]: 2026/09/21 14:45:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign lb # [ 142.392534] niks3-server[1112]: 2026/09/21 14:45:57 INFO Signed narinfos id=48 count=1 lb # [ 142.398565] niks3-server[1112]: 2026/09/21 14:45:57 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete lb # [ 142.409657] niks3-server[1112]: 2026/09/21 14:45:58 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls lb # [ 142.418760] niks3-server[1112]: 2026/09/21 14:45:58 INFO Completed upload id=48 worker2 # [ 142.384396] nix-grpc-daemon[1485]: time=2026-09-21T14:45:57.987Z level=INFO msg="Upload complete. (286ms)" worker2 # [ 142.405386] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=1 worker2 # [ 142.420311] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:58Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=0 worker2 # [ 142.426867] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:58Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 142.432193] nix-grpc-daemon[1485]: time=2026-09-21T14:45:58.019Z level=WARN msg="Failed to register uploaded object" key=daks7y2c761g04qpzsjx25629wigr8js.narinfo error="server returned 404: object is not pending upload\n" worker2 # [ 142.438826] nix-daemon[838]: reaped child process 1611, status = succeeded worker2 # [ 142.448072] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 142.462452] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 142.481328] nix-grpc-daemon[1485]: time=2026-09-21T14:45:58.084Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 142.491895] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33404 client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'... worker2 # [ 142.514903] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 142.667379] nix-daemon[838]: accepted connection from pid 1638, user nix-grpc-daemon (trusted) worker2 # [ 142.689270] nix-daemon[838]: reaped child process 1641, status = succeeded worker2 # [ 142.790029] nix-daemon[838]: accepted connection from pid 1643, user nix-grpc-daemon (trusted) worker2 # [ 142.810260] nix-daemon[838]: reaped child process 1646, status = succeeded lb # [ 142.835933] niks3-server[1112]: 2026/09/21 14:45:58 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 142.823870] nix-grpc-daemon[1485]: time=2026-09-21T14:45:58.426Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 142.825589] nix-grpc-daemon[1485]: time=2026-09-21T14:45:58.426Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)" worker2 # [ 142.859734] nix-grpc-daemon[1485]: time=2026-09-21T14:45:58.462Z level=INFO msg="Uploading 1 narinfos" lb # [ 142.884081] niks3-server[1112]: 2026/09/21 14:45:58 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst lb # [ 142.899189] niks3-server[1112]: 2026/09/21 14:45:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign lb # [ 142.911394] niks3-server[1112]: 2026/09/21 14:45:58 INFO Signed narinfos id=49 count=1 worker2 # [ 142.891920] nix-grpc-daemon[1485]: time=2026-09-21T14:45:58.494Z level=INFO msg="Upload complete. (360ms)" lb # [ 142.918305] niks3-server[1112]: 2026/09/21 14:45:58 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls lb # [ 142.923866] niks3-server[1112]: 2026/09/21 14:45:58 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete lb # [ 142.928872] niks3-server[1112]: 2026/09/21 14:45:58 INFO Completed upload id=49 lb # [ 142.931763] niks3-server[1112]: 2026/09/21 14:45:58 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo worker2 # [ 142.899330] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33404 duration_s=0 paths=1 nar_bytes_in=768 worker2 # [ 142.913270] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 142.923333] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 142.926331] nix-daemon[838]: reaped child process 1636, status = succeeded client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051... worker2 # [ 142.939919] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker2 # [ 144.017945] nix-daemon[838]: reaped child process 1650, status = succeeded worker2 # [ 144.120767] nix-daemon[838]: accepted connection from pid 1666, user nix-grpc-daemon (trusted) worker2 # [ 144.139818] nix-daemon[838]: reaped child process 1669, status = succeeded lb # [ 144.171069] niks3-server[1112]: 2026/09/21 14:45:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 144.158320] nix-grpc-daemon[1485]: time=2026-09-21T14:45:59.759Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 144.159999] nix-grpc-daemon[1485]: time=2026-09-21T14:45:59.759Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)" lb # [ 144.210469] niks3-server[1112]: 2026/09/21 14:45:59 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst worker2 # [ 144.201755] nix-grpc-daemon[1485]: time=2026-09-21T14:45:59.804Z level=INFO msg="Uploading 1 narinfos" lb # [ 144.226012] niks3-server[1112]: 2026/09/21 14:45:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign lb # [ 144.237389] niks3-server[1112]: 2026/09/21 14:45:59 INFO Signed narinfos id=50 count=1 lb # [ 144.243362] niks3-server[1112]: 2026/09/21 14:45:59 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker2 # [ 144.229096] nix-grpc-daemon[1485]: time=2026-09-21T14:45:59.832Z level=INFO msg="Upload complete. (278ms)" lb # [ 144.254679] niks3-server[1112]: 2026/09/21 14:45:59 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls lb # [ 144.262441] niks3-server[1112]: 2026/09/21 14:45:59 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 144.268527] niks3-server[1112]: 2026/09/21 14:45:59 INFO Completed upload id=50 lb # [ 144.272204] niks3-server[1112]: 2026/09/21 14:45:59 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo worker2 # [ 144.245168] nix-grpc-daemon[1407]: ts=2026-09-21T14:45:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=1 worker2 # [ 144.261632] nix-daemon[838]: reaped child process 1648, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.83 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds) worker1: must succeed: pkill -f 'read -t [9]0 x' worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds) client: waiting for success: ! systemctl is-active sw worker1 # [ 144.577458] nix-daemon[852]: reaped child process 1709, status = succeeded worker1 # [ 144.668899] nix-daemon[852]: accepted connection from pid 1750, user nix-grpc-daemon (trusted) worker1 # [ 144.689295] nix-daemon[852]: reaped child process 1753, status = succeeded lb # [ 144.694433] niks3-server[1112]: 2026/09/21 14:46:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 144.699877] nix-grpc-daemon[1600]: time=2026-09-21T14:46:00.307Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 144.701562] nix-grpc-daemon[1600]: time=2026-09-21T14:46:00.307Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)" lb # [ 144.728115] niks3-server[1112]: 2026/09/21 14:46:00 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst lb # [ 144.746606] niks3-server[1112]: 2026/09/21 14:46:00 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker1 # [ 144.751873] nix-grpc-daemon[1600]: time=2026-09-21T14:46:00.360Z level=INFO msg="Uploading 1 narinfos" lb # [ 144.756909] niks3-server[1112]: 2026/09/21 14:46:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign lb # [ 144.765728] niks3-server[1112]: 2026/09/21 14:46:00 INFO Signed narinfos id=51 count=1 lb # [ 144.767746] niks3-server[1112]: 2026/09/21 14:46:00 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls lb # [ 144.774986] niks3-server[1112]: 2026/09/21 14:46:00 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete worker1 # [ 144.776550] nix-grpc-daemon[1600]: time=2026-09-21T14:46:00.385Z level=INFO msg="Upload complete. (220ms)" lb # [ 144.782488] niks3-server[1112]: 2026/09/21 14:46:00 INFO Completed upload id=51 lb # [ 144.786229] niks3-server[1112]: 2026/09/21 14:46:00 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo worker1 # [ 144.789006] nix-grpc-daemon[1476]: ts=2026-09-21T14:46:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38582 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=1 worker1 # [ 144.802020] nix-daemon[852]: reaped child process 1707, status = succeeded client # [ 144.818383] systemd[1]: sw.service: Deactivated successfully. client # [ 144.823485] systemd[1]: sw.service: Consumed 193ms CPU time over 9.026s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic. worker1 # [ 145.610744] nix-grpc-daemon[1476]: ts=2026-09-21T14:46:01Z level=info event=signal_exit client: (finished: waiting for success: ! systemctl is-active sw, in 1.09 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds) worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service lb # [ 146.890531] envoy[888]: conn peer=[2001:db8:1::1]:52756 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9621 tx=4090 ms=5585 lb # [ 146.895528] envoy[888]: conn peer=[2001:db8:1::1]:52750 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1405 ms=8808 worker1 # [ 147.611470] nix-grpc-daemon[1476]: ts=2026-09-21T14:46:03Z level=info event=scheduler_restarting worker2 # [ 147.590246] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:45410 worker2 # [ 147.595070] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:03Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 147.636981] nix-grpc-daemon[1476]: ts=2026-09-21T14:46:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 147.662337] nix-daemon[852]: reaped child process 1588, status = succeeded worker1 # [ 148.015026] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 148.023208] systemd[1]: nix-grpc-daemon.service: Consumed 1.147s CPU time over 42.577s wall clock time, 36.1M memory peak, 26.5K incoming IP traffic, 37.8K outgoing IP traffic. worker1 # [ 148.043265] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 148.191016] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:03Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 148.193355] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:03Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 148.197116] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:03Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 148.203625] nix-grpc-daemon[1767]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 148.220216] nix-grpc-daemon[1767]: E0000 00:00:1790001963.810066 1767 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.56 seconds) lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 148.249572] nix-grpc-daemon[1767]: E0000 00:00:1790001963.812067 1767 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 148.251839] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:03Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 148.254602] nix-grpc-daemon[1767]: E0000 00:00:1790001963.821492 1807 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 148.257679] nix-grpc-daemon[1767]: E0000 00:00:1790001963.840491 1807 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 148.260154] nix-grpc-daemon[1767]: E0000 00:00:1790001963.840654 1807 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 148.262451] nix-grpc-daemon[1767]: E0000 00:00:1790001963.857988 1807 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 148.264808] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 148.271392] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 148.254038] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:03Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:45410 worker2 # [ 148.259837] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:03Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.14 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 13.77 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker1 # [ 149.720778] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 149.735694] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 149.755450] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 149.763376] nix-grpc-daemon[1767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 1/5) worker1 # [ 149.801993] nix-grpc-daemon[1767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 2/5) worker1 # [ 149.821685] nix-grpc-daemon[1767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 304 ms (attempt 3/5) worker1 # [ 150.128156] nix-grpc-daemon[1767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 410 ms (attempt 4/5) worker1 # [ 150.540643] nix-grpc-daemon[1767]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 150.549329] nix-grpc-daemon[1767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 150.571804] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:06Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=0 worker2 # [ 150.577876] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:06Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 150.583811] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:06Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=0 worker2 # [ 150.589141] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:06Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=0 worker1 # [ 150.637768] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 150.652803] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 150.667302] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 150.681622] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 150.711218] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:06Z level=info event=niks3_push_started pid=1827 worker1 # [ 150.858280] nix-grpc-daemon[1827]: time=2026-09-21T14:46:06.466Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 150.963237] nix-daemon[852]: accepted connection from pid 1839, user nix-grpc-daemon (trusted) worker1 # [ 150.978889] nix-daemon[852]: reaped child process 1842, status = succeeded worker1 # [ 151.083215] nix-daemon[852]: accepted connection from pid 1844, user nix-grpc-daemon (trusted) worker1 # [ 151.098345] nix-daemon[852]: reaped child process 1847, status = succeeded lb # [ 151.110231] niks3-server[1112]: 2026/09/21 14:46:06 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 151.118104] nix-grpc-daemon[1827]: time=2026-09-21T14:46:06.725Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 151.119569] nix-grpc-daemon[1827]: time=2026-09-21T14:46:06.725Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)" lb # [ 151.149011] niks3-server[1112]: 2026/09/21 14:46:06 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst worker1 # [ 151.164134] nix-grpc-daemon[1827]: time=2026-09-21T14:46:06.772Z level=INFO msg="Uploading 1 narinfos" lb # [ 151.167863] niks3-server[1112]: 2026/09/21 14:46:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign lb # [ 151.179478] niks3-server[1112]: 2026/09/21 14:46:06 INFO Signed narinfos id=52 count=1 lb # [ 151.196200] niks3-server[1112]: 2026/09/21 14:46:06 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls lb # [ 151.206221] niks3-server[1112]: 2026/09/21 14:46:06 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 151.212264] niks3-server[1112]: 2026/09/21 14:46:06 INFO Completed upload id=52 worker1 # [ 151.207735] nix-grpc-daemon[1827]: time=2026-09-21T14:46:06.816Z level=INFO msg="Upload complete. (345ms)" lb # [ 151.215466] niks3-server[1112]: 2026/09/21 14:46:06 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo worker1 # [ 151.219838] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 151.236612] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 151.246112] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 151.250181] nix-daemon[852]: reaped child process 1825, status = succeeded worker1 # [ 151.262016] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 152.155733] nix-daemon[852]: reaped child process 1851, status = succeeded worker1 # [ 152.240775] nix-daemon[852]: accepted connection from pid 1868, user nix-grpc-daemon (trusted) worker1 # [ 152.258781] nix-daemon[852]: reaped child process 1871, status = succeeded lb # [ 152.271365] niks3-server[1112]: 2026/09/21 14:46:07 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 152.280083] nix-grpc-daemon[1827]: time=2026-09-21T14:46:07.885Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 152.281858] nix-grpc-daemon[1827]: time=2026-09-21T14:46:07.885Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)" lb # [ 152.304089] niks3-server[1112]: 2026/09/21 14:46:07 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker1 # [ 152.329323] nix-grpc-daemon[1827]: time=2026-09-21T14:46:07.937Z level=INFO msg="Uploading 1 narinfos" lb # [ 152.332894] niks3-server[1112]: 2026/09/21 14:46:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign lb # [ 152.343744] niks3-server[1112]: 2026/09/21 14:46:07 INFO Signed narinfos id=53 count=1 lb # [ 152.349366] niks3-server[1112]: 2026/09/21 14:46:07 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst lb # [ 152.362049] niks3-server[1112]: 2026/09/21 14:46:07 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete worker1 # [ 152.357760] nix-grpc-daemon[1827]: time=2026-09-21T14:46:07.966Z level=INFO msg="Upload complete. (282ms)" lb # [ 152.372158] niks3-server[1112]: 2026/09/21 14:46:07 INFO Completed upload id=53 worker2 # [ 152.348447] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:07Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 152.370383] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=6 outputs=1 lb # [ 152.378390] niks3-server[1112]: 2026/09/21 14:46:07 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo worker2 # [ 152.359771] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:07Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=0 lb # [ 152.386563] niks3-server[1112]: 2026/09/21 14:46:07 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls worker1 # [ 152.396241] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 152.400516] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 152.406624] nix-daemon[852]: reaped child process 1849, status = succeeded worker1 # [ 152.418217] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 152.424352] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 152.554352] nix-daemon[852]: accepted connection from pid 1875, user nix-grpc-daemon (trusted) worker1 # [ 152.573798] nix-daemon[852]: reaped child process 1878, status = succeeded worker1 # [ 152.672690] nix-daemon[852]: accepted connection from pid 1880, user nix-grpc-daemon (trusted) worker1 # [ 152.688928] nix-daemon[852]: reaped child process 1883, status = succeeded lb # [ 152.698447] niks3-server[1112]: 2026/09/21 14:46:08 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 152.703365] nix-grpc-daemon[1827]: time=2026-09-21T14:46:08.311Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 152.704991] nix-grpc-daemon[1827]: time=2026-09-21T14:46:08.311Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)" lb # [ 152.725873] niks3-server[1112]: 2026/09/21 14:46:08 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst worker1 # [ 152.735616] nix-grpc-daemon[1827]: time=2026-09-21T14:46:08.344Z level=INFO msg="Uploading 1 narinfos" lb # [ 152.740863] niks3-server[1112]: 2026/09/21 14:46:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign lb # [ 152.752517] niks3-server[1112]: 2026/09/21 14:46:08 INFO Signed narinfos id=54 count=1 lb # [ 152.761216] niks3-server[1112]: 2026/09/21 14:46:08 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls worker1 # [ 152.768354] nix-grpc-daemon[1827]: time=2026-09-21T14:46:08.376Z level=INFO msg="Upload complete. (337ms)" lb # [ 152.773545] niks3-server[1112]: 2026/09/21 14:46:08 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete lb # [ 152.782565] niks3-server[1112]: 2026/09/21 14:46:08 INFO Completed upload id=54 lb # [ 152.785381] niks3-server[1112]: 2026/09/21 14:46:08 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo worker1 # [ 152.776125] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 152.789325] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 152.795249] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 152.799963] nix-daemon[852]: reaped child process 1873, status = succeeded worker1 # [ 152.810202] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 153.708636] nix-daemon[852]: reaped child process 1887, status = succeeded worker1 # [ 153.774271] nix-daemon[852]: accepted connection from pid 1903, user nix-grpc-daemon (trusted) worker1 # [ 153.792817] nix-daemon[852]: reaped child process 1906, status = succeeded lb # [ 153.803461] niks3-server[1112]: 2026/09/21 14:46:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 153.808407] nix-grpc-daemon[1827]: time=2026-09-21T14:46:09.416Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 153.810765] nix-grpc-daemon[1827]: time=2026-09-21T14:46:09.418Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)" lb # [ 153.840109] niks3-server[1112]: 2026/09/21 14:46:09 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst worker1 # [ 153.853431] nix-grpc-daemon[1827]: time=2026-09-21T14:46:09.461Z level=INFO msg="Uploading 1 narinfos" lb # [ 153.857493] niks3-server[1112]: 2026/09/21 14:46:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign lb # [ 153.868212] niks3-server[1112]: 2026/09/21 14:46:09 INFO Signed narinfos id=55 count=1 lb # [ 153.874852] niks3-server[1112]: 2026/09/21 14:46:09 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv lb # [ 153.886988] niks3-server[1112]: 2026/09/21 14:46:09 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls worker1 # [ 153.888914] nix-grpc-daemon[1827]: time=2026-09-21T14:46:09.497Z level=INFO msg="Upload complete. (267ms)" lb # [ 153.895770] niks3-server[1112]: 2026/09/21 14:46:09 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete lb # [ 153.900837] niks3-server[1112]: 2026/09/21 14:46:09 INFO Completed upload id=55 lb # [ 153.904422] niks3-server[1112]: 2026/09/21 14:46:09 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo worker2 # [ 153.878437] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:09Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 153.901135] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=7 outputs=1 worker2 # [ 153.890921] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:09Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=0 worker1 # [ 153.920426] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 153.931520] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 153.937916] nix-daemon[852]: reaped child process 1885, status = succeeded worker1 # [ 153.945486] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 153.959964] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 154.099431] nix-daemon[852]: accepted connection from pid 1910, user nix-grpc-daemon (trusted) worker1 # [ 154.116811] nix-daemon[852]: reaped child process 1913, status = succeeded worker1 # [ 154.219249] nix-daemon[852]: accepted connection from pid 1915, user nix-grpc-daemon (trusted) worker1 # [ 154.234026] nix-daemon[852]: reaped child process 1918, status = succeeded lb # [ 154.245165] niks3-server[1112]: 2026/09/21 14:46:09 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 154.250729] nix-grpc-daemon[1827]: time=2026-09-21T14:46:09.859Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 154.252285] nix-grpc-daemon[1827]: time=2026-09-21T14:46:09.859Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)" lb # [ 154.271540] niks3-server[1112]: 2026/09/21 14:46:09 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst worker1 # [ 154.288087] nix-grpc-daemon[1827]: time=2026-09-21T14:46:09.896Z level=INFO msg="Uploading 1 narinfos" lb # [ 154.285303] niks3-server[1112]: 2026/09/21 14:46:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign lb # [ 154.297466] niks3-server[1112]: 2026/09/21 14:46:09 INFO Signed narinfos id=56 count=1 lb # [ 154.304260] niks3-server[1112]: 2026/09/21 14:46:09 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls lb # [ 154.315701] niks3-server[1112]: 2026/09/21 14:46:09 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 154.329737] niks3-server[1112]: 2026/09/21 14:46:09 INFO Completed upload id=56 worker1 # [ 154.325665] nix-grpc-daemon[1827]: time=2026-09-21T14:46:09.934Z level=INFO msg="Upload complete. (356ms)" lb # [ 154.331825] niks3-server[1112]: 2026/09/21 14:46:09 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo worker1 # [ 154.332268] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 154.345824] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 154.352357] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 154.357586] nix-daemon[852]: reaped child process 1908, status = succeeded worker1 # [ 154.371393] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 155.608001] nix-daemon[852]: reaped child process 1922, status = succeeded worker1 # [ 155.726023] nix-daemon[852]: accepted connection from pid 1938, user nix-grpc-daemon (trusted) worker1 # [ 155.743901] nix-daemon[852]: reaped child process 1941, status = succeeded lb # [ 155.755548] niks3-server[1112]: 2026/09/21 14:46:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 155.764052] nix-grpc-daemon[1827]: time=2026-09-21T14:46:11.370Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 155.765687] nix-grpc-daemon[1827]: time=2026-09-21T14:46:11.370Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)" lb # [ 155.789816] niks3-server[1112]: 2026/09/21 14:46:11 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv lb # [ 155.802036] niks3-server[1112]: 2026/09/21 14:46:11 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 155.816120] niks3-server[1112]: 2026/09/21 14:46:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign worker1 # [ 155.818067] nix-grpc-daemon[1827]: time=2026-09-21T14:46:11.426Z level=INFO msg="Uploading 1 narinfos" lb # [ 155.824289] niks3-server[1112]: 2026/09/21 14:46:11 INFO Signed narinfos id=57 count=1 lb # [ 155.832416] niks3-server[1112]: 2026/09/21 14:46:11 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls lb # [ 155.846125] niks3-server[1112]: 2026/09/21 14:46:11 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete lb # [ 155.853491] niks3-server[1112]: 2026/09/21 14:46:11 INFO Completed upload id=57 worker1 # [ 155.847946] nix-grpc-daemon[1827]: time=2026-09-21T14:46:11.456Z level=INFO msg="Upload complete. (266ms)" lb # [ 155.855794] niks3-server[1112]: 2026/09/21 14:46:11 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo worker2 # [ 155.838391] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:11Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 155.859891] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=8 outputs=1 worker2 # [ 155.850738] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:11Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=0 worker1 # [ 155.880324] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 155.889860] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 155.895275] nix-daemon[852]: reaped child process 1920, status = succeeded worker1 # [ 155.901668] nix-grpc-daemon[1827]: time=2026-09-21T14:46:11.510Z level=INFO msg="All 1 paths already cached" worker1 # [ 155.910151] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 155.923454] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 156.065732] nix-daemon[852]: accepted connection from pid 1945, user nix-grpc-daemon (trusted) worker1 # [ 156.082274] nix-daemon[852]: reaped child process 1948, status = succeeded worker1 # [ 156.183585] nix-daemon[852]: accepted connection from pid 1950, user nix-grpc-daemon (trusted) worker1 # [ 156.197958] nix-daemon[852]: reaped child process 1953, status = succeeded lb # [ 156.209793] niks3-server[1112]: 2026/09/21 14:46:11 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 156.214706] nix-grpc-daemon[1827]: time=2026-09-21T14:46:11.823Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 156.216576] nix-grpc-daemon[1827]: time=2026-09-21T14:46:11.823Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)" lb # [ 156.235995] niks3-server[1112]: 2026/09/21 14:46:11 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst worker1 # [ 156.244223] nix-grpc-daemon[1827]: time=2026-09-21T14:46:11.852Z level=INFO msg="Uploading 1 narinfos" lb # [ 156.249875] niks3-server[1112]: 2026/09/21 14:46:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign lb # [ 156.261504] niks3-server[1112]: 2026/09/21 14:46:11 INFO Signed narinfos id=58 count=1 lb # [ 156.267679] niks3-server[1112]: 2026/09/21 14:46:11 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls lb # [ 156.276777] niks3-server[1112]: 2026/09/21 14:46:11 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete worker1 # [ 156.278657] nix-grpc-daemon[1827]: time=2026-09-21T14:46:11.887Z level=INFO msg="Upload complete. (342ms)" lb # [ 156.283835] niks3-server[1112]: 2026/09/21 14:46:11 INFO Completed upload id=58 lb # [ 156.290203] niks3-server[1112]: 2026/09/21 14:46:11 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo worker1 # [ 156.286828] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 156.300403] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 156.306264] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 156.310732] nix-daemon[852]: reaped child process 1943, status = succeeded worker1 # [ 156.321876] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) lb # [ 156.892519] envoy[888]: conn peer=[2001:db8:1::3]:50074 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2175 tx=710 ms=41965 worker1 # [ 157.559574] nix-daemon[852]: reaped child process 1957, status = succeeded worker1 # [ 157.655382] nix-daemon[852]: accepted connection from pid 1973, user nix-grpc-daemon (trusted) worker1 # [ 157.674156] nix-daemon[852]: reaped child process 1976, status = succeeded lb # [ 157.683709] niks3-server[1112]: 2026/09/21 14:46:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 157.689498] nix-grpc-daemon[1827]: time=2026-09-21T14:46:13.297Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 157.693742] nix-grpc-daemon[1827]: time=2026-09-21T14:46:13.299Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)" lb # [ 157.726541] niks3-server[1112]: 2026/09/21 14:46:13 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst worker1 # [ 157.734906] nix-grpc-daemon[1827]: time=2026-09-21T14:46:13.343Z level=INFO msg="Uploading 1 narinfos" lb # [ 157.738775] niks3-server[1112]: 2026/09/21 14:46:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign lb # [ 157.750460] niks3-server[1112]: 2026/09/21 14:46:13 INFO Signed narinfos id=59 count=1 lb # [ 157.756361] niks3-server[1112]: 2026/09/21 14:46:13 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv lb # [ 157.764267] niks3-server[1112]: 2026/09/21 14:46:13 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls worker1 # [ 157.770993] nix-grpc-daemon[1827]: time=2026-09-21T14:46:13.379Z level=INFO msg="Upload complete. (237ms)" lb # [ 157.776636] niks3-server[1112]: 2026/09/21 14:46:13 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete lb # [ 157.783931] niks3-server[1112]: 2026/09/21 14:46:13 INFO Completed upload id=59 worker2 # [ 157.759333] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:13Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=1 lb # [ 157.789786] niks3-server[1112]: 2026/09/21 14:46:13 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo worker2 # [ 157.772130] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:13Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=0 worker1 # [ 157.785341] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=9 outputs=1 worker1 # [ 157.800544] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 157.806448] nix-daemon[852]: reaped child process 1955, status = succeeded worker1 # [ 157.814400] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 157.826739] nix-grpc-daemon[1827]: time=2026-09-21T14:46:13.435Z level=INFO msg="All 1 paths already cached" worker1 # [ 157.835353] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 157.848862] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 157.991569] nix-daemon[852]: accepted connection from pid 1980, user nix-grpc-daemon (trusted) worker1 # [ 158.008891] nix-daemon[852]: reaped child process 1983, status = succeeded worker1 # [ 158.112491] nix-daemon[852]: accepted connection from pid 1985, user nix-grpc-daemon (trusted) worker1 # [ 158.127555] nix-daemon[852]: reaped child process 1988, status = succeeded lb # [ 158.138008] niks3-server[1112]: 2026/09/21 14:46:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 158.144772] nix-grpc-daemon[1827]: time=2026-09-21T14:46:13.753Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 158.146254] nix-grpc-daemon[1827]: time=2026-09-21T14:46:13.753Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)" lb # [ 158.170519] niks3-server[1112]: 2026/09/21 14:46:13 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst lb # [ 158.184885] niks3-server[1112]: 2026/09/21 14:46:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign worker1 # [ 158.185687] nix-grpc-daemon[1827]: time=2026-09-21T14:46:13.794Z level=INFO msg="Uploading 1 narinfos" lb # [ 158.195319] niks3-server[1112]: 2026/09/21 14:46:13 INFO Signed narinfos id=60 count=1 lb # [ 158.200649] niks3-server[1112]: 2026/09/21 14:46:13 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls lb # [ 158.209557] niks3-server[1112]: 2026/09/21 14:46:13 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete worker1 # [ 158.210764] nix-grpc-daemon[1827]: time=2026-09-21T14:46:13.819Z level=INFO msg="Upload complete. (348ms)" lb # [ 158.216575] niks3-server[1112]: 2026/09/21 14:46:13 INFO Completed upload id=60 lb # [ 158.220542] niks3-server[1112]: 2026/09/21 14:46:13 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo worker1 # [ 158.220284] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 158.234335] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 158.243194] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 158.246075] nix-daemon[852]: reaped child process 1978, status = succeeded worker1 # [ 158.258930] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 159.761692] nix-daemon[852]: reaped child process 1992, status = succeeded worker1 # [ 159.833731] nix-daemon[852]: accepted connection from pid 2008, user nix-grpc-daemon (trusted) worker1 # [ 159.853765] nix-daemon[852]: reaped child process 2011, status = succeeded lb # [ 159.856906] niks3-server[1112]: 2026/09/21 14:46:15 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 159.863195] nix-grpc-daemon[1827]: time=2026-09-21T14:46:15.470Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 159.864952] nix-grpc-daemon[1827]: time=2026-09-21T14:46:15.470Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)" lb # [ 159.888793] niks3-server[1112]: 2026/09/21 14:46:15 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv lb # [ 159.904129] niks3-server[1112]: 2026/09/21 14:46:15 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst worker1 # [ 159.914096] nix-grpc-daemon[1827]: time=2026-09-21T14:46:15.522Z level=INFO msg="Uploading 1 narinfos" lb # [ 159.917622] niks3-server[1112]: 2026/09/21 14:46:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign lb # [ 159.928669] niks3-server[1112]: 2026/09/21 14:46:15 INFO Signed narinfos id=61 count=1 lb # [ 159.936128] niks3-server[1112]: 2026/09/21 14:46:15 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls lb # [ 159.950515] niks3-server[1112]: 2026/09/21 14:46:15 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete lb # [ 159.958586] niks3-server[1112]: 2026/09/21 14:46:15 INFO Completed upload id=61 worker1 # [ 159.954011] nix-grpc-daemon[1827]: time=2026-09-21T14:46:15.562Z level=INFO msg="Upload complete. (284ms)" lb # [ 159.961738] niks3-server[1112]: 2026/09/21 14:46:15 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo worker2 # [ 159.944700] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:15Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 159.969439] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=10 outputs=1 worker2 # [ 159.956570] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:15Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=0 worker1 # [ 159.985447] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 159.996946] nix-daemon[852]: reaped child process 1990, status = succeeded worker1 # [ 159.999225] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 160.010941] nix-grpc-daemon[1827]: time=2026-09-21T14:46:15.619Z level=INFO msg="All 1 paths already cached" worker1 # [ 160.019494] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 160.032273] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 160.180085] nix-daemon[852]: accepted connection from pid 2015, user nix-grpc-daemon (trusted) worker1 # [ 160.208231] nix-daemon[852]: reaped child process 2018, status = succeeded worker1 # [ 160.300261] nix-daemon[852]: accepted connection from pid 2020, user nix-grpc-daemon (trusted) worker1 # [ 160.315663] nix-daemon[852]: reaped child process 2023, status = succeeded lb # [ 160.326243] niks3-server[1112]: 2026/09/21 14:46:15 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 160.330668] nix-grpc-daemon[1827]: time=2026-09-21T14:46:15.939Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 160.332202] nix-grpc-daemon[1827]: time=2026-09-21T14:46:15.939Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)" lb # [ 160.352260] niks3-server[1112]: 2026/09/21 14:46:15 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst worker1 # [ 160.370873] nix-grpc-daemon[1827]: time=2026-09-21T14:46:15.979Z level=INFO msg="Uploading 1 narinfos" lb # [ 160.368238] niks3-server[1112]: 2026/09/21 14:46:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign lb # [ 160.381407] niks3-server[1112]: 2026/09/21 14:46:15 INFO Signed narinfos id=62 count=1 lb # [ 160.388254] niks3-server[1112]: 2026/09/21 14:46:15 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls worker1 # [ 160.396890] nix-grpc-daemon[1827]: time=2026-09-21T14:46:16.005Z level=INFO msg="Upload complete. (351ms)" lb # [ 160.402070] niks3-server[1112]: 2026/09/21 14:46:15 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete worker1 # [ 160.404126] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 160.412120] niks3-server[1112]: 2026/09/21 14:46:16 INFO Completed upload id=62 lb # [ 160.415249] niks3-server[1112]: 2026/09/21 14:46:16 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo worker1 # [ 160.419256] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 160.426276] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 160.431209] nix-daemon[852]: reaped child process 2013, status = succeeded worker1 # [ 160.442777] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 162.084757] nix-daemon[852]: reaped child process 2027, status = succeeded worker1 # [ 162.193426] nix-daemon[852]: accepted connection from pid 2043, user nix-grpc-daemon (trusted) worker1 # [ 162.212355] nix-daemon[852]: reaped child process 2046, status = succeeded lb # [ 162.221880] niks3-server[1112]: 2026/09/21 14:46:17 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 162.228391] nix-grpc-daemon[1827]: time=2026-09-21T14:46:17.835Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 162.230789] nix-grpc-daemon[1827]: time=2026-09-21T14:46:17.835Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)" lb # [ 162.258604] niks3-server[1112]: 2026/09/21 14:46:17 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst worker1 # [ 162.267764] nix-grpc-daemon[1827]: time=2026-09-21T14:46:17.876Z level=INFO msg="Uploading 1 narinfos" lb # [ 162.272133] niks3-server[1112]: 2026/09/21 14:46:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign lb # [ 162.283009] niks3-server[1112]: 2026/09/21 14:46:17 INFO Signed narinfos id=63 count=1 lb # [ 162.288965] niks3-server[1112]: 2026/09/21 14:46:17 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv lb # [ 162.296830] niks3-server[1112]: 2026/09/21 14:46:17 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls lb # [ 162.302901] niks3-server[1112]: 2026/09/21 14:46:17 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete lb # [ 162.311129] niks3-server[1112]: 2026/09/21 14:46:17 INFO Completed upload id=63 worker1 # [ 162.306420] nix-grpc-daemon[1827]: time=2026-09-21T14:46:17.914Z level=INFO msg="Upload complete. (250ms)" lb # [ 162.314245] niks3-server[1112]: 2026/09/21 14:46:17 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo worker2 # [ 162.297337] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:17Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 162.317059] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=11 outputs=1 worker2 # [ 162.310170] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:17Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=0 worker1 # [ 162.335518] nix-daemon[852]: reaped child process 2025, status = succeeded worker1 # [ 162.372293] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 162.386611] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 162.400630] nix-grpc-daemon[1827]: time=2026-09-21T14:46:18.009Z level=INFO msg="All 2 paths already cached" worker1 # [ 162.409648] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 162.423380] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 162.578764] nix-daemon[852]: accepted connection from pid 2050, user nix-grpc-daemon (trusted) worker1 # [ 162.598842] nix-daemon[852]: reaped child process 2053, status = succeeded worker1 # [ 162.700855] nix-daemon[852]: accepted connection from pid 2055, user nix-grpc-daemon (trusted) worker1 # [ 162.718274] nix-daemon[852]: reaped child process 2058, status = succeeded lb # [ 162.726653] niks3-server[1112]: 2026/09/21 14:46:18 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 162.731417] nix-grpc-daemon[1827]: time=2026-09-21T14:46:18.339Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 162.733019] nix-grpc-daemon[1827]: time=2026-09-21T14:46:18.339Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)" lb # [ 162.753291] niks3-server[1112]: 2026/09/21 14:46:18 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst worker1 # [ 162.763888] nix-grpc-daemon[1827]: time=2026-09-21T14:46:18.372Z level=INFO msg="Uploading 1 narinfos" lb # [ 162.767805] niks3-server[1112]: 2026/09/21 14:46:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign lb # [ 162.780085] niks3-server[1112]: 2026/09/21 14:46:18 INFO Signed narinfos id=64 count=1 lb # [ 162.787550] niks3-server[1112]: 2026/09/21 14:46:18 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls lb # [ 162.800562] niks3-server[1112]: 2026/09/21 14:46:18 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete worker1 # [ 162.803186] nix-grpc-daemon[1827]: time=2026-09-21T14:46:18.411Z level=INFO msg="Upload complete. (363ms)" lb # [ 162.810193] niks3-server[1112]: 2026/09/21 14:46:18 INFO Completed upload id=64 lb # [ 162.814845] niks3-server[1112]: 2026/09/21 14:46:18 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo worker1 # [ 162.812256] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 162.824452] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 162.831865] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 162.837842] nix-daemon[852]: reaped child process 2048, status = succeeded worker1 # [ 162.852368] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 163.942975] nix-daemon[852]: reaped child process 2062, status = succeeded worker1 # [ 164.011460] nix-daemon[852]: accepted connection from pid 2078, user nix-grpc-daemon (trusted) worker1 # [ 164.028987] nix-daemon[852]: reaped child process 2081, status = succeeded lb # [ 164.039894] niks3-server[1112]: 2026/09/21 14:46:19 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 164.048424] nix-grpc-daemon[1827]: time=2026-09-21T14:46:19.654Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 164.050021] nix-grpc-daemon[1827]: time=2026-09-21T14:46:19.654Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)" lb # [ 164.084708] niks3-server[1112]: 2026/09/21 14:46:19 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst worker1 # [ 164.091127] nix-grpc-daemon[1827]: time=2026-09-21T14:46:19.699Z level=INFO msg="Uploading 1 narinfos" lb # [ 164.097679] niks3-server[1112]: 2026/09/21 14:46:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign lb # [ 164.108975] niks3-server[1112]: 2026/09/21 14:46:19 INFO Signed narinfos id=65 count=1 lb # [ 164.111308] niks3-server[1112]: 2026/09/21 14:46:19 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv lb # [ 164.119916] niks3-server[1112]: 2026/09/21 14:46:19 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls worker1 # [ 164.124191] nix-grpc-daemon[1827]: time=2026-09-21T14:46:19.732Z level=INFO msg="Upload complete. (262ms)" lb # [ 164.128625] niks3-server[1112]: 2026/09/21 14:46:19 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete lb # [ 164.133920] niks3-server[1112]: 2026/09/21 14:46:19 INFO Completed upload id=65 lb # [ 164.138532] niks3-server[1112]: 2026/09/21 14:46:19 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo worker2 # [ 164.111690] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:19Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 164.137077] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=12 outputs=1 worker1 # [ 164.156350] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 164.163423] nix-daemon[852]: reaped child process 2060, status = succeeded worker1 # [ 164.167882] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 164.176586] nix-grpc-daemon[1827]: time=2026-09-21T14:46:19.785Z level=INFO msg="All 2 paths already cached" worker1 # [ 164.184487] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 164.196318] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 164.341177] nix-daemon[852]: accepted connection from pid 2085, user nix-grpc-daemon (trusted) worker1 # [ 164.356998] nix-daemon[852]: reaped child process 2088, status = succeeded worker1 # [ 164.459256] nix-daemon[852]: accepted connection from pid 2090, user nix-grpc-daemon (trusted) worker1 # [ 164.475045] nix-daemon[852]: reaped child process 2093, status = succeeded lb # [ 164.485613] niks3-server[1112]: 2026/09/21 14:46:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 164.490063] nix-grpc-daemon[1827]: time=2026-09-21T14:46:20.098Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 164.491504] nix-grpc-daemon[1827]: time=2026-09-21T14:46:20.098Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)" lb # [ 164.515180] niks3-server[1112]: 2026/09/21 14:46:20 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst worker1 # [ 164.522483] nix-grpc-daemon[1827]: time=2026-09-21T14:46:20.130Z level=INFO msg="Uploading 1 narinfos" lb # [ 164.531098] niks3-server[1112]: 2026/09/21 14:46:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign lb # [ 164.539365] niks3-server[1112]: 2026/09/21 14:46:20 INFO Signed narinfos id=66 count=1 lb # [ 164.547426] niks3-server[1112]: 2026/09/21 14:46:20 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete worker1 # [ 164.549165] nix-grpc-daemon[1827]: time=2026-09-21T14:46:20.157Z level=INFO msg="Upload complete. (340ms)" lb # [ 164.555545] niks3-server[1112]: 2026/09/21 14:46:20 INFO Completed upload id=66 lb # [ 164.558522] niks3-server[1112]: 2026/09/21 14:46:20 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo lb # [ 164.563565] niks3-server[1112]: 2026/09/21 14:46:20 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls worker1 # [ 164.556108] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 164.569419] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 164.572933] nix-daemon[852]: reaped child process 2083, status = succeeded worker1 # [ 164.579581] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 164.592236] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 165.664287] nix-daemon[852]: reaped child process 2097, status = succeeded worker1 # [ 165.767923] nix-daemon[852]: accepted connection from pid 2113, user nix-grpc-daemon (trusted) worker1 # [ 165.788696] nix-daemon[852]: reaped child process 2116, status = succeeded lb # [ 165.794926] niks3-server[1112]: 2026/09/21 14:46:21 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 165.801687] nix-grpc-daemon[1827]: time=2026-09-21T14:46:21.408Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 165.803272] nix-grpc-daemon[1827]: time=2026-09-21T14:46:21.409Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)" lb # [ 165.831928] niks3-server[1112]: 2026/09/21 14:46:21 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv lb # [ 165.843358] niks3-server[1112]: 2026/09/21 14:46:21 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst worker1 # [ 165.853406] nix-grpc-daemon[1827]: time=2026-09-21T14:46:21.461Z level=INFO msg="Uploading 1 narinfos" lb # [ 165.857439] niks3-server[1112]: 2026/09/21 14:46:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign lb # [ 165.864667] niks3-server[1112]: 2026/09/21 14:46:21 INFO Signed narinfos id=67 count=1 lb # [ 165.866990] niks3-server[1112]: 2026/09/21 14:46:21 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls lb # [ 165.879182] niks3-server[1112]: 2026/09/21 14:46:21 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete lb # [ 165.887991] niks3-server[1112]: 2026/09/21 14:46:21 INFO Completed upload id=67 worker1 # [ 165.883197] nix-grpc-daemon[1827]: time=2026-09-21T14:46:21.491Z level=INFO msg="Upload complete. (281ms)" lb # [ 165.891836] niks3-server[1112]: 2026/09/21 14:46:21 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo worker2 # [ 165.874664] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:21Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 165.896626] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=13 outputs=1 worker1 # [ 165.914119] nix-daemon[852]: reaped child process 2095, status = succeeded worker1 # [ 165.939955] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 165.955265] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 165.969491] nix-grpc-daemon[1827]: time=2026-09-21T14:46:21.577Z level=INFO msg="All 2 paths already cached" worker1 # [ 165.978325] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 165.991952] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 166.144120] nix-daemon[852]: accepted connection from pid 2120, user nix-grpc-daemon (trusted) worker1 # [ 166.161967] nix-daemon[852]: reaped child process 2123, status = succeeded worker1 # [ 166.262694] nix-daemon[852]: accepted connection from pid 2125, user nix-grpc-daemon (trusted) worker1 # [ 166.278193] nix-daemon[852]: reaped child process 2128, status = succeeded lb # [ 166.289749] niks3-server[1112]: 2026/09/21 14:46:21 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 166.294143] nix-grpc-daemon[1827]: time=2026-09-21T14:46:21.902Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 166.295592] nix-grpc-daemon[1827]: time=2026-09-21T14:46:21.902Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)" lb # [ 166.315725] niks3-server[1112]: 2026/09/21 14:46:21 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst worker1 # [ 166.325789] nix-grpc-daemon[1827]: time=2026-09-21T14:46:21.934Z level=INFO msg="Uploading 1 narinfos" lb # [ 166.330866] niks3-server[1112]: 2026/09/21 14:46:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign lb # [ 166.342781] niks3-server[1112]: 2026/09/21 14:46:21 INFO Signed narinfos id=68 count=1 lb # [ 166.348646] niks3-server[1112]: 2026/09/21 14:46:21 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls lb # [ 166.356207] niks3-server[1112]: 2026/09/21 14:46:21 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete worker1 # [ 166.356767] nix-grpc-daemon[1827]: time=2026-09-21T14:46:21.965Z level=INFO msg="Upload complete. (348ms)" lb # [ 166.364114] niks3-server[1112]: 2026/09/21 14:46:21 INFO Completed upload id=68 worker1 # [ 166.364147] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 166.367950] niks3-server[1112]: 2026/09/21 14:46:21 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo worker1 # [ 166.375979] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 166.379462] nix-daemon[852]: reaped child process 2118, status = succeeded worker1 # [ 166.386328] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 166.398631] nix-daemon[852]: accepted connection from pid 1767, user nix-grpc-daemon (trusted) worker1 # [ 167.628201] nix-daemon[852]: reaped child process 2132, status = succeeded worker1 # [ 167.728384] nix-daemon[852]: accepted connection from pid 2148, user nix-grpc-daemon (trusted) worker1 # [ 167.746048] nix-daemon[852]: reaped child process 2151, status = succeeded lb # [ 167.757882] niks3-server[1112]: 2026/09/21 14:46:23 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 167.767160] nix-grpc-daemon[1827]: time=2026-09-21T14:46:23.372Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 167.768769] nix-grpc-daemon[1827]: time=2026-09-21T14:46:23.372Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)" lb # [ 167.797141] niks3-server[1112]: 2026/09/21 14:46:23 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv lb # [ 167.810308] niks3-server[1112]: 2026/09/21 14:46:23 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst worker1 # [ 167.817388] nix-grpc-daemon[1827]: time=2026-09-21T14:46:23.425Z level=INFO msg="Uploading 1 narinfos" lb # [ 167.824440] niks3-server[1112]: 2026/09/21 14:46:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign lb # [ 167.832858] niks3-server[1112]: 2026/09/21 14:46:23 INFO Signed narinfos id=69 count=1 lb # [ 167.835329] niks3-server[1112]: 2026/09/21 14:46:23 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls lb # [ 167.846359] niks3-server[1112]: 2026/09/21 14:46:23 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 167.851716] niks3-server[1112]: 2026/09/21 14:46:23 INFO Completed upload id=69 worker1 # [ 167.847862] nix-grpc-daemon[1827]: time=2026-09-21T14:46:23.456Z level=INFO msg="Upload complete. (240ms)" lb # [ 167.855326] niks3-server[1112]: 2026/09/21 14:46:23 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo worker1 # [ 167.861143] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34656 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=14 outputs=1 worker1 # [ 167.877979] nix-daemon[852]: reaped child process 2130, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.43 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 168.052829] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 865a0a652f284df391777fa094053b01 client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2 # [ 168.227412] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:33404 client # [ 168.256153] nix[1411]: don't know how to build these paths: client # [ 168.257254] nix[1411]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv worker2 # [ 168.245397] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds) worker2 # [ 168.266362] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:23Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=0 client # [ 168.284734] nix[1411]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code lb # [ 176.893472] envoy[888]: conn peer=[2001:db8:1::1]:55428 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27131 tx=11378 ms=18162 client # [ 198.289739] nix[1411]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu} client # [ 198.302185] nix[1411]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed client # [ 198.330757] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 198.341621] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 198.345451] systemd[1]: gpu.service: Consumed 185ms CPU time over 30.287s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.03 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.21 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.21 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1 # [ 200.242789] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:55Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2 client # don't know how to build these paths: client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker1 # [ 201.225143] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:34656 worker1 # [ 201.233310] nix-grpc-daemon[1767]: ts=2026-09-21T14:46:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:34656 worker2 # [ 201.238936] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:56Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=0 worker2 # [ 201.244585] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:56Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 201.284608] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 201.299316] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33404 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'... worker2 # [ 201.315583] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 201.331004] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 201.490620] nix-daemon[838]: accepted connection from pid 1703, user nix-grpc-daemon (trusted) worker2 # [ 201.508872] nix-daemon[838]: reaped child process 1706, status = succeeded worker2 # [ 201.616510] nix-daemon[838]: accepted connection from pid 1708, user nix-grpc-daemon (trusted) worker2 # [ 201.634059] nix-daemon[838]: reaped child process 1711, status = succeeded lb # [ 201.664163] niks3-server[1112]: 2026/09/21 14:46:57 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 201.649810] nix-grpc-daemon[1485]: time=2026-09-21T14:46:57.252Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 201.651281] nix-grpc-daemon[1485]: time=2026-09-21T14:46:57.252Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)" lb # [ 201.704456] niks3-server[1112]: 2026/09/21 14:46:57 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst lb # [ 201.717057] niks3-server[1112]: 2026/09/21 14:46:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign worker2 # [ 201.697404] nix-grpc-daemon[1485]: time=2026-09-21T14:46:57.300Z level=INFO msg="Uploading 1 narinfos" lb # [ 201.723847] niks3-server[1112]: 2026/09/21 14:46:57 INFO Signed narinfos id=70 count=1 lb # [ 201.738700] niks3-server[1112]: 2026/09/21 14:46:57 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls lb # [ 201.750773] niks3-server[1112]: 2026/09/21 14:46:57 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete lb # [ 201.758135] niks3-server[1112]: 2026/09/21 14:46:57 INFO Completed upload id=70 worker2 # [ 201.732032] nix-grpc-daemon[1485]: time=2026-09-21T14:46:57.334Z level=INFO msg="Upload complete. (384ms)" lb # [ 201.759997] niks3-server[1112]: 2026/09/21 14:46:57 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo worker2 # [ 201.743856] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33404 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 201.758552] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 201.765138] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 201.770313] nix-daemon[838]: reaped child process 1701, status = succeeded client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051... client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker2 # [ 201.783254] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 203.414760] nix-daemon[838]: reaped child process 1715, status = succeeded worker2 # [ 203.520088] nix-daemon[838]: accepted connection from pid 1731, user nix-grpc-daemon (trusted) worker2 # [ 203.536474] nix-daemon[838]: reaped child process 1734, status = succeeded lb # [ 203.569274] niks3-server[1112]: 2026/09/21 14:46:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 203.555111] nix-grpc-daemon[1485]: time=2026-09-21T14:46:59.155Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 203.556720] nix-grpc-daemon[1485]: time=2026-09-21T14:46:59.155Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)" lb # [ 203.605788] niks3-server[1112]: 2026/09/21 14:46:59 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 203.619639] niks3-server[1112]: 2026/09/21 14:46:59 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker2 # [ 203.602678] nix-grpc-daemon[1485]: time=2026-09-21T14:46:59.205Z level=INFO msg="Uploading 1 narinfos" lb # [ 203.630939] niks3-server[1112]: 2026/09/21 14:46:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign lb # [ 203.638791] niks3-server[1112]: 2026/09/21 14:46:59 INFO Signed narinfos id=71 count=1 lb # [ 203.645472] niks3-server[1112]: 2026/09/21 14:46:59 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete worker2 # [ 203.626728] nix-grpc-daemon[1485]: time=2026-09-21T14:46:59.229Z level=INFO msg="Upload complete. (233ms)" lb # [ 203.653328] niks3-server[1112]: 2026/09/21 14:46:59 INFO Completed upload id=71 lb # [ 203.657287] niks3-server[1112]: 2026/09/21 14:46:59 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo lb # [ 203.661609] niks3-server[1112]: 2026/09/21 14:46:59 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls worker2 # [ 203.641241] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=15 outputs=1 worker2 # [ 203.657740] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:59Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=0 worker2 # [ 203.662768] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:59Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 203.667541] nix-daemon[838]: reaped child process 1713, status = succeeded worker2 # [ 203.703673] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 203.718553] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 203.732404] nix-grpc-daemon[1485]: time=2026-09-21T14:46:59.335Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'... worker2 # [ 203.743214] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 203.756733] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 203.911064] nix-daemon[838]: accepted connection from pid 1740, user nix-grpc-daemon (trusted) worker2 # [ 203.928676] nix-daemon[838]: reaped child process 1743, status = succeeded worker2 # [ 204.033879] nix-daemon[838]: accepted connection from pid 1745, user nix-grpc-daemon (trusted) worker2 # [ 204.049586] nix-daemon[838]: reaped child process 1748, status = succeeded lb # [ 204.081179] niks3-server[1112]: 2026/09/21 14:46:59 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 204.065377] nix-grpc-daemon[1485]: time=2026-09-21T14:46:59.668Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 204.066892] nix-grpc-daemon[1485]: time=2026-09-21T14:46:59.668Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)" lb # [ 204.107692] niks3-server[1112]: 2026/09/21 14:46:59 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst worker2 # [ 204.101807] nix-grpc-daemon[1485]: time=2026-09-21T14:46:59.704Z level=INFO msg="Uploading 1 narinfos" lb # [ 204.126090] niks3-server[1112]: 2026/09/21 14:46:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign lb # [ 204.136906] niks3-server[1112]: 2026/09/21 14:46:59 INFO Signed narinfos id=72 count=1 lb # [ 204.144538] niks3-server[1112]: 2026/09/21 14:46:59 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls lb # [ 204.155418] niks3-server[1112]: 2026/09/21 14:46:59 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete worker2 # [ 204.135980] nix-grpc-daemon[1485]: time=2026-09-21T14:46:59.738Z level=INFO msg="Upload complete. (362ms)" lb # [ 204.162113] niks3-server[1112]: 2026/09/21 14:46:59 INFO Completed upload id=72 lb # [ 204.166285] niks3-server[1112]: 2026/09/21 14:46:59 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo worker2 # [ 204.144977] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33404 duration_s=0 paths=1 nar_bytes_in=632 worker2 # [ 204.158237] nix-grpc-daemon[1407]: ts=2026-09-21T14:46:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 204.167008] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 204.170449] nix-daemon[838]: reaped child process 1738, status = succeeded client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051... worker2 # [ 204.182339] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 205.656658] nix-daemon[838]: reaped child process 1752, status = succeeded worker2 # [ 205.762530] nix-daemon[838]: accepted connection from pid 1768, user nix-grpc-daemon (trusted) worker2 # [ 205.782642] nix-daemon[838]: reaped child process 1771, status = succeeded lb # [ 205.813130] niks3-server[1112]: 2026/09/21 14:47:01 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 205.800420] nix-grpc-daemon[1485]: time=2026-09-21T14:47:01.400Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 205.802062] nix-grpc-daemon[1485]: time=2026-09-21T14:47:01.400Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)" lb # [ 205.849709] niks3-server[1112]: 2026/09/21 14:47:01 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv lb # [ 205.866580] niks3-server[1112]: 2026/09/21 14:47:01 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst lb # [ 205.879345] niks3-server[1112]: 2026/09/21 14:47:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign lb # [ 205.884269] niks3-server[1112]: 2026/09/21 14:47:01 INFO Signed narinfos id=73 count=1 worker2 # [ 205.859653] nix-grpc-daemon[1485]: time=2026-09-21T14:47:01.462Z level=INFO msg="Uploading 1 narinfos" lb # [ 205.891979] niks3-server[1112]: 2026/09/21 14:47:01 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls lb # [ 205.900687] niks3-server[1112]: 2026/09/21 14:47:01 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 205.908307] niks3-server[1112]: 2026/09/21 14:47:01 INFO Completed upload id=73 worker2 # [ 205.882971] nix-grpc-daemon[1485]: time=2026-09-21T14:47:01.485Z level=INFO msg="Upload complete. (287ms)" lb # [ 205.910787] niks3-server[1112]: 2026/09/21 14:47:01 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo worker2 # [ 205.901015] nix-grpc-daemon[1407]: ts=2026-09-21T14:47:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=16 outputs=1 worker2 # [ 205.916377] nix-grpc-daemon[1407]: ts=2026-09-21T14:47:01Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=0 worker2 # [ 205.922830] nix-grpc-daemon[1407]: ts=2026-09-21T14:47:01Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 205.927277] nix-daemon[838]: reaped child process 1750, status = succeeded worker2 # [ 205.943693] nix-grpc-daemon[1407]: ts=2026-09-21T14:47:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 205.958936] nix-grpc-daemon[1407]: ts=2026-09-21T14:47:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 205.973316] nix-grpc-daemon[1485]: time=2026-09-21T14:47:01.576Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'... worker2 # [ 205.983651] nix-grpc-daemon[1407]: ts=2026-09-21T14:47:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 205.997541] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 206.157427] nix-daemon[838]: accepted connection from pid 1777, user nix-grpc-daemon (trusted) worker2 # [ 206.183205] nix-daemon[838]: reaped child process 1780, status = succeeded worker2 # [ 206.278623] nix-daemon[838]: accepted connection from pid 1782, user nix-grpc-daemon (trusted) worker2 # [ 206.293553] nix-daemon[838]: reaped child process 1785, status = succeeded lb # [ 206.325384] niks3-server[1112]: 2026/09/21 14:47:01 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 206.309504] nix-grpc-daemon[1485]: time=2026-09-21T14:47:01.912Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 206.311014] nix-grpc-daemon[1485]: time=2026-09-21T14:47:01.912Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)" lb # [ 206.351590] niks3-server[1112]: 2026/09/21 14:47:01 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst worker2 # [ 206.341106] nix-grpc-daemon[1485]: time=2026-09-21T14:47:01.943Z level=INFO msg="Uploading 1 narinfos" lb # [ 206.365452] niks3-server[1112]: 2026/09/21 14:47:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign lb # [ 206.375607] niks3-server[1112]: 2026/09/21 14:47:01 INFO Signed narinfos id=74 count=1 lb # [ 206.383679] niks3-server[1112]: 2026/09/21 14:47:01 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls lb # [ 206.396211] niks3-server[1112]: 2026/09/21 14:47:01 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete worker2 # [ 206.376129] nix-grpc-daemon[1485]: time=2026-09-21T14:47:01.979Z level=INFO msg="Upload complete. (362ms)" lb # [ 206.405659] niks3-server[1112]: 2026/09/21 14:47:02 INFO Completed upload id=74 lb # [ 206.408753] niks3-server[1112]: 2026/09/21 14:47:02 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo worker2 # [ 206.384746] nix-grpc-daemon[1407]: ts=2026-09-21T14:47:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33404 duration_s=0 paths=1 nar_bytes_in=760 worker2 # [ 206.392593] nix-grpc-daemon[1407]: ts=2026-09-21T14:47:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 worker2 # [ 206.404410] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) worker2 # [ 206.406966] nix-daemon[838]: reaped child process 1775, status = succeeded worker2 # [ 206.417217] nix-daemon[838]: accepted connection from pid 1407, user nix-grpc-daemon (trusted) client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051... client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv lb # [ 206.896668] envoy[888]: conn peer=[2001:db8:1::1]:57888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=500 ms=30067 client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 208.326502] nix-daemon[838]: reaped child process 1789, status = succeeded worker2 # [ 208.398904] nix-daemon[838]: accepted connection from pid 1810, user nix-grpc-daemon (trusted) worker2 # [ 208.417468] nix-daemon[838]: reaped child process 1813, status = succeeded lb # [ 208.446871] niks3-server[1112]: 2026/09/21 14:47:04 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 208.435761] nix-grpc-daemon[1485]: time=2026-09-21T14:47:04.037Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 208.437651] nix-grpc-daemon[1485]: time=2026-09-21T14:47:04.037Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)" lb # [ 208.488085] niks3-server[1112]: 2026/09/21 14:47:04 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv lb # [ 208.498434] niks3-server[1112]: 2026/09/21 14:47:04 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst worker2 # [ 208.485608] nix-grpc-daemon[1485]: time=2026-09-21T14:47:04.088Z level=INFO msg="Uploading 1 narinfos" lb # [ 208.513394] niks3-server[1112]: 2026/09/21 14:47:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign lb # [ 208.522327] niks3-server[1112]: 2026/09/21 14:47:04 INFO Signed narinfos id=75 count=1 lb # [ 208.529718] niks3-server[1112]: 2026/09/21 14:47:04 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls worker2 # [ 208.511546] nix-grpc-daemon[1485]: time=2026-09-21T14:47:04.114Z level=INFO msg="Upload complete. (281ms)" lb # [ 208.538239] niks3-server[1112]: 2026/09/21 14:47:04 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete lb # [ 208.543900] niks3-server[1112]: 2026/09/21 14:47:04 INFO Completed upload id=75 lb # [ 208.547967] niks3-server[1112]: 2026/09/21 14:47:04 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo worker2 # [ 208.524783] nix-grpc-daemon[1407]: ts=2026-09-21T14:47:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33404 duration_s=2 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=17 outputs=1 worker2 # [ 208.544257] nix-daemon[838]: reaped child process 1787, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 7.58 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 209.253254] nix-grpc-daemon[1767]: ts=2026-09-21T14:47:04Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 11.39 seconds) (finished: run the VM test script, in 211.01 seconds) test script finished in 211.26s cleanup kill QemuMachine (pid 50) client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) client # [2026-09-21T14:47:05Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-21T14:47:05Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-21T14:47:05Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 49) lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) lb # [2026-09-21T14:47:05Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-21T14:47:05Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-21T14:47:05Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker1 # [2026-09-21T14:47:06Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-21T14:47:06Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-21T14:47:06Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker2 # [2026-09-21T14:47:06Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-21T14:47:06Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-21T14:47:06Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.19 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh