tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 900.0 seconds run the VM test script start all VMs client: starting vm lb: starting vm worker1: starting vm worker2: starting vm client # Disk image does not exist, creating the virtualisation disk image... worker2 # Disk image does not exist, creating the virtualisation disk image... client # Formatting '/build/vm-state-client/tmp.89eV63jx49', fmt=raw size=1073741824 client # mke2fs 1.47.4 (6-Mar-2025) worker2: QEMU running (pid 51) worker2 # Formatting '/build/vm-state-worker2/tmp.EvP4wNsNzk', fmt=raw size=1073741824 client: QEMU running (pid 49) worker1 # Disk image does not exist, creating the virtualisation disk image... worker2 # mke2fs 1.47.4 (6-Mar-2025) lb # Disk image does not exist, creating the virtualisation disk image... lb: QEMU running (pid 50) worker2 # Discarding device blocks: 0/262144 done lb # Formatting '/build/vm-state-lb/tmp.zUFWG6t6bZ', fmt=raw size=1073741824 client # Discarding device blocks: 0/262144 done worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes worker2 # Filesystem UUID: 218d393b-2fac-4c4a-8d56-08b2222e1a82 client # Creating filesystem with 262144 4k blocks and 65536 inodes worker1: QEMU running (pid 52) worker1 # Formatting '/build/vm-state-worker1/tmp.ilbWMUSjL6', fmt=raw size=1073741824 client # Filesystem UUID: 386ec444-c406-47a2-b2dd-0117e896c256 lb # mke2fs 1.47.4 (6-Mar-2025) worker2 # Superblock backups stored on blocks: worker1 # mke2fs 1.47.4 (6-Mar-2025) client # Superblock backups stored on blocks: lb # Discarding device blocks: 0/262144 done worker1 # Discarding device blocks: 0/262144 done client # 32768, 98304, 163840, 229376 lb # Creating filesystem with 262144 4k blocks and 65536 inodes worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes client # lb # Filesystem UUID: c0390e1f-1753-470d-be95-7d18f0ff2ba3 worker1 # Filesystem UUID: f896596c-c5ce-4568-8087-8dcfd162026c client # Allocating group tables: 0/8 done lb # Superblock backups stored on blocks: worker1 # Superblock backups stored on blocks: client # Writing inode tables: 0/8 done lb # 32768, 98304, 163840, 229376 worker1 # 32768, 98304, 163840, 229376 client # Creating journal (8192 blocks): done lb # worker1 # client # Writing superblocks and filesystem accounting information: 0/8 done lb # Allocating group tables: 0/8 done worker1 # Allocating group tables: 0/8 done client # lb # Writing inode tables: 0/8 done worker1 # Writing inode tables: 0/8 done client # Virtualisation disk image created. lb # Creating journal (8192 blocks): done worker1 # Creating journal (8192 blocks): done client # Starting virtiofs daemons... lb # Writing superblocks and filesystem accounting information: 0/8 done worker1 # Writing superblocks and filesystem accounting information: 0/8 done client # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # worker1 # client # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # Virtualisation disk image created. worker1 # Virtualisation disk image created. client # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # Starting virtiofs daemons... worker1 # Starting virtiofs daemons... client # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) lb # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether lb # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection... lb # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection... client # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker1 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) client # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker1 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether client # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests lb # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection... worker1 # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # 32768, 98304, 163840, 229376 (finished: start all VMs, in 0.51 seconds) lb # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests worker2 # client: waiting for unit network-addresses-eth1.service lb # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests worker2 # Allocating group tables: 0/8 done client: waiting for the VM to finish booting lb # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests worker1 # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests worker2 # Writing inode tables: 0/8 done worker2 # Creating journal (8192 blocks): done worker2 # Writing superblocks and filesystem accounting information: 0/8 done worker2 # worker2 # Virtualisation disk image created. worker2 # Starting virtiofs daemons... worker2 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1) worker2 # [2026-09-23T05:12:35Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether worker2 # [2026-09-23T05:12:35Z INFO virtiofsd] Waiting for vhost-user socket connection... worker2 # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests worker2 # [2026-09-23T05:12:35Z INFO virtiofsd] Client connected, servicing requests client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 client # [ 0.000000] KASLR enabled client # [ 0.000000] random: crng init done client # [ 0.000000] Machine model: linux,dummy-virt worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] client # [ 0.000000] efi: UEFI not found. worker1 # [ 0.000000] KASLR enabled worker1 # [ 0.000000] random: crng init done client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT worker1 # [ 0.000000] Machine model: linux,dummy-virt worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] efi: UEFI not found. worker2 # [ 0.000000] KASLR enabled client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] random: crng init done worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] Machine model: linux,dummy-virt client # [ 0.000000] Zone ranges: worker2 # [ 0.000000] efi: UEFI not found. worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] DMA32 empty worker1 # [ 0.000000] Zone ranges: client # [ 0.000000] Normal empty client # [ 0.000000] Device empty worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] Movable zone start for each node worker1 # [ 0.000000] DMA32 empty worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] worker1 # [ 0.000000] Normal empty client # [ 0.000000] Early memory node ranges worker2 # [ 0.000000] Zone ranges: worker1 # [ 0.000000] Device empty client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] Movable zone start for each node worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] DMA32 empty worker1 # [ 0.000000] Early memory node ranges worker2 # [ 0.000000] Normal empty client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] Device empty worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] Movable zone start for each node worker2 # [ 0.000000] Early memory node ranges worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] psci: probing for conduit method from DT. client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] psci: Trusted OS migration not required worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker2 # [ 0.000000] psci: probing for conduit method from DT. worker1 # [ 0.000000] psci: Trusted OS migration not required worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware. client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) worker1 # [ 0.000000] psci: SMC Calling Convention v1.1 worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 worker2 # [ 0.000000] psci: Trusted OS migration not required worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] Detected PIPT I-cache on CPU0 worker2 # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] Detected PIPT I-cache on CPU0 worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) client # [ 0.000000] CPU features: detected: Spectre-v4 worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 client # [ 0.000000] CPU features: detected: Spectre-BHB worker2 # [ 0.000000] Detected PIPT I-cache on CPU0 worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker1 # [ 0.000000] CPU features: detected: Spectre-BHB client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker2 # [ 0.000000] CPU features: detected: Spectre-v4 client # [ 0.000000] alternatives: applying boot alternatives worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker2 # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000000] alternatives: applying boot alternatives worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 worker2 # [ 0.000000] alternatives: applying boot alternatives client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test/init regInfo=/nix/store/i5mcvvp2ppsizw0yf6wy574nqpq3ajr4-closure-info/registration console=ttyAMA0,115200n8 console=tty0 worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0hxczffswy9q3fzf9cy2bm1icq25s5b8-nixos-system-worker1-test/init regInfo=/nix/store/gxyq0wqq5b5k7nykh0671gkp38b0z5pq-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/i5mcvvp2ppsizw0yf6wy574nqpq3ajr4-closure-info/registration", will be passed to user space. worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gjdzrza82g2nqw3dgr89lmwyjzkwq204-nixos-system-worker2-test/init regInfo=/nix/store/6zp3n5xwb020l9g406n4c13fq3abx4ma-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/gxyq0wqq5b5k7nykh0671gkp38b0z5pq-closure-info/registration", will be passed to user space. client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/6zp3n5xwb020l9g406n4c13fq3abx4ma-closure-info/registration", will be passed to user space. client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) client # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker1 # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB client # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] software IO TLB: area num 1. worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker1 # [ 0.000000] Fallback order for Node 0: 0 worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB client # [ 0.000000] Policy zone: DMA worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker1 # [ 0.000000] Policy zone: DMA worker2 # [ 0.000000] Fallback order for Node 0: 0 client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 client # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.000000] Policy zone: DMA worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker1 # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 client # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker2 # [ 0.000000] allocated 3145728 bytes of page_ext client # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.000000] rcu: Hierarchical RCU implementation. worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000000] rcu: RCU event tracing is enabled. worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups client # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.000000] rcu: Hierarchical RCU implementation. worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.000000] Rude variant of Tasks RCU enabled. worker2 # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled. client # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.000000] Rude variant of Tasks RCU enabled. worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled. client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled. lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] worker2 # [ 0.000000] Rude variant of Tasks RCU enabled. worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled. lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026 worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] KASLR enabled worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. lb # [ 0.000000] random: crng init done worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] Machine model: linux,dummy-virt worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] efi: UEFI not found. worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff] client # [ 0.000000] GICv3: 256 SPIs implemented worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f] client # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] Zone ranges: worker1 # [ 0.000000] GICv3: 256 SPIs implemented worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. client # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.000000] DMA32 empty worker2 # [ 0.000000] GICv3: 256 SPIs implemented lb # [ 0.000000] Normal empty client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented lb # [ 0.000000] Device empty worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 lb # [ 0.000000] Movable zone start for each node worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq lb # [ 0.000000] Early memory node ranges client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff] worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff] worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000 client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] psci: probing for conduit method from DT. worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1) client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] psci: PSCIv1.3 detected in firmware. worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1) lb # [ 0.000000] psci: Trusted OS migration not required worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000 lb # [ 0.000000] psci: SMC Calling Convention v1.1 worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000 lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 lb # [ 0.000000] Detected PIPT I-cache on CPU0 worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.000000] CPU features: detected: GICv3 CPU interface client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns client # [ 0.000033] arm-pv: using stolen time PV lb # [ 0.000000] CPU features: detected: Spectre-v4 worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns lb # [ 0.000000] CPU features: detected: Spectre-BHB worker1 # [ 0.000033] arm-pv: using stolen time PV worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns client # [ 0.000490] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 worker2 # [ 0.000030] arm-pv: using stolen time PV client # [ 0.000687] Console: colour dummy device 80x25 worker1 # [ 0.000567] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 client # [ 0.000694] printk: legacy console [tty0] enabled worker1 # [ 0.000772] Console: colour dummy device 80x25 lb # [ 0.000000] alternatives: applying boot alternatives worker2 # [ 0.000468] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) worker1 # [ 0.000779] printk: legacy console [tty0] enabled worker2 # [ 0.000678] Console: colour dummy device 80x25 client # [ 0.000916] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker2 # [ 0.000686] printk: legacy console [tty0] enabled client # [ 0.000923] pid_max: default: 32768 minimum: 301 worker1 # [ 0.000966] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) worker1 # [ 0.000973] pid_max: default: 32768 minimum: 301 client # [ 0.000999] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.000889] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) client # [ 0.001179] landlock: Up and running. worker1 # [ 0.001049] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker2 # [ 0.000896] pid_max: default: 32768 minimum: 301 client # [ 0.001182] Yama: becoming mindful. worker1 # [ 0.001184] landlock: Up and running. lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kgjzg78npi7z07am2szqxkr41f06jp5w-nixos-system-lb-test/init regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration console=ttyAMA0,115200n8 console=tty0 client # [ 0.001784] LSM support for eBPF active worker1 # [ 0.001186] Yama: becoming mindful. worker2 # [ 0.000983] LSM: initializing lsm=capability,landlock,yama,bpf,ima worker1 # [ 0.001683] LSM support for eBPF active worker2 # [ 0.001164] landlock: Up and running. client # [ 0.001951] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.001167] Yama: becoming mindful. worker1 # [ 0.001859] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.001777] LSM support for eBPF active lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration", will be passed to user space. client # [ 0.001984] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker1 # [ 0.001895] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.003858] rcu: Hierarchical SRCU implementation. worker2 # [ 0.001921] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes worker1 # [ 0.003904] rcu: Hierarchical SRCU implementation. client # [ 0.003864] rcu: Max phase no-delay instances is 1000. worker2 # [ 0.001954] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear) worker1 # [ 0.003910] rcu: Max phase no-delay instances is 1000. client # [ 0.005197] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.003853] rcu: Hierarchical SRCU implementation. worker1 # [ 0.005192] fsl-mc MSI: its@8080000 domain created client # [ 0.005289] EFI services will not be available. lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear) worker2 # [ 0.003859] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.005281] EFI services will not be available. client # [ 0.005359] smp: Bringing up secondary CPUs ... worker2 # [ 0.005136] fsl-mc MSI: its@8080000 domain created lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB client # [ 0.005367] smp: Brought up 1 node, 1 CPU worker1 # [ 0.005374] smp: Bringing up secondary CPUs ... lb # [ 0.000000] software IO TLB: area num 1. worker2 # [ 0.005223] EFI services will not be available. worker1 # [ 0.005382] smp: Brought up 1 node, 1 CPU client # [ 0.005370] SMP: Total of 1 processors activated. worker2 # [ 0.005292] smp: Bringing up secondary CPUs ... worker1 # [ 0.005385] SMP: Total of 1 processors activated. client # [ 0.005376] CPU: All CPU(s) started at EL1 lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB worker2 # [ 0.005300] smp: Brought up 1 node, 1 CPU worker1 # [ 0.005392] CPU: All CPU(s) started at EL1 client # [ 0.005387] CPU features: detected: Branch Target Identification worker2 # [ 0.005303] SMP: Total of 1 processors activated. lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB) worker1 # [ 0.005403] CPU features: detected: Branch Target Identification client # [ 0.005392] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.005309] CPU: All CPU(s) started at EL1 lb # [ 0.000000] Fallback order for Node 0: 0 worker1 # [ 0.005407] CPU features: detected: ARMv8.4 Translation Table Level worker2 # [ 0.005320] CPU features: detected: Branch Target Identification lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216 client # [ 0.005395] CPU features: detected: Instruction cache invalidation not required for I/D coherence worker2 # [ 0.005325] CPU features: detected: ARMv8.4 Translation Table Level worker1 # [ 0.005410] CPU features: detected: Instruction cache invalidation not required for I/D coherence lb # [ 0.000000] Policy zone: DMA client # [ 0.005399] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker1 # [ 0.005414] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker2 # [ 0.005327] CPU features: detected: Instruction cache invalidation not required for I/D coherence lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off client # [ 0.005403] CPU features: detected: Common not Private translations worker1 # [ 0.005418] CPU features: detected: Common not Private translations client # [ 0.005406] CPU features: detected: CRC32 instructions lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 worker2 # [ 0.005331] CPU features: detected: Data cache clean to the PoU not required for I/D coherence worker1 # [ 0.005421] CPU features: detected: CRC32 instructions lb # [ 0.000000] allocated 3145728 bytes of page_ext worker2 # [ 0.005335] CPU features: detected: Common not Private translations client # [ 0.005409] CPU features: detected: Data cache clean to Point of Deep Persistence lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages worker1 # [ 0.005424] CPU features: detected: Data cache clean to Point of Deep Persistence worker2 # [ 0.005338] CPU features: detected: CRC32 instructions client # [ 0.005412] CPU features: detected: Data cache clean to Point of Persistence lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups worker1 # [ 0.005427] CPU features: detected: Data cache clean to Point of Persistence worker2 # [ 0.005341] CPU features: detected: Data cache clean to Point of Deep Persistence lb # [ 0.000000] rcu: Hierarchical RCU implementation. client # [ 0.005415] CPU features: detected: Data independent timing control (DIT) lb # [ 0.000000] rcu: RCU event tracing is enabled. worker1 # [ 0.005430] CPU features: detected: Data independent timing control (DIT) client # [ 0.005418] CPU features: detected: E0PD worker2 # [ 0.005344] CPU features: detected: Data cache clean to Point of Persistence worker1 # [ 0.005433] CPU features: detected: E0PD lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1. client # [ 0.005421] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.005348] CPU features: detected: Data independent timing control (DIT) worker1 # [ 0.005436] CPU features: detected: Enhanced Counter Virtualization lb # [ 0.000000] Trampoline variant of Tasks RCU enabled. worker2 # [ 0.005351] CPU features: detected: E0PD client # [ 0.005424] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) lb # [ 0.000000] Rude variant of Tasks RCU enabled. worker1 # [ 0.005439] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) worker2 # [ 0.005353] CPU features: detected: Enhanced Counter Virtualization client # [ 0.005428] CPU features: detected: Enhanced Virtualization Traps lb # [ 0.000000] Tracing variant of Tasks RCU enabled. worker1 # [ 0.005443] CPU features: detected: Enhanced Virtualization Traps worker2 # [ 0.005356] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.005431] CPU features: detected: Fine Grained Traps lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. worker1 # [ 0.005445] CPU features: detected: Fine Grained Traps worker2 # [ 0.005360] CPU features: detected: Enhanced Virtualization Traps client # [ 0.005434] CPU features: detected: Generic authentication (architected QARMA5 algorithm) lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1 worker2 # [ 0.005363] CPU features: detected: Fine Grained Traps worker1 # [ 0.005449] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.005439] CPU features: detected: RCpc load-acquire (LDAPR) worker1 # [ 0.005454] CPU features: detected: RCpc load-acquire (LDAPR) lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker2 # [ 0.005366] CPU features: detected: Generic authentication (architected QARMA5 algorithm) client # [ 0.005442] CPU features: detected: LSE atomic instructions worker1 # [ 0.005457] CPU features: detected: LSE atomic instructions worker2 # [ 0.005371] CPU features: detected: RCpc load-acquire (LDAPR) client # [ 0.005445] CPU features: detected: Privileged Access Never lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.005460] CPU features: detected: Privileged Access Never client # [ 0.005448] CPU features: detected: PMUv3 worker2 # [ 0.005374] CPU features: detected: LSE atomic instructions worker1 # [ 0.005462] CPU features: detected: PMUv3 client # [ 0.005451] CPU features: detected: RAS Extension Support worker2 # [ 0.005377] CPU features: detected: Privileged Access Never lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1. worker1 # [ 0.005465] CPU features: detected: RAS Extension Support worker2 # [ 0.005379] CPU features: detected: PMUv3 client # [ 0.005454] CPU features: detected: RASv1p1 Extension Support lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 worker1 # [ 0.005468] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.005382] CPU features: detected: RAS Extension Support lb # [ 0.000000] GICv3: 256 SPIs implemented client # [ 0.005456] CPU features: detected: Random Number Generator worker1 # [ 0.005471] CPU features: detected: Random Number Generator lb # [ 0.000000] GICv3: 0 Extended SPIs implemented worker2 # [ 0.005385] CPU features: detected: RASv1p1 Extension Support client # [ 0.005459] CPU features: detected: Speculation barrier (SB) lb # [ 0.000000] Root IRQ handler: gic_handle_irq worker1 # [ 0.005474] CPU features: detected: Speculation barrier (SB) worker2 # [ 0.005387] CPU features: detected: Random Number Generator client # [ 0.005462] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI worker1 # [ 0.005476] CPU features: detected: Stage-2 Force Write-Back worker2 # [ 0.005390] CPU features: detected: Speculation barrier (SB) lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 client # [ 0.005465] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.005479] CPU features: detected: TLB range maintenance instructions worker2 # [ 0.005393] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 client # [ 0.005470] CPU features: detected: Speculative Store Bypassing Safe (SSBS) worker2 # [ 0.005395] CPU features: detected: TLB range maintenance instructions worker1 # [ 0.005485] CPU features: detected: Speculative Store Bypassing Safe (SSBS) lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] client # [ 0.005508] alternatives: applying system-wide alternatives worker1 # [ 0.005522] alternatives: applying system-wide alternatives worker2 # [ 0.005400] CPU features: detected: Speculative Store Bypassing Safe (SSBS) client # [ 0.008543] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1) worker1 # [ 0.008600] CPU features: detected: BBM Level 2 without TLB conflict abort worker2 # [ 0.005437] alternatives: applying system-wide alternatives worker2 # [ 0.008383] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1) client # [ 0.008729] Memory: 1405632K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved) lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000 worker1 # [ 0.008753] Memory: 1405652K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved) client # [ 0.009169] devtmpfs: initialized worker1 # [ 0.009168] devtmpfs: initialized lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000 worker2 # [ 0.008576] Memory: 1405672K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved) client # [ 0.011023] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker2 # [ 0.008976] devtmpfs: initialized worker1 # [ 0.011058] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. client # [ 0.011052] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker2 # [ 0.010839] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.011081] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.011240] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns worker2 # [ 0.010862] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). worker1 # [ 0.011273] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.011245] 0 pages in range for non-PLT usage lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). worker1 # [ 0.011278] 0 pages in range for non-PLT usage worker2 # [ 0.011049] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL client # [ 0.011246] 508272 pages in range for PLT usage worker1 # [ 0.011279] 508272 pages in range for PLT usage worker2 # [ 0.011053] 0 pages in range for non-PLT usage client # [ 0.011355] pinctrl core: initialized pinctrl subsystem worker1 # [ 0.011401] pinctrl core: initialized pinctrl subsystem lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns client # [ 0.012087] DMI not present or invalid. worker2 # [ 0.011054] 508272 pages in range for PLT usage worker1 # [ 0.012198] DMI not present or invalid. worker2 # [ 0.011171] pinctrl core: initialized pinctrl subsystem client # [ 0.015536] NET: Registered PF_NETLINK/PF_ROUTE protocol family lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns worker1 # [ 0.015629] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.011985] DMI not present or invalid. lb # [ 0.000030] arm-pv: using stolen time PV client # [ 0.017819] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker2 # [ 0.015388] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker1 # [ 0.017918] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations lb # [ 0.000391] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) client # [ 0.018028] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.017653] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations worker1 # [ 0.018109] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations lb # [ 0.000593] Console: colour dummy device 80x25 worker2 # [ 0.017846] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations client # [ 0.018237] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.000601] printk: legacy console [tty0] enabled worker1 # [ 0.018315] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations client # [ 0.018261] audit: initializing netlink subsys (disabled) worker1 # [ 0.018339] audit: initializing netlink subsys (disabled) worker2 # [ 0.018051] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations lb # [ 0.000795] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) client # [ 0.018898] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.018979] thermal_sys: Registered thermal governor 'fair_share' worker2 # [ 0.018075] audit: initializing netlink subsys (disabled) lb # [ 0.000802] pid_max: default: 32768 minimum: 301 client # [ 0.018900] thermal_sys: Registered thermal governor 'bang_bang' worker2 # [ 0.018647] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.018981] thermal_sys: Registered thermal governor 'bang_bang' lb # [ 0.000883] LSM: initializing lsm=capability,landlock,yama,bpf,ima client # [ 0.018904] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.018649] thermal_sys: Registered thermal governor 'bang_bang' worker1 # [ 0.018985] thermal_sys: Registered thermal governor 'step_wise' lb # [ 0.001063] landlock: Up and running. lb # [ 0.001066] Yama: becoming mindful. client # [ 0.018907] thermal_sys: Registered thermal governor 'user_space' worker2 # [ 0.018653] thermal_sys: Registered thermal governor 'step_wise' worker1 # [ 0.018988] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.001526] LSM support for eBPF active worker2 # [ 0.018656] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.001677] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) client # [ 0.018913] thermal_sys: Registered thermal governor 'power_allocator' worker1 # [ 0.018994] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.001709] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear) worker2 # [ 0.018661] thermal_sys: Registered thermal governor 'power_allocator' client # [ 0.018948] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 worker1 # [ 0.019021] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.003524] rcu: Hierarchical SRCU implementation. client # [ 0.018956] cpuidle: using governor ladder worker1 # [ 0.019029] cpuidle: using governor ladder worker2 # [ 0.018694] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 client # [ 0.018962] cpuidle: using governor menu lb # [ 0.003530] rcu: Max phase no-delay instances is 1000. worker1 # [ 0.019035] cpuidle: using governor menu worker2 # [ 0.018702] cpuidle: using governor ladder lb # [ 0.004777] fsl-mc MSI: its@8080000 domain created worker2 # [ 0.018708] cpuidle: using governor menu client # [ 0.019159] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.019262] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.004873] EFI services will not be available. client # [ 0.019174] ASID allocator initialised with 65536 entries worker2 # [ 0.018907] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. worker1 # [ 0.019279] ASID allocator initialised with 65536 entries lb # [ 0.004983] smp: Bringing up secondary CPUs ... client # [ 0.020376] Serial: AMBA PL011 UART driver lb # [ 0.004992] smp: Brought up 1 node, 1 CPU worker1 # [ 0.020521] Serial: AMBA PL011 UART driver worker2 # [ 0.018922] ASID allocator initialised with 65536 entries lb # [ 0.004995] SMP: Total of 1 processors activated. worker2 # [ 0.020139] Serial: AMBA PL011 UART driver client # [ 0.025722] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.005000] CPU: All CPU(s) started at EL1 worker1 # [ 0.025807] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 client # [ 0.025859] printk: console [ttyAMA0] enabled worker1 # [ 0.025975] printk: console [ttyAMA0] enabled worker2 # [ 0.025456] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.005010] CPU features: detected: Branch Target Identification worker2 # [ 0.025597] printk: console [ttyAMA0] enabled lb # [ 0.005015] CPU features: detected: ARMv8.4 Translation Table Level lb # [ 0.005018] CPU features: detected: Instruction cache invalidation not required for I/D coherence lb # [ 0.005021] CPU features: detected: Data cache clean to the PoU not required for I/D coherence lb # [ 0.005025] CPU features: detected: Common not Private translations lb # [ 0.005028] CPU features: detected: CRC32 instructions lb # [ 0.005031] CPU features: detected: Data cache clean to Point of Deep Persistence lb # [ 0.005034] CPU features: detected: Data cache clean to Point of Persistence client # [ 0.144262] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.005037] CPU features: detected: Data independent timing control (DIT) worker1 # [ 0.143172] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.005040] CPU features: detected: E0PD worker2 # [ 0.141729] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages client # [ 0.144281] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page worker1 # [ 0.143192] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.005043] CPU features: detected: Enhanced Counter Virtualization worker2 # [ 0.141748] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page client # [ 0.144287] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker1 # [ 0.143197] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages worker2 # [ 0.141754] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages lb # [ 0.005046] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) client # [ 0.144291] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page worker1 # [ 0.143202] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.005050] CPU features: detected: Enhanced Virtualization Traps worker2 # [ 0.141759] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page client # [ 0.144296] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages worker1 # [ 0.143206] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.005052] CPU features: detected: Fine Grained Traps worker2 # [ 0.141763] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages client # [ 0.144300] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page worker1 # [ 0.143211] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.005056] CPU features: detected: Generic authentication (architected QARMA5 algorithm) worker2 # [ 0.141767] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page client # [ 0.144304] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages worker1 # [ 0.143215] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.005060] CPU features: detected: RCpc load-acquire (LDAPR) worker2 # [ 0.141772] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages client # [ 0.144308] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page worker1 # [ 0.143219] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.005063] CPU features: detected: LSE atomic instructions worker2 # [ 0.141776] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.005066] CPU features: detected: Privileged Access Never lb # [ 0.005068] CPU features: detected: PMUv3 lb # [ 0.005072] CPU features: detected: RAS Extension Support client # [ 0.151699] fbcon: Taking over console worker1 # [ 0.150627] fbcon: Taking over console client # [ 0.151717] ACPI: Interpreter disabled. lb # [ 0.005074] CPU features: detected: RASv1p1 Extension Support worker2 # [ 0.149010] fbcon: Taking over console worker1 # [ 0.150643] ACPI: Interpreter disabled. worker2 # [ 0.149026] ACPI: Interpreter disabled. lb # [ 0.005077] CPU features: detected: Random Number Generator lb # [ 0.005080] CPU features: detected: Speculation barrier (SB) lb # [ 0.005082] CPU features: detected: Stage-2 Force Write-Back lb # [ 0.005085] CPU features: detected: TLB range maintenance instructions lb # [ 0.005090] CPU features: detected: Speculative Store Bypassing Safe (SSBS) lb # [ 0.005126] alternatives: applying system-wide alternatives lb # [ 0.008077] CPU features: detected: BBM Level 2 without TLB conflict abort lb # [ 0.008279] Memory: 1405640K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved) lb # [ 0.008641] devtmpfs: initialized client # [ 0.160712] iommu: Default domain type: Translated lb # [ 0.010454] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear) worker1 # [ 0.159663] iommu: Default domain type: Translated worker2 # [ 0.158065] iommu: Default domain type: Translated client # [ 0.160722] iommu: DMA domain TLB invalidation policy: strict mode worker1 # [ 0.159675] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.010476] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear). client # [ 0.161116] SCSI subsystem initialized worker2 # [ 0.158076] iommu: DMA domain TLB invalidation policy: strict mode worker1 # [ 0.160051] SCSI subsystem initialized worker2 # [ 0.158432] SCSI subsystem initialized lb # [ 0.010667] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL lb # [ 0.010672] 0 pages in range for non-PLT usage client # [ 0.163088] usbcore: registered new interface driver usbfs worker1 # [ 0.161997] usbcore: registered new interface driver usbfs worker2 # [ 0.160358] usbcore: registered new interface driver usbfs lb # [ 0.010673] 508272 pages in range for PLT usage client # [ 0.163120] usbcore: registered new interface driver hub worker1 # [ 0.162035] usbcore: registered new interface driver hub lb # [ 0.010785] pinctrl core: initialized pinctrl subsystem worker2 # [ 0.160390] usbcore: registered new interface driver hub client # [ 0.163137] usbcore: registered new device driver usb worker1 # [ 0.162051] usbcore: registered new device driver usb lb # [ 0.011532] DMI not present or invalid. worker2 # [ 0.160406] usbcore: registered new device driver usb client # [ 0.163400] pps_core: LinuxPPS API ver. 1 registered worker1 # [ 0.162315] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.014568] NET: Registered PF_NETLINK/PF_ROUTE protocol family worker2 # [ 0.160687] pps_core: LinuxPPS API ver. 1 registered client # [ 0.163406] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker1 # [ 0.162321] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.016858] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations client # [ 0.163423] PTP clock support registered worker2 # [ 0.160694] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti worker1 # [ 0.162331] PTP clock support registered client # [ 0.163473] EDAC MC: Ver: 3.0.0 worker2 # [ 0.160703] PTP clock support registered worker1 # [ 0.162389] EDAC MC: Ver: 3.0.0 lb # [ 0.017033] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations worker2 # [ 0.160753] EDAC MC: Ver: 3.0.0 client # [ 0.168047] scmi_core: SCMI protocol bus registered worker1 # [ 0.166938] scmi_core: SCMI protocol bus registered lb # [ 0.017237] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations worker2 # [ 0.165204] scmi_core: SCMI protocol bus registered lb # [ 0.017257] audit: initializing netlink subsys (disabled) worker2 # [ 0.166149] FPGA manager framework client # [ 0.169022] FPGA manager framework worker1 # [ 0.167925] FPGA manager framework client # [ 0.169949] vgaarb: loaded lb # [ 0.018109] thermal_sys: Registered thermal governor 'fair_share' worker1 # [ 0.168895] vgaarb: loaded worker2 # [ 0.167089] vgaarb: loaded lb # [ 0.018111] thermal_sys: Registered thermal governor 'bang_bang' client # [ 0.170575] clocksource: Switched to clocksource arch_sys_counter worker1 # [ 0.169529] clocksource: Switched to clocksource arch_sys_counter lb # [ 0.018114] thermal_sys: Registered thermal governor 'step_wise' worker2 # [ 0.167719] clocksource: Switched to clocksource arch_sys_counter client # [ 0.176872] VFS: Disk quotas dquot_6.6.0 worker1 # [ 0.175851] VFS: Disk quotas dquot_6.6.0 worker2 # [ 0.174054] VFS: Disk quotas dquot_6.6.0 client # [ 0.176935] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker1 # [ 0.175901] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) worker2 # [ 0.174100] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) client # [ 0.178668] netfs: FS-Cache loaded worker2 # [ 0.175702] netfs: FS-Cache loaded worker1 # [ 0.177558] netfs: FS-Cache loaded client # [ 0.178804] pnp: PnP ACPI: disabled worker1 # [ 0.177680] pnp: PnP ACPI: disabled worker2 # [ 0.176256] pnp: PnP ACPI: disabled lb # [ 0.018117] thermal_sys: Registered thermal governor 'user_space' lb # [ 0.018121] thermal_sys: Registered thermal governor 'power_allocator' lb # [ 0.018147] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1 lb # [ 0.018155] cpuidle: using governor ladder client # [ 0.182558] NET: Registered PF_INET protocol family lb # [ 0.018161] cpuidle: using governor menu worker1 # [ 0.181515] NET: Registered PF_INET protocol family worker2 # [ 0.179625] NET: Registered PF_INET protocol family lb # [ 0.018346] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. lb # [ 0.018361] ASID allocator initialised with 65536 entries lb # [ 0.019569] Serial: AMBA PL011 UART driver lb # [ 0.024866] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 lb # [ 0.025029] printk: console [ttyAMA0] enabled client # [ 0.183465] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker1 # [ 0.182419] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.180511] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) lb # [ 0.144791] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages lb # [ 0.144811] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page lb # [ 0.144817] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages lb # [ 0.144821] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page lb # [ 0.144826] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages lb # [ 0.144830] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page lb # [ 0.144834] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages lb # [ 0.144838] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page lb # [ 0.152361] fbcon: Taking over console lb # [ 0.152377] ACPI: Interpreter disabled. lb # [ 0.154260] iommu: Default domain type: Translated lb # [ 0.154270] iommu: DMA domain TLB invalidation policy: strict mode lb # [ 0.156072] SCSI subsystem initialized lb # [ 0.163172] usbcore: registered new interface driver usbfs lb # [ 0.163205] usbcore: registered new interface driver hub lb # [ 0.163222] usbcore: registered new device driver usb lb # [ 0.163507] pps_core: LinuxPPS API ver. 1 registered lb # [ 0.163513] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti lb # [ 0.163530] PTP clock support registered lb # [ 0.163580] EDAC MC: Ver: 3.0.0 lb # [ 0.168217] scmi_core: SCMI protocol bus registered lb # [ 0.169197] FPGA manager framework lb # [ 0.170197] vgaarb: loaded lb # [ 0.170853] clocksource: Switched to clocksource arch_sys_counter lb # [ 0.172888] VFS: Disk quotas dquot_6.6.0 lb # [ 0.172923] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) lb # [ 0.176636] netfs: FS-Cache loaded client # [ 0.212928] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.176775] pnp: PnP ACPI: disabled worker2 # [ 0.209920] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.212982] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) worker2 # [ 0.209968] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.213016] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) worker2 # [ 0.209996] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.213099] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) worker2 # [ 0.210072] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) client # [ 0.213242] TCP: Hash tables configured (established 16384 bind 16384) worker2 # [ 0.210215] TCP: Hash tables configured (established 16384 bind 16384) worker1 # [ 0.212147] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) client # [ 0.213344] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.210301] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker1 # [ 0.212200] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) client # [ 0.213381] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.210339] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.212230] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) client # [ 0.213416] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker2 # [ 0.210377] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.180649] NET: Registered PF_INET protocol family client # [ 0.213527] NET: Registered PF_UNIX/PF_LOCAL protocol family worker1 # [ 0.212312] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.180917] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear) worker2 # [ 0.210510] NET: Registered PF_UNIX/PF_LOCAL protocol family client # [ 0.213554] NET: Registered PF_XDP protocol family worker2 # [ 0.210529] NET: Registered PF_XDP protocol family worker1 # [ 0.212455] TCP: Hash tables configured (established 16384 bind 16384) client # [ 0.213572] PCI: CLS 0 bytes, default 64 worker2 # [ 0.210548] PCI: CLS 0 bytes, default 64 client # [ 0.213820] Trying to unpack rootfs image as initramfs... worker1 # [ 0.212549] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) worker2 # [ 0.210809] Trying to unpack rootfs image as initramfs... worker1 # [ 0.212588] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.212625] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) worker1 # [ 0.212758] NET: Registered PF_UNIX/PF_LOCAL protocol family worker1 # [ 0.212779] NET: Registered PF_XDP protocol family worker1 # [ 0.212797] PCI: CLS 0 bytes, default 64 worker1 # [ 0.213054] Trying to unpack rootfs image as initramfs... client # [ 0.228316] kvm [1]: HYP mode not available worker2 # [ 0.225291] kvm [1]: HYP mode not available worker1 # [ 0.231237] kvm [1]: HYP mode not available lb # [ 0.211122] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear) lb # [ 0.211172] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) lb # [ 0.211203] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear) lb # [ 0.211279] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear) lb # [ 0.211423] TCP: Hash tables configured (established 16384 bind 16384) lb # [ 0.211509] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear) lb # [ 0.211547] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.211583] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear) lb # [ 0.211676] NET: Registered PF_UNIX/PF_LOCAL protocol family lb # [ 0.211713] NET: Registered PF_XDP protocol family lb # [ 0.211733] PCI: CLS 0 bytes, default 64 lb # [ 0.211969] Trying to unpack rootfs image as initramfs... lb # [ 0.234888] kvm [1]: HYP mode not available client # [ 0.335159] Initialise system trusted keyrings worker2 # [ 0.332322] Initialise system trusted keyrings client # [ 0.335915] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker2 # [ 0.333070] workingset: timestamp_bits=42 max_order=19 bucket_order=0 client # [ 0.337253] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker2 # [ 0.334428] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker2 # [ 0.335210] 9p: Installing v9fs 9p2000 file system support worker1 # [ 0.343020] Initialise system trusted keyrings worker1 # [ 0.343779] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker1 # [ 0.345049] squashfs: version 4.0 (2009/01/31) Phillip Lougher client # [ 0.342645] 9p: Installing v9fs 9p2000 file system support worker1 # [ 0.345900] 9p: Installing v9fs 9p2000 file system support client # [ 0.363477] Key type asymmetric registered client # [ 0.363504] Asymmetric key parser 'x509' registered client # [ 0.363583] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.365785] io scheduler mq-deadline registered client # [ 0.365797] io scheduler kyber registered worker1 # [ 0.365893] Key type asymmetric registered worker2 # [ 0.363955] Key type asymmetric registered worker1 # [ 0.365922] Asymmetric key parser 'x509' registered worker2 # [ 0.363984] Asymmetric key parser 'x509' registered worker1 # [ 0.366008] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker2 # [ 0.364059] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) worker2 # [ 0.366256] io scheduler mq-deadline registered worker2 # [ 0.366269] io scheduler kyber registered worker1 # [ 0.373613] io scheduler mq-deadline registered worker1 # [ 0.373638] io scheduler kyber registered client # [ 0.378755] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker2 # [ 0.375873] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker2 # [ 0.377331] ledtrig-cpu: registered to indicate activity on CPUs worker2 # [ 0.377745] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.377763] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 lb # [ 0.343388] Initialise system trusted keyrings worker2 # [ 0.377775] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.380185] ledtrig-cpu: registered to indicate activity on CPUs lb # [ 0.344137] workingset: timestamp_bits=42 max_order=19 bucket_order=0 worker2 # [ 0.377784] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.380580] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker1 # [ 0.385713] pl061_gpio 9030000.pl061: PL061 GPIO chip registered lb # [ 0.345402] squashfs: version 4.0 (2009/01/31) Phillip Lougher worker2 # [ 0.377805] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.380596] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 lb # [ 0.346146] 9p: Installing v9fs 9p2000 file system support worker2 # [ 0.377832] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.380609] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 worker2 # [ 0.377926] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 client # [ 0.380617] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.377937] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.377943] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.380655] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits worker2 # [ 0.377949] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.380682] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] worker2 # [ 0.377954] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] client # [ 0.380757] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker2 # [ 0.378042] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint client # [ 0.380766] pci_bus 0000:00: root bus resource [bus 00-ff] worker1 # [ 0.387118] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.380772] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker2 # [ 0.378480] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.387543] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: worker2 # [ 0.378663] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.380778] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker2 # [ 0.378679] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.387562] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 client # [ 0.380783] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.378709] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.387574] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.380868] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.378725] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.381327] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.387582] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 worker2 # [ 0.379184] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.381516] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.379367] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.387619] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.381533] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.379383] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.387647] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.381563] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.379413] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.381580] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.387726] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker1 # [ 0.387736] pci_bus 0000:00: root bus resource [bus 00-ff] client # [ 0.382046] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.398895] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.387743] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] client # [ 0.382232] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.399086] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker1 # [ 0.387748] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] client # [ 0.382248] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.399102] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.382278] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.387753] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.399132] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.382767] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint worker1 # [ 0.387809] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.399586] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.382954] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker1 # [ 0.388256] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.382970] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.388444] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.382999] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.388460] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.366844] Key type asymmetric registered client # [ 0.383453] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.388491] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.383647] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.388507] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.383663] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.388973] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.383693] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.389159] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.383713] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.389175] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.384167] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker1 # [ 0.389205] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.384358] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.389697] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.384389] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.389886] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] worker1 # [ 0.389902] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.384847] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.385049] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.389932] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.385079] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.374920] Asymmetric key parser 'x509' registered worker1 # [ 0.390395] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker1 # [ 0.390589] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.375006] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) client # [ 0.385488] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker1 # [ 0.390603] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.376616] io scheduler mq-deadline registered client # [ 0.385671] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.376628] io scheduler kyber registered worker1 # [ 0.390633] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.385939] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.390652] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] client # [ 0.386127] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.391114] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint client # [ 0.386157] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.391307] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.391337] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.411853] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.391809] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker2 # [ 0.411889] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.392001] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.411919] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.392031] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.411938] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.392475] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker2 # [ 0.412469] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker1 # [ 0.392658] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker2 # [ 0.412662] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.412691] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.392912] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.393100] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.413146] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint worker1 # [ 0.393130] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.387018] pl061_gpio 9030000.pl061: PL061 GPIO chip registered worker2 # [ 0.413333] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.426768] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.413362] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.426976] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.413770] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint client # [ 0.427008] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.413951] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] lb # [ 0.388422] ledtrig-cpu: registered to indicate activity on CPUs client # [ 0.427552] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.414238] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.388813] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: client # [ 0.427743] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.414429] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.427774] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.388831] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 worker2 # [ 0.414460] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.428242] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.414913] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint lb # [ 0.388842] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 client # [ 0.428527] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker2 # [ 0.415099] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.388851] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 client # [ 0.428545] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.415129] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.388874] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits client # [ 0.428576] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.415583] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.433655] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.429054] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker2 # [ 0.415788] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.388901] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] client # [ 0.429240] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.433868] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.388988] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 worker2 # [ 0.415818] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.429256] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.433901] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.388997] pci_bus 0000:00: root bus resource [bus 00-ff] worker2 # [ 0.416286] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint client # [ 0.429287] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.389003] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] worker1 # [ 0.434421] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker2 # [ 0.416582] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] client # [ 0.429874] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned lb # [ 0.389009] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] worker1 # [ 0.434615] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.416599] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.429886] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned lb # [ 0.389013] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] worker2 # [ 0.416628] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.434645] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.429891] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.389070] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint worker2 # [ 0.417097] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.435112] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker2 # [ 0.417281] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] worker1 # [ 0.435415] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] client # [ 0.429937] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.389509] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint worker2 # [ 0.417297] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.435434] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.389697] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] client # [ 0.429986] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned worker2 # [ 0.417326] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.435463] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.389713] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.430034] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker2 # [ 0.417925] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned lb # [ 0.389743] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.435954] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint client # [ 0.430084] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker2 # [ 0.417936] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned worker1 # [ 0.436144] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.389760] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.436160] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.430133] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker2 # [ 0.417941] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.390221] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint worker1 # [ 0.436192] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.390404] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] client # [ 0.430182] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.417986] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned worker1 # [ 0.436797] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned lb # [ 0.390420] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.430235] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.418033] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.390449] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.436808] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned client # [ 0.430283] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.418081] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned worker1 # [ 0.436813] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned client # [ 0.430330] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.418128] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned worker1 # [ 0.436860] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.409534] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint client # [ 0.430393] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.418174] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned worker1 # [ 0.436908] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.409724] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f] client # [ 0.430439] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.418220] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.409740] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.436956] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned client # [ 0.430461] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned lb # [ 0.409769] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.418265] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker1 # [ 0.437005] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned client # [ 0.430483] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker2 # [ 0.418310] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned lb # [ 0.410236] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint client # [ 0.430505] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.437054] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.410419] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] worker2 # [ 0.418356] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned client # [ 0.430527] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.410435] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.437103] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned worker2 # [ 0.418428] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned client # [ 0.430548] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned lb # [ 0.410465] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.437151] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned worker2 # [ 0.418474] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned client # [ 0.430570] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.410483] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref] worker1 # [ 0.437197] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned worker2 # [ 0.418495] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.418517] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker1 # [ 0.437244] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned worker2 # [ 0.418538] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker1 # [ 0.437329] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned worker2 # [ 0.418559] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker1 # [ 0.437384] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.418580] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker1 # [ 0.437407] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.418601] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker1 # [ 0.437428] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned worker2 # [ 0.418625] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker1 # [ 0.437449] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned worker2 # [ 0.418647] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker1 # [ 0.437471] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned worker2 # [ 0.418669] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker1 # [ 0.437493] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned worker2 # [ 0.418691] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker1 # [ 0.437514] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned worker2 # [ 0.418714] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned worker2 # [ 0.418735] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned worker2 # [ 0.418756] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned client # [ 0.466688] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned worker2 # [ 0.418777] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned client # [ 0.466716] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned worker2 # [ 0.418797] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned client # [ 0.466740] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned worker2 # [ 0.418818] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.418839] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned client # [ 0.466764] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned worker2 # [ 0.418864] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] client # [ 0.466788] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.423088] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint worker2 # [ 0.418874] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] client # [ 0.466811] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.423299] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] worker2 # [ 0.418879] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] client # [ 0.466834] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.423330] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker2 # [ 0.419701] pci 0000:00:07.0: enabling device (0000 -> 0002) client # [ 0.466856] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.423842] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint client # [ 0.466879] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.424034] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] client # [ 0.466902] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.424064] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] client # [ 0.466926] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned worker1 # [ 0.473650] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned client # [ 0.466962] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.424491] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint worker1 # [ 0.473696] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned client # [ 0.466972] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.424674] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff] worker1 # [ 0.473721] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned client # [ 0.466977] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.424936] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint client # [ 0.467824] pci 0000:00:07.0: enabling device (0000 -> 0002) worker1 # [ 0.473746] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned lb # [ 0.425124] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.473770] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.425153] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.473794] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.425604] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.473817] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.425795] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.473838] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.425825] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.473866] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned worker2 # [ 0.479901] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 58792 usecs worker1 # [ 0.473888] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned lb # [ 0.426285] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint worker1 # [ 0.473911] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.426472] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff] worker1 # [ 0.473943] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.426502] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] worker1 # [ 0.473952] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.426999] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint worker1 # [ 0.473957] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.427306] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] worker1 # [ 0.474822] pci 0000:00:07.0: enabling device (0000 -> 0002) lb # [ 0.427324] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.427353] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.427808] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint lb # [ 0.428001] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] lb # [ 0.428016] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] lb # [ 0.428048] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] lb # [ 0.428643] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned lb # [ 0.428655] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned lb # [ 0.428661] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned lb # [ 0.428707] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned lb # [ 0.428754] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned lb # [ 0.428803] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned lb # [ 0.428850] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned lb # [ 0.428897] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned lb # [ 0.428943] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned lb # [ 0.428989] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned client # [ 0.499870] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) lb # [ 0.429036] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned client # [ 0.502091] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.429083] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned lb # [ 0.429162] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned lb # [ 0.429208] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned worker2 # [ 0.500754] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) worker1 # [ 0.502793] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) lb # [ 0.429230] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned worker2 # [ 0.502972] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker1 # [ 0.505006] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) lb # [ 0.429251] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned lb # [ 0.429272] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned lb # [ 0.429294] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned lb # [ 0.429316] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned lb # [ 0.429337] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned lb # [ 0.429363] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned lb # [ 0.429384] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned lb # [ 0.429406] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned client # [ 0.512487] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.429428] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned lb # [ 0.429450] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned lb # [ 0.429472] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned lb # [ 0.429493] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned lb # [ 0.429514] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned lb # [ 0.429534] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned lb # [ 0.429555] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned worker2 # [ 0.514226] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) lb # [ 0.429576] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned lb # [ 0.429602] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] lb # [ 0.429612] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] lb # [ 0.429616] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] lb # [ 0.430441] pci 0000:00:07.0: enabling device (0000 -> 0002) client # [ 0.518710] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) client # [ 0.520851] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker1 # [ 0.523163] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker1 # [ 0.525322] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.486984] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 55221 usecs worker2 # [ 0.524695] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) worker1 # [ 0.529107] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker2 # [ 0.527056] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) client # [ 0.530827] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker2 # [ 0.530871] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) client # [ 0.532703] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.536638] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker2 # [ 0.539085] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) client # [ 0.542673] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) client # [ 0.545343] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker1 # [ 0.542719] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker1 # [ 0.544824] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) lb # [ 0.508292] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) lb # [ 0.510481] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) worker2 # [ 0.549156] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) worker2 # [ 0.552648] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker1 # [ 0.555116] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) worker2 # [ 0.554579] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) worker1 # [ 0.557092] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) client # [ 0.555220] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) client # [ 0.558506] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) lb # [ 0.521921] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) worker2 # [ 0.566222] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) worker1 # [ 0.568947] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.571777] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled client # [ 0.574597] msm_serial: driver initialized client # [ 0.574742] SuperH (H)SCI(F) driver initialized client # [ 0.574796] STM32 USART driver initialized lb # [ 0.536380] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) lb # [ 0.538414] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002) worker1 # [ 0.581669] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled worker2 # [ 0.581014] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.542215] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002) worker1 # [ 0.584208] msm_serial: driver initialized worker1 # [ 0.584356] SuperH (H)SCI(F) driver initialized worker1 # [ 0.584408] STM32 USART driver initialized lb # [ 0.544958] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) worker2 # [ 0.583782] msm_serial: driver initialized worker2 # [ 0.583915] SuperH (H)SCI(F) driver initialized worker2 # [ 0.583970] STM32 USART driver initialized lb # [ 0.555036] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) lb # [ 0.557183] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002) client # [ 0.609168] loop: module loaded client # [ 0.609361] virtio_blk virtio2: 1/0/0 default/read/poll queues client # [ 0.610237] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 0.567082] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) lb # [ 0.570028] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) client # [ 0.615243] megasas: 07.734.00.00-rc1 client # [ 0.615926] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.616353] loop: module loaded worker1 # [ 0.618610] loop: module loaded worker2 # [ 0.616547] virtio_blk virtio2: 1/0/0 default/read/poll queues worker1 # [ 0.618809] virtio_blk virtio2: 1/0/0 default/read/poll queues worker2 # [ 0.617335] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) worker1 # [ 0.619749] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 0.580069] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled lb # [ 0.582825] msm_serial: driver initialized client # [ 0.626623] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 client # [ 0.626653] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.628384] Using buffer write method lb # [ 0.587146] SuperH (H)SCI(F) driver initialized client # [ 0.628459] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.587215] STM32 USART driver initialized worker1 # [ 0.630289] megasas: 07.734.00.00-rc1 worker2 # [ 0.624407] megasas: 07.734.00.00-rc1 worker1 # [ 0.631003] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker2 # [ 0.625159] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] worker1 # [ 0.633333] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.627289] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.633360] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.627315] Intel/Sharp Extended Query Table at 0x0031 worker2 # [ 0.628943] Using buffer write method worker2 # [ 0.629006] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] client # [ 0.639271] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.630800] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker2 # [ 0.630823] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.639300] Intel/Sharp Extended Query Table at 0x0031 client # [ 0.640714] Using buffer write method client # [ 0.640743] Concatenating MTD devices: client # [ 0.640747] (0): "0.flash" client # [ 0.640751] (1): "0.flash" client # [ 0.640764] into device "0.flash" worker1 # [ 0.642604] Using buffer write method worker1 # [ 0.642679] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] worker2 # [ 0.648317] Using buffer write method worker2 # [ 0.648353] Concatenating MTD devices: worker2 # [ 0.648358] (0): "0.flash" worker2 # [ 0.648366] (1): "0.flash" worker2 # [ 0.648370] into device "0.flash" worker1 # [ 0.653586] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 worker1 # [ 0.653634] Intel/Sharp Extended Query Table at 0x0031 worker1 # [ 0.655245] Using buffer write method worker1 # [ 0.655275] Concatenating MTD devices: worker1 # [ 0.655279] (0): "0.flash" worker1 # [ 0.655283] (1): "0.flash" worker1 # [ 0.655292] into device "0.flash" lb # [ 0.619125] loop: module loaded lb # [ 0.619328] virtio_blk virtio2: 1/0/0 default/read/poll queues lb # [ 0.620175] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) lb # [ 0.627511] megasas: 07.734.00.00-rc1 lb # [ 0.628284] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] lb # [ 0.630719] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.630744] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.632379] Using buffer write method lb # [ 0.632444] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] lb # [ 0.634242] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 lb # [ 0.634265] Intel/Sharp Extended Query Table at 0x0031 lb # [ 0.652588] Using buffer write method lb # [ 0.652647] Concatenating MTD devices: lb # [ 0.652652] (0): "0.flash" lb # [ 0.652659] (1): "0.flash" lb # [ 0.652662] into device "0.flash" client # [ 0.920147] Freeing initrd memory: 26972K worker2 # [ 0.920036] Freeing initrd memory: 26944K client # [ 0.926292] tun: Universal TUN/TAP device driver, 1.6 worker2 # [ 0.926132] tun: Universal TUN/TAP device driver, 1.6 client # [ 0.930421] thunder_xcv, ver 1.0 client # [ 0.930473] thunder_bgx, ver 1.0 client # [ 0.930496] nicpf, ver 1.0 worker1 # [ 0.929675] Freeing initrd memory: 26976K worker2 # [ 0.930042] thunder_xcv, ver 1.0 worker2 # [ 0.930089] thunder_bgx, ver 1.0 client # [ 0.932281] e1000: Intel(R) PRO/1000 Network Driver worker2 # [ 0.930112] nicpf, ver 1.0 worker2 # [ 0.930672] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.932292] e1000: Copyright (c) 1999-2006 Intel Corporation. worker2 # [ 0.930679] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.932316] e1000e: Intel(R) PRO/1000 Network Driver worker1 # [ 0.935812] tun: Universal TUN/TAP device driver, 1.6 worker2 # [ 0.930703] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.932324] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker2 # [ 0.930711] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. client # [ 0.932353] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 0.930741] igb: Intel(R) Gigabit Ethernet Network Driver client # [ 0.932358] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 0.930746] igb: Copyright (c) 2007-2014 Intel Corporation. client # [ 0.932381] igbvf: Intel(R) Gigabit Virtual Function Network Driver worker2 # [ 0.930788] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 0.932386] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. client # [ 0.932520] sky2: driver version 1.30 worker2 # [ 0.930794] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker2 # [ 0.930935] sky2: driver version 1.30 client # [ 0.934203] usbcore: registered new interface driver usb-storage client # [ 0.934300] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.939543] thunder_xcv, ver 1.0 worker1 # [ 0.939611] thunder_bgx, ver 1.0 client # [ 0.934313] usbserial: USB Serial support registered for generic worker1 # [ 0.939631] nicpf, ver 1.0 worker1 # [ 0.940187] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.935264] ehci-pci 0000:00:07.0: EHCI Host Controller worker2 # [ 0.940435] ehci-pci 0000:00:07.0: EHCI Host Controller worker1 # [ 0.940194] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.935290] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker2 # [ 0.940466] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.940216] e1000e: Intel(R) PRO/1000 Network Driver client # [ 0.935496] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker2 # [ 0.940703] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.940226] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. worker1 # [ 0.940254] igb: Intel(R) Gigabit Ethernet Network Driver worker2 # [ 0.943062] usbcore: registered new interface driver usb-storage client # [ 0.946689] hv_vmbus: registering driver hyperv_keyboard worker1 # [ 0.940260] igb: Copyright (c) 2007-2014 Intel Corporation. worker2 # [ 0.943145] usbcore: registered new interface driver usbserial_generic worker1 # [ 0.940282] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 0.948041] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker2 # [ 0.943159] usbserial: USB Serial support registered for generic worker1 # [ 0.940288] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. client # [ 0.949073] hub 1-0:1.0: USB hub found worker1 # [ 0.940417] sky2: driver version 1.30 client # [ 0.949554] hub 1-0:1.0: 6 ports detected worker2 # [ 0.946016] hv_vmbus: registering driver hyperv_keyboard worker2 # [ 0.947536] rtc-pl031 9010000.pl031: registered as rtc0 client # [ 0.950698] rtc-pl031 9010000.pl031: registered as rtc0 worker1 # [ 0.949119] usbcore: registered new interface driver usb-storage worker2 # [ 0.947563] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:12:37 UTC (1790140357) client # [ 0.950724] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:12:37 UTC (1790140357) worker1 # [ 0.949219] usbcore: registered new interface driver usbserial_generic client # [ 0.951044] i2c_dev: i2c /dev entries driver worker2 # [ 0.949533] i2c_dev: i2c /dev entries driver worker1 # [ 0.949232] usbserial: USB Serial support registered for generic worker1 # [ 0.950294] ehci-pci 0000:00:07.0: EHCI Host Controller worker1 # [ 0.950320] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 worker1 # [ 0.950526] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 worker1 # [ 0.955527] hv_vmbus: registering driver hyperv_keyboard client # [ 0.956379] sdhci: Secure Digital Host Controller Interface driver worker1 # [ 0.957123] rtc-pl031 9010000.pl031: registered as rtc0 worker2 # [ 0.953258] sdhci: Secure Digital Host Controller Interface driver client # [ 0.956391] sdhci: Copyright(c) Pierre Ossman worker2 # [ 0.953271] sdhci: Copyright(c) Pierre Ossman worker1 # [ 0.957151] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:12:37 UTC (1790140357) client # [ 0.956652] Synopsys Designware Multimedia Card Interface Driver worker1 # [ 0.957449] i2c_dev: i2c /dev entries driver worker2 # [ 0.953541] Synopsys Designware Multimedia Card Interface Driver client # [ 0.957037] sdhci-pltfm: SDHCI platform and OF driver helper worker2 # [ 0.953923] sdhci-pltfm: SDHCI platform and OF driver helper worker2 # [ 0.955452] hid: raw HID events driver (C) Jiri Kosina lb # [ 0.919500] Freeing initrd memory: 26948K worker2 # [ 0.955693] usbcore: registered new interface driver usbhid client # [ 0.961323] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.955700] usbhid: USB HID core driver client # [ 0.961563] usbcore: registered new interface driver usbhid worker2 # [ 0.960246] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 client # [ 0.961570] usbhid: USB HID core driver worker2 # [ 0.961284] hub 1-0:1.0: USB hub found worker2 # [ 0.961915] hub 1-0:1.0: 6 ports detected client # [ 0.964547] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker1 # [ 0.962787] sdhci: Secure Digital Host Controller Interface driver worker1 # [ 0.962798] sdhci: Copyright(c) Pierre Ossman client # [ 0.966095] drop_monitor: Initializing network drop monitor service lb # [ 0.925658] tun: Universal TUN/TAP device driver, 1.6 worker1 # [ 0.963062] Synopsys Designware Multimedia Card Interface Driver client # [ 0.966285] NET: Registered PF_INET6 protocol family worker1 # [ 0.963427] sdhci-pltfm: SDHCI platform and OF driver helper client # [ 0.969379] Segment Routing with IPv6 worker1 # [ 0.964960] hid: raw HID events driver (C) Jiri Kosina worker2 # [ 0.963939] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available client # [ 0.969398] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.965195] usbcore: registered new interface driver usbhid worker2 # [ 0.965495] drop_monitor: Initializing network drop monitor service client # [ 0.969425] NET: Registered PF_PACKET protocol family worker1 # [ 0.965202] usbhid: USB HID core driver worker2 # [ 0.965655] NET: Registered PF_INET6 protocol family worker2 # [ 0.968759] Segment Routing with IPv6 worker1 # [ 0.970086] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 worker2 # [ 0.968777] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.970424] hub 1-0:1.0: USB hub found lb # [ 0.929528] thunder_xcv, ver 1.0 worker2 # [ 0.968809] NET: Registered PF_PACKET protocol family worker1 # [ 0.970504] hub 1-0:1.0: 6 ports detected lb # [ 0.929577] thunder_bgx, ver 1.0 lb # [ 0.929601] nicpf, ver 1.0 lb # [ 0.930176] e1000: Intel(R) PRO/1000 Network Driver client # [ 0.971293] 9pnet: Installing 9P2000 support lb # [ 0.930183] e1000: Copyright (c) 1999-2006 Intel Corporation. client # [ 0.971341] Key type dns_resolver registered worker2 # [ 0.970442] 9pnet: Installing 9P2000 support worker1 # [ 0.973336] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available worker2 # [ 0.970494] Key type dns_resolver registered lb # [ 0.930208] e1000e: Intel(R) PRO/1000 Network Driver lb # [ 0.930216] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. lb # [ 0.930247] igb: Intel(R) Gigabit Ethernet Network Driver worker1 # [ 0.975972] drop_monitor: Initializing network drop monitor service lb # [ 0.930253] igb: Copyright (c) 2007-2014 Intel Corporation. worker1 # [ 0.976160] NET: Registered PF_INET6 protocol family client # [ 0.977953] registered taskstats version 1 lb # [ 0.930292] igbvf: Intel(R) Gigabit Virtual Function Network Driver client # [ 0.978100] Loading compiled-in X.509 certificates lb # [ 0.930298] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. worker1 # [ 0.978120] Segment Routing with IPv6 lb # [ 0.930435] sky2: driver version 1.30 worker1 # [ 0.978137] In-situ OAM (IOAM) with IPv6 worker1 # [ 0.978164] NET: Registered PF_PACKET protocol family worker1 # [ 0.979789] 9pnet: Installing 9P2000 support worker2 # [ 0.977236] registered taskstats version 1 lb # [ 0.939361] usbcore: registered new interface driver usb-storage worker1 # [ 0.979840] Key type dns_resolver registered worker2 # [ 0.977406] Loading compiled-in X.509 certificates lb # [ 0.939501] usbcore: registered new interface driver usbserial_generic lb # [ 0.939520] usbserial: USB Serial support registered for generic lb # [ 0.940162] hv_vmbus: registering driver hyperv_keyboard lb # [ 0.944488] ehci-pci 0000:00:07.0: EHCI Host Controller client # [ 0.986851] Demotion targets for Node 0: null client # [ 0.986963] Key type .fscrypt registered lb # [ 0.944518] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 1 client # [ 0.986970] Key type fscrypt-provisioning registered lb # [ 0.944690] ehci-pci 0000:00:07.0: irq 16, io mem 0x10086000 client # [ 0.987075] ima: No TPM chip found, activating TPM-bypass! client # [ 0.987095] ima: Allocated hash algorithm: sha1 worker2 # [ 0.986349] Demotion targets for Node 0: null client # [ 0.987119] ima: No architecture policies found worker2 # [ 0.986466] Key type .fscrypt registered lb # [ 0.947272] rtc-pl031 9010000.pl031: registered as rtc0 worker1 # [ 0.986572] registered taskstats version 1 worker2 # [ 0.986474] Key type fscrypt-provisioning registered worker1 # [ 0.986753] Loading compiled-in X.509 certificates lb # [ 0.947303] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:12:37 UTC (1790140357) client # [ 0.991497] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker2 # [ 0.986575] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.947646] i2c_dev: i2c /dev entries driver worker2 # [ 0.986594] ima: Allocated hash algorithm: sha1 worker2 # [ 0.986615] ima: No architecture policies found worker2 # [ 0.990755] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker1 # [ 0.995854] Demotion targets for Node 0: null lb # [ 0.952999] sdhci: Secure Digital Host Controller Interface driver worker1 # [ 0.995992] Key type .fscrypt registered lb # [ 0.953010] sdhci: Copyright(c) Pierre Ossman worker1 # [ 0.995999] Key type fscrypt-provisioning registered lb # [ 0.953284] Synopsys Designware Multimedia Card Interface Driver worker1 # [ 0.996101] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.953661] sdhci-pltfm: SDHCI platform and OF driver helper worker1 # [ 0.996121] ima: Allocated hash algorithm: sha1 worker1 # [ 0.996144] ima: No architecture policies found lb # [ 0.954899] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.00 lb # [ 0.955217] hub 1-0:1.0: USB hub found lb # [ 0.955239] hub 1-0:1.0: 6 ports detected worker1 # [ 1.000509] input: gpio-keys as /devices/platform/gpio-keys/input/input0 lb # [ 0.960511] hid: raw HID events driver (C) Jiri Kosina lb # [ 0.960776] usbcore: registered new interface driver usbhid lb # [ 0.960783] usbhid: USB HID core driver lb # [ 0.963777] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available lb # [ 0.965391] drop_monitor: Initializing network drop monitor service lb # [ 0.965529] NET: Registered PF_INET6 protocol family client # [ 1.010065] clk: Disabling unused clocks lb # [ 0.968622] Segment Routing with IPv6 client # [ 1.010097] PM: genpd: Disabling unused power domains lb # [ 0.968656] In-situ OAM (IOAM) with IPv6 lb # [ 0.968683] NET: Registered PF_PACKET protocol family lb # [ 0.970333] 9pnet: Installing 9P2000 support lb # [ 0.970380] Key type dns_resolver registered worker2 # [ 1.008414] clk: Disabling unused clocks worker2 # [ 1.008448] PM: genpd: Disabling unused power domains client # [ 1.014269] Freeing unused kernel memory: 4736K client # [ 1.015225] Run /init as init process worker2 # [ 1.012702] Freeing unused kernel memory: 4736K worker2 # [ 1.012921] Run /init as init process lb # [ 0.977084] registered taskstats version 1 lb # [ 0.977228] Loading compiled-in X.509 certificates worker1 # [ 1.019740] clk: Disabling unused clocks worker1 # [ 1.019772] PM: genpd: Disabling unused power domains worker1 # [ 1.024066] Freeing unused kernel memory: 4736K worker1 # [ 1.024259] Run /init as init process lb # [ 0.986511] Demotion targets for Node 0: null client # [ 1.028981] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.986662] Key type .fscrypt registered lb # [ 0.986669] Key type fscrypt-provisioning registered lb # [ 0.986776] ima: No TPM chip found, activating TPM-bypass! lb # [ 0.986796] ima: Allocated hash algorithm: sha1 lb # [ 0.986819] ima: No architecture policies found worker2 # [ 1.030074] systemd[1]: Successfully made /usr/ read-only. lb # [ 0.990940] input: gpio-keys as /devices/platform/gpio-keys/input/input0 worker1 # [ 1.039869] systemd[1]: Successfully made /usr/ read-only. lb # [ 1.008844] clk: Disabling unused clocks lb # [ 1.008873] PM: genpd: Disabling unused power domains lb # [ 1.012998] Freeing unused kernel memory: 4736K lb # [ 1.013205] Run /init as init process lb # [ 1.029223] systemd[1]: Successfully made /usr/ read-only. client # [ 1.194697] usb 1-1: new high-speed USB device number 2 using ehci-pci worker2 # [ 1.207788] usb 1-1: new high-speed USB device number 2 using ehci-pci worker1 # [ 1.217589] usb 1-1: new high-speed USB device number 2 using ehci-pci lb # [ 1.202972] usb 1-1: new high-speed USB device number 2 using ehci-pci client # [ 1.349096] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.357966] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker1 # [ 1.367576] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 worker2 # [ 1.364767] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.371036] systemd[1]: Detected virtualization qemu. client # [ 1.363898] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 1.372049] systemd[1]: Detected architecture arm64. worker2 # [ 1.372976] systemd[1]: Running in initrd. client # [ 1.376262] systemd[1]: Detected virtualization qemu. worker2 # [ 1.374316] systemd[1]: Initializing machine ID from random generator. worker2 # [ 1.375683] systemd[1]: Hostname set to . client # [ 1.378442] systemd[1]: Detected architecture arm64. worker1 # [ 1.374541] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 1.379230] systemd[1]: Detected virtualization qemu. client # [ 1.380474] systemd[1]: Running in initrd. worker1 # [ 1.380113] systemd[1]: Detected architecture arm64. worker1 # [ 1.380809] systemd[1]: Running in initrd. worker1 # [ 1.381905] systemd[1]: Initializing machine ID from random generator. worker1 # [ 1.382983] systemd[1]: Hostname set to . client # [ 1.383127] systemd[1]: Initializing machine ID from random generator. client # [ 1.386010] systemd[1]: Hostname set to . lb # [ 1.357582] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 lb # [ 1.363786] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 1.363848] systemd[1]: Detected virtualization qemu. lb # [ 1.363939] systemd[1]: Detected architecture arm64. lb # [ 1.363963] systemd[1]: Running in initrd. lb # [ 1.364980] systemd[1]: Initializing machine ID from random generator. lb # [ 1.365314] systemd[1]: Hostname set to . client # [ 1.430936] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker2 # [ 1.439908] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 worker1 # [ 1.453735] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 lb # [ 1.443147] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input0 client # [ 1.558744] usb 1-2: new high-speed USB device number 3 using ehci-pci worker2 # [ 1.559810] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.573637] usb 1-2: new high-speed USB device number 3 using ehci-pci lb # [ 1.562957] usb 1-2: new high-speed USB device number 3 using ehci-pci worker1 # [ 1.626250] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 1.652467] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.688315] systemd[1]: Queued start job for default target Initrd Default Target. worker1 # [ 1.699770] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.701008] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 1.702462] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker1 # [ 1.703368] systemd[1]: Reached target Path Units. worker1 # [ 1.704058] systemd[1]: Reached target Slice Units. worker1 # [ 1.704831] systemd[1]: Reached target Swaps. worker1 # [ 1.705458] systemd[1]: Reached target Timer Units. worker1 # [ 1.706369] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.707430] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.708550] systemd[1]: Listening on Journal Sockets. worker1 # [ 1.709384] systemd[1]: Listening on udev Control Socket. worker1 # [ 1.710351] systemd[1]: Listening on udev Kernel Socket. worker1 # [ 1.711126] systemd[1]: Reached target Socket Units. worker1 # [ 1.713451] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 1.714543] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 1.713868] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 1.716604] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.725030] systemd[1]: Mounting Kernel Configuration File System... client # [ 1.721951] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.727988] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker1 # [ 1.732034] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 client # [ 1.733017] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 1.691005] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 1.739912] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker2 # [ 1.740896] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.750357] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.753755] systemd[1]: Starting Journal Service... worker2 # [ 1.751651] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 1.753225] systemd[1]: Expecting device /dev/disk/by-label/nixos... worker2 # [ 1.754497] systemd[1]: Reached target Path Units. worker2 # [ 1.755410] systemd[1]: Reached target Slice Units. worker2 # [ 1.756379] systemd[1]: Reached target Swaps. worker2 # [ 1.757183] systemd[1]: Reached target Timer Units. worker2 # [ 1.758319] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 1.761702] systemd[1]: Starting Load Kernel Modules... worker2 # [ 1.759657] systemd[1]: Listening on Journal Socket (/dev/log). worker1 # [ 1.762577] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.761091] systemd[1]: Listening on Journal Sockets. worker2 # [ 1.762192] systemd[1]: Listening on udev Control Socket. worker2 # [ 1.763295] systemd[1]: Listening on udev Kernel Socket. worker2 # [ 1.764317] systemd[1]: Reached target Socket Units. lb # [ 1.723672] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 worker2 # [ 1.767042] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.728450] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input0 worker2 # [ 1.768374] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 1.774975] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.779906] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 1.784933] systemd[1]: Starting Journal Service... worker1 # [ 1.793693] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 1.794748] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 1.803635] systemd[1]: Starting Load Kernel Modules... worker1 # [ 1.806051] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 1.804669] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 1.817087] systemd-journald[72]: Collecting audit messages is disabled. client # [ 1.824782] systemd[1]: Queued start job for default target Initrd Default Target. client # [ 1.832576] systemd[1]: Created slice Slice /system/modprobe. client # [ 1.833580] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.834631] systemd[1]: Expecting device /dev/disk/by-label/nixos... client # [ 1.835476] systemd[1]: Reached target Path Units. worker1 # [ 1.832562] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. client # [ 1.836112] systemd[1]: Reached target Slice Units. client # [ 1.836752] systemd[1]: Reached target Swaps. client # [ 1.837334] systemd[1]: Reached target Timer Units. client # [ 1.838125] systemd[1]: Listening on D-Bus System Message Bus Socket. client # [ 1.839179] systemd[1]: Listening on Journal Socket (/dev/log). client # [ 1.840049] systemd[1]: Listening on Journal Sockets. client # [ 1.840807] systemd[1]: Listening on udev Control Socket. worker2 # [ 1.836320] systemd[1]: Starting Coldplug All udev Devices... client # [ 1.841625] systemd[1]: Listening on udev Kernel Socket. client # [ 1.842326] systemd[1]: Reached target Socket Units. worker1 # [ 1.841603] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev client # [ 1.844498] systemd[1]: Starting Create List of Static Device Nodes... client # [ 1.845445] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 1.806737] systemd[1]: Queued start job for default target Initrd Default Target. worker2 # [ 1.847897] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.854480] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 1.857407] systemd-journald[72]: Collecting audit messages is disabled. worker1 # [ 1.858605] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 lb # [ 1.818700] systemd[1]: Created slice Slice /system/modprobe. worker1 # [ 1.858863] [drm] features: -virgl +edid -resource_blob -host_visible worker1 # [ 1.858874] [drm] features: -context_init lb # [ 1.819994] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 1.863011] systemd[1]: Starting Journal Service... worker1 # [ 1.859624] [drm] number of scanouts: 1 worker1 # [ 1.859643] [drm] number of cap sets: 0 lb # [ 1.821331] systemd[1]: Expecting device /dev/disk/by-label/nixos... lb # [ 1.822398] systemd[1]: Reached target Path Units. lb # [ 1.823269] systemd[1]: Reached target Slice Units. lb # [ 1.824102] systemd[1]: Reached target Swaps. lb # [ 1.824850] systemd[1]: Reached target Timer Units. worker2 # [ 1.863913] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.825893] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 1.864419] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 1.866330] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.869901] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 1.827163] systemd[1]: Listening on Journal Socket (/dev/log). lb # [ 1.827311] systemd[1]: Listening on Journal Sockets. lb # [ 1.827451] systemd[1]: Listening on udev Control Socket. lb # [ 1.827566] systemd[1]: Listening on udev Kernel Socket. lb # [ 1.827590] systemd[1]: Reached target Socket Units. lb # [ 1.833637] systemd[1]: Starting Create List of Static Device Nodes... lb # [ 1.834804] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 1.878757] systemd[1]: Starting Load Kernel Modules... client # [ 1.878883] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.882352] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker1 # [ 1.881952] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker1 # [ 1.881974] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 lb # [ 1.846294] systemd[1]: Mounting Kernel Configuration File System... client # [ 1.892658] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 1.901850] Console: switching to colour frame buffer device 160x50 worker2 # [ 1.907861] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 1.915384] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 lb # [ 1.875183] systemd[1]: Starting Journal Service... worker2 # [ 1.915636] [drm] features: -virgl +edid -resource_blob -host_visible worker2 # [ 1.915645] [drm] features: -context_init client # [ 1.918834] systemd[1]: Finished Create List of Static Device Nodes. client # [ 1.919877] systemd[1]: Mounted Kernel Configuration File System. client # [ 1.923085] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 1.883061] systemd[1]: Starting Load Kernel Modules... lb # [ 1.884040] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 1.928425] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 1.891126] systemd[1]: Starting Coldplug All udev Devices... worker2 # [ 1.931190] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 1.934542] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 1.936298] systemd[1]: Finished Create Static Device Nodes in /dev. worker1 # [ 1.936571] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 1.936619] systemd[1]: Reached target Local File Systems. worker1 # [ 1.940760] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 1.941706] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. worker2 # [ 1.940506] [drm] number of scanouts: 1 worker2 # [ 1.940538] [drm] number of cap sets: 0 client # [ 1.949013] systemd-journald[72]: Collecting audit messages is disabled. worker2 # [ 1.944151] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker2 # [ 1.944167] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 lb # [ 1.911049] systemd[1]: Finished Create List of Static Device Nodes. lb # [ 1.912051] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 1.953876] systemd[1]: Finished Load Kernel Modules. client # [ 1.958656] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev lb # [ 1.919547] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker1 # [ 1.961945] systemd[1]: Starting Apply Kernel Variables... client # [ 1.975961] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 client # [ 1.976212] [drm] features: -virgl +edid -resource_blob -host_visible client # [ 1.976223] [drm] features: -context_init client # [ 1.977014] [drm] number of scanouts: 1 worker2 # [ 1.968039] Console: switching to colour frame buffer device 160x50 client # [ 1.977034] [drm] number of cap sets: 0 worker2 # [ 1.974863] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device worker1 # [ 1.999373] systemd[1]: Started Journal Service. client # [ 1.998984] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic lb # [ 1.960480] systemd-journald[72]: Collecting audit messages is disabled. client # [ 1.999010] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 worker1 # [ 1.990675] systemd-modules-load[73]: Inserted module 'dm_mod' client # [ 2.005201] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker1 # [ 1.991967] systemd-modules-load[73]: Module 'virtio_balloon' is built in worker2 # [ 2.000948] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 2.001195] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 2.001231] systemd[1]: Reached target Local File Systems. worker2 # [ 2.008194] systemd[1]: Starting Rule-based Manager for Device Events and Files... client # [ 2.010985] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 2.004440] systemd-modules-load[73]: Module 'virtio_console' is built in worker1 # [ 2.005595] systemd-modules-load[73]: Inserted module 'virtio_gpu' worker2 # [ 2.015839] systemd[1]: Finished Load Kernel Modules. worker1 # [ 2.006619] systemd-modules-load[73]: Module 'virtio_rng' is built in worker1 # [ 2.007607] systemd[1]: Starting Create System Files and Directories... lb # [ 1.983685] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. lb # [ 1.985414] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev worker2 # [ 2.014734] systemd-modules-load[73]: Inserted module 'dm_mod' client # [ 2.026881] Console: switching to colour frame buffer device 160x50 worker2 # [ 2.020189] systemd-modules-load[73]: Module 'virtio_balloon' is built in lb # [ 1.991608] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 2.021324] systemd-modules-load[73]: Module 'virtio_console' is built in worker2 # [ 2.022449] systemd-modules-load[73]: Inserted module 'virtio_gpu' worker2 # [ 2.023416] systemd-modules-load[73]: Module 'virtio_rng' is built in worker1 # [ 2.024154] systemd[1]: Finished Apply Kernel Variables. lb # [ 1.999334] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 2.044414] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 2.044619] systemd[1]: Started Journal Service. lb # [ 2.011716] [drm] pci: virtio-gpu-pci detected at 0000:00:05.0 lb # [ 2.011962] [drm] features: -virgl +edid -resource_blob -host_visible lb # [ 2.011973] [drm] features: -context_init client # [ 2.056191] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device lb # [ 2.012681] [drm] number of scanouts: 1 lb # [ 2.012700] [drm] number of cap sets: 0 lb # [ 2.019285] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic worker1 # [ 2.050915] systemd-udevd[78]: Using default interface naming scheme 'v261'. lb # [ 2.019302] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 0 client # [ 2.063980] systemd[1]: Finished Load Kernel Modules. worker2 # [ 2.054010] systemd[1]: Starting Create System Files and Directories... client # [ 2.075123] systemd[1]: Starting Apply Kernel Variables... worker1 # [ 2.070082] systemd[1]: Finished Create System Files and Directories. client # [ 2.090713] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 2.090957] systemd[1]: Reached target Preparation for Local File Systems. client # [ 2.090994] systemd[1]: Reached target Local File Systems. client # [ 2.096332] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 2.088914] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 2.088918] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.054276] Console: switching to colour frame buffer device 160x50 lb # [ 2.061060] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device lb # [ 2.067026] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 2.067268] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 2.067300] systemd[1]: Reached target Local File Systems. lb # [ 2.075360] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 2.105653] systemd[1]: Finished Create System Files and Directories. client # [ 2.125649] systemd[1]: Started Journal Service. worker2 # [ 2.113373] systemd-udevd[78]: Using default interface naming scheme 'v261'. client # [ 2.115697] systemd-modules-load[73]: Inserted module 'dm_mod' client # [ 2.121485] systemd-modules-load[73]: Module 'virtio_balloon' is built in client # [ 2.132320] systemd-modules-load[73]: Module 'virtio_console' is built in lb # [ 2.103163] systemd[1]: Finished Load Kernel Modules. lb # [ 2.107283] systemd[1]: Starting Apply Kernel Variables... client # [ 2.136304] systemd-modules-load[73]: Inserted module 'virtio_gpu' worker2 # [ 2.140730] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 2.144370] systemd-modules-load[73]: Module 'virtio_rng' is built in client # [ 2.146704] systemd[1]: Starting Create System Files and Directories... client # [ 2.154035] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 2.168130] systemd[1]: Starting Virtual Console Setup... client # [ 2.173672] systemd[1]: Finished Create System Files and Directories. lb # [ 2.151079] systemd[1]: Started Journal Service. lb # [ 2.139451] systemd-modules-load[73]: Inserted module 'dm_mod' client # [ 2.185462] systemd-udevd[80]: Using default interface naming scheme 'v261'. lb # [ 2.144258] systemd-modules-load[73]: Module 'virtio_balloon' is built in lb # [ 2.145412] systemd-modules-load[73]: Module 'virtio_console' is built in lb # [ 2.146482] systemd-modules-load[73]: Inserted module 'virtio_gpu' lb # [ 2.147488] systemd-modules-load[73]: Module 'virtio_rng' is built in worker2 # [ 2.199884] systemd[1]: Starting Virtual Console Setup... lb # [ 2.164515] systemd[1]: Starting Create System Files and Directories... client # [ 2.217105] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 2.175755] systemd-udevd[78]: Using default interface naming scheme 'v261'. lb # [ 2.181350] systemd[1]: Finished Apply Kernel Variables. lb # [ 2.193304] systemd[1]: Finished Create System Files and Directories. worker1 # [ 2.238285] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 2.241994] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.252475] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 2.256163] systemd[1]: Finished Virtual Console Setup. lb # [ 2.217988] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 2.280467] systemd[1]: Starting Virtual Console Setup... lb # [ 2.273939] systemd[1]: Starting Virtual Console Setup... client # [ 2.328557] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 2.332182] systemd[1]: Finished Virtual Console Setup. lb # [ 2.324514] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 2.327940] systemd[1]: Finished Virtual Console Setup. worker2 # [ 2.837316] systemd[1]: Finished Coldplug All udev Devices. worker2 # [ 2.838292] systemd[1]: Reached target System Initialization. worker2 # [ 2.839123] systemd[1]: Reached target Basic System. worker1 # [ 2.853164] systemd[1]: Finished Coldplug All udev Devices. worker1 # [ 2.854758] systemd[1]: Reached target System Initialization. worker1 # [ 2.855613] systemd[1]: Reached target Basic System. client # [ 2.932784] systemd[1]: Finished Coldplug All udev Devices. client # [ 2.933767] systemd[1]: Reached target System Initialization. client # [ 2.934600] systemd[1]: Reached target Basic System. worker1 # [ 2.976258] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line. lb # [ 2.939443] systemd[1]: Finished Coldplug All udev Devices. lb # [ 2.940466] systemd[1]: Reached target System Initialization. lb # [ 2.941261] systemd[1]: Reached target Basic System. worker2 # [ 3.019699] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.031643] (udev-worker)[89]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 3.040205] (udev-worker)[89]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 3.045637] (udev-worker)[100]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 3.052236] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line. client # [ 3.080547] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 3.100424] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.107308] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 3.108321] systemd[1]: Reached target Initrd Root Device. worker1 # [ 3.114049] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 3.118395] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.076414] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. lb # [ 3.081015] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line. lb # [ 3.091377] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 3.142092] systemd[1]: Found device /dev/disk/by-label/nixos. worker2 # [ 3.147892] systemd[1]: Reached target Initrd Root Device. worker2 # [ 3.151883] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker1 # [ 3.167437] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks worker1 # [ 3.175316] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker1 # [ 3.180174] systemd[1]: Mounting /sysroot... worker2 # [ 3.205358] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks client # [ 3.206749] systemd[1]: Found device /dev/disk/by-label/nixos. client # [ 3.208732] systemd[1]: Reached target Initrd Root Device. client # [ 3.211332] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... worker2 # [ 3.212210] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. worker2 # [ 3.215826] systemd[1]: Mounting /sysroot... worker1 # [ 3.229233] EXT4-fs (vda): mounted filesystem f896596c-c5ce-4568-8087-8dcfd162026c r/w with ordered data mode. Quota mode: none. worker1 # [ 3.220962] systemd[1]: Mounted /sysroot. worker1 # [ 3.222849] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.224200] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.210536] systemd[1]: Found device /dev/disk/by-label/nixos. worker1 # [ 3.255487] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 3.265884] EXT4-fs (vda): mounted filesystem 218d393b-2fac-4c4a-8d56-08b2222e1a82 r/w with ordered data mode. Quota mode: none. lb # [ 3.214807] systemd[1]: Reached target Initrd Root Device. worker1 # [ 3.259601] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.218345] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... client # [ 3.261709] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.261965] systemd[1]: Mounted /sysroot. worker2 # [ 3.263273] systemd[1]: Reached target Initrd Root File System. worker1 # [ 3.264763] systemd[1]: Reloading... worker2 # [ 3.267420] systemd[1]: Starting Mountpoints Configured in the Real Root... client # [ 3.269477] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. client # [ 3.271747] systemd[1]: Mounting /sysroot... worker2 # [ 3.299407] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. worker2 # [ 3.303934] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.263025] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks worker2 # [ 3.309992] systemd[1]: Reloading... lb # [ 3.271179] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. client # [ 3.327134] EXT4-fs (vda): mounted filesystem 386ec444-c406-47a2-b2dd-0117e896c256 r/w with ordered data mode. Quota mode: none. lb # [ 3.272778] systemd[1]: Mounting /sysroot... client # [ 3.321232] systemd[1]: Mounted /sysroot. client # [ 3.323163] systemd[1]: Reached target Initrd Root File System. client # [ 3.324450] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.323324] EXT4-fs (vda): mounted filesystem c0390e1f-1753-470d-be95-7d18f0ff2ba3 r/w with ordered data mode. Quota mode: none. client # [ 3.355357] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.315550] systemd[1]: Mounted /sysroot. client # [ 3.358909] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.317563] systemd[1]: Reached target Initrd Root File System. client # [ 3.363762] systemd[1]: Reloading... lb # [ 3.324132] systemd[1]: Starting Mountpoints Configured in the Real Root... lb # [ 3.347864] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload. lb # [ 3.354658] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)... lb # [ 3.359304] systemd[1]: Reloading... worker1 # [ 3.473876] systemd[1]: Reloading finished in 211 ms. worker1 # [ 3.510956] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... worker1 # [ 3.514755] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... worker1 # [ 3.522716] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.524087] systemd[1]: Reloading finished in 213 ms. worker1 # [ 3.525310] systemd[1]: Finished Mountpoints Configured in the Real Root. worker1 # [ 3.527455] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker2 # [ 3.554233] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... worker2 # [ 3.557255] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... worker2 # [ 3.562078] systemd[1]: initrd-parse-etc.service: Deactivated successfully. worker2 # [ 3.564606] systemd[1]: Finished Mountpoints Configured in the Real Root. worker2 # [ 3.567435] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. client # [ 3.577503] systemd[1]: Reloading finished in 215 ms. client # [ 3.608388] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... client # [ 3.613017] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... lb # [ 3.576103] systemd[1]: Reloading finished in 215 ms. client # [ 3.619525] systemd[1]: initrd-parse-etc.service: Deactivated successfully. client # [ 3.622607] systemd[1]: Finished Mountpoints Configured in the Real Root. client # [ 3.624636] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. lb # [ 3.596744] systemd-sysroot-fstab-check[118]: Requesting initrd-fs.target/start/replace... lb # [ 3.601757] systemd-sysroot-fstab-check[118]: Requesting swap.target/start/replace... lb # [ 3.609096] systemd[1]: initrd-parse-etc.service: Deactivated successfully. lb # [ 3.612086] systemd[1]: Finished Mountpoints Configured in the Real Root. lb # [ 3.616158] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. worker1 # [ 3.767680] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.779548] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.793174] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.826625] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.831420] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.834884] systemd[1]: Mounting /sysroot/tmp/xchg... worker2 # [ 3.847244] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.858182] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.865658] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 3.877981] systemd[1]: Mounting /sysroot/tmp/shared... client # [ 3.883248] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.887336] systemd[1]: Mounted /sysroot/run. client # [ 3.896134] systemd[1]: Mounting /sysroot/nix/.rw-store... worker1 # [ 3.909465] fuse: init (API version 7.45) worker2 # [ 3.898603] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 3.907944] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.928079] virtiofs virtio6: discovered new tag: nix-store worker1 # [ 3.928927] virtiofs virtio6: virtio_fs_setup_dax: No cache capability client # [ 3.925881] systemd[1]: Mounting /sysroot/tmp/shared... worker1 # [ 3.945032] virtiofs virtio7: discovered new tag: shared lb # [ 3.892459] systemd[1]: Mounting /sysroot/nix/.ro-store... worker1 # [ 3.948831] virtiofs virtio7: virtio_fs_setup_dax: No cache capability worker1 # [ 3.955219] virtiofs virtio8: discovered new tag: xchg worker1 # [ 3.956052] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker1 # [ 3.950249] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 3.912676] systemd[1]: Mounting /sysroot/nix/.rw-store... worker2 # [ 3.958594] systemd[1]: Mounted /sysroot/nix/.rw-store. lb # [ 3.916694] systemd[1]: Mounting /sysroot/run... worker1 # [ 3.962524] systemd[1]: Mounted /sysroot/tmp/shared. worker1 # [ 3.969279] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 3.979459] fuse: init (API version 7.45) worker1 # [ 3.975500] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 3.935466] systemd[1]: Mounting /sysroot/tmp/shared... worker2 # [ 3.992463] virtiofs virtio6: discovered new tag: nix-store client # [ 3.983879] systemd[1]: Mounting /sysroot/tmp/xchg... worker2 # [ 3.993330] virtiofs virtio6: virtio_fs_setup_dax: No cache capability client # [ 4.003531] fuse: init (API version 7.45) worker2 # [ 3.991005] systemd[1]: Mounted /sysroot/run. worker1 # [ 3.995293] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker1 # [ 3.997019] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 4.010805] virtiofs virtio7: discovered new tag: shared worker2 # [ 4.011637] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 4.015447] virtiofs virtio6: discovered new tag: nix-store client # [ 4.016289] virtiofs virtio6: virtio_fs_setup_dax: No cache capability lb # [ 3.963050] systemd[1]: Mounting /sysroot/tmp/xchg... client # [ 4.013115] systemd[1]: Mounted /sysroot/nix/.rw-store. worker2 # [ 4.023411] virtiofs virtio8: discovered new tag: xchg worker2 # [ 4.028472] virtiofs virtio8: virtio_fs_setup_dax: No cache capability client # [ 4.033020] virtiofs virtio7: discovered new tag: shared client # [ 4.033830] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 4.041276] virtiofs virtio8: discovered new tag: xchg client # [ 4.042069] virtiofs virtio8: virtio_fs_setup_dax: No cache capability worker2 # [ 4.029281] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 4.035731] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 4.042085] systemd[1]: Mounted /sysroot/tmp/shared. worker2 # [ 4.045228] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 4.007604] systemd[1]: Mounted /sysroot/nix/.rw-store. client # [ 4.061815] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 4.069015] systemd[1]: Mounted /sysroot/nix/.ro-store. worker2 # [ 4.070209] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 4.071689] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.044803] fuse: init (API version 7.45) client # [ 4.074022] systemd[1]: Mounted /sysroot/run. client # [ 4.076161] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.050775] virtiofs virtio6: discovered new tag: nix-store lb # [ 4.059574] virtiofs virtio6: virtio_fs_setup_dax: No cache capability lb # [ 4.048725] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 4.099324] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 4.076646] virtiofs virtio7: discovered new tag: shared lb # [ 4.077420] virtiofs virtio7: virtio_fs_setup_dax: No cache capability client # [ 4.110500] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 4.111630] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 4.085841] virtiofs virtio8: discovered new tag: xchg lb # [ 4.086664] virtiofs virtio8: virtio_fs_setup_dax: No cache capability lb # [ 4.077828] systemd[1]: Mounted /sysroot/run. lb # [ 4.089668] systemd[1]: Mounted /sysroot/tmp/shared. lb # [ 4.110807] systemd[1]: Mounted /sysroot/nix/.ro-store. lb # [ 4.113071] systemd[1]: Mounted /sysroot/tmp/xchg. lb # [ 4.119095] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 4.121478] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 4.439483] (udev-worker)[91]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.446568] (udev-worker)[91]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.484325] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 4.486705] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 4.487976] systemd[1]: Stopping Virtual Console Setup... worker1 # [ 4.492322] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.509470] (udev-worker)[97]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker1 # [ 4.517692] systemd-vconsole-setup[152]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 4.516604] (udev-worker)[97]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker1 # [ 4.521237] systemd[1]: Finished Virtual Console Setup. client # [ 4.527713] (udev-worker)[93]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory client # [ 4.534864] (udev-worker)[93]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory worker2 # [ 4.547594] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.548892] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.550300] systemd[1]: Stopping Virtual Console Setup... worker2 # [ 4.552675] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.569137] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.570421] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 4.571443] systemd[1]: Stopped Virtual Console Setup. client # [ 4.572140] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 4.572738] systemd[1]: Starting Virtual Console Setup... client # [ 4.576092] systemd[1]: Stopping Virtual Console Setup... client # [ 4.576931] systemd[1]: Starting Virtual Console Setup... client # [ 4.585287] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 4.588172] systemd[1]: Stopped Virtual Console Setup. client # [ 4.592189] systemd[1]: Starting Virtual Console Setup... worker2 # [ 4.594632] systemd-vconsole-setup[154]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 4.551954] (udev-worker)[90]: mtd0ro: Failed to find and pin callout binary "/nix/store/gnadg5slifq2r638xvz4pv6vrsdl9xkj-systemd-261.2/lib/udev/mtd_probe": No such file or directory worker2 # [ 4.598425] systemd[1]: Finished Virtual Console Setup. lb # [ 4.560214] (udev-worker)[90]: mtd0ro: /etc/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory client # [ 4.621687] systemd-vconsole-setup[152]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 4.625196] systemd[1]: Finished Virtual Console Setup. lb # [ 4.597051] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 4.599433] systemd[1]: Stopped Virtual Console Setup. lb # [ 4.604335] systemd[1]: Stopping Virtual Console Setup... lb # [ 4.605185] systemd[1]: Starting Virtual Console Setup... lb # [ 4.629659] systemd-vconsole-setup[154]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 4.633341] systemd[1]: Finished Virtual Console Setup. worker1 # [ 4.768706] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 4.832528] systemd[1]: Mounting /sysroot/nix/store... worker1 # [ 4.836343] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 4.839401] systemd[1]: Reached target Initrd File Systems. worker1 # [ 4.844977] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.853548] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... client # [ 4.885661] systemd[1]: Mounting /sysroot/nix/store... worker2 # [ 4.901397] systemd[1]: Mounted /sysroot/nix/store. worker2 # [ 4.904718] systemd[1]: Reached target Initrd File Systems. worker2 # [ 4.909237] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.912853] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 4.916768] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker2 # [ 4.917619] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 4.924581] systemd[1]: Finished Find NixOS closure. worker1 # [ 4.928088] systemd[1]: Reached target Initrd Default Target. worker1 # [ 4.932369] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... lb # [ 4.895242] systemd[1]: Mounting /sysroot/nix/store... client # [ 4.952049] systemd[1]: Mounted /sysroot/nix/store. client # [ 4.955722] systemd[1]: Reached target Initrd File Systems. client # [ 4.959728] systemd[1]: Starting Find NixOS closure... worker1 # [ 4.960842] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 4.964128] systemd[1]: Stopped target Basic System. worker1 # [ 4.965457] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 4.967210] systemd[1]: Stopped target Path Units. worker2 # [ 4.966428] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. client # [ 4.968320] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker1 # [ 4.970424] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker2 # [ 4.969897] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker1 # [ 4.972495] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker1 # [ 4.977166] systemd[1]: Stopped target Slice Units. worker1 # [ 4.980209] systemd[1]: Stopped target Socket Units. worker2 # [ 4.980817] systemd[1]: Finished Find NixOS closure. worker2 # [ 4.982722] systemd[1]: Reached target Initrd Default Target. worker1 # [ 4.985093] systemd[1]: Stopped target System Initialization. worker2 # [ 4.986131] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 4.987077] systemd[1]: Stopped target Swaps. worker1 # [ 4.993094] systemd[1]: Stopped target Timer Units. worker1 # [ 4.996245] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 4.997212] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 4.999663] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 5.002908] systemd[1]: Stopped Find NixOS closure. client # [ 5.010402] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 5.012210] systemd[1]: Starting rw-sysroot-nix-store.service... lb # [ 4.969940] systemd[1]: Mounted /sysroot/nix/store. worker1 # [ 5.013437] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 5.014736] systemd[1]: Stopped target Initrd Default Target. worker1 # [ 5.016151] systemd[1]: Stopped Apply Kernel Variables. lb # [ 4.974087] systemd[1]: Reached target Initrd File Systems. client # [ 5.015698] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker2 # [ 5.017622] systemd[1]: Stopped target Basic System. worker1 # [ 5.018025] systemd[1]: systemd-modules-load.service: Deactivated successfully. lb # [ 4.976447] systemd[1]: Starting Find NixOS closure... worker2 # [ 5.019144] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 5.023214] systemd[1]: Stopped Load Kernel Modules. worker2 # [ 5.023083] systemd[1]: Stopped target Path Units. worker2 # [ 5.024004] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. worker1 # [ 5.025728] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker1 # [ 5.027053] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. worker2 # [ 5.028186] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 5.029130] systemd[1]: Finished Find NixOS closure. worker2 # [ 5.029634] systemd[1]: Stopped target Slice Units. lb # [ 4.988329] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... worker2 # [ 5.031624] systemd[1]: Stopped target Socket Units. client # [ 5.032256] systemd[1]: Reached target Initrd Default Target. worker1 # [ 5.032375] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. worker1 # [ 5.033565] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 5.033867] systemd[1]: Stopped target System Initialization. worker1 # [ 5.034547] systemd[1]: Stopped target Local File Systems. client # [ 5.034909] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker1 # [ 5.035338] systemd[1]: Stopped target Preparation for Local File Systems. worker1 # [ 5.038530] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 5.040721] systemd[1]: Stopped target Swaps. worker1 # [ 5.041513] systemd[1]: Stopped Coldplug All udev Devices. worker2 # [ 5.041703] systemd[1]: Stopped target Timer Units. worker1 # [ 5.042486] systemd[1]: Stopping Rule-based Manager for Device Events and Files... worker1 # [ 5.043534] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 5.044199] systemd[1]: dbus.socket: Deactivated successfully. worker1 # [ 5.044680] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 5.045431] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker1 # [ 5.046391] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. worker2 # [ 5.046494] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 5.047323] systemd[1]: systemd-udevd.service: Deactivated successfully. worker1 # [ 5.052346] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 5.054218] systemd[1]: systemd-udevd.service: Consumed 1.476s CPU time over 3.090s wall clock time, 22.5M memory peak. worker2 # [ 5.054248] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker1 # [ 5.060395] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. worker1 # [ 5.061657] systemd[1]: Closed udev Control Socket. worker2 # [ 5.062132] systemd[1]: Stopped Find NixOS closure. worker1 # [ 5.062426] systemd[1]: Starting Cleanup udev Database... worker1 # [ 5.064183] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 5.064422] systemd[1]: Starting rw-sysroot-nix-store.service... worker1 # [ 5.068165] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 5.068633] systemd[1]: Stopped target Initrd Default Target. worker2 # [ 5.068454] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker1 # [ 5.069194] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 5.069939] systemd[1]: Stopped target Basic System. worker1 # [ 5.070342] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. client # [ 5.070949] systemd[1]: Stopped target Initrd Root Device. worker1 # [ 5.072204] systemd[1]: kmod-static-nodes.service: Deactivated successfully. client # [ 5.073368] systemd[1]: Stopped target Path Units. worker2 # [ 5.073212] systemd[1]: Stopped Apply Kernel Variables. worker2 # [ 5.074805] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 5.075154] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. lb # [ 5.032745] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. worker1 # [ 5.076370] systemd[1]: Stopped Create List of Static Device Nodes. worker1 # [ 5.077272] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.077314] systemd[1]: Stopped Load Kernel Modules. worker1 # [ 5.078286] systemd[1]: Finished rw-sysroot-nix-store.service. worker2 # [ 5.079114] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. client # [ 5.080225] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. worker2 # [ 5.081555] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. client # [ 5.084194] systemd[1]: Stopped target Slice Units. lb # [ 5.040314] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. worker2 # [ 5.084281] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 5.086550] systemd[1]: Stopped target Socket Units. worker2 # [ 5.086550] systemd[1]: Stopped Create System Files and Directories. worker2 # [ 5.088456] systemd[1]: Stopped target Local File Systems. client # [ 5.089776] systemd[1]: Stopped target System Initialization. worker2 # [ 5.092253] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 5.093196] systemd[1]: Stopped target Swaps. worker2 # [ 5.093284] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. worker2 # [ 5.094292] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 5.051844] systemd[1]: Finished Find NixOS closure. worker2 # [ 5.095653] systemd[1]: Stopping Rule-based Manager for Device Events and Files... client # [ 5.097045] systemd[1]: Stopped target Timer Units. worker2 # [ 5.096979] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 5.054926] systemd[1]: Reached target Initrd Default Target. worker2 # [ 5.098005] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 5.098725] systemd[1]: initrd-cleanup.service: Deactivated successfully. worker2 # [ 5.099640] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 5.100208] systemd[1]: dbus.socket: Deactivated successfully. worker2 # [ 5.100671] systemd[1]: systemd-udevd.service: Deactivated successfully. worker2 # [ 5.101581] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker1 # [ 5.102117] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 5.060447] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... worker2 # [ 5.102580] systemd[1]: systemd-udevd.service: Consumed 1.506s CPU time over 3.080s wall clock time, 25.3M memory peak. worker1 # [ 5.104521] systemd[1]: Finished Cleanup udev Database. worker2 # [ 5.103972] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. client # [ 5.105042] systemd[1]: Closed D-Bus System Message Bus Socket. worker1 # [ 5.106150] systemd[1]: Reached target Switch Root. client # [ 5.107241] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. worker2 # [ 5.108250] systemd[1]: Finished rw-sysroot-nix-store.service. worker1 # [ 5.108876] systemd[1]: Starting NixOS Activation... client # [ 5.109245] systemd[1]: Stopped Find NixOS closure. worker2 # [ 5.109149] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. client # [ 5.110956] systemd[1]: Starting rw-sysroot-nix-store.service... worker2 # [ 5.111650] systemd[1]: Closed udev Control Socket. client # [ 5.115374] systemd[1]: systemd-sysctl.service: Deactivated successfully. worker2 # [ 5.116339] systemd[1]: Starting Cleanup udev Database... worker2 # [ 5.117121] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. worker2 # [ 5.118179] systemd[1]: Stopped Create Static Device Nodes in /dev. worker2 # [ 5.119026] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. client # [ 5.120440] systemd[1]: Stopped Apply Kernel Variables. client # [ 5.121304] systemd[1]: systemd-modules-load.service: Deactivated successfully. client # [ 5.122467] systemd[1]: Stopped Load Kernel Modules. client # [ 5.123179] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. worker2 # [ 5.124200] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. worker2 # [ 5.125209] systemd[1]: kmod-static-nodes.service: Deactivated successfully. worker2 # [ 5.126177] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 5.131107] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 5.089423] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.133604] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. lb # [ 5.092147] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 5.136412] systemd[1]: Stopped Create System Files and Directories. client # [ 5.138438] systemd[1]: Stopped target Local File Systems. client # [ 5.139646] systemd[1]: Stopped target Preparation for Local File Systems. lb # [ 5.096321] systemd[1]: Stopped target Initrd Default Target. lb # [ 5.098442] systemd[1]: Stopped target Basic System. client # [ 5.143669] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. lb # [ 5.101467] systemd[1]: Stopped target Initrd Root Device. client # [ 5.145016] systemd[1]: Stopped Coldplug All udev Devices. lb # [ 5.102879] systemd[1]: Stopped target Path Units. client # [ 5.145827] systemd[1]: Stopping Rule-based Manager for Device Events and Files... client # [ 5.146882] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 5.147896] systemd[1]: Stopped Virtual Console Setup. lb # [ 5.105218] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. client # [ 5.148958] systemd[1]: initrd-cleanup.service: Deactivated successfully. client # [ 5.149884] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. client # [ 5.150801] systemd[1]: systemd-udevd.service: Deactivated successfully. client # [ 5.151715] systemd[1]: Stopped Rule-based Manager for Device Events and Files. worker2 # [ 5.153475] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 5.113170] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. client # [ 5.156247] systemd[1]: systemd-udevd.service: Consumed 1.449s CPU time over 3.044s wall clock time, 25.4M memory peak. worker2 # [ 5.156554] systemd[1]: Finished Cleanup udev Database. worker2 # [ 5.157674] systemd[1]: Reached target Switch Root. client # [ 5.158710] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 5.116509] systemd[1]: Stopped target Slice Units. worker2 # [ 5.159351] systemd[1]: Starting NixOS Activation... client # [ 5.161056] systemd[1]: Closed udev Control Socket. lb # [ 5.118304] systemd[1]: Stopped target Socket Units. lb # [ 5.121231] systemd[1]: Stopped target System Initialization. client # [ 5.164246] systemd[1]: Starting Cleanup udev Database... client # [ 5.165112] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 5.124587] systemd[1]: Stopped target Swaps. client # [ 5.168157] systemd[1]: Stopped Create Static Device Nodes in /dev. client # [ 5.169079] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 5.127141] systemd[1]: Stopped target Timer Units. lb # [ 5.128435] systemd[1]: dbus.socket: Deactivated successfully. client # [ 5.172142] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 5.130187] systemd[1]: Closed D-Bus System Message Bus Socket. client # [ 5.173169] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 5.131249] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. client # [ 5.176191] systemd[1]: Stopped Create List of Static Device Nodes. client # [ 5.177095] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. lb # [ 5.136273] systemd[1]: Stopped Find NixOS closure. lb # [ 5.137201] systemd[1]: Starting rw-sysroot-nix-store.service... client # [ 5.180251] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 5.141653] systemd[1]: systemd-sysctl.service: Deactivated successfully. lb # [ 5.146297] systemd[1]: Stopped Apply Kernel Variables. lb # [ 5.148472] systemd[1]: systemd-modules-load.service: Deactivated successfully. worker1 # [ 5.191072] initrd-nixos-activation-start[178]: booting system configuration /nix/store/0hxczffswy9q3fzf9cy2bm1icq25s5b8-nixos-system-worker1-test lb # [ 5.152575] systemd[1]: Stopped Load Kernel Modules. lb # [ 5.154197] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. lb # [ 5.156893] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. lb # [ 5.160231] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. client # [ 5.203938] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 5.162364] systemd[1]: Stopped Create System Files and Directories. lb # [ 5.163631] systemd[1]: Stopped target Local File Systems. lb # [ 5.164716] systemd[1]: Stopped target Preparation for Local File Systems. client # [ 5.207517] systemd[1]: Finished Cleanup udev Database. lb # [ 5.165688] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. client # [ 5.208782] systemd[1]: Reached target Switch Root. lb # [ 5.166669] systemd[1]: Stopped Coldplug All udev Devices. client # [ 5.209967] systemd[1]: Starting NixOS Activation... lb # [ 5.167441] systemd[1]: Stopping Rule-based Manager for Device Events and Files... lb # [ 5.172293] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 5.173376] systemd[1]: Stopped Virtual Console Setup. lb # [ 5.176302] systemd[1]: systemd-udevd.service: Deactivated successfully. lb # [ 5.177511] systemd[1]: Stopped Rule-based Manager for Device Events and Files. lb # [ 5.180231] systemd[1]: systemd-udevd.service: Consumed 1.453s CPU time over 3.083s wall clock time, 22M memory peak. worker1 # [ 5.225781] initrd-nixos-activation-start[178]: running activation script... lb # [ 5.184205] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. lb # [ 5.185241] systemd[1]: Closed udev Control Socket. lb # [ 5.185935] systemd[1]: Starting Cleanup udev Database... lb # [ 5.188178] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. lb # [ 5.189294] systemd[1]: Stopped Create Static Device Nodes in /dev. lb # [ 5.192255] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. lb # [ 5.196226] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. lb # [ 5.197245] systemd[1]: kmod-static-nodes.service: Deactivated successfully. lb # [ 5.198193] systemd[1]: Stopped Create List of Static Device Nodes. lb # [ 5.200243] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. worker2 # [ 5.244769] initrd-nixos-activation-start[177]: booting system configuration /nix/store/gjdzrza82g2nqw3dgr89lmwyjzkwq204-nixos-system-worker2-test lb # [ 5.204328] systemd[1]: Finished rw-sysroot-nix-store.service. lb # [ 5.223797] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. lb # [ 5.226955] systemd[1]: Finished Cleanup udev Database. lb # [ 5.228786] systemd[1]: Reached target Switch Root. lb # [ 5.232615] systemd[1]: Starting NixOS Activation... worker2 # [ 5.278399] initrd-nixos-activation-start[177]: running activation script... client # [ 5.294155] initrd-nixos-activation-start[177]: booting system configuration /nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test client # [ 5.326894] initrd-nixos-activation-start[177]: running activation script... lb # [ 5.307867] initrd-nixos-activation-start[178]: booting system configuration /nix/store/kgjzg78npi7z07am2szqxkr41f06jp5w-nixos-system-lb-test lb # [ 5.341086] initrd-nixos-activation-start[178]: running activation script... worker1 # [ 5.486704] initrd-nixos-activation-start[201]: setting up /etc... worker2 # [ 5.524567] initrd-nixos-activation-start[200]: setting up /etc... client # [ 5.562331] initrd-nixos-activation-start[200]: setting up /etc... worker1 # [ 5.607079] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker1 # [ 5.610026] systemd[1]: Finished NixOS Activation. worker1 # [ 5.611249] systemd[1]: Starting Switch Root... lb # [ 5.571915] initrd-nixos-activation-start[201]: setting up /etc... worker1 # [ 5.635523] systemd[1]: Switching root. worker2 # [ 5.652436] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. worker2 # [ 5.655227] systemd[1]: Finished NixOS Activation. worker2 # [ 5.660247] systemd[1]: Starting Switch Root... worker2 # [ 5.679755] systemd[1]: Switching root. client # [ 5.682142] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. client # [ 5.685141] systemd[1]: Finished NixOS Activation. client # [ 5.688160] systemd[1]: Starting Switch Root... client # [ 5.707848] systemd[1]: Switching root. lb # [ 5.689755] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. lb # [ 5.692598] systemd[1]: Finished NixOS Activation. lb # [ 5.696194] systemd[1]: Starting Switch Root... lb # [ 5.711803] systemd[1]: Switching root. worker1 # [ 5.815258] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker2 # [ 5.857148] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). client # [ 5.892456] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). lb # [ 5.889167] systemd-journald[72]: Received SIGTERM from PID 1 (systemd). worker1 # [ 6.344614] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker1 # [ 6.357652] systemd[1]: Detected virtualization qemu. worker1 # [ 6.360780] systemd[1]: Detected architecture arm64. worker1 # [ 6.364591] systemd[1]: Detected first boot. worker2 # [ 6.353563] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) worker2 # [ 6.366659] systemd[1]: Detected virtualization qemu. worker1 # [ 6.370529] systemd[1]: Initializing machine ID from random generator. worker2 # [ 6.370240] systemd[1]: Detected architecture arm64. worker2 # [ 6.374291] systemd[1]: Detected first boot. worker2 # [ 6.379369] systemd[1]: Initializing machine ID from random generator. client # [ 6.422528] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) client # [ 6.435369] systemd[1]: Detected virtualization qemu. client # [ 6.438696] systemd[1]: Detected architecture arm64. client # [ 6.442738] systemd[1]: Detected first boot. client # [ 6.448791] systemd[1]: Initializing machine ID from random generator. lb # [ 6.414308] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) lb # [ 6.427263] systemd[1]: Detected virtualization qemu. lb # [ 6.430428] systemd[1]: Detected architecture arm64. lb # [ 6.434344] systemd[1]: Detected first boot. lb # [ 6.440299] systemd[1]: Initializing machine ID from random generator. client # [ 6.649703] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker1 # [ 6.697532] systemd[1]: bpf-restrict-fs: LSM BPF program attached worker2 # [ 6.708365] systemd[1]: bpf-restrict-fs: LSM BPF program attached lb # [ 6.771469] systemd[1]: bpf-restrict-fs: LSM BPF program attached client # [ 6.827920] systemd[1]: Applying preset policy. worker1 # [ 6.867670] systemd[1]: Applying preset policy. worker2 # [ 6.928542] systemd[1]: Applying preset policy. lb # [ 6.931631] systemd[1]: Applying preset policy. client # [ 7.082417] systemd[1]: Populated /etc with preset unit settings. worker1 # [ 7.125092] systemd[1]: Populated /etc with preset unit settings. worker2 # [ 7.215928] systemd[1]: Populated /etc with preset unit settings. lb # [ 7.192547] systemd[1]: Populated /etc with preset unit settings. client # [ 7.308567] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 7.310292] systemd[1]: Stopped initrd-switch-root.service. client # [ 7.314505] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 7.318989] systemd[1]: Created slice Slice /system/getty. client # [ 7.320916] systemd[1]: Created slice User and Session Slice. client # [ 7.322180] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 7.324836] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 7.327208] systemd[1]: Expecting device /dev/hvc0... client # [ 7.328221] systemd[1]: Expecting device /dev/ttyAMA0... client # [ 7.330678] systemd[1]: Reached target Local Encrypted Volumes. client # [ 7.331829] systemd[1]: Stopped target initrd-fs.target. client # [ 7.333387] systemd[1]: Stopped target initrd-root-fs.target. client # [ 7.335081] systemd[1]: Stopped target initrd-switch-root.target. client # [ 7.338351] systemd[1]: Reached target Virtual Machines and Containers. client # [ 7.339602] systemd[1]: Reached target Path Units. client # [ 7.341080] systemd[1]: Reached target Remote File Systems. client # [ 7.342701] systemd[1]: Reached target Slice Units. client # [ 7.345066] systemd[1]: Reached target Swaps. client # [ 7.347482] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 7.350432] systemd[1]: Listening on Process Core Dump Socket. client # [ 7.353551] systemd[1]: Listening on Credential Encryption/Decryption. client # [ 7.355970] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.354386] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 7.357206] systemd[1]: Listening on Hostname Service Socket. worker1 # [ 7.357108] systemd[1]: Stopped initrd-switch-root.service. client # [ 7.361450] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 7.361259] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. client # [ 7.363680] systemd[1]: Listening on Journal Audit Socket. client # [ 7.367449] systemd[1]: Listening on Console Output Muting Service Socket. client # [ 7.369039] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 7.365935] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.367084] systemd[1]: Created slice User and Session Slice. client # [ 7.370709] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.367468] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker1 # [ 7.367829] systemd[1]: Started Forward Password Requests to Wall Directory Watch. client # [ 7.373245] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 7.368095] systemd[1]: Expecting device /dev/hvc0... worker1 # [ 7.368338] systemd[1]: Expecting device /dev/ttyAMA0... worker1 # [ 7.368577] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 7.368819] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 7.369056] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 7.369289] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 7.369527] systemd[1]: Reached target Virtual Machines and Containers. client # [ 7.379201] systemd[1]: Listening on Disk Repartitioning Service Socket. worker1 # [ 7.378035] systemd[1]: Reached target Path Units. client # [ 7.380586] systemd[1]: Listening on udev Control Socket. worker1 # [ 7.380324] systemd[1]: Reached target Remote File Systems. client # [ 7.382252] systemd[1]: Listening on udev Varlink Socket. worker1 # [ 7.382254] systemd[1]: Reached target Slice Units. worker1 # [ 7.384043] systemd[1]: Reached target Swaps. client # [ 7.386108] systemd[1]: Mounting Huge Pages File System... worker1 # [ 7.387629] systemd[1]: Listening on Query the User Interactively for a Password. client # [ 7.389748] systemd[1]: Mounting POSIX Message Queue File System... worker1 # [ 7.390656] systemd[1]: Listening on Process Core Dump Socket. worker1 # [ 7.392880] systemd[1]: Listening on Credential Encryption/Decryption. worker1 # [ 7.395298] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.396541] systemd[1]: Listening on Hostname Service Socket. client # [ 7.399462] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 7.400993] systemd[1]: Starting Journal Log Access Socket... worker1 # [ 7.403286] systemd[1]: Listening on Journal Audit Socket. worker1 # [ 7.406838] systemd[1]: Listening on Console Output Muting Service Socket. worker1 # [ 7.408414] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker1 # [ 7.410155] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.412734] systemd[1]: Mounting Kernel Trace File System... worker1 # [ 7.413649] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker1 # [ 7.419611] systemd[1]: Listening on Disk Repartitioning Service Socket. client # [ 7.420965] systemd[1]: Starting Create List of Static Device Nodes... worker1 # [ 7.421000] systemd[1]: Listening on udev Control Socket. worker1 # [ 7.422796] systemd[1]: Listening on udev Varlink Socket. client # [ 7.424072] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 7.427625] systemd[1]: Mounting Huge Pages File System... worker1 # [ 7.437234] systemd[1]: Mounting POSIX Message Queue File System... client # [ 7.444013] systemd[1]: Mounting Kernel Configuration File System... client # [ 7.446460] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.447324] systemd[1]: Mounting Kernel Debug File System... worker2 # [ 7.449188] systemd[1]: initrd-switch-root.service: Deactivated successfully. client # [ 7.452991] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker2 # [ 7.450977] systemd[1]: Stopped initrd-switch-root.service. client # [ 7.457779] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.455334] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 7.460143] systemd[1]: Created slice Slice /system/getty. worker1 # [ 7.463844] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 7.462302] systemd[1]: Created slice User and Session Slice. worker2 # [ 7.464639] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. worker2 # [ 7.467014] systemd[1]: Started Forward Password Requests to Wall Directory Watch. lb # [ 7.429136] systemd[1]: initrd-switch-root.service: Deactivated successfully. worker2 # [ 7.469322] systemd[1]: Expecting device /dev/hvc0... lb # [ 7.431030] systemd[1]: Stopped initrd-switch-root.service. worker2 # [ 7.471199] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.473280] systemd[1]: Reached target Local Encrypted Volumes. worker1 # [ 7.474152] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 7.474441] systemd[1]: Stopped target initrd-fs.target. worker1 # [ 7.474566] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 7.435651] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. worker2 # [ 7.476039] systemd[1]: Stopped target initrd-root-fs.target. worker2 # [ 7.478559] systemd[1]: Stopped target initrd-switch-root.target. lb # [ 7.439660] systemd[1]: Created slice Slice /system/getty. lb # [ 7.441550] systemd[1]: Created slice User and Session Slice. worker2 # [ 7.479698] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 7.482161] systemd[1]: Reached target Path Units. lb # [ 7.442844] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. client # [ 7.487561] systemd[1]: Mounting FUSE Control File System... worker2 # [ 7.483142] systemd[1]: Reached target Remote File Systems. lb # [ 7.445472] systemd[1]: Started Forward Password Requests to Wall Directory Watch. worker2 # [ 7.485551] systemd[1]: Reached target Slice Units. worker2 # [ 7.487080] systemd[1]: Reached target Swaps. lb # [ 7.447777] systemd[1]: Expecting device /dev/hvc0... client # [ 7.489499] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.449611] systemd[1]: Expecting device /dev/ttyAMA0... worker2 # [ 7.490179] systemd[1]: Listening on Query the User Interactively for a Password. lb # [ 7.451639] systemd[1]: Reached target Local Encrypted Volumes. lb # [ 7.452760] systemd[1]: Stopped target initrd-fs.target. worker2 # [ 7.493344] systemd[1]: Listening on Process Core Dump Socket. lb # [ 7.454347] systemd[1]: Stopped target initrd-root-fs.target. worker1 # [ 7.493894] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 7.495683] systemd[1]: Listening on Credential Encryption/Decryption. lb # [ 7.456060] systemd[1]: Stopped target initrd-switch-root.target. worker1 # [ 7.494345] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm lb # [ 7.458533] systemd[1]: Reached target Virtual Machines and Containers. worker2 # [ 7.498140] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.494628] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.494913] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker2 # [ 7.499377] systemd[1]: Listening on Hostname Service Socket. lb # [ 7.459831] systemd[1]: Reached target Path Units. lb # [ 7.462081] systemd[1]: Reached target Remote File Systems. lb # [ 7.463208] systemd[1]: Reached target Slice Units. lb # [ 7.465540] systemd[1]: Reached target Swaps. worker2 # [ 7.504399] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 7.506158] systemd[1]: Listening on Journal Audit Socket. lb # [ 7.468071] systemd[1]: Listening on Query the User Interactively for a Password. worker2 # [ 7.509183] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 7.471118] systemd[1]: Listening on Process Core Dump Socket. worker2 # [ 7.510857] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. lb # [ 7.473288] systemd[1]: Listening on Credential Encryption/Decryption. worker2 # [ 7.512554] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.475641] systemd[1]: Listening on Factory Reset Management. worker1 # [ 7.516758] systemd[1]: Mounting FUSE Control File System... lb # [ 7.476810] systemd[1]: Listening on Hostname Service Socket. worker2 # [ 7.515873] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki client # [ 7.520300] systemd[1]: Starting Journal Service... worker1 # [ 7.518173] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 lb # [ 7.480552] systemd[1]: Starting Journal Log Access Socket... worker2 # [ 7.522024] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.482854] systemd[1]: Listening on Journal Audit Socket. worker2 # [ 7.523440] systemd[1]: Listening on udev Control Socket. worker2 # [ 7.525250] systemd[1]: Listening on udev Varlink Socket. lb # [ 7.486988] systemd[1]: Listening on Console Output Muting Service Socket. lb # [ 7.489044] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. worker2 # [ 7.529228] systemd[1]: Mounting Huge Pages File System... lb # [ 7.491591] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.535876] systemd[1]: Starting Load Kernel Modules... lb # [ 7.493175] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki worker2 # [ 7.536542] systemd[1]: Mounting POSIX Message Queue File System... lb # [ 7.499486] systemd[1]: Listening on Disk Repartitioning Service Socket. lb # [ 7.500849] systemd[1]: Listening on udev Control Socket. worker1 # [ 7.543862] systemd[1]: Starting Journal Service... lb # [ 7.502584] systemd[1]: Listening on udev Varlink Socket. lb # [ 7.506281] systemd[1]: Mounting Huge Pages File System... worker2 # [ 7.547950] systemd[1]: Mounting Kernel Debug File System... lb # [ 7.511551] systemd[1]: Mounting POSIX Message Queue File System... client # [ 7.554288] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... worker2 # [ 7.559982] systemd[1]: Mounting Kernel Trace File System... lb # [ 7.523700] systemd[1]: Mounting Kernel Debug File System... worker1 # [ 7.571332] systemd[1]: Starting Load Kernel Modules... lb # [ 7.533782] systemd[1]: Mounting Kernel Trace File System... worker2 # [ 7.573061] systemd[1]: Starting Create List of Static Device Nodes... worker2 # [ 7.575895] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker1 # [ 7.581804] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... client # [ 7.583548] systemd[1]: Starting Remount Root and Kernel File Systems... lb # [ 7.543562] systemd[1]: Starting Create List of Static Device Nodes... client # [ 7.583998] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.545977] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs worker2 # [ 7.587946] systemd[1]: Mounting Kernel Configuration File System... worker2 # [ 7.591878] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.594528] systemd[1]: Starting Remount Root and Kernel File Systems... client # [ 7.596149] systemd[1]: Starting Coldplug All udev Devices... worker1 # [ 7.596770] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.597348] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore client # [ 7.602077] systemd[1]: Listening on Journal Log Access Socket. worker2 # [ 7.599433] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 7.610129] systemd[1]: Starting Coldplug All udev Devices... lb # [ 7.572222] systemd[1]: Mounting Kernel Configuration File System... lb # [ 7.573581] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm worker1 # [ 7.616514] systemd[1]: Listening on Journal Log Access Socket. lb # [ 7.580081] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore worker1 # [ 7.620669] systemd[1]: Mounted Huge Pages File System. worker1 # [ 7.621169] systemd[1]: Mounted POSIX Message Queue File System. client # [ 7.624541] systemd-journald[270]: Collecting audit messages is enabled. worker2 # [ 7.620235] systemd[1]: Mounting FUSE Control File System... lb # [ 7.583817] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 7.627048] systemd[1]: Mounted Huge Pages File System. worker1 # [ 7.625328] systemd[1]: Mounted Kernel Debug File System. worker2 # [ 7.620716] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 worker1 # [ 7.627792] systemd[1]: Mounted Kernel Trace File System. worker1 # [ 7.631568] systemd[1]: Mounted Kernel Configuration File System. worker1 # [ 7.632046] systemd[1]: Mounted FUSE Control File System. client # [ 7.634502] systemd[1]: Mounted POSIX Message Queue File System. client # [ 7.637544] systemd[1]: Mounted Kernel Debug File System. client # [ 7.638030] systemd[1]: Mounted Kernel Trace File System. lb # [ 7.607287] systemd[1]: Mounting FUSE Control File System... client # [ 7.637744] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.648599] systemd[1]: Starting Journal Service... worker1 # [ 7.649918] systemd[1]: Finished Create List of Static Device Nodes. client # [ 7.640417] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 7.611086] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 7.657829] systemd[1]: Finished Create List of Static Device Nodes. client # [ 7.659806] systemd[1]: Started Journal Service. worker1 # [ 7.658010] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.655694] systemd-modules-load[271]: Module 'atkbd' is built in worker2 # [ 7.670522] systemd[1]: Starting Load Kernel Modules... client # [ 7.660552] systemd-modules-load[271]: Module 'loop' is built in lb # [ 7.641091] systemd[1]: Starting Journal Service... client # [ 7.672406] systemd[1]: Mounted Kernel Configuration File System. client # [ 7.681508] systemd[1]: Finished Load Kernel Modules. worker2 # [ 7.693565] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... lb # [ 7.657789] systemd[1]: Starting Load Kernel Modules... worker2 # [ 7.699026] systemd[1]: Starting Remount Root and Kernel File Systems... worker2 # [ 7.701169] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 7.696810] systemd[1]: Starting Apply Kernel Variables... worker2 # [ 7.712835] systemd[1]: Starting Coldplug All udev Devices... lb # [ 7.675700] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... client # [ 7.708336] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... worker2 # [ 7.718808] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 7.723081] systemd-journald[271]: Collecting audit messages is enabled. worker2 # [ 7.722671] systemd[1]: Mounted Huge Pages File System. lb # [ 7.684609] systemd[1]: Starting Remount Root and Kernel File Systems... lb # [ 7.686254] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.725847] systemd[1]: Mounted POSIX Message Queue File System. client # [ 7.716576] systemd-oomd[272]: No swap; memory pressure usage will be degraded worker2 # [ 7.729002] systemd[1]: Mounted Kernel Debug File System. client # [ 7.722011] systemd[1]: Mounted FUSE Control File System. worker2 # [ 7.732492] systemd[1]: Mounted Kernel Trace File System. worker2 # [ 7.734987] systemd[1]: Mounted Kernel Configuration File System. worker2 # [ 7.735456] systemd[1]: Mounted FUSE Control File System. client # [ 7.727789] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.[ 7.743525] EXT4-fs (vda): re-mounted 386ec444-c406-47a2-b2dd-0117e896c256. client # worker1 # [ 7.744959] systemd[1]: Finished Load Kernel Modules. worker1 # [ 7.755202] systemd[1]: Starting Firewall... worker2 # [ 7.753571] systemd[1]: Finished Create List of Static Device Nodes. client # [ 7.744988] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 7.757675] EXT4-fs (vda): re-mounted f896596c-c5ce-4568-8087-8dcfd162026c. worker2 # [ 7.759004] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... client # [ 7.750306] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 7.764158] systemd[1]: Starting Apply Kernel Variables... lb # [ 7.723595] systemd[1]: Starting Coldplug All udev Devices... lb # [ 7.726427] systemd[1]: Listening on Journal Log Access Socket. worker1 # [ 7.767813] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 7.768428] systemd[1]: Listening on Disk Image Download Service Socket. worker1 # [ 7.768716] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore client # [ 7.759635] systemd[1]: Starting Flush Journal to Persistent Storage... client # [ 7.760988] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 7.743876] systemd[1]: Mounted Huge Pages File System. client # [ 7.772206] systemd[1]: Starting Load/Save OS Random Seed... client # [ 7.773178] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 7.788222] systemd[1]: Starting Load/Save OS Random Seed... lb # [ 7.748468] systemd[1]: Mounted POSIX Message Queue File System. worker1 # [ 7.790874] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os lb # [ 7.754026] systemd[1]: Mounted Kernel Debug File System. worker1 # [ 7.804663] systemd[1]: Started Journal Service. lb # [ 7.765618] systemd[1]: Mounted Kernel Trace File System. lb # [ 7.768480] systemd[1]: Finished Create List of Static Device Nodes. worker1 # [ 7.801292] systemd[1]: Queued start job for default target Multi-User System. worker1 # [ 7.802499] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 7.779629] systemd[1]: Mounted Kernel Configuration File System. lb # [ 7.781617] systemd[1]: Mounted FUSE Control File System. worker1 # [ 7.813542] systemd-modules-load[272]: Module 'atkbd' is built in worker2 # [ 7.828173] systemd-journald[270]: Collecting audit messages is enabled. lb # [ 7.787172] systemd-journald[271]: Collecting audit messages is enabled. worker1 # [ 7.827556] systemd-modules-load[272]: Module 'loop' is built in lb # [ 7.804336] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... lb # [ 7.806447] systemd[1]: Finished Load Kernel Modules. worker2 # [ 7.844419] systemd[1]: Finished Load Kernel Modules. worker1 # [ 7.837752] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 7.811052] EXT4-fs (vda): re-mounted c0390e1f-1753-470d-be95-7d18f0ff2ba3. lb # [ 7.802492] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.857004] systemd[1]: Starting Firewall... lb # [ 7.822273] systemd[1]: Starting Firewall... client # [ 7.867576] systemd-journald[270]: Received client request to flush runtime journal. lb # [ 7.811423] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 7.815103] systemd-modules-load[272]: Module 'atkbd' is built in worker2 # [ 7.870652] systemd[1]: Starting Apply Kernel Variables... lb # [ 7.820440] systemd-modules-load[272]: Module 'loop' is built in worker2 # [ 7.879895] EXT4-fs (vda): re-mounted 218d393b-2fac-4c4a-8d56-08b2222e1a82. lb # [ 7.833117] systemd-oomd[273]: No swap; memory pressure usage will be degraded worker2 # [ 7.889707] systemd[1]: Started Journal Service. lb # [ 7.852049] systemd[1]: Starting Apply Kernel Variables... lb # [ 7.857855] systemd[1]: Started Journal Service. worker2 # [ 7.886985] systemd[1]: Queued start job for default target Multi-User System. worker2 # [ 7.888267] systemd[1]: systemd-journald.service: Deactivated successfully. lb # [ 7.851663] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 7.905009] systemd-modules-load[271]: Module 'atkbd' is built in lb # [ 7.864102] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 7.907155] systemd-oomd[274]: No swap; memory pressure usage will be degraded client # [ 7.911857] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 7.918526] systemd-modules-load[271]: Module 'loop' is built in client # [ 7.920880] systemd[1]: Reached target First Boot Complete. client # [ 7.923649] systemd[1]: Finished Flush Journal to Persistent Storage. client # [ 7.925676] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 7.923793] systemd[1]: Finished Load/Save OS Random Seed. worker1 # [ 7.930448] systemd[1]: Reached target First Boot Complete. worker2 # [ 7.930339] systemd[1]: Finished Remount Root and Kernel File Systems. worker1 # [ 7.931283] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. lb # [ 7.898051] systemd[1]: Listening on Disk Image Download Service Socket. worker2 # [ 7.942627] systemd[1]: Listening on Disk Image Download Service Socket. lb # [ 7.906415] systemd[1]: Starting Flush Journal to Persistent Storage... lb # [ 7.907467] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore worker1 # [ 7.951696] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 7.957116] systemd[1]: Starting Flush Journal to Persistent Storage... worker1 # [ 7.971212] systemd-journald[271]: Received client request to flush runtime journal. lb # [ 7.919991] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 7.965242] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 7.924808] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker2 # [ 7.983110] systemd[1]: Starting Load/Save OS Random Seed... worker2 # [ 7.996105] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os worker1 # [ 8.013025] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 8.014206] systemd[1]: Finished Apply Kernel Variables. worker1 # [ 8.022202] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 8.036950] systemd-oomd[273]: No swap; memory pressure usage will be degraded lb # [ 8.012060] systemd-journald[271]: Received client request to flush runtime journal. worker2 # [ 8.056454] systemd-journald[270]: Received client request to flush runtime journal. lb # [ 8.068853] systemd[1]: Finished Load/Save OS Random Seed. lb # [ 8.069818] systemd[1]: Reached target First Boot Complete. lb # [ 8.070640] systemd[1]: Finished Apply Kernel Variables. lb # [ 8.071402] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. worker2 # [ 8.121895] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. lb # [ 8.080881] systemd[1]: Starting Create Static Device Nodes in /dev... worker2 # [ 8.123243] systemd[1]: Starting Create Static Device Nodes in /dev... lb # [ 8.082927] systemd[1]: Finished Flush Journal to Persistent Storage. worker2 # [ 8.136310] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. worker2 # [ 8.138511] systemd[1]: Finished Load/Save OS Random Seed. worker2 # [ 8.139409] systemd[1]: Reached target First Boot Complete. worker2 # [ 8.142533] systemd[1]: Finished Apply Kernel Variables. worker2 # [ 8.143350] systemd[1]: Finished Flush Journal to Persistent Storage. worker1 # [ 8.184309] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 8.184853] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. client # [ 8.188937] systemd[1]: Starting Create Static Device Nodes in /dev... worker1 # [ 8.190411] systemd[1]: Reached target Preparation for Local File Systems. worker1 # [ 8.200995] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 8.250989] systemd[1]: Finished Create Static Device Nodes in /dev. worker2 # [ 8.253840] systemd[1]: Reached target Preparation for Local File Systems. worker2 # [ 8.261061] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 8.313094] systemd-udevd[310]: Using default interface naming scheme 'v261'. worker1 # [ 8.346027] systemd[1]: Mounting /run/wrappers... worker2 # [ 8.373217] systemd-udevd[311]: Using default interface naming scheme 'v261'. lb # [ 8.341054] systemd[1]: Finished Create Static Device Nodes in /dev. lb # [ 8.348111] systemd[1]: Reached target Preparation for Local File Systems. lb # [ 8.352957] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker1 # [ 8.411895] systemd[1]: Mounted /run/wrappers. worker1 # [ 8.425348] systemd[1]: Reached target Local File Systems. worker1 # [ 8.432646] systemd[1]: Listening on Boot Loader Control Service Socket. worker1 # [ 8.443850] systemd[1]: Starting register-nix-paths.service... worker2 # [ 8.448286] systemd[1]: Mounting /run/wrappers... worker1 # [ 8.454472] systemd[1]: Starting Create SUID/SGID Wrappers... worker1 # [ 8.462176] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker1 # [ 8.466280] systemd[1]: Starting Save Transient machine-id to Disk... lb # [ 8.425329] systemd[1]: Mounting /run/wrappers... worker1 # [ 8.467706] systemd[1]: Starting Create System Files and Directories... worker1 # [ 8.483690] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 8.484260] systemd[1]: Finished Coldplug All udev Devices. client # [ 8.494069] systemd[1]: Finished Create Static Device Nodes in /dev. client # [ 8.496563] systemd[1]: Reached target Preparation for Local File Systems. client # [ 8.500369] systemd[1]: Mounting /run/wrappers... worker2 # [ 8.505192] systemd[1]: Mounted /run/wrappers. client # [ 8.508285] systemd[1]: Starting Rule-based Manager for Device Events and Files... worker2 # [ 8.512948] systemd[1]: Reached target Local File Systems. worker2 # [ 8.517255] systemd[1]: Listening on Boot Loader Control Service Socket. worker2 # [ 8.527816] systemd[1]: Starting register-nix-paths.service... lb # [ 8.497686] systemd[1]: Mounted /run/wrappers. worker2 # [ 8.540223] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 8.498534] systemd[1]: Reached target Local File Systems. lb # [ 8.502542] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 8.548930] systemd[1]: Mounted /run/wrappers. lb # [ 8.507523] systemd[1]: Starting register-nix-paths.service... client # [ 8.550419] systemd[1]: Reached target Local File Systems. worker2 # [ 8.550879] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 8.554954] systemd[1]: Listening on Boot Loader Control Service Socket. client # [ 8.557038] systemd[1]: Starting register-nix-paths.service... lb # [ 8.514777] systemd[1]: Starting Create SUID/SGID Wrappers... lb # [ 8.515732] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. worker2 # [ 8.560462] systemd[1]: Starting Save Transient machine-id to Disk... worker2 # [ 8.561428] systemd[1]: Starting Create System Files and Directories... worker2 # [ 8.562389] systemd[1]: Started Rule-based Manager for Device Events and Files. lb # [ 8.520589] systemd[1]: Starting Save Transient machine-id to Disk... client # [ 8.563846] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 8.565307] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. client # [ 8.568454] systemd[1]: Starting Save Transient machine-id to Disk... lb # [ 8.534730] systemd[1]: Starting Create System Files and Directories... worker1 # [ 8.579022] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 8.584802] systemd[1]: Starting Create System Files and Directories... worker1 # [ 8.585365] systemd[1]: Finished Save Transient machine-id to Disk. lb # [ 8.544624] systemd-udevd[308]: Using default interface naming scheme 'v261'. client # [ 8.599095] systemd-udevd[302]: Using default interface naming scheme 'v261'. client # [ 8.646854] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. client # [ 8.656991] systemd[1]: Finished Save Transient machine-id to Disk. worker2 # [ 8.668632] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. worker2 # [ 8.680252] systemd[1]: Finished Save Transient machine-id to Disk. worker1 # [ 8.692542] systemd[1]: Finished Create System Files and Directories. worker1 # [ 8.698100] systemd[1]: Starting Rebuild Journal Catalog... worker1 # [ 8.704890] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 8.664422] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. lb # [ 8.683155] systemd[1]: Finished Save Transient machine-id to Disk. client # [ 8.745228] systemd[1]: Finished Create System Files and Directories. client # [ 8.746355] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 8.760127] systemd[1]: Starting Rebuild Journal Catalog... client # [ 8.763687] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker2 # [ 8.788219] systemd[1]: Finished Create System Files and Directories. worker2 # [ 8.793689] systemd[1]: Starting Rebuild Journal Catalog... worker2 # [ 8.802106] systemd[1]: Starting Record System Boot/Shutdown in UTMP... worker1 # [ 8.811472] systemd[1]: Finished Record System Boot/Shutdown in UTMP. lb # [ 8.783307] systemd[1]: Finished Create System Files and Directories. lb # [ 8.796164] systemd[1]: Starting Rebuild Journal Catalog... lb # [ 8.807325] systemd[1]: Starting Record System Boot/Shutdown in UTMP... lb # [ 8.814529] systemd[1]: Started Rule-based Manager for Device Events and Files. client # [ 8.872929] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker1 # [ 8.876301] systemd[1]: Finished Rebuild Journal Catalog. worker1 # [ 8.883503] systemd[1]: Starting Update is Completed... client # [ 8.914720] systemd[1]: Finished Rebuild Journal Catalog. client # [ 8.917605] systemd[1]: Starting Update is Completed... worker2 # [ 8.918037] systemd[1]: Finished Record System Boot/Shutdown in UTMP. client # [ 8.981741] systemd[1]: Finished Update is Completed. worker1 # [ 8.989835] systemd[1]: Finished Update is Completed. worker2 # [ 8.999692] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 8.960129] systemd[1]: Finished Record System Boot/Shutdown in UTMP. worker2 # [ 9.004276] systemd[1]: Starting Update is Completed... lb # [ 9.000291] systemd[1]: Finished Rebuild Journal Catalog. lb # [ 9.008908] systemd[1]: Starting Update is Completed... worker2 # [ 9.103199] systemd[1]: Finished Update is Completed. lb # [ 9.104439] systemd[1]: Finished Update is Completed. client # [ 9.203426] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs client # [ 9.233947] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 9.455395] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. client # [ 9.492642] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. client # [ 9.495533] systemd[1]: Finished Create SUID/SGID Wrappers. worker1 # [ 9.499351] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 9.499442] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker1 # [ 9.504821] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 9.505059] systemd[1]: Finished Create SUID/SGID Wrappers. client # [ 9.521978] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. lb # [ 9.529605] systemd[1]: Finished Coldplug All udev Devices. lb # [ 9.603031] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 9.671751] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse client # [ 9.750028] (udev-worker)[345]: Network interface NamePolicy= disabled on kernel command line. client # [ 9.754047] (udev-worker)[350]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. client # [ 9.761363] (udev-worker)[350]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 9.826957] systemd[1]: Finished Firewall. worker1 # [ 9.879462] systemd[1]: Finished Firewall. worker1 # [ 9.881506] systemd[1]: Finished Coldplug All udev Devices. lb # [ 9.893514] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. lb # [ 9.901618] systemd[1]: Finished Create SUID/SGID Wrappers. lb # [ 9.926035] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker1 # [ 9.971110] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 9.936478] systemd[1]: Finished register-nix-paths.service. lb # [ 9.941829] systemd[1]: Reached target System Initialization. lb # [ 9.942796] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 9.943813] systemd[1]: Started niks3 garbage collection timer. lb # [ 9.950658] systemd[1]: Started Daily Cleanup of Temporary Directories. lb # [ 9.951639] systemd[1]: Reached target Timer Units. lb # [ 9.955213] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 9.999722] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse lb # [ 9.961107] systemd[1]: Listening on niks3 server socket. lb # [ 9.961930] systemd[1]: Listening on Nix Daemon Socket. lb # [ 9.962685] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. lb # [ 9.963890] systemd[1]: Reached target Socket Units. lb # [ 9.971390] systemd[1]: Reached target Basic System. client # [ 10.015672] systemd[1]: Condition check resulted in Virtio network device being skipped. lb # [ 9.976395] systemd[1]: Starting Import lastlog data into lastlog2 database... client # [ 10.018851] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore lb # [ 9.977454] systemd[1]: Starting Name Service Cache Daemon (nsncd)... client # [ 10.024757] systemd[1]: Starting Create SUID/SGID Wrappers... client # [ 10.025709] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. lb # [ 9.984833] systemd[1]: Starting Post-Boot Actions... client # [ 10.026961] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 client # [ 10.035470] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore lb # [ 10.002367] systemd[1]: Started Reset console on configuration changes. client # [ 10.045409] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 10.046784] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os client # [ 10.049615] systemd[1]: Finished register-nix-paths.service. client # [ 10.069075] mousedev: PS/2 mouse device common for all mice lb # [ 10.020163] systemd[1]: Starting resolvconf update... worker2 # [ 10.081561] systemd[1]: Finished Coldplug All udev Devices. lb # [ 10.062587] systemd[1]: Started rustfs.service. worker2 # [ 10.129879] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs lb # [ 10.110371] systemd[1]: Starting rustfs-bucket.service... worker2 # [ 10.163535] systemd[1]: Load Kernel Module fuse skipped, unmet condition check ConditionKernelModuleLoaded=!fuse worker1 # [ 10.166994] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. lb # [ 10.130801] nsncd[449]: Sep 23 05:12:46.694 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 10.149499] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 10.150482] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. lb # [ 10.151607] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.200642] systemd[1]: Finished register-nix-paths.service. worker1 # [ 10.204869] systemd[1]: Reached target System Initialization. worker1 # [ 10.207459] systemd[1]: Started Discard unused filesystem blocks once a week. lb # [ 10.166027] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.212985] systemd[1]: Started Daily Cleanup of Temporary Directories. worker1 # [ 10.225160] systemd[1]: Reached target Timer Units. lb # [ 10.184540] systemd[1]: Started backdoor.service. worker1 # [ 10.227304] systemd[1]: Listening on D-Bus System Message Bus Socket. worker1 # [ 10.235145] systemd[1]: Listening on Nix Daemon Socket. worker1 # [ 10.243359] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 10.247650] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 10.263451] systemd[1]: Reached target Socket Units. worker2 # [ 10.268109] systemd[1]: Finished register-nix-paths.service. lb # [ 10.228456] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 10.272344] systemd[1]: Reached target Basic System. worker2 # [ 10.273635] systemd[1]: Reached target System Initialization. worker2 # [ 10.276378] systemd[1]: Started Discard unused filesystem blocks once a week. worker1 # [ 10.277195] systemd[1]: Starting Import lastlog data into lastlog2 database... worker2 # [ 10.278664] systemd[1]: Started Daily Cleanup of Temporary Directories. worker2 # [ 10.281474] systemd[1]: Reached target Timer Units. worker1 # [ 10.286295] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 10.289006] systemd[1]: Listening on D-Bus System Message Bus Socket. worker2 # [ 10.300222] systemd[1]: Listening on Nix Daemon Socket. lb # [ 10.258102] systemd[1]: Starting User Login Management... worker1 # [ 10.302488] systemd[1]: Starting Post-Boot Actions... worker1 # [ 10.313175] systemd[1]: Started Reset console on configuration changes. worker2 # [ 10.313393] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 10.323639] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. worker1 # [ 10.325467] systemd[1]: Starting resolvconf update... worker2 # [ 10.332355] systemd[1]: Reached target Socket Units. worker1 # [ 10.336497] systemd[1]: Starting D-Bus System Message Bus... lb # [ 10.294245] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.337898] systemd[1]: Reached target Basic System. worker2 # [ 10.344471] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 10.345509] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker1 # [ 10.357272] systemd[1]: Started backdoor.service. worker2 # [ 10.357403] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 10.370749] systemd[1]: Starting Post-Boot Actions... worker2 # [ 10.378229] systemd[1]: Started Reset console on configuration changes. worker2 # [ 10.389414] systemd[1]: Starting resolvconf update... worker1 # [ 10.392684] systemd[1]: Finished Post-Boot Actions. worker2 # [ 10.400348] systemd[1]: Starting D-Bus System Message Bus... worker1 # [ 10.419229] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # connecting to host... worker1 # [ 10.426796] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.431297] systemd[1]: Finished Post-Boot Actions. worker1 # [ 10.437526] nsncd[490]: Sep 23 05:12:46.977 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.457134] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.462068] systemd[1]: Starting User Login Management... worker2 # [ 10.468633] systemd[1]: Started Name Service Cache Daemon (nsncd). worker2 # [ 10.476407] nsncd[489]: Sep 23 05:12:47.032 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.490045] systemd[1]: Finished Import lastlog data into lastlog2 database. worker2 # [ 10.490438] systemd[1]: Reached target Host and Network Name Lookups. worker1 # connecting to host... worker2 # [ 10.500213] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 10.502011] dbus-broker-launch[494]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.503413] systemd[1]: Starting User Login Management... worker1 # [ 10.515120] dbus-broker-launch[494]: NSS returned no entry for 'systemd-timesync' worker1 # [ 10.522919] dbus-broker-launch[494]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" worker2 # [ 10.527581] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. worker2 # [ 10.535436] systemd[1]: Finished Import lastlog data into lastlog2 database. worker1 # [ 10.541179] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 10.549026] dbus-broker-launch[493]: Looking up NSS user entry for 'systemd-timesync'... client # [ 10.555801] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. worker2 # [ 10.556776] dbus-broker-launch[493]: NSS returned no entry for 'systemd-timesync' client # [ 10.559274] systemd[1]: Finished Create SUID/SGID Wrappers. worker2 # [ 10.562859] dbus-broker-launch[493]: Invalid user-name in /nix/store/flsxa7d757b7zpdiv9wsbs42iawcisak-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client # [ 10.565098] systemd[1]: Reached target System Initialization. client # [ 10.565982] systemd[1]: Started Discard unused filesystem blocks once a week. client # [ 10.567032] systemd[1]: Started Daily Cleanup of Temporary Directories. client # [ 10.567980] systemd[1]: Reached target Timer Units. client # [ 10.575459] systemd[1]: Listening on D-Bus System Message Bus Socket. lb # [ 10.527868] systemd[1]: Finished Import lastlog data into lastlog2 database. client # [ 10.579216] systemd[1]: Listening on Nix Daemon Socket. worker2 # [ 10.580078] systemd[1]: Started D-Bus System Message Bus. client # [ 10.585167] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. client # [ 10.589053] systemd[1]: Reached target Socket Units. client # [ 10.589826] systemd[1]: Reached target Basic System. client # [ 10.590563] systemd[1]: Started backdoor.service. client # [ 10.591257] systemd[1]: Starting Import lastlog data into lastlog2 database... worker1 # [ 10.601823] dbus-broker-launch[494]: Ready client # [ 10.607964] systemd[1]: Starting nftables firewall... lb # [ 10.572982] dbus-broker-launch[472]: Looking up NSS user entry for 'systemd-timesync'... worker2 # [ 10.625588] dbus-broker-launch[493]: Ready client # [ 10.629724] systemd[1]: Starting Name Service Cache Daemon (nsncd)... lb # [ 10.587057] dbus-broker-launch[472]: NSS returned no entry for 'systemd-timesync' lb # [ 10.592720] dbus-broker-launch[472]: Invalid user-name in /nix/store/4yiprls1sll47jid3xzq0m9nmi1qz2ww-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client # [ 10.645659] systemd[1]: Starting Post-Boot Actions... lb # [ 10.607142] (udev-worker)[377]: Network interface NamePolicy= disabled on kernel command line. worker1 # [ 10.655588] systemd-logind[503]: New seat seat0. worker2 # [ 10.656831] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. worker2 # [ 10.662899] systemd[1]: Started backdoor.service. worker1 # [ 10.667912] systemd[1]: Started User Login Management. client # [ 10.668622] systemd[1]: Started Reset console on configuration changes. worker1 # [ 10.672258] systemd[1]: Starting linger-users.service... lb # [ 10.635418] (udev-worker)[352]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker1 # [ 10.680455] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 10.694768] systemd[1]: Stopping Host and Network Name Lookups... worker1 # [ 10.695658] systemd[1]: Stopped target User and Group Name Lookups. worker2 # [ 10.696617] systemd-logind[501]: New seat seat0. lb # [ 10.662673] systemd[1]: Started D-Bus System Message Bus. worker2 # [ 10.706512] systemd[1]: Started User Login Management. worker1 # [ 10.707760] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 10.709305] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 10.710208] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 10.711007] systemd[1]: Stopped Name Service Cache Daemon (nsncd). lb # [ 10.668324] (udev-worker)[352]: Network interface NamePolicy= disabled on kernel command line. client # [ 10.712144] systemd[1]: Starting resolvconf update... worker1 # [ 10.711835] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 10.715878] systemd[1]: Starting linger-users.service... client # connecting to host... worker2 # [ 10.745656] systemd[1]: Stopped target Host and Network Name Lookups. worker1 # [ 10.747493] (udev-worker)[435]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 10.754357] systemd[1]: Stopping Host and Network Name Lookups... lb # [ 10.709402] systemd-logind[475]: New seat seat0. worker1 # [ 10.757677] (udev-worker)[435]: Network interface NamePolicy= disabled on kernel command line. lb # [ 10.716267] systemd[1]: Started User Login Management. worker2 # [ 10.765178] systemd[1]: Stopped target User and Group Name Lookups. worker1 # [ 10.772468] (udev-worker)[345]: Network interface NamePolicy= disabled on kernel command line. lb # [ 10.732220] systemd[1]: Starting linger-users.service... worker2 # [ 10.776844] systemd[1]: Stopping User and Group Name Lookups... worker2 # [ 10.782116] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker1 # [ 10.784178] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 10.787032] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 10.788339] systemd[1]: Finished linger-users.service. lb # [ 10.747362] dbus-broker-launch[472]: Ready worker1 # [ 10.797371] systemd[1]: Started Name Service Cache Daemon (nsncd). client: Guest shell says: b'Spawning backdoor root shell...\n' worker2 # [ 10.797954] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker2 # [ 10.801928] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # connecting to host... worker1 # [ 10.798663] nsncd[562]: Sep 23 05:12:47.353 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker1 # [ 10.808436] systemd[1]: Reached target Host and Network Name Lookups. worker1 # [ 10.809335] systemd[1]: Reached target User and Group Name Lookups. client # [ 10.829258] systemd[1]: Finished Post-Boot Actions. lb # [ 10.781945] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 10.833255] nsncd[454]: Sep 23 05:12:47.392 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" worker2 # [ 10.826009] systemd[1]: linger-users.service: Deactivated successfully. worker1 # [ 10.832199] systemd[1]: Finished resolvconf update. worker2 # [ 10.826966] systemd[1]: Finished linger-users.service. lb # [ 10.794321] systemd[1]: Stopping Host and Network Name Lookups... client # [ 10.835213] systemd[1]: Started Name Service Cache Daemon (nsncd). worker1 # [ 10.833012] systemd[1]: Reached target Preparation for Network. worker2 # [ 10.879648] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 10.795247] systemd[1]: Stopped target User and Group Name Lookups. client: connected to guest root shell worker2 # [ 10.882581] nsncd[559]: Sep 23 05:12:47.443 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 10.805407] systemd[1]: Stopping User and Group Name Lookups... client: (connecting took 11.32 seconds) worker2 # [ 10.889402] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 10.806240] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... client: (finished: waiting for the VM to finish booting, in 11.32 seconds) worker2 # [ 10.890321] systemd[1]: Reached target User and Group Name Lookups. lb # [ 10.807050] systemd[1]: nscd.service: Deactivated successfully. client # [ 10.848738] systemd[1]: Reached target Host and Network Name Lookups. worker2 # [ 10.911737] systemd[1]: Finished resolvconf update. lb # [ 10.820446] systemd[1]: Stopped Name Service Cache Daemon (nsncd). worker1 # [ 10.845133] systemd[1]: Starting DHCP Client... client # [ 10.849744] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 10.916241] systemd[1]: Reached target Preparation for Network. lb # [ 10.872782] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker1 # [ 10.845907] systemd[1]: Starting Extra networking commands.... client # [ 10.861061] systemd[1]: Starting D-Bus System Message Bus... worker2 # [ 10.922379] systemd[1]: Starting DHCP Client... client # [ 10.879546] systemd[1]: Starting User Login Management... worker2 # [ 10.923140] systemd[1]: Starting Extra networking commands.... client # [ 10.902043] systemd[1]: Finished Import lastlog data into lastlog2 database. lb # [ 10.883782] systemd[1]: linger-users.service: Deactivated successfully. lb # [ 10.892756] systemd[1]: Finished linger-users.service. worker2 # [ 10.981829] (udev-worker)[342]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. worker2 # [ 11.000128] (udev-worker)[342]: Network interface NamePolicy= disabled on kernel command line. worker2 # [ 11.015483] (udev-worker)[344]: Network interface NamePolicy= disabled on kernel command line. lb # [ 10.981853] systemd[1]: Finished resolvconf update. lb # [ 10.997759] nsncd[567]: Sep 23 05:12:47.562 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" lb # [ 11.007158] systemd[1]: Starting DHCP Client... lb # [ 11.013716] systemd[1]: Started Name Service Cache Daemon (nsncd). lb # [ 11.017142] systemd[1]: Reached target Host and Network Name Lookups. lb # [ 11.018279] systemd[1]: Reached target User and Group Name Lookups. worker1 # [ 11.076502] dhcpcd[598]: dhcpcd-10.3.2 starting worker1 # [ 11.087555] dhcpcd[638]: dev: loaded udev client # [ 11.094020] dbus-broker-launch[481]: Looking up NSS user entry for 'systemd-timesync'... worker1 # [ 11.136151] 8021q: 802.1Q VLAN Support v1.8 client # [ 11.136131] dbus-broker-launch[481]: NSS returned no entry for 'systemd-timesync' client # [ 11.137251] dbus-broker-launch[481]: Invalid user-name in /nix/store/6wvgzhp3xj1v1xybf1ixpbqi5gif5fbv-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" client # [ 11.155763] systemd-logind[483]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker2 # [ 11.159138] dhcpcd[596]: dhcpcd-10.3.2 starting client # [ 11.164380] systemd-logind[483]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.165580] systemd-logind[483]: New seat seat0. client # [ 11.166554] systemd[1]: Started User Login Management. worker2 # [ 11.173186] dhcpcd[636]: dev: loaded udev client # [ 11.181307] systemd[1]: Starting linger-users.service... worker1 # [ 11.198583] systemd[1]: Finished Extra networking commands.. worker2 # [ 11.218534] 8021q: 802.1Q VLAN Support v1.8 client # [ 11.208827] systemd[1]: Started D-Bus System Message Bus. worker1 # [ 11.208650] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 11.209856] systemd[1]: Reached target Network. worker1 # [ 11.220138] systemd[1]: Starting Address configuration of eth1... worker1 # [ 11.230656] systemd[1]: Starting Permit User Sessions... worker1 # [ 11.253881] cfg80211: Loading compiled-in X.509 certificates for regulatory database client # [ 11.258968] systemd[1]: Stopped target Host and Network Name Lookups. client # [ 11.265745] systemd[1]: Stopping Host and Network Name Lookups... client # [ 11.266795] systemd[1]: Stopped target User and Group Name Lookups. client # [ 11.267653] systemd[1]: Stopping User and Group Name Lookups... worker1 # [ 11.289479] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker1 # [ 11.292081] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 11.281482] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... worker2 # [ 11.281275] systemd[1]: Finished Extra networking commands.. client # [ 11.282449] systemd[1]: nscd.service: Deactivated successfully. worker1 # [ 11.295207] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker1 # [ 11.295563] cfg80211: failed to load regulatory.db worker2 # [ 11.294391] systemd[1]: Reached target Network. client # [ 11.296210] systemd[1]: Stopped Name Service Cache Daemon (nsncd). client # [ 11.297510] dbus-broker-launch[481]: Ready worker2 # [ 11.297795] systemd[1]: Starting Permit User Sessions... client # [ 11.313032] systemd[1]: linger-users.service: Deactivated successfully. worker2 # [ 11.325187] cfg80211: Loading compiled-in X.509 certificates for regulatory database client # [ 11.318178] systemd[1]: Finished linger-users.service. lb # [ 11.276268] systemd-logind[475]: Watching system buttons on /dev/input/event0 (gpio-keys) worker1 # [ 11.324438] systemd[1]: Finished Permit User Sessions. lb # [ 11.290651] systemd[1]: Condition check resulted in Virtio network device being skipped. worker1 # [ 11.334544] systemd[1]: Started Getty on tty1. worker1 # [ 11.335263] systemd[1]: Reached target Login Prompts. lb # [ 11.302343] dhcpcd[612]: dhcpcd-10.3.2 starting worker1 # [ 11.353078] systemd-logind[503]: Watching system buttons on /dev/input/event0 (gpio-keys) client # [ 11.354741] systemd[1]: Starting Name Service Cache Daemon (nsncd)... worker2 # [ 11.369039] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' worker2 # [ 11.369588] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' lb # [ 11.317363] dhcpcd[625]: dev: loaded udev worker2 # [ 11.372712] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 worker2 # [ 11.373065] cfg80211: failed to load regulatory.db worker1 # [ 11.389504] 8021q: adding VLAN 0 to HW filter on device eth1 worker2 # [ 11.380472] systemd[1]: Condition check resulted in Virtio network device being skipped. worker2 # [ 11.386569] systemd[1]: Starting Address configuration of eth1... worker2 # [ 11.397221] systemd[1]: Finished Permit User Sessions. worker2 # [ 11.398121] systemd[1]: Started Getty on tty1. worker1 # [ 11.398385] network-addresses-eth1-start[652]: adding address 192.168.1.3/24... done worker2 # [ 11.398767] systemd[1]: Reached target Login Prompts. worker1 # [ 11.418640] 8021q: adding VLAN 0 to HW filter on device eth0 worker1 # [ 11.409560] dhcpcd[638]: eth0: waiting for carrier lb # [ 11.381883] 8021q: 802.1Q VLAN Support v1.8 worker1 # [ 11.414467] dhcpcd[638]: eth0: waiting for carrier worker1 # [ 11.417822] dhcpcd[638]: eth0: carrier acquired client # [ 11.419388] systemd[1]: Finished resolvconf update. worker1 # [ 11.420886] network-addresses-eth1-start[652]: adding address 2001:db8:1::3/64... done client # [ 11.429438] systemd[1]: Starting DHCP Client... worker1 # [ 11.435925] dhcpcd[638]: DUID 00:01:00:01:32:46:1c:4f:52:54:00:12:34:56 worker1 # [ 11.439514] dhcpcd[638]: eth0: IAID 00:12:34:56 worker1 # [ 11.443554] dhcpcd[638]: eth0: adding address fe80::5054:ff:fe12:3456 worker1 # [ 11.445230] systemd[1]: Finished Address configuration of eth1. client # [ 11.446471] systemd[1]: Started Name Service Cache Daemon (nsncd). client # [ 11.451194] nsncd[554]: Sep 23 05:12:48.013 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" client # [ 11.459138] systemd[1]: Reached target Host and Network Name Lookups. client # [ 11.462764] systemd[1]: Reached target User and Group Name Lookups. worker2 # [ 11.486509] 8021q: adding VLAN 0 to HW filter on device eth0 worker2 # [ 11.478134] dhcpcd[636]: eth0: waiting for carrier worker2 # [ 11.483038] dhcpcd[636]: eth0: waiting for carrier worker2 # [ 11.487012] dhcpcd[636]: eth0: carrier acquired worker2 # [ 11.512841] 8021q: adding VLAN 0 to HW filter on device eth1 lb # [ 11.461238] systemd[1]: Finished Firewall. worker2 # [ 11.505552] dhcpcd[636]: DUID 00:01:00:01:32:46:1c:50:52:54:00:12:34:56 worker2 # [ 11.506537] dhcpcd[636]: eth0: IAID 00:12:34:56 worker2 # [ 11.507167] dhcpcd[636]: eth0: adding address fe80::5054:ff:fe12:3456 lb # [ 11.467160] systemd[1]: Reached target Preparation for Network. worker2 # [ 11.517993] network-addresses-eth1-start[658]: adding address 192.168.1.4/24... done lb # [ 11.480538] systemd[1]: Starting Address configuration of eth1... worker2 # [ 11.530407] network-addresses-eth1-start[658]: adding address 2001:db8:1::4/64... done lb # [ 11.491250] systemd[1]: Starting Extra networking commands.... worker2 # [ 11.548480] systemd[1]: Finished Address configuration of eth1. worker1 # [ 11.585145] mousedev: PS/2 mouse device common for all mice lb # [ 11.565636] cfg80211: Loading compiled-in X.509 certificates for regulatory database worker1 # [ 11.604221] systemd-logind[503]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) worker2 # [ 11.613959] systemd-logind[501]: Watching system buttons on /dev/input/event0 (gpio-keys) lb # [ 11.611200] mousedev: PS/2 mouse device common for all mice client # [ 11.650410] dhcpcd[580]: dhcpcd-10.3.2 starting lb # [ 11.629670] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' lb # [ 11.630163] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 11.659681] dhcpcd[587]: dev: loaded udev lb # [ 11.636190] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 lb # [ 11.636523] cfg80211: failed to load regulatory.db worker2 # [ 11.694901] mousedev: PS/2 mouse device common for all mice client # [ 11.717562] 8021q: 802.1Q VLAN Support v1.8 lb # [ 11.685750] 8021q: adding VLAN 0 to HW filter on device eth1 worker2 # [ 11.743226] systemd-logind[501]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 11.705500] network-addresses-eth1-start[638]: adding address 192.168.1.2/24... done lb # [ 11.735425] network-addresses-eth1-start[638]: adding address 2001:db8:1::2/64... done lb # [ 11.739941] dhcpcd[654]: /nix/store/1ki7bgdr0niilkg9jbkpcdaggbv0kb8a-openresolv-3.17.4/sbin/.resolvconf-wrapped: line 1250: kill: (643) - Operation not permitted lb # [ 11.753029] dhcpcd[654]: .resolvconf-wrapped: clearing stale lock pid 643 client # [ 11.838379] cfg80211: Loading compiled-in X.509 certificates for regulatory database lb # [ 11.783460] systemd[1]: Finished Address configuration of eth1. lb # [ 11.835225] 8021q: adding VLAN 0 to HW filter on device eth0 lb # [ 11.822293] dhcpcd[625]: eth0: waiting for carrier lb # [ 11.827849] dhcpcd[625]: libudev: received NULL device lb # [ 11.832462] dhcpcd[625]: libudev: received NULL device lb # [ 11.833991] dhcpcd[625]: eth0: carrier acquired lb # [ 11.852698] dhcpcd[625]: DUID 00:01:00:01:32:46:1c:50:52:54:00:12:34:56 lb # [ 11.853740] dhcpcd[625]: eth0: IAID 00:12:34:56 lb # [ 11.854369] dhcpcd[625]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 11.912832] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' client # [ 11.913348] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' client # [ 11.916575] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 client # [ 11.916919] cfg80211: failed to load regulatory.db client # [ 11.958704] systemd[1]: Finished nftables firewall. client # [ 11.964420] systemd[1]: Reached target Preparation for Network. client # [ 11.974305] systemd[1]: Starting Address configuration of eth1... client # [ 11.980647] systemd[1]: Starting Extra networking commands.... lb # [ 11.990523] systemd[1]: Finished Extra networking commands.. lb # [ 11.996960] systemd[1]: Reached target Network. client # [ 12.055611] 8021q: adding VLAN 0 to HW filter on device eth0 client # [ 12.042704] dhcpcd[587]: eth0: waiting for carrier client # [ 12.046884] dhcpcd[587]: eth0: carrier acquired lb # [ 12.008385] systemd[1]: Starting PostgreSQL Server... lb # [ 12.019109] systemd[1]: Starting Permit User Sessions... client # [ 12.067508] dhcpcd[587]: DUID 00:01:00:01:32:46:1c:50:52:54:00:12:34:56 client # [ 12.071410] dhcpcd[587]: eth0: IAID 00:12:34:56 client # [ 12.075134] dhcpcd[587]: eth0: adding address fe80::5054:ff:fe12:3456 client # [ 12.131322] 8021q: adding VLAN 0 to HW filter on device eth1 client # [ 12.148675] network-addresses-eth1-start[618]: adding address 192.168.1.1/24... done client # [ 12.172988] network-addresses-eth1-start[618]: adding address 2001:db8:1::1/64... done lb # [ 12.134537] systemd[1]: Finished Permit User Sessions. lb # [ 12.146245] systemd[1]: Started Getty on tty1. lb # [ 12.148927] systemd[1]: Reached target Login Prompts. client # [ 12.201413] systemd[1]: Finished Address configuration of eth1. lb # [ 12.223421] systemd-logind[475]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) lb # [ 12.227128] dhcpcd[625]: eth0: soliciting a DHCP lease lb # [ 12.232531] dhcpcd[625]: eth0: offered 10.0.2.15 from 10.0.2.2 lb # [ 12.242346] dhcpcd[625]: eth0: probing address 10.0.2.15/24 client # [ 12.305039] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker1 # [ 12.321878] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 client # [ 12.316239] systemd[1]: Finished Extra networking commands.. client # [ 12.318679] systemd[1]: Reached target Network. client # [ 12.325431] systemd[1]: Starting Permit User Sessions... client # [ 12.408433] systemd[1]: Finished Permit User Sessions. worker2 # [ 12.429277] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 lb # [ 12.388704] postgresql-pre-start[732]: The files belonging to this database system will be owned by user "postgres". lb # [ 12.390188] postgresql-pre-start[732]: This user must also own the server process. client # [ 12.434863] systemd[1]: Started Getty on tty1. client # [ 12.435644] systemd[1]: Reached target Login Prompts. client: (finished: waiting for unit network-addresses-eth1.service, in 12.82 seconds) client: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; } lb # [ 12.408130] postgresql-pre-start[732]: The database cluster will be initialized with locale "en_US.UTF-8". lb # [ 12.409476] postgresql-pre-start[732]: The default database encoding has accordingly been set to "UTF8". lb # [ 12.410666] postgresql-pre-start[732]: The default text search configuration will be set to "english". lb # [ 12.411852] postgresql-pre-start[732]: Data page checksums are enabled. lb # [ 12.418971] postgresql-pre-start[732]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok lb # [ 12.421892] postgresql-pre-start[732]: creating subdirectories ... ok lb # [ 12.422780] postgresql-pre-start[732]: selecting dynamic shared memory implementation ... posix client: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.1/ || { ip -4 addr >&2; false; }, in 0.10 seconds) lb: waiting for unit network-addresses-eth1.service lb: waiting for the VM to finish booting lb: Guest shell says: b'Spawning backdoor root shell...\n' lb: connected to guest root shell lb: (connecting took 0.00 seconds) lb: (finished: waiting for the VM to finish booting, in 0.00 seconds) worker1 # [ 12.552662] dhcpcd[638]: eth0: soliciting a DHCP lease worker1 # [ 12.560945] dhcpcd[638]: eth0: offered 10.0.2.15 from 10.0.2.2 worker1 # [ 12.568329] dhcpcd[638]: eth0: probing address 10.0.2.15/24 worker2 # [ 12.570408] dhcpcd[636]: eth0: soliciting a DHCP lease worker2 # [ 12.576598] dhcpcd[636]: eth0: offered 10.0.2.15 from 10.0.2.2 worker2 # [ 12.584280] dhcpcd[636]: eth0: probing address 10.0.2.15/24 lb # [ 12.564476] postgresql-pre-start[732]: selecting default "max_connections" ... 100 worker1 # [ 12.661400] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb: (finished: waiting for unit network-addresses-eth1.service, in 0.12 seconds) lb: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; } worker1 # [ 12.670777] systemd[1]: Starting Virtual Console Setup... worker1 # [ 12.694069] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker1 # [ 12.695231] systemd[1]: Stopped Virtual Console Setup. worker1 # [ 12.703047] systemd[1]: Starting Virtual Console Setup... worker1 # [ 12.762651] systemd-logind[503]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 12.766484] systemd[1]: Starting Virtual Console Setup... lb: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.2/ || { ip -4 addr >&2; false; }, in 0.10 seconds) worker1: waiting for unit network-addresses-eth1.service worker1: waiting for the VM to finish booting worker1: Guest shell says: b'Spawning backdoor root shell...\n' worker1: connected to guest root shell worker1: (connecting took 0.00 seconds) worker1: (finished: waiting for the VM to finish booting, in 0.00 seconds) client # [ 12.779948] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. client # [ 12.783819] systemd[1]: Stopped Virtual Console Setup. lb # [ 12.741426] postgresql-pre-start[732]: selecting default "shared_buffers" ... 128MB worker2 # [ 12.785958] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker2 # [ 12.795539] systemd[1]: Starting Virtual Console Setup... client # [ 12.796314] systemd[1]: Starting Virtual Console Setup... client # [ 12.822521] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. worker1: (finished: waiting for unit network-addresses-eth1.service, in 0.06 seconds) worker1: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; } worker2 # [ 12.824406] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. worker2 # [ 12.825585] systemd[1]: Stopped Virtual Console Setup. worker2 # [ 12.830577] systemd[1]: Starting Virtual Console Setup... worker1: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.3/ || { ip -4 addr >&2; false; }, in 0.04 seconds) worker2: waiting for unit network-addresses-eth1.service worker2: waiting for the VM to finish booting worker2: Guest shell says: b'Spawning backdoor root shell...\n' worker2: connected to guest root shell worker2: (connecting took 0.00 seconds) worker2: (finished: waiting for the VM to finish booting, in 0.00 seconds) client # [ 12.879216] systemd-logind[483]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker1 # [ 12.906190] systemd-vconsole-setup[694]: Configuration of first virtual console was skipped, ignoring remaining ones. worker1 # [ 12.911298] systemd[1]: Finished Virtual Console Setup. worker2 # [ 12.911371] systemd-logind[501]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) worker2: (finished: waiting for unit network-addresses-eth1.service, in 0.08 seconds) worker2: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; } client # [ 12.967565] systemd-vconsole-setup[698]: Configuration of first virtual console was skipped, ignoring remaining ones. client # [ 12.971718] systemd[1]: Finished Virtual Console Setup. worker2: (finished: must succeed: ip -4 addr show | grep -qF 192.168.1.4/ || { ip -4 addr >&2; false; }, in 0.04 seconds) lb: waiting for unit niks3.service worker2 # [ 13.036281] systemd-vconsole-setup[694]: Configuration of first virtual console was skipped, ignoring remaining ones. worker2 # [ 13.041631] systemd[1]: Finished Virtual Console Setup. client # [ 13.187317] dhcpcd[587]: eth0: soliciting a DHCP lease client # [ 13.193384] dhcpcd[587]: eth0: offered 10.0.2.15 from 10.0.2.2 client # [ 13.200530] dhcpcd[587]: eth0: probing address 10.0.2.15/24 lb # [ 13.377553] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:06.0/virtio5/input/input3 worker2 # [ 13.658979] dhcpcd[636]: eth0: soliciting an IPv6 router worker2 # [ 13.662718] dhcpcd[636]: eth0: Router Advertisement from fe80::2 worker2 # [ 13.665818] dhcpcd[636]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker2 # [ 13.668990] dhcpcd[636]: eth0: adding route to fec0::/64 worker2 # [ 13.671404] dhcpcd[636]: eth0: adding default route via fe80::2 lb # [ 13.742347] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. lb # [ 13.751133] systemd[1]: Starting Virtual Console Setup... worker1 # [ 13.811889] dhcpcd[638]: eth0: soliciting an IPv6 router worker1 # [ 13.815485] dhcpcd[638]: eth0: Router Advertisement from fe80::2 worker1 # [ 13.818496] dhcpcd[638]: eth0: adding address fec0::5054:ff:fe12:3456/64 worker1 # [ 13.821582] dhcpcd[638]: eth0: adding route to fec0::/64 worker1 # [ 13.823886] dhcpcd[638]: eth0: adding default route via fe80::2 lb # [ 13.792934] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. lb # [ 13.797132] systemd[1]: Stopped Virtual Console Setup. lb # [ 13.809898] systemd[1]: Starting Virtual Console Setup... lb # [ 13.871436] systemd-logind[475]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) client # [ 13.922781] dhcpcd[587]: eth0: soliciting an IPv6 router client # [ 13.926188] dhcpcd[587]: eth0: Router Advertisement from fe80::2 client # [ 13.929613] dhcpcd[587]: eth0: adding address fec0::5054:ff:fe12:3456/64 client # [ 13.932541] dhcpcd[587]: eth0: adding route to fec0::/64 client # [ 13.934835] dhcpcd[587]: eth0: adding default route via fe80::2 lb # [ 14.065177] systemd-vconsole-setup[763]: Configuration of first virtual console was skipped, ignoring remaining ones. lb # [ 14.069373] systemd[1]: Finished Virtual Console Setup. lb # [ 14.362422] dhcpcd[625]: eth0: soliciting an IPv6 router lb # [ 14.363303] dhcpcd[625]: eth0: Router Advertisement from fe80::2 lb # [ 14.368123] dhcpcd[625]: eth0: adding address fec0::5054:ff:fe12:3456/64 lb # [ 14.369033] dhcpcd[625]: eth0: adding route to fec0::/64 lb # [ 14.369751] dhcpcd[625]: eth0: adding default route via fe80::2 lb # [ 14.465354] postgresql-pre-start[732]: selecting default time zone ... UTC lb # [ 14.468284] postgresql-pre-start[732]: creating configuration files ... ok lb # [ 14.695445] postgresql-pre-start[732]: running bootstrap script ... ok lb # [ 15.317688] postgresql-pre-start[732]: performing post-bootstrap initialization ... ok lb # [ 15.532745] postgresql-pre-start[732]: syncing data to disk ... ok lb # [ 15.533696] postgresql-pre-start[732]: initdb: warning: enabling "trust" authentication for local connections lb # [ 15.534915] postgresql-pre-start[732]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. lb # [ 15.537132] postgresql-pre-start[732]: Success. You can now start the database server using: lb # [ 15.538281] postgresql-pre-start[732]: pg_ctl -D /var/lib/postgresql/18 -l logfile start lb # [ 15.637728] postgres[792]: [792] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit lb # [ 15.640943] postgres[792]: [792] LOG: listening on IPv6 address "::1", port 5432 lb # [ 15.642181] postgres[792]: [792] LOG: listening on IPv4 address "127.0.0.1", port 5432 lb # [ 15.644407] postgres[792]: [792] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" lb # [ 15.658068] postgres[801]: [801] LOG: database system was shut down at 2026-09-23 05:12:51 GMT lb # [ 15.663001] postgres[792]: [792] LOG: database system is ready to accept connections lb # [ 15.669026] systemd[1]: Started PostgreSQL Server. lb # [ 15.676785] systemd[1]: Starting PostgreSQL Setup Scripts... lb # [ 15.872433] postgresql-setup-start[812]: CREATE DATABASE lb # [ 15.916232] postgresql-setup-start[817]: CREATE ROLE lb # [ 15.936499] postgresql-setup-start[819]: ALTER DATABASE lb # [ 15.942558] systemd[1]: Finished PostgreSQL Setup Scripts. lb # [ 15.944708] systemd[1]: Reached target PostgreSQL. worker1 # [ 17.373911] dhcpcd[638]: eth0: leased 10.0.2.15 for 86400 seconds worker1 # [ 17.377392] dhcpcd[638]: eth0: adding route to 10.0.2.0/24 worker1 # [ 17.379805] dhcpcd[638]: eth0: adding default route via 10.0.2.2 worker1 # [ 17.527766] systemd[1]: Started DHCP Client. worker1 # [ 17.530044] systemd[1]: Reached target Multi-User System. worker1 # [ 17.531838] systemd[1]: Startup finished in 1.015s (kernel) + 4.893s (initrd) + 11.622s (userspace) = 17.531s. lb # [ 17.533230] dhcpcd[625]: eth0: leased 10.0.2.15 for 86400 seconds lb # [ 17.538257] dhcpcd[625]: eth0: adding route to 10.0.2.0/24 lb # [ 17.546416] dhcpcd[625]: eth0: adding default route via 10.0.2.2 worker2 # [ 17.589597] dhcpcd[636]: eth0: leased 10.0.2.15 for 86400 seconds worker2 # [ 17.593804] dhcpcd[636]: eth0: adding route to 10.0.2.0/24 worker2 # [ 17.599664] dhcpcd[636]: eth0: adding default route via 10.0.2.2 lb # [ 17.702895] systemd[1]: Started DHCP Client. lb # [ 17.706098] systemd[1]: Reached target Network is Online. worker2 # [ 17.750604] systemd[1]: Started DHCP Client. lb # [ 17.710896] systemd[1]: Started Envoy reverse proxy. worker2 # [ 17.753284] systemd[1]: Reached target Multi-User System. worker2 # [ 17.754214] systemd[1]: Startup finished in 1.005s (kernel) + 4.915s (initrd) + 11.831s (userspace) = 17.752s. lb # [ 17.716766] systemd[1]: Started mock-oidc.service. client # [ 17.930592] dhcpcd[587]: eth0: leased 10.0.2.15 for 86400 seconds client # [ 17.934913] dhcpcd[587]: eth0: adding route to 10.0.2.0/24 client # [ 17.940424] dhcpcd[587]: eth0: adding default route via 10.0.2.2 lb # [ 18.041322] mock-oidc-server[901]: Mock OIDC Server running client # [ 18.083528] systemd[1]: Started DHCP Client. client # [ 18.085961] systemd[1]: Reached target Multi-User System. lb # [ 18.042919] mock-oidc-server[901]: OIDC Address: 192.168.1.2:8080 lb # [ 18.045309] mock-oidc-server[901]: Issue Address: 0.0.0.0:8081 client # [ 18.087283] systemd[1]: Startup finished in 1.004s (kernel) + 4.977s (initrd) + 12.104s (userspace) = 18.087s. lb # [ 18.046479] mock-oidc-server[901]: Issuer: http://192.168.1.2:8080/oidc lb # [ 18.047702] mock-oidc-server[901]: JWKS: http://192.168.1.2:8080/oidc/.well-known/jwks.json lb # [ 18.049755] mock-oidc-server[901]: Discovery: http://192.168.1.2:8080/oidc/.well-known/openid-configuration lb # [ 18.051351] mock-oidc-server[901]: Issue tokens: http://0.0.0.0:8081/issue?sub=... lb # [ 18.141414] envoy[900]: [2026-09-23 05:12:54.708][900][info][main] [source/server/server.cc:467] initializing epoch 0 (base id=0, hot restart version=11.120) lb # [ 18.146803] envoy[900]: [2026-09-23 05:12:54.710][900][info][main] [source/server/server.cc:469] statically linked extensions: lb # [ 18.152237] envoy[900]: [2026-09-23 05:12:54.710][900][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file lb # [ 18.154188] envoy[900]: [2026-09-23 05:12:54.710][900][info][main] [source/server/server.cc:471] envoy.config.validators: envoy.config.validators.minimum_clusters, envoy.config.validators.minimum_clusters_validator lb # [ 18.159777] envoy[900]: [2026-09-23 05:12:54.710][900][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest lb # [ 18.166857] envoy[900]: [2026-09-23 05:12:54.710][900][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector lb # [ 18.200337] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name lb # [ 18.202880] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 18.213471] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override lb # [ 18.222889] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.http.forward_client_cert_details: forward_client_cert lb # [ 18.225612] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.compression.decompressor: envoy.compression.brotli.decompressor, envoy.compression.gzip.decompressor, envoy.compression.zstd.decompressor lb # [ 18.229525] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle lb # [ 18.231596] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.quic.connection_id_generator: envoy.quic.connection_id_generator.quic_lb, envoy.quic.deterministic_connection_id_generator lb # [ 18.234829] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth2 lb # [ 18.238015] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated lb # [ 18.239840] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule lb # [ 18.243012] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind lb # [ 18.244989] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm lb # [ 18.252449] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router lb # [ 18.254209] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.upstream_options: envoy.extensions.upstreams.http.reverse_tunnel.v3.ReverseTunnelUpstreamCodecOptions, envoy.extensions.upstreams.http.v3.HttpProtocolOptions, envoy.extensions.upstreams.tcp.v3.TcpProtocolOptions, envoy.upstreams.http.http_protocol_options, envoy.upstreams.tcp.tcp_protocol_options lb # [ 18.261036] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override lb # [ 18.263945] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input lb # [ 18.266930] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap lb # [ 18.272257] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.matching.transport_socket.input: envoy.matching.inputs.endpoint_metadata, envoy.matching.inputs.locality_metadata, envoy.matching.inputs.transport_socket_filter_state lb # [ 18.275098] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso lb # [ 18.277884] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher lb # [ 18.279776] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource lb # [ 18.283882] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter lb # [ 18.286236] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default lb # [ 18.287872] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.thrift_proxy.protocols: auto, binary, binary/non-strict, compact, twitter lb # [ 18.290046] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re2 lb # [ 18.291720] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts lb # [ 18.295297] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.internal_redirect_predicates: envoy.internal_redirect_predicates.allow_listed_routes, envoy.internal_redirect_predicates.previous_routes, envoy.internal_redirect_predicates.safe_cross_scheme lb # [ 18.298925] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.matching.input_matchers: envoy.matching.input_matchers.generic_request_matcher, envoy.matching.matchers.cel_matcher, envoy.matching.matchers.consistent_hashing, envoy.matching.matchers.dynamic_modules, envoy.matching.matchers.ip, envoy.matching.matchers.metadata_matcher, envoy.matching.matchers.runtime_fraction lb # [ 18.303866] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.health_checkers: envoy.health_checkers.dynamic_modules, envoy.health_checkers.grpc, envoy.health_checkers.http, envoy.health_checkers.redis, envoy.health_checkers.tcp, envoy.health_checkers.thrift lb # [ 18.307838] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation lb # [ 18.310305] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.retry_priorities: envoy.retry_priorities.previous_priorities lb # [ 18.312442] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.access_loggers: envoy.access_loggers.dynamic_modules, envoy.access_loggers.file, envoy.access_loggers.fluentd, envoy.access_loggers.http_grpc, envoy.access_loggers.open_telemetry, envoy.access_loggers.stats, envoy.access_loggers.stderr, envoy.access_loggers.stdout, envoy.access_loggers.tcp_grpc, envoy.access_loggers.wasm, envoy.file_access_log, envoy.fluentd_access_log, envoy.http_grpc_access_log, envoy.open_telemetry_access_log, envoy.stderr_access_log, envoy.stdout_access_log, envoy.tcp_grpc_access_log, envoy.wasm_access_log lb # [ 18.320584] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable lb # [ 18.324659] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default lb # [ 18.326533] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.srds_factory: envoy.srds_factory.default lb # [ 18.328409] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.access_loggers.extension_filters: envoy.access_loggers.extension_filters.cel, envoy.access_loggers.extension_filters.process_ratelimit lb # [ 18.331047] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router worker2 # [ 18.412218] systemd[1]: Reached target Network is Online. lb # [ 18.366119] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port lb # [ 18.376259] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr worker1 # [ 18.418459] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 18.378093] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy lb # [ 18.382566] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche worker2 # [ 18.425564] systemd[1]: Starting scheduler-token.service... lb # [ 18.385025] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header lb # [ 18.387611] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed lb # [ 18.390202] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple lb # [ 18.392838] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory lb # [ 18.395722] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default lb # [ 18.397900] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] network.connection.client: default, envoy_internal lb # [ 18.399586] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo lb # [ 18.401460] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.upstreams: envoy.filters.connection_pools.tcp.generic lb # [ 18.403202] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value lb # [ 18.406867] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.network.dns_resolver: envoy.network.dns_resolver.cares, envoy.network.dns_resolver.getaddrinfo, envoy.network.dns_resolver.hickory lb # [ 18.409673] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls lb # [ 18.415870] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua lb # [ 18.417817] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 18.420466] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params lb # [ 18.431291] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input lb # [ 18.433489] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata lb # [ 18.435929] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed lb # [ 18.437936] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin lb # [ 18.441534] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats lb # [ 18.443984] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection lb # [ 18.449389] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service lb # [ 18.453871] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality lb # [ 18.461341] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.common.key_value: envoy.key_value.file_based lb # [ 18.463016] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid lb # [ 18.464739] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.generic_proxy.codecs: envoy.generic_proxy.codecs.dubbo, envoy.generic_proxy.codecs.http1 lb # [ 18.467030] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response lb # [ 18.469305] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default lb # [ 18.471132] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v8 lb # [ 18.473092] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.quic.proof_source: envoy.quic.proof_source.filter_chain lb # [ 18.474845] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe lb # [ 18.477952] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.thrift_proxy.filters: envoy.filters.thrift.header_to_metadata, envoy.filters.thrift.payload_to_metadata, envoy.filters.thrift.rate_limit, envoy.filters.thrift.router lb # [ 18.481783] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router lb # [ 18.486509] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor lb # [ 18.489521] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.content_parsers: envoy.content_parsers.json lb # [ 18.491740] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.stats_sinks: envoy.dog_statsd, envoy.graphite_statsd, envoy.metrics_service, envoy.open_telemetry_stat_sink, envoy.stat_sinks.dog_statsd, envoy.stat_sinks.dynamic_modules, envoy.stat_sinks.graphite_statsd, envoy.stat_sinks.hystrix, envoy.stat_sinks.metrics_service, envoy.stat_sinks.open_telemetry, envoy.stat_sinks.statsd, envoy.stat_sinks.wasm, envoy.statsd worker2 # [ 18.542519] systemd[1]: scheduler-token.service: Deactivated successfully. lb # [ 18.497174] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.filters.network: envoy.echo, envoy.ext_authz, envoy.filters.network.connection_limit, envoy.filters.network.direct_response, envoy.filters.network.dubbo_proxy, envoy.filters.network.dynamic_modules, envoy.filters.network.echo, envoy.filters.network.ext_authz, envoy.filters.network.ext_proc, envoy.filters.network.generic_proxy, envoy.filters.network.geoip, envoy.filters.network.http_connection_manager, envoy.filters.network.local_ratelimit, envoy.filters.network.mongo_proxy, envoy.filters.network.ratelimit, envoy.filters.network.rbac, envoy.filters.network.redis_proxy, envoy.filters.network.reverse_tunnel, envoy.filters.network.reverse_tunnel_drain_aware_http_connection_manager, envoy.filters.network.set_filter_state, envoy.filters.network.sni_cluster, envoy.filters.network.sni_dynamic_forward_proxy, envoy.filters.network.tcp_bandwidth_limit, envoy.filters.network.tcp_proxy, envoy.filters.network.thrift_proxy, envoy.filters.network.wasm, envoy.filters.network.zookeeper_proxy, envoy.http_connection_manager, envoy.mongo_proxy, envoy.ratelimit, envoy.redis_proxy, envoy.tcp_proxy worker2 # [ 18.552525] systemd[1]: Finished scheduler-token.service. lb # [ 18.510746] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.rbac.matchers: envoy.rbac.matchers.upstream_ip_port lb # [ 18.512617] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian2 lb # [ 18.514236] envoy[900]: [2026-09-23 05:12:54.711][900][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation worker2 # [ 18.554411] systemd[1]: scheduler-token.service: Consumed 75ms CPU time over 122ms wall clock time, 3.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb # [ 18.516583] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple lb # [ 18.518786] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default worker2 # [ 18.562027] systemd[1]: Starting Nix worker-protocol over gRPC... lb # [ 18.520878] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default lb # [ 18.522789] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher lb # [ 18.525425] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san lb # [ 18.532786] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.tls.certificate_mappers: envoy.tls.certificate_mappers.sni, envoy.tls.certificate_mappers.static_name lb # [ 18.535006] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default lb # [ 18.536845] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default lb # [ 18.538628] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.guarddog_actions: envoy.watchdog.abort_action, envoy.watchdog.profile_action lb # [ 18.547635] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection lb # [ 18.549619] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder lb # [ 18.551831] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config lb # [ 18.555621] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.tls.certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret lb # [ 18.557756] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.formatter: envoy.formatter.cel, envoy.formatter.dynamic_modules, envoy.formatter.file_content, envoy.formatter.generic_secret, envoy.formatter.metadata, envoy.formatter.req_without_query lb # [ 18.560897] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher lb # [ 18.563282] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.filters.udp_listener: envoy.filters.udp.dns_filter, envoy.filters.udp_listener.dynamic_modules, envoy.filters.udp_listener.udp_proxy lb # [ 18.566075] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff lb # [ 18.569817] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based lb # [ 18.574024] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case lb # [ 18.576363] envoy[900]: [2026-09-23 05:12:54.712][900][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls lb # [ 18.580889] envoy[900]: [2026-09-23 05:12:54.726][900][info][main] [source/server/server.cc:531] HTTP header map info: worker1 # [ 18.624627] nix-grpc-daemon[768]: ts=2026-09-23T05:12:55Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 18.634078] nix-grpc-daemon[768]: ts=2026-09-23T05:12:55Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 18.636258] nix-grpc-daemon[768]: ts=2026-09-23T05:12:55Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 lb # [ 18.582219] envoy[900]: [2026-09-23 05:12:54.726][900][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id lb # [ 18.597832] envoy[900]: [2026-09-23 05:12:54.726][900][info][main] [source/server/server.cc:534] request trailer map: 120 bytes: worker1 # [ 18.648067] nix-grpc-daemon[768]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 18.599311] envoy[900]: [2026-09-23 05:12:54.726][900][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id lb # [ 18.608318] envoy[900]: [2026-09-23 05:12:54.726][900][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status lb # [ 18.610044] envoy[900]: [2026-09-23 05:12:54.787][900][info][main] [source/server/server.cc:968] runtime: {} worker1 # [ 18.651130] nix-grpc-daemon[768]: E0000 00:00:1790140375.204014 768 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb # [ 18.611287] envoy[900]: [2026-09-23 05:12:54.789][900][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288. lb # [ 18.613000] envoy[900]: [2026-09-23 05:12:54.789][900][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:9901 lb # [ 18.614576] envoy[900]: [2026-09-23 05:12:54.789][900][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:9901 worker1 # [ 18.656991] nix-grpc-daemon[768]: E0000 00:00:1790140375.212913 768 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 18.616099] envoy[900]: [2026-09-23 05:12:54.790][900][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration lb # [ 18.617684] envoy[900]: [2026-09-23 05:12:54.790][900][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s) lb # [ 18.619203] envoy[900]: [2026-09-23 05:12:54.790][900][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s) lb # [ 18.620742] envoy[900]: [2026-09-23 05:12:54.932][900][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s) lb # [ 18.622217] envoy[900]: [2026-09-23 05:12:54.948][900][info][config] [source/server/configuration_impl.cc:164] loading stats configuration lb # [ 18.623768] envoy[900]: [2026-09-23 05:12:54.949][900][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. lb # [ 18.627015] envoy[900]: [2026-09-23 05:12:54.949][900][info][main] [source/server/server.cc:1077] starting main dispatch loop worker1 # [ 18.679765] nix-grpc-daemon[768]: ts=2026-09-23T05:12:55Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 18.684316] systemd[1]: Started Nix worker-protocol over gRPC. lb # [ 18.645897] envoy[900]: [2026-09-23 05:12:55.213][900][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization lb # [ 18.647663] envoy[900]: [2026-09-23 05:12:55.213][900][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized lb # [ 18.650021] envoy[900]: [2026-09-23 05:12:55.213][900][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager lb # [ 18.655597] envoy[900]: [2026-09-23 05:12:55.220][900][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers worker1 # [ 18.689835] nix-grpc-daemon[768]: E0000 00:00:1790140375.244646 811 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 18.708466] nix-grpc-daemon[768]: E0000 00:00:1790140375.264196 811 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 18.712067] nix-grpc-daemon[768]: E0000 00:00:1790140375.267280 811 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.713839] nix-grpc-daemon[771]: ts=2026-09-23T05:12:55Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 18.723364] nix-grpc-daemon[771]: ts=2026-09-23T05:12:55Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 18.725948] nix-grpc-daemon[771]: ts=2026-09-23T05:12:55Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 18.731280] nix-grpc-daemon[768]: E0000 00:00:1790140375.286223 811 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.737361] nix-grpc-daemon[771]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 18.740307] nix-grpc-daemon[768]: ts=2026-09-23T05:12:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.740164] nix-grpc-daemon[771]: E0000 00:00:1790140375.300906 771 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 18.742606] nix-grpc-daemon[768]: ts=2026-09-23T05:12:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 18.746904] nix-grpc-daemon[771]: E0000 00:00:1790140375.310615 771 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.775434] nix-grpc-daemon[771]: ts=2026-09-23T05:12:55Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 18.780594] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 18.781469] nix-grpc-daemon[771]: E0000 00:00:1790140375.338024 813 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.797143] nix-grpc-daemon[771]: E0000 00:00:1790140375.360899 813 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.817865] nix-grpc-daemon[771]: E0000 00:00:1790140375.361485 813 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 18.823787] nix-grpc-daemon[771]: E0000 00:00:1790140375.377651 813 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 18.829656] nix-grpc-daemon[771]: ts=2026-09-23T05:12:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 18.831276] nix-grpc-daemon[771]: ts=2026-09-23T05:12:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 18.943913] nix-grpc-daemon[768]: ts=2026-09-23T05:12:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 18.949482] nix-grpc-daemon[768]: ts=2026-09-23T05:12:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.031103] nix-grpc-daemon[771]: ts=2026-09-23T05:12:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.036981] nix-grpc-daemon[771]: ts=2026-09-23T05:12:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.150154] nix-grpc-daemon[768]: ts=2026-09-23T05:12:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.155668] nix-grpc-daemon[768]: ts=2026-09-23T05:12:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.238393] nix-grpc-daemon[771]: ts=2026-09-23T05:12:55Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.243943] nix-grpc-daemon[771]: ts=2026-09-23T05:12:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.356759] nix-grpc-daemon[768]: ts=2026-09-23T05:12:55Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.362197] nix-grpc-daemon[768]: ts=2026-09-23T05:12:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.445540] nix-grpc-daemon[771]: ts=2026-09-23T05:12:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.451005] nix-grpc-daemon[771]: ts=2026-09-23T05:12:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.563276] nix-grpc-daemon[768]: ts=2026-09-23T05:12:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.569546] nix-grpc-daemon[768]: ts=2026-09-23T05:12:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.652507] nix-grpc-daemon[771]: ts=2026-09-23T05:12:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.657971] nix-grpc-daemon[771]: ts=2026-09-23T05:12:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.772553] nix-grpc-daemon[768]: ts=2026-09-23T05:12:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.777059] nix-grpc-daemon[768]: ts=2026-09-23T05:12:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 19.861248] nix-grpc-daemon[771]: ts=2026-09-23T05:12:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 19.865559] nix-grpc-daemon[771]: ts=2026-09-23T05:12:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 19.973769] nix-grpc-daemon[768]: ts=2026-09-23T05:12:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 19.979487] nix-grpc-daemon[768]: ts=2026-09-23T05:12:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.062350] nix-grpc-daemon[771]: ts=2026-09-23T05:12:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.068352] nix-grpc-daemon[771]: ts=2026-09-23T05:12:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.180574] nix-grpc-daemon[768]: ts=2026-09-23T05:12:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.186109] nix-grpc-daemon[768]: ts=2026-09-23T05:12:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.269964] nix-grpc-daemon[771]: ts=2026-09-23T05:12:56Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.276373] nix-grpc-daemon[771]: ts=2026-09-23T05:12:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.387074] nix-grpc-daemon[768]: ts=2026-09-23T05:12:56Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.392879] nix-grpc-daemon[768]: ts=2026-09-23T05:12:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.478266] nix-grpc-daemon[771]: ts=2026-09-23T05:12:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.484392] nix-grpc-daemon[771]: ts=2026-09-23T05:12:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.593885] nix-grpc-daemon[768]: ts=2026-09-23T05:12:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.599520] nix-grpc-daemon[768]: ts=2026-09-23T05:12:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.686223] nix-grpc-daemon[771]: ts=2026-09-23T05:12:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.691671] nix-grpc-daemon[771]: ts=2026-09-23T05:12:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 20.802577] nix-grpc-daemon[768]: ts=2026-09-23T05:12:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 20.806854] nix-grpc-daemon[768]: ts=2026-09-23T05:12:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 20.894266] nix-grpc-daemon[771]: ts=2026-09-23T05:12:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 20.899914] nix-grpc-daemon[771]: ts=2026-09-23T05:12:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.003124] nix-grpc-daemon[768]: ts=2026-09-23T05:12:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.008921] nix-grpc-daemon[768]: ts=2026-09-23T05:12:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.101380] nix-grpc-daemon[771]: ts=2026-09-23T05:12:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.107078] nix-grpc-daemon[771]: ts=2026-09-23T05:12:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.210089] nix-grpc-daemon[768]: ts=2026-09-23T05:12:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.215655] nix-grpc-daemon[768]: ts=2026-09-23T05:12:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.308254] nix-grpc-daemon[771]: ts=2026-09-23T05:12:57Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.312264] nix-grpc-daemon[771]: ts=2026-09-23T05:12:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.416611] nix-grpc-daemon[768]: ts=2026-09-23T05:12:57Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.422060] nix-grpc-daemon[768]: ts=2026-09-23T05:12:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.513806] nix-grpc-daemon[771]: ts=2026-09-23T05:12:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.519602] nix-grpc-daemon[771]: ts=2026-09-23T05:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.623146] nix-grpc-daemon[768]: ts=2026-09-23T05:12:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.629311] nix-grpc-daemon[768]: ts=2026-09-23T05:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.721122] nix-grpc-daemon[771]: ts=2026-09-23T05:12:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.726720] nix-grpc-daemon[771]: ts=2026-09-23T05:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 21.833063] nix-grpc-daemon[768]: ts=2026-09-23T05:12:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 21.837408] nix-grpc-daemon[768]: ts=2026-09-23T05:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 21.929848] nix-grpc-daemon[771]: ts=2026-09-23T05:12:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 21.934079] nix-grpc-daemon[771]: ts=2026-09-23T05:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.033127] nix-grpc-daemon[768]: ts=2026-09-23T05:12:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.038590] nix-grpc-daemon[768]: ts=2026-09-23T05:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.130712] nix-grpc-daemon[771]: ts=2026-09-23T05:12:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.136092] nix-grpc-daemon[771]: ts=2026-09-23T05:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.239683] nix-grpc-daemon[768]: ts=2026-09-23T05:12:58Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.245464] nix-grpc-daemon[768]: ts=2026-09-23T05:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.337339] nix-grpc-daemon[771]: ts=2026-09-23T05:12:58Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.342611] nix-grpc-daemon[771]: ts=2026-09-23T05:12:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.446856] nix-grpc-daemon[768]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.452613] nix-grpc-daemon[768]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.543885] nix-grpc-daemon[771]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.549331] nix-grpc-daemon[771]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.653528] nix-grpc-daemon[768]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.659132] nix-grpc-daemon[768]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.750488] nix-grpc-daemon[771]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.756798] nix-grpc-daemon[771]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 22.862652] nix-grpc-daemon[768]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 22.867030] nix-grpc-daemon[768]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 22.961809] nix-grpc-daemon[771]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 22.964113] nix-grpc-daemon[771]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.063458] nix-grpc-daemon[768]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.069507] nix-grpc-daemon[768]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.163113] nix-grpc-daemon[771]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.168659] nix-grpc-daemon[771]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.270553] nix-grpc-daemon[768]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.276364] nix-grpc-daemon[768]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.369960] nix-grpc-daemon[771]: ts=2026-09-23T05:12:59Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.375311] nix-grpc-daemon[771]: ts=2026-09-23T05:12:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.477372] nix-grpc-daemon[768]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.482962] nix-grpc-daemon[768]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.576686] nix-grpc-daemon[771]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.582015] nix-grpc-daemon[771]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.684139] nix-grpc-daemon[768]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.689978] nix-grpc-daemon[768]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.783062] nix-grpc-daemon[771]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.788551] nix-grpc-daemon[771]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 23.893075] nix-grpc-daemon[768]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 23.897353] nix-grpc-daemon[768]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 23.991892] nix-grpc-daemon[771]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 23.996300] nix-grpc-daemon[771]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.093790] nix-grpc-daemon[768]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.099474] nix-grpc-daemon[768]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.193248] nix-grpc-daemon[771]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.198644] nix-grpc-daemon[771]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.300777] nix-grpc-daemon[768]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.306096] nix-grpc-daemon[768]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.400276] nix-grpc-daemon[771]: ts=2026-09-23T05:13:00Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.406201] nix-grpc-daemon[771]: ts=2026-09-23T05:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.507023] nix-grpc-daemon[768]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.512479] nix-grpc-daemon[768]: ts=2026-09-23T05:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.607249] nix-grpc-daemon[771]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.615493] nix-grpc-daemon[771]: ts=2026-09-23T05:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.713145] nix-grpc-daemon[768]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.718549] nix-grpc-daemon[768]: ts=2026-09-23T05:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 24.813767] nix-grpc-daemon[771]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 24.819900] nix-grpc-daemon[771]: ts=2026-09-23T05:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 24.921534] nix-grpc-daemon[768]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 24.925787] nix-grpc-daemon[768]: ts=2026-09-23T05:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.023083] nix-grpc-daemon[771]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.027560] nix-grpc-daemon[771]: ts=2026-09-23T05:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.122400] nix-grpc-daemon[768]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.128246] nix-grpc-daemon[768]: ts=2026-09-23T05:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.224963] nix-grpc-daemon[771]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.231487] nix-grpc-daemon[771]: ts=2026-09-23T05:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.329057] nix-grpc-daemon[768]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.335111] nix-grpc-daemon[768]: ts=2026-09-23T05:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.432585] nix-grpc-daemon[771]: ts=2026-09-23T05:13:01Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.438272] nix-grpc-daemon[771]: ts=2026-09-23T05:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.536207] nix-grpc-daemon[768]: ts=2026-09-23T05:13:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.541671] nix-grpc-daemon[768]: ts=2026-09-23T05:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.639710] nix-grpc-daemon[771]: ts=2026-09-23T05:13:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.645530] nix-grpc-daemon[771]: ts=2026-09-23T05:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.742945] nix-grpc-daemon[768]: ts=2026-09-23T05:13:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.748709] nix-grpc-daemon[768]: ts=2026-09-23T05:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 25.846663] nix-grpc-daemon[771]: ts=2026-09-23T05:13:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 25.852442] nix-grpc-daemon[771]: ts=2026-09-23T05:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 25.951303] nix-grpc-daemon[768]: ts=2026-09-23T05:13:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 25.956894] nix-grpc-daemon[768]: ts=2026-09-23T05:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.056196] nix-grpc-daemon[771]: ts=2026-09-23T05:13:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.060544] nix-grpc-daemon[771]: ts=2026-09-23T05:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.157911] nix-grpc-daemon[768]: ts=2026-09-23T05:13:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.163515] nix-grpc-daemon[768]: ts=2026-09-23T05:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.257543] nix-grpc-daemon[771]: ts=2026-09-23T05:13:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.263312] nix-grpc-daemon[771]: ts=2026-09-23T05:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.364732] nix-grpc-daemon[768]: ts=2026-09-23T05:13:02Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.370333] nix-grpc-daemon[768]: ts=2026-09-23T05:13:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.464526] nix-grpc-daemon[771]: ts=2026-09-23T05:13:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.470373] nix-grpc-daemon[771]: ts=2026-09-23T05:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.571042] nix-grpc-daemon[768]: ts=2026-09-23T05:13:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.576652] nix-grpc-daemon[768]: ts=2026-09-23T05:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.671504] nix-grpc-daemon[771]: ts=2026-09-23T05:13:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.677322] nix-grpc-daemon[771]: ts=2026-09-23T05:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.777458] nix-grpc-daemon[768]: ts=2026-09-23T05:13:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.782851] nix-grpc-daemon[768]: ts=2026-09-23T05:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 26.878360] nix-grpc-daemon[771]: ts=2026-09-23T05:13:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 26.884162] nix-grpc-daemon[771]: ts=2026-09-23T05:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 26.985101] nix-grpc-daemon[768]: ts=2026-09-23T05:13:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 26.990574] nix-grpc-daemon[768]: ts=2026-09-23T05:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.086847] nix-grpc-daemon[771]: ts=2026-09-23T05:13:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.092704] nix-grpc-daemon[771]: ts=2026-09-23T05:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.191532] nix-grpc-daemon[768]: ts=2026-09-23T05:13:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.197018] nix-grpc-daemon[768]: ts=2026-09-23T05:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.294173] nix-grpc-daemon[771]: ts=2026-09-23T05:13:03Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.299872] nix-grpc-daemon[771]: ts=2026-09-23T05:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.398271] nix-grpc-daemon[768]: ts=2026-09-23T05:13:03Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.403739] nix-grpc-daemon[768]: ts=2026-09-23T05:13:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.501442] nix-grpc-daemon[771]: ts=2026-09-23T05:13:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.507369] nix-grpc-daemon[771]: ts=2026-09-23T05:13:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.605120] nix-grpc-daemon[768]: ts=2026-09-23T05:13:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.610734] nix-grpc-daemon[768]: ts=2026-09-23T05:13:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.708656] nix-grpc-daemon[771]: ts=2026-09-23T05:13:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.714699] nix-grpc-daemon[771]: ts=2026-09-23T05:13:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 27.811524] nix-grpc-daemon[768]: ts=2026-09-23T05:13:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 27.817191] nix-grpc-daemon[768]: ts=2026-09-23T05:13:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 27.915979] nix-grpc-daemon[771]: ts=2026-09-23T05:13:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 27.925590] nix-grpc-daemon[771]: ts=2026-09-23T05:13:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.020219] nix-grpc-daemon[768]: ts=2026-09-23T05:13:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.024440] nix-grpc-daemon[768]: ts=2026-09-23T05:13:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.121338] nix-grpc-daemon[771]: ts=2026-09-23T05:13:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.126266] nix-grpc-daemon[771]: ts=2026-09-23T05:13:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.220926] nix-grpc-daemon[768]: ts=2026-09-23T05:13:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.226339] nix-grpc-daemon[768]: ts=2026-09-23T05:13:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.322581] nix-grpc-daemon[771]: ts=2026-09-23T05:13:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.328786] nix-grpc-daemon[771]: ts=2026-09-23T05:13:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 28.427250] nix-grpc-daemon[768]: ts=2026-09-23T05:13:04Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.432835] nix-grpc-daemon[768]: ts=2026-09-23T05:13:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 28.421805] rustfs-bucket-start[1002]: mb s3://farm lb # [ 28.436858] systemd[1]: Finished rustfs-bucket.service. lb # [ 28.449790] systemd[1]: Starting niks3 server... worker2 # [ 28.530297] nix-grpc-daemon[771]: ts=2026-09-23T05:13:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.536648] nix-grpc-daemon[771]: ts=2026-09-23T05:13:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 28.587314] postgres[1018]: [1018] ERROR: relation "goose_db_version" does not exist at character 36 lb # [ 28.592041] postgres[1018]: [1018] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC worker1 # [ 28.633710] nix-grpc-daemon[768]: ts=2026-09-23T05:13:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.640776] nix-grpc-daemon[768]: ts=2026-09-23T05:13:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 28.675118] niks3-server[1013]: 2026/09/23 05:13:05 OK 20241026095416_initial_model.sql (74.22ms) lb # [ 28.681282] niks3-server[1013]: 2026/09/23 05:13:05 OK 20251210153512_drop_unused_gin_index.sql (6.55ms) lb # [ 28.688704] niks3-server[1013]: 2026/09/23 05:13:05 OK 20251218171726_add_pins.sql (7.35ms) lb # [ 28.695149] niks3-server[1013]: 2026/09/23 05:13:05 OK 20260628120000_add_object_size_and_stats.sql (6.4ms) worker2 # [ 28.737898] nix-grpc-daemon[771]: ts=2026-09-23T05:13:05Z level=info event=scheduler_connected addr=lb:50051 lb # [ 28.702880] niks3-server[1013]: 2026/09/23 05:13:05 OK 20260905000000_add_claims.sql (7.69ms) worker2 # [ 28.743607] nix-grpc-daemon[771]: ts=2026-09-23T05:13:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 28.708274] niks3-server[1013]: 2026/09/23 05:13:05 OK 20260920000000_drop_claims.sql (5.26ms) lb # [ 28.709586] niks3-server[1013]: 2026/09/23 05:13:05 goose: successfully migrated database to version: 20260920000000 lb # [ 28.715231] niks3-server[1013]: 2026/09/23 05:13:05 OK 1_commit_pending_closure.sql (6.92ms) lb # [ 28.719034] niks3-server[1013]: 2026/09/23 05:13:05 OK 2_object_stats_trigger.sql (3.74ms) lb # [ 28.721078] niks3-server[1013]: 2026/09/23 05:13:05 goose: up to current file version: 2 lb # [ 28.722778] niks3-server[1013]: 2026/09/23 05:13:05 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 28.743135] niks3-server[1013]: 2026/09/23 05:13:05 INFO Read proxy enabled — serving cache objects from S3 lb # [ 28.747623] niks3-server[1013]: 2026/09/23 05:13:05 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 28.754699] niks3-server[1013]: 2026/09/23 05:13:05 INFO systemd watchdog enabled interval=15s lb # [ 28.757133] niks3-server[1013]: 2026/09/23 05:13:05 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 28.759232] systemd[1]: Started niks3 server. lb # [ 28.760182] systemd[1]: Reached target Multi-User System. lb # [ 28.761028] systemd[1]: Startup finished in 1.002s (kernel) + 4.976s (initrd) + 22.777s (userspace) = 28.756s. worker1 # [ 28.840359] nix-grpc-daemon[768]: ts=2026-09-23T05:13:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 28.845315] nix-grpc-daemon[768]: ts=2026-09-23T05:13:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 28.944615] nix-grpc-daemon[771]: ts=2026-09-23T05:13:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 28.950454] nix-grpc-daemon[771]: ts=2026-09-23T05:13:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.043988] nix-grpc-daemon[768]: ts=2026-09-23T05:13:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.048101] nix-grpc-daemon[768]: ts=2026-09-23T05:13:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.153924] nix-grpc-daemon[771]: ts=2026-09-23T05:13:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.158788] nix-grpc-daemon[771]: ts=2026-09-23T05:13:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.244917] nix-grpc-daemon[768]: ts=2026-09-23T05:13:05Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.250170] nix-grpc-daemon[768]: ts=2026-09-23T05:13:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.360155] nix-grpc-daemon[771]: ts=2026-09-23T05:13:05Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.366134] nix-grpc-daemon[771]: ts=2026-09-23T05:13:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.450967] nix-grpc-daemon[768]: ts=2026-09-23T05:13:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.456371] nix-grpc-daemon[768]: ts=2026-09-23T05:13:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.567360] nix-grpc-daemon[771]: ts=2026-09-23T05:13:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.573455] nix-grpc-daemon[771]: ts=2026-09-23T05:13:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 29.657239] nix-grpc-daemon[768]: ts=2026-09-23T05:13:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.662810] nix-grpc-daemon[768]: ts=2026-09-23T05:13:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 29.774852] nix-grpc-daemon[771]: ts=2026-09-23T05:13:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 29.780908] nix-grpc-daemon[771]: ts=2026-09-23T05:13:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: waiting for unit niks3.service, in 16.86 seconds) lb: waiting for TCP port 5751 on localhost worker1 # [ 29.863290] nix-grpc-daemon[768]: ts=2026-09-23T05:13:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 29.868664] nix-grpc-daemon[768]: ts=2026-09-23T05:13:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded! lb: (finished: waiting for TCP port 5751 on localhost, in 0.08 seconds) lb: waiting for unit envoy.service lb: (finished: waiting for unit envoy.service, in 0.05 seconds) lb: waiting for TCP port 50051 on localhost worker2 # [ 29.982092] nix-grpc-daemon[771]: ts=2026-09-23T05:13:06Z level=info event=scheduler_connected addr=lb:50051 lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded! worker2 # [ 29.988193] nix-grpc-daemon[771]: ts=2026-09-23T05:13:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 29.951272] envoy[900]: conn peer=[::1]:35250 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=4 lb: (finished: waiting for TCP port 50051 on localhost, in 0.04 seconds) worker1 # [ 30.076167] nix-grpc-daemon[768]: ts=2026-09-23T05:13:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.079066] nix-grpc-daemon[768]: ts=2026-09-23T05:13:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.171168] systemd[1]: Starting scheduler-token.service... worker2 # [ 30.189625] nix-grpc-daemon[771]: ts=2026-09-23T05:13:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.193149] nix-grpc-daemon[771]: ts=2026-09-23T05:13:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.249255] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 30.253539] systemd[1]: Finished scheduler-token.service. worker2 # [ 30.254931] systemd[1]: scheduler-token.service: Consumed 41ms CPU time over 81ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 30.276456] nix-grpc-daemon[768]: ts=2026-09-23T05:13:06Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.281785] nix-grpc-daemon[768]: ts=2026-09-23T05:13:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.08 seconds) subtest: one node schedules and both builders hold a WorkerSession on it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2 # [ 30.394031] nix-grpc-daemon[771]: ts=2026-09-23T05:13:06Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.399301] nix-grpc-daemon[771]: ts=2026-09-23T05:13:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.482663] nix-grpc-daemon[768]: ts=2026-09-23T05:13:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.487977] nix-grpc-daemon[768]: ts=2026-09-23T05:13:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.600502] nix-grpc-daemon[771]: ts=2026-09-23T05:13:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.605841] nix-grpc-daemon[771]: ts=2026-09-23T05:13:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.688808] nix-grpc-daemon[768]: ts=2026-09-23T05:13:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.694047] nix-grpc-daemon[768]: ts=2026-09-23T05:13:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 30.807501] nix-grpc-daemon[771]: ts=2026-09-23T05:13:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 30.812782] nix-grpc-daemon[771]: ts=2026-09-23T05:13:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 30.894856] nix-grpc-daemon[768]: ts=2026-09-23T05:13:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 30.900270] nix-grpc-daemon[768]: ts=2026-09-23T05:13:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.014031] nix-grpc-daemon[771]: ts=2026-09-23T05:13:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.019484] nix-grpc-daemon[771]: ts=2026-09-23T05:13:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.103020] nix-grpc-daemon[768]: ts=2026-09-23T05:13:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.107194] nix-grpc-daemon[768]: ts=2026-09-23T05:13:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.222657] nix-grpc-daemon[771]: ts=2026-09-23T05:13:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.236309] nix-grpc-daemon[771]: ts=2026-09-23T05:13:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.303503] nix-grpc-daemon[768]: ts=2026-09-23T05:13:07Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.307225] nix-grpc-daemon[768]: ts=2026-09-23T05:13:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 31.423447] nix-grpc-daemon[771]: ts=2026-09-23T05:13:07Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.428888] nix-grpc-daemon[771]: ts=2026-09-23T05:13:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 31.508239] nix-grpc-daemon[768]: ts=2026-09-23T05:13:08Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.513476] nix-grpc-daemon[768]: ts=2026-09-23T05:13:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.630084] nix-grpc-daemon[771]: ts=2026-09-23T05:13:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.635367] nix-grpc-daemon[771]: ts=2026-09-23T05:13:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.714117] nix-grpc-daemon[768]: ts=2026-09-23T05:13:08Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.720367] nix-grpc-daemon[768]: ts=2026-09-23T05:13:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 31.836661] nix-grpc-daemon[771]: ts=2026-09-23T05:13:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 31.841862] nix-grpc-daemon[771]: ts=2026-09-23T05:13:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 31.920670] nix-grpc-daemon[768]: ts=2026-09-23T05:13:08Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 31.925843] nix-grpc-daemon[768]: ts=2026-09-23T05:13:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.042372] nix-grpc-daemon[771]: ts=2026-09-23T05:13:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.044747] nix-grpc-daemon[771]: ts=2026-09-23T05:13:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.128583] nix-grpc-daemon[768]: ts=2026-09-23T05:13:08Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.132896] nix-grpc-daemon[768]: ts=2026-09-23T05:13:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.247951] nix-grpc-daemon[771]: ts=2026-09-23T05:13:08Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.252130] nix-grpc-daemon[771]: ts=2026-09-23T05:13:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.329372] nix-grpc-daemon[768]: ts=2026-09-23T05:13:08Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.334898] nix-grpc-daemon[768]: ts=2026-09-23T05:13:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.448805] nix-grpc-daemon[771]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.454143] nix-grpc-daemon[771]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 32.535854] nix-grpc-daemon[768]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.541587] nix-grpc-daemon[768]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.655559] nix-grpc-daemon[771]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.661151] nix-grpc-daemon[771]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.742203] nix-grpc-daemon[768]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.747562] nix-grpc-daemon[768]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 32.862147] nix-grpc-daemon[771]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 32.867807] nix-grpc-daemon[771]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 32.948664] nix-grpc-daemon[768]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 32.954788] nix-grpc-daemon[768]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.069258] nix-grpc-daemon[771]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.075028] nix-grpc-daemon[771]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.157561] nix-grpc-daemon[768]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.161767] nix-grpc-daemon[768]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.278132] nix-grpc-daemon[771]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.282524] nix-grpc-daemon[771]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.358356] nix-grpc-daemon[768]: ts=2026-09-23T05:13:09Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.363844] nix-grpc-daemon[768]: ts=2026-09-23T05:13:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.478960] nix-grpc-daemon[771]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.484651] nix-grpc-daemon[771]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 33.564714] nix-grpc-daemon[768]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.570125] nix-grpc-daemon[768]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 33.685886] nix-grpc-daemon[771]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.691394] nix-grpc-daemon[771]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.770769] nix-grpc-daemon[768]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.776253] nix-grpc-daemon[768]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 33.892463] nix-grpc-daemon[771]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 33.897973] nix-grpc-daemon[771]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 33.976889] nix-grpc-daemon[768]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 33.982343] nix-grpc-daemon[768]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.099176] nix-grpc-daemon[771]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.104779] nix-grpc-daemon[771]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.185196] nix-grpc-daemon[768]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.189423] nix-grpc-daemon[768]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.307863] nix-grpc-daemon[771]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.312346] nix-grpc-daemon[771]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.385812] nix-grpc-daemon[768]: ts=2026-09-23T05:13:10Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.391185] nix-grpc-daemon[768]: ts=2026-09-23T05:13:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.508871] nix-grpc-daemon[771]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.514359] nix-grpc-daemon[771]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.593078] nix-grpc-daemon[768]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.602180] nix-grpc-daemon[768]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 34.715500] nix-grpc-daemon[771]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.721055] nix-grpc-daemon[771]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 34.797724] nix-grpc-daemon[768]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 34.803329] nix-grpc-daemon[768]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 34.922097] nix-grpc-daemon[771]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 34.927389] nix-grpc-daemon[771]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.003996] nix-grpc-daemon[768]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.009372] nix-grpc-daemon[768]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.128565] nix-grpc-daemon[771]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.133941] nix-grpc-daemon[771]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.211897] nix-grpc-daemon[768]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.216232] nix-grpc-daemon[768]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.336912] nix-grpc-daemon[771]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.341098] nix-grpc-daemon[771]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.412383] nix-grpc-daemon[768]: ts=2026-09-23T05:13:11Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.417549] nix-grpc-daemon[768]: ts=2026-09-23T05:13:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.538092] nix-grpc-daemon[771]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.543640] nix-grpc-daemon[771]: ts=2026-09-23T05:13:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.618425] nix-grpc-daemon[768]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.623754] nix-grpc-daemon[768]: ts=2026-09-23T05:13:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 35.744763] nix-grpc-daemon[771]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:50051 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker2 # [ 35.750033] nix-grpc-daemon[771]: ts=2026-09-23T05:13:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 35.824876] nix-grpc-daemon[768]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 35.830261] nix-grpc-daemon[768]: ts=2026-09-23T05:13:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 35.951199] nix-grpc-daemon[771]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 35.956679] nix-grpc-daemon[771]: ts=2026-09-23T05:13:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.031275] nix-grpc-daemon[768]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.037002] nix-grpc-daemon[768]: ts=2026-09-23T05:13:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.157778] nix-grpc-daemon[771]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.163059] nix-grpc-daemon[771]: ts=2026-09-23T05:13:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.240236] nix-grpc-daemon[768]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.244384] nix-grpc-daemon[768]: ts=2026-09-23T05:13:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.366093] nix-grpc-daemon[771]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.370318] nix-grpc-daemon[771]: ts=2026-09-23T05:13:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.440675] nix-grpc-daemon[768]: ts=2026-09-23T05:13:12Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.446046] nix-grpc-daemon[768]: ts=2026-09-23T05:13:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.566916] nix-grpc-daemon[771]: ts=2026-09-23T05:13:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.572358] nix-grpc-daemon[771]: ts=2026-09-23T05:13:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 36.646874] nix-grpc-daemon[768]: ts=2026-09-23T05:13:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.652433] nix-grpc-daemon[768]: ts=2026-09-23T05:13:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 36.773612] nix-grpc-daemon[771]: ts=2026-09-23T05:13:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.779534] nix-grpc-daemon[771]: ts=2026-09-23T05:13:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1 # [ 36.853164] nix-grpc-daemon[768]: ts=2026-09-23T05:13:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 36.858583] nix-grpc-daemon[768]: ts=2026-09-23T05:13:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 36.981088] nix-grpc-daemon[771]: ts=2026-09-23T05:13:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 36.986471] nix-grpc-daemon[771]: ts=2026-09-23T05:13:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.059616] nix-grpc-daemon[768]: ts=2026-09-23T05:13:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.065186] nix-grpc-daemon[768]: ts=2026-09-23T05:13:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.187660] nix-grpc-daemon[771]: ts=2026-09-23T05:13:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.193107] nix-grpc-daemon[771]: ts=2026-09-23T05:13:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.267732] nix-grpc-daemon[768]: ts=2026-09-23T05:13:13Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.271945] nix-grpc-daemon[768]: ts=2026-09-23T05:13:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.396449] nix-grpc-daemon[771]: ts=2026-09-23T05:13:13Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.400588] nix-grpc-daemon[771]: ts=2026-09-23T05:13:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.468443] nix-grpc-daemon[768]: ts=2026-09-23T05:13:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.473587] nix-grpc-daemon[768]: ts=2026-09-23T05:13:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.597173] nix-grpc-daemon[771]: ts=2026-09-23T05:13:14Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.602575] nix-grpc-daemon[771]: ts=2026-09-23T05:13:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 37.674625] nix-grpc-daemon[768]: ts=2026-09-23T05:13:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.679927] nix-grpc-daemon[768]: ts=2026-09-23T05:13:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 37.803566] nix-grpc-daemon[771]: ts=2026-09-23T05:13:14Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 37.809015] nix-grpc-daemon[771]: ts=2026-09-23T05:13:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 37.880835] nix-grpc-daemon[768]: ts=2026-09-23T05:13:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 37.886595] nix-grpc-daemon[768]: ts=2026-09-23T05:13:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker2 # [ 38.010322] nix-grpc-daemon[771]: ts=2026-09-23T05:13:14Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.016252] nix-grpc-daemon[771]: ts=2026-09-23T05:13:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.087004] nix-grpc-daemon[768]: ts=2026-09-23T05:13:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.092733] nix-grpc-daemon[768]: ts=2026-09-23T05:13:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.217631] nix-grpc-daemon[771]: ts=2026-09-23T05:13:14Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.223263] nix-grpc-daemon[771]: ts=2026-09-23T05:13:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.295770] nix-grpc-daemon[768]: ts=2026-09-23T05:13:14Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.300169] nix-grpc-daemon[768]: ts=2026-09-23T05:13:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.426650] nix-grpc-daemon[771]: ts=2026-09-23T05:13:14Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.431142] nix-grpc-daemon[771]: ts=2026-09-23T05:13:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.496666] nix-grpc-daemon[768]: ts=2026-09-23T05:13:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.501988] nix-grpc-daemon[768]: ts=2026-09-23T05:13:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.627963] nix-grpc-daemon[771]: ts=2026-09-23T05:13:15Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.633688] nix-grpc-daemon[771]: ts=2026-09-23T05:13:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 38.703046] nix-grpc-daemon[768]: ts=2026-09-23T05:13:15Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 38.708833] nix-grpc-daemon[768]: ts=2026-09-23T05:13:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 38.795325] nix-grpc-daemon[771]: ts=2026-09-23T05:13:15Z level=info event=scheduler_take_over lb # [ 38.755637] niks3-server[1013]: 2026/09/23 05:13:15 INFO lead: acquired remote=192.168.1.4:57726 worker2 # [ 38.834720] nix-grpc-daemon[771]: ts=2026-09-23T05:13:15Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.856216] nix-grpc-daemon[771]: ts=2026-09-23T05:13:15Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 38.862047] nix-grpc-daemon[771]: ts=2026-09-23T05:13:15Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:45476 worker2 # [ 38.868340] nix-grpc-daemon[771]: ts=2026-09-23T05:13:15Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 38.909840] nix-grpc-daemon[768]: ts=2026-09-23T05:13:15Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 38.912719] nix-grpc-daemon[771]: ts=2026-09-23T05:13:15Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:45476 worker2 # [ 38.918069] nix-grpc-daemon[771]: ts=2026-09-23T05:13:15Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open' worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds) (finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 9.33 seconds) subtest: balancer auth: client cert, bearer token, nothing, unknown CN worker2 # [ 39.941711] nix-grpc-daemon[771]: ts=2026-09-23T05:13:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 39.955012] systemd[1]: Started Nix Daemon. worker2 # [ 40.131258] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 40.153899] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5) worker2 # [ 40.172563] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 2/5) worker2 # [ 40.179562] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 252 ms (attempt 3/5) worker2 # [ 40.436978] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 223 ms (attempt 4/5) worker2 # [ 40.662629] nix-grpc-daemon[771]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 40.668375] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 40.702825] nix-grpc-daemon[768]: ts=2026-09-23T05:13:17Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 40.717563] systemd[1]: Started Nix Daemon. worker1 # [ 40.905311] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 40.924909] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 1/5) worker1 # [ 40.964842] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 107 ms (attempt 2/5) worker1 # [ 41.076866] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 231 ms (attempt 3/5) worker1 # [ 41.310158] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 710 ms (attempt 4/5) worker1 # [ 42.024846] nix-grpc-daemon[768]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 42.029827] nix-grpc-daemon[768]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 42.044300] nix-grpc-daemon[768]: ts=2026-09-23T05:13:18Z level=debug event=path_info_miss path=b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test error="error: path '/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 42.060529] nix-grpc-daemon[771]: ts=2026-09-23T05:13:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 42.074546] nix-grpc-daemon[771]: ts=2026-09-23T05:13:18Z level=debug event=path_info_miss path=b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test error="error: path '/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.08 seconds) client: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key client: (finished: must succeed: install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.04 seconds) worker1 # [ 42.403083] nix-grpc-daemon[768]: ts=2026-09-23T05:13:18Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 42.409199] nix-grpc-daemon[768]: ts=2026-09-23T05:13:18Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:37640 worker1 # [ 42.424808] nix-grpc-daemon[768]: ts=2026-09-23T05:13:18Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:37640 worker2 # [ 42.442150] nix-grpc-daemon[771]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:45480 worker1 # [ 42.437070] nix-grpc-daemon[768]: ts=2026-09-23T05:13:18Z level=debug event=path_info_miss path=b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test error="error: path '/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test' is required, but there is no substituter that can build it" worker2 # [ 42.456340] nix-grpc-daemon[771]: ts=2026-09-23T05:13:19Z level=debug event=path_info_miss path=b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test error="error: path '/nix/store/b1xy1s6qy6gkzf9w9b4q2blrhdavjf85-nixos-system-client-test' is required, but there is no substituter that can build it" client: must succeed: rm -r /var/lib/nix-grpc-store client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.03 seconds) worker2 # [ 42.688875] nix-grpc-daemon[771]: ts=2026-09-23T05:13:19Z level=info event=denied method=QueryMissing cn=- role=none worker1 # [ 42.869316] nix-grpc-daemon[768]: ts=2026-09-23T05:13:19Z level=info event=denied method=QueryMissing cn=stranger role=none (finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.22 seconds) subtest: DAG build is scheduled across workers and lands in the cache client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 43.098453] nix-grpc-daemon[771]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 43.108390] nix-grpc-daemon[771]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 43.138635] nix-grpc-daemon[771]: ts=2026-09-23T05:13:19Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=0 worker2 # [ 43.146190] nix-grpc-daemon[771]: ts=2026-09-23T05:13:19Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 43.192413] nix-grpc-daemon[771]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 43.211085] nix-grpc-daemon[771]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 client # copying 1 paths to 'lb:50051'... worker2 # [ 43.228125] nix-grpc-daemon[771]: ts=2026-09-23T05:13:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'... worker2 # [ 43.254532] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 43.292518] nix-grpc-daemon[771]: ts=2026-09-23T05:13:19Z level=info event=niks3_push_started pid=873 worker2 # [ 43.487986] nix-grpc-daemon[873]: time=2026-09-23T05:13:20.049Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 43.605115] nix-daemon[858]: accepted connection from pid 885, user nix-grpc-daemon (trusted) worker2 # [ 43.628423] nix-daemon[858]: reaped child process 888, status = succeeded lb # [ 43.590309] niks3-server[1013]: 2026/09/23 05:13:20 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 43.653344] nix-grpc-daemon[873]: time=2026-09-23T05:13:20.216Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 43.654954] nix-grpc-daemon[873]: time=2026-09-23T05:13:20.216Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)" lb # [ 43.648316] niks3-server[1013]: 2026/09/23 05:13:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign lb # [ 43.669319] niks3-server[1013]: 2026/09/23 05:13:20 INFO Signed narinfos id=1 count=1 worker2 # [ 43.716129] nix-grpc-daemon[873]: time=2026-09-23T05:13:20.279Z level=INFO msg="Uploading 1 narinfos" lb # [ 43.688129] niks3-server[1013]: 2026/09/23 05:13:20 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst lb # [ 43.698756] niks3-server[1013]: 2026/09/23 05:13:20 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls lb # [ 43.708777] niks3-server[1013]: 2026/09/23 05:13:20 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete lb # [ 43.719365] niks3-server[1013]: 2026/09/23 05:13:20 INFO Completed upload id=1 worker2 # [ 43.762222] nix-grpc-daemon[873]: time=2026-09-23T05:13:20.325Z level=INFO msg="Upload complete. (275ms)" worker2 # [ 43.776449] nix-grpc-daemon[771]: ts=2026-09-23T05:13:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_in=504 worker2 # [ 43.797712] nix-grpc-daemon[771]: ts=2026-09-23T05:13:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 43.806744] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 43.815462] nix-daemon[858]: reaped child process 871, status = succeeded client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051... worker2 # [ 43.826939] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) client # farm-a> worker2: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-t1 worker2 # [ 44.933723] nix-daemon[858]: reaped child process 892, status = succeeded worker2 # [ 45.003088] nix-daemon[858]: accepted connection from pid 909, user nix-grpc-daemon (trusted) worker2 # [ 45.024863] nix-daemon[858]: reaped child process 912, status = succeeded lb # [ 44.982251] niks3-server[1013]: 2026/09/23 05:13:21 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 45.043240] nix-grpc-daemon[873]: time=2026-09-23T05:13:21.604Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 45.046143] nix-grpc-daemon[873]: time=2026-09-23T05:13:21.604Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)" lb # [ 45.036613] niks3-server[1013]: 2026/09/23 05:13:21 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker2 # [ 45.089872] nix-grpc-daemon[873]: time=2026-09-23T05:13:21.653Z level=INFO msg="Uploading 1 narinfos" lb # [ 45.050111] niks3-server[1013]: 2026/09/23 05:13:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign lb # [ 45.068358] niks3-server[1013]: 2026/09/23 05:13:21 INFO Signed narinfos id=2 count=1 lb # [ 45.084413] niks3-server[1013]: 2026/09/23 05:13:21 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst lb # [ 45.098872] niks3-server[1013]: 2026/09/23 05:13:21 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete worker2 # [ 45.145379] nix-grpc-daemon[873]: time=2026-09-23T05:13:21.709Z level=INFO msg="Upload complete. (304ms)" lb # [ 45.105864] niks3-server[1013]: 2026/09/23 05:13:21 INFO Completed upload id=2 lb # [ 45.108372] niks3-server[1013]: 2026/09/23 05:13:21 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo lb # [ 45.113349] niks3-server[1013]: 2026/09/23 05:13:21 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls worker2 # [ 45.169369] nix-grpc-daemon[771]: ts=2026-09-23T05:13:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=1 worker2 # [ 45.188274] nix-grpc-daemon[771]: ts=2026-09-23T05:13:21Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=0 worker2 # [ 45.192442] nix-grpc-daemon[771]: ts=2026-09-23T05:13:21Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 45.196810] nix-daemon[858]: reaped child process 890, status = succeeded worker2 # [ 45.220322] nix-grpc-daemon[771]: ts=2026-09-23T05:13:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 45.236538] nix-grpc-daemon[771]: ts=2026-09-23T05:13:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 45.251934] nix-grpc-daemon[873]: time=2026-09-23T05:13:21.815Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 45.263156] nix-grpc-daemon[771]: ts=2026-09-23T05:13:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'... worker2 # [ 45.285916] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 45.447971] nix-daemon[858]: accepted connection from pid 918, user nix-grpc-daemon (trusted) worker2 # [ 45.469291] nix-daemon[858]: reaped child process 921, status = succeeded lb # [ 45.430867] niks3-server[1013]: 2026/09/23 05:13:21 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 45.483115] nix-grpc-daemon[873]: time=2026-09-23T05:13:22.046Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 45.484880] nix-grpc-daemon[873]: time=2026-09-23T05:13:22.046Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)" lb # [ 45.463865] niks3-server[1013]: 2026/09/23 05:13:22 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls worker2 # [ 45.519555] nix-grpc-daemon[873]: time=2026-09-23T05:13:22.083Z level=INFO msg="Uploading 1 narinfos" lb # [ 45.477174] niks3-server[1013]: 2026/09/23 05:13:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign lb # [ 45.483559] niks3-server[1013]: 2026/09/23 05:13:22 INFO Signed narinfos id=3 count=1 lb # [ 45.496309] niks3-server[1013]: 2026/09/23 05:13:22 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete lb # [ 45.552220] niks3-server[1013]: 2026/09/23 05:13:22 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst lb # [ 45.562085] niks3-server[1013]: 2026/09/23 05:13:22 INFO Completed upload id=3 worker2 # [ 45.604569] nix-grpc-daemon[873]: time=2026-09-23T05:13:22.168Z level=INFO msg="Upload complete. (300ms)" worker2 # [ 45.615144] nix-grpc-daemon[771]: ts=2026-09-23T05:13:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 45.579886] niks3-server[1013]: 2026/09/23 05:13:22 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo worker2 # [ 45.632294] nix-grpc-daemon[771]: ts=2026-09-23T05:13:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 45.644517] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 45.658668] nix-daemon[858]: reaped child process 916, status = succeeded client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051... client # farm-b> worker2: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker2 # [ 45.668151] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-t1 worker2 # [ 46.894724] nix-daemon[858]: reaped child process 925, status = succeeded worker2 # [ 46.999057] nix-daemon[858]: accepted connection from pid 941, user nix-grpc-daemon (trusted) worker2 # [ 47.019910] nix-daemon[858]: reaped child process 944, status = succeeded lb # [ 46.982090] niks3-server[1013]: 2026/09/23 05:13:23 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 47.083503] nix-grpc-daemon[873]: time=2026-09-23T05:13:23.645Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 47.086507] nix-grpc-daemon[873]: time=2026-09-23T05:13:23.645Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)" lb # [ 47.078302] niks3-server[1013]: 2026/09/23 05:13:23 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker2 # [ 47.132499] nix-grpc-daemon[873]: time=2026-09-23T05:13:23.695Z level=INFO msg="Uploading 1 narinfos" lb # [ 47.093850] niks3-server[1013]: 2026/09/23 05:13:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign lb # [ 47.108122] niks3-server[1013]: 2026/09/23 05:13:23 INFO Signed narinfos id=4 count=1 lb # [ 47.111066] niks3-server[1013]: 2026/09/23 05:13:23 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst lb # [ 47.119136] niks3-server[1013]: 2026/09/23 05:13:23 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls lb # [ 47.124782] niks3-server[1013]: 2026/09/23 05:13:23 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete lb # [ 47.134638] niks3-server[1013]: 2026/09/23 05:13:23 INFO Completed upload id=4 worker2 # [ 47.166490] nix-grpc-daemon[873]: time=2026-09-23T05:13:23.730Z level=INFO msg="Upload complete. (287ms)" lb # [ 47.140470] niks3-server[1013]: 2026/09/23 05:13:23 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo worker2 # [ 47.184751] nix-grpc-daemon[771]: ts=2026-09-23T05:13:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=1 worker2 # [ 47.206072] nix-grpc-daemon[771]: ts=2026-09-23T05:13:23Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=0 worker2 # [ 47.211856] nix-grpc-daemon[771]: ts=2026-09-23T05:13:23Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 47.217903] nix-daemon[858]: reaped child process 923, status = succeeded worker2 # [ 47.234936] nix-grpc-daemon[771]: ts=2026-09-23T05:13:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 47.250462] nix-grpc-daemon[771]: ts=2026-09-23T05:13:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 47.265296] nix-grpc-daemon[873]: time=2026-09-23T05:13:23.829Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... worker2 # [ 47.276934] nix-grpc-daemon[771]: ts=2026-09-23T05:13:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'... worker2 # [ 47.299125] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 47.456159] nix-daemon[858]: accepted connection from pid 950, user nix-grpc-daemon (trusted) worker2 # [ 47.475334] nix-daemon[858]: reaped child process 953, status = succeeded lb # [ 47.439925] niks3-server[1013]: 2026/09/23 05:13:24 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 47.491658] nix-grpc-daemon[873]: time=2026-09-23T05:13:24.055Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 47.493527] nix-grpc-daemon[873]: time=2026-09-23T05:13:24.055Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)" lb # [ 47.470097] niks3-server[1013]: 2026/09/23 05:13:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign worker2 # [ 47.518485] nix-grpc-daemon[873]: time=2026-09-23T05:13:24.082Z level=INFO msg="Uploading 1 narinfos" lb # [ 47.483103] niks3-server[1013]: 2026/09/23 05:13:24 INFO Signed narinfos id=5 count=1 lb # [ 47.491415] niks3-server[1013]: 2026/09/23 05:13:24 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls lb # [ 47.509766] niks3-server[1013]: 2026/09/23 05:13:24 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete lb # [ 47.518814] niks3-server[1013]: 2026/09/23 05:13:24 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst lb # [ 47.526105] niks3-server[1013]: 2026/09/23 05:13:24 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo lb # [ 47.529634] niks3-server[1013]: 2026/09/23 05:13:24 INFO Completed upload id=5 worker2 # [ 47.568638] nix-grpc-daemon[873]: time=2026-09-23T05:13:24.131Z level=INFO msg="Upload complete. (251ms)" worker2 # [ 47.578564] nix-grpc-daemon[771]: ts=2026-09-23T05:13:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 47.594982] nix-grpc-daemon[771]: ts=2026-09-23T05:13:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 47.607224] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 47.614615] nix-daemon[858]: reaped child process 948, status = succeeded client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051... worker2 # [ 47.628232] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1 worker2 # [ 48.308693] nix-daemon[858]: reaped child process 957, status = succeeded worker2 # [ 48.411872] nix-daemon[858]: accepted connection from pid 973, user nix-grpc-daemon (trusted) lb # [ 48.368963] envoy[900]: conn peer=[2001:db8:1::1]:56996 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=2166 lb # [ 48.374017] envoy[900]: conn peer=[2001:db8:1::1]:58558 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=76 lb # [ 48.378610] envoy[900]: conn peer=[2001:db8:1::1]:58572 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=16 lb # [ 48.383180] envoy[900]: conn peer=[2001:db8:1::1]:58586 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=13 worker2 # [ 48.434717] nix-daemon[858]: reaped child process 976, status = succeeded lb # [ 48.391433] niks3-server[1013]: 2026/09/23 05:13:24 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 48.448537] nix-grpc-daemon[873]: time=2026-09-23T05:13:25.010Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 48.450436] nix-grpc-daemon[873]: time=2026-09-23T05:13:25.010Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)" lb # [ 48.435477] niks3-server[1013]: 2026/09/23 05:13:25 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst lb # [ 48.447758] niks3-server[1013]: 2026/09/23 05:13:25 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker2 # [ 48.500267] nix-grpc-daemon[873]: time=2026-09-23T05:13:25.063Z level=INFO msg="Uploading 1 narinfos" lb # [ 48.457536] niks3-server[1013]: 2026/09/23 05:13:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign lb # [ 48.464468] niks3-server[1013]: 2026/09/23 05:13:25 INFO Signed narinfos id=6 count=1 lb # [ 48.466389] niks3-server[1013]: 2026/09/23 05:13:25 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls lb # [ 48.477564] niks3-server[1013]: 2026/09/23 05:13:25 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete lb # [ 48.483388] niks3-server[1013]: 2026/09/23 05:13:25 INFO Completed upload id=6 worker2 # [ 48.525012] nix-grpc-daemon[873]: time=2026-09-23T05:13:25.089Z level=INFO msg="Upload complete. (282ms)" lb # [ 48.486162] niks3-server[1013]: 2026/09/23 05:13:25 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo worker2 # [ 48.541189] nix-grpc-daemon[771]: ts=2026-09-23T05:13:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=1 worker2 # [ 48.558093] nix-daemon[858]: reaped child process 955, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 5.69 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.20 seconds) client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds) client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top client # copying 3 paths... client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'... client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'... client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.23 seconds) (finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.23 seconds) subtest: repeat is answered Cached by the scheduler without building worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker1 # finding garbage collector roots... worker1 # deleting specified paths... worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker1 # deleting unused links... worker1 # note: hard linking is currently saving 0.0 KiB worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.25 seconds) worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top worker2 # finding garbage collector roots... worker2 # deleting specified paths... worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' worker2 # deleting unused links... worker2 # note: hard linking is currently saving 0.0 KiB worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2 client # don't know how to build these paths: client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker1 # [ 49.820304] nix-grpc-daemon[768]: ts=2026-09-23T05:13:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 49.827462] nix-grpc-daemon[768]: ts=2026-09-23T05:13:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37640 worker2 # [ 49.861095] nix-grpc-daemon[771]: ts=2026-09-23T05:13:26Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=1 client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.30 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.15 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: repeat is answered Cached by the scheduler without building, in 1.19 seconds) subtest: inputs already in the cache are not sent to the scheduler lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2 client # don't know how to build these paths: client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 50.701887] nix-grpc-daemon[771]: ts=2026-09-23T05:13:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45480 worker1 # [ 50.705685] nix-grpc-daemon[768]: ts=2026-09-23T05:13:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37640 worker2 # [ 50.722587] nix-grpc-daemon[771]: ts=2026-09-23T05:13:27Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=0 worker2 # [ 50.730894] nix-grpc-daemon[771]: ts=2026-09-23T05:13:27Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 50.770405] nix-grpc-daemon[771]: ts=2026-09-23T05:13:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 50.787048] nix-grpc-daemon[771]: ts=2026-09-23T05:13:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 50.802809] nix-grpc-daemon[873]: time=2026-09-23T05:13:27.365Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'... worker2 # [ 50.813871] nix-grpc-daemon[771]: ts=2026-09-23T05:13:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 50.829290] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 50.987703] nix-daemon[858]: accepted connection from pid 1005, user nix-grpc-daemon (trusted) worker2 # [ 51.011099] nix-daemon[858]: reaped child process 1008, status = succeeded lb # [ 50.972658] niks3-server[1013]: 2026/09/23 05:13:27 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 51.025613] nix-grpc-daemon[873]: time=2026-09-23T05:13:27.589Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 51.027208] nix-grpc-daemon[873]: time=2026-09-23T05:13:27.589Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)" lb # [ 51.010286] niks3-server[1013]: 2026/09/23 05:13:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign worker2 # [ 51.063735] nix-grpc-daemon[873]: time=2026-09-23T05:13:27.627Z level=INFO msg="Uploading 1 narinfos" lb # [ 51.023811] niks3-server[1013]: 2026/09/23 05:13:27 INFO Signed narinfos id=7 count=1 lb # [ 51.033723] niks3-server[1013]: 2026/09/23 05:13:27 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls lb # [ 51.044245] niks3-server[1013]: 2026/09/23 05:13:27 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst worker2 # [ 51.100809] nix-grpc-daemon[873]: time=2026-09-23T05:13:27.664Z level=INFO msg="Upload complete. (253ms)" lb # [ 51.060123] niks3-server[1013]: 2026/09/23 05:13:27 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete lb # [ 51.068095] niks3-server[1013]: 2026/09/23 05:13:27 INFO Completed upload id=7 lb # [ 51.071529] niks3-server[1013]: 2026/09/23 05:13:27 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo worker2 # [ 51.116157] nix-grpc-daemon[771]: ts=2026-09-23T05:13:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 51.128499] nix-grpc-daemon[771]: ts=2026-09-23T05:13:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 51.137359] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 51.143285] nix-daemon[858]: reaped child process 1003, status = succeeded worker2 # [ 51.154200] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051... client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-t1b worker2 # [ 52.386097] nix-daemon[858]: reaped child process 1012, status = succeeded worker2 # [ 52.503692] nix-daemon[858]: accepted connection from pid 1028, user nix-grpc-daemon (trusted) worker2 # [ 52.531738] nix-daemon[858]: reaped child process 1031, status = succeeded lb # [ 52.490324] niks3-server[1013]: 2026/09/23 05:13:29 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 52.543992] nix-grpc-daemon[873]: time=2026-09-23T05:13:29.105Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 52.545664] nix-grpc-daemon[873]: time=2026-09-23T05:13:29.105Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)" lb # [ 52.533687] niks3-server[1013]: 2026/09/23 05:13:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign worker2 # [ 52.586611] nix-grpc-daemon[873]: time=2026-09-23T05:13:29.150Z level=INFO msg="Uploading 1 narinfos" lb # [ 52.552785] niks3-server[1013]: 2026/09/23 05:13:29 INFO Signed narinfos id=8 count=1 lb # [ 52.556002] niks3-server[1013]: 2026/09/23 05:13:29 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst lb # [ 52.575828] niks3-server[1013]: 2026/09/23 05:13:29 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker2 # [ 52.626676] nix-grpc-daemon[873]: time=2026-09-23T05:13:29.190Z level=INFO msg="Upload complete. (260ms)" lb # [ 52.587755] niks3-server[1013]: 2026/09/23 05:13:29 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls lb # [ 52.593219] niks3-server[1013]: 2026/09/23 05:13:29 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete lb # [ 52.598012] niks3-server[1013]: 2026/09/23 05:13:29 INFO Completed upload id=8 lb # [ 52.601134] niks3-server[1013]: 2026/09/23 05:13:29 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo worker2 # [ 52.642123] nix-grpc-daemon[771]: ts=2026-09-23T05:13:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 worker2 # [ 52.659747] nix-daemon[858]: reaped child process 1010, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.20 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.16 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds) (finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.72 seconds) subtest: outputs read back through any worker carry the cache signature client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2 worker2 # [ 53.251828] nix-grpc-daemon[771]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45480 client # don't know how to build these paths: client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker2 # [ 53.261872] nix-grpc-daemon[771]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 53.286191] nix-grpc-daemon[771]: ts=2026-09-23T05:13:29Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=0 worker2 # [ 53.292623] nix-grpc-daemon[771]: ts=2026-09-23T05:13:29Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 53.298189] nix-grpc-daemon[768]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 53.315492] nix-grpc-daemon[768]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37640 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'... worker1 # [ 53.332211] nix-grpc-daemon[768]: ts=2026-09-23T05:13:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 53.348134] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 53.378612] nix-grpc-daemon[768]: ts=2026-09-23T05:13:29Z level=info event=niks3_push_started pid=862 worker1 # [ 53.555512] nix-grpc-daemon[862]: time=2026-09-23T05:13:30.111Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 53.673192] nix-daemon[835]: accepted connection from pid 875, user nix-grpc-daemon (trusted) worker1 # [ 53.693312] nix-daemon[835]: reaped child process 878, status = succeeded lb # [ 53.657312] niks3-server[1013]: 2026/09/23 05:13:30 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 53.711631] nix-grpc-daemon[862]: time=2026-09-23T05:13:30.266Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 53.715620] nix-grpc-daemon[862]: time=2026-09-23T05:13:30.266Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)" worker1 # [ 53.753199] nix-grpc-daemon[862]: time=2026-09-23T05:13:30.308Z level=INFO msg="Uploading 1 narinfos" lb # [ 53.700140] niks3-server[1013]: 2026/09/23 05:13:30 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst lb # [ 53.713723] niks3-server[1013]: 2026/09/23 05:13:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign lb # [ 53.726377] niks3-server[1013]: 2026/09/23 05:13:30 INFO Signed narinfos id=9 count=1 lb # [ 53.736368] niks3-server[1013]: 2026/09/23 05:13:30 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls lb # [ 53.739728] niks3-server[1013]: 2026/09/23 05:13:30 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete worker1 # [ 53.789105] nix-grpc-daemon[862]: time=2026-09-23T05:13:30.345Z level=INFO msg="Upload complete. (229ms)" lb # [ 53.747152] niks3-server[1013]: 2026/09/23 05:13:30 INFO Completed upload id=9 lb # [ 53.750080] niks3-server[1013]: 2026/09/23 05:13:30 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo worker1 # [ 53.797500] nix-grpc-daemon[768]: ts=2026-09-23T05:13:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 53.811582] nix-grpc-daemon[768]: ts=2026-09-23T05:13:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 53.818215] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 53.823046] nix-daemon[835]: reaped child process 860, status = succeeded client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051... worker1 # [ 53.834246] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) client # farm-a> node-a: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-sig worker1 # [ 55.471359] nix-daemon[835]: reaped child process 882, status = succeeded worker1 # [ 55.570727] nix-daemon[835]: accepted connection from pid 898, user nix-grpc-daemon (trusted) worker1 # [ 55.591501] nix-daemon[835]: reaped child process 901, status = succeeded lb # [ 55.551478] niks3-server[1013]: 2026/09/23 05:13:32 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 55.604391] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.160Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 55.605859] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.160Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)" lb # [ 55.602295] niks3-server[1013]: 2026/09/23 05:13:32 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls worker1 # [ 55.655020] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.210Z level=INFO msg="Uploading 1 narinfos" lb # [ 55.610352] niks3-server[1013]: 2026/09/23 05:13:32 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst lb # [ 55.623415] niks3-server[1013]: 2026/09/23 05:13:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign lb # [ 55.631719] niks3-server[1013]: 2026/09/23 05:13:32 INFO Signed narinfos id=10 count=1 lb # [ 55.640545] niks3-server[1013]: 2026/09/23 05:13:32 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete worker1 # [ 55.688238] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.244Z level=INFO msg="Upload complete. (241ms)" lb # [ 55.646416] niks3-server[1013]: 2026/09/23 05:13:32 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv lb # [ 55.650556] niks3-server[1013]: 2026/09/23 05:13:32 INFO Completed upload id=10 lb # [ 55.658643] niks3-server[1013]: 2026/09/23 05:13:32 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo worker1 # [ 55.700941] nix-grpc-daemon[768]: ts=2026-09-23T05:13:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=1 worker2 # [ 55.709421] nix-grpc-daemon[771]: ts=2026-09-23T05:13:32Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=0 worker1 # [ 55.718298] nix-daemon[835]: reaped child process 880, status = succeeded worker2 # [ 55.715649] nix-grpc-daemon[771]: ts=2026-09-23T05:13:32Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 55.755468] nix-grpc-daemon[768]: ts=2026-09-23T05:13:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 55.771680] nix-grpc-daemon[768]: ts=2026-09-23T05:13:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 55.788079] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.344Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'... worker1 # [ 55.799216] nix-grpc-daemon[768]: ts=2026-09-23T05:13:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 55.814430] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 55.965660] nix-daemon[835]: accepted connection from pid 905, user nix-grpc-daemon (trusted) worker1 # [ 55.984953] nix-daemon[835]: reaped child process 908, status = succeeded lb # [ 55.948663] niks3-server[1013]: 2026/09/23 05:13:32 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 56.000271] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.556Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 56.002123] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.556Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)" lb # [ 55.978084] niks3-server[1013]: 2026/09/23 05:13:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign worker1 # [ 56.027694] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.583Z level=INFO msg="Uploading 1 narinfos" lb # [ 55.991302] niks3-server[1013]: 2026/09/23 05:13:32 INFO Signed narinfos id=11 count=1 lb # [ 56.001952] niks3-server[1013]: 2026/09/23 05:13:32 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls lb # [ 56.014943] niks3-server[1013]: 2026/09/23 05:13:32 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst worker1 # [ 56.065747] nix-grpc-daemon[862]: time=2026-09-23T05:13:32.621Z level=INFO msg="Upload complete. (235ms)" lb # [ 56.023884] niks3-server[1013]: 2026/09/23 05:13:32 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete lb # [ 56.029272] niks3-server[1013]: 2026/09/23 05:13:32 INFO Completed upload id=11 worker1 # [ 56.072934] nix-grpc-daemon[768]: ts=2026-09-23T05:13:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 56.035358] niks3-server[1013]: 2026/09/23 05:13:32 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo worker1 # [ 56.084296] nix-grpc-daemon[768]: ts=2026-09-23T05:13:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 56.097129] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 56.102576] nix-daemon[835]: reaped child process 903, status = succeeded client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051... worker1 # [ 56.113658] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-sig worker1 # [ 57.572866] nix-daemon[835]: reaped child process 912, status = succeeded worker1 # [ 57.696456] nix-daemon[835]: accepted connection from pid 928, user nix-grpc-daemon (trusted) worker1 # [ 57.718006] nix-daemon[835]: reaped child process 931, status = succeeded lb # [ 57.675394] niks3-server[1013]: 2026/09/23 05:13:34 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 57.728520] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.284Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 57.730118] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.284Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)" lb # [ 57.720290] niks3-server[1013]: 2026/09/23 05:13:34 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst worker1 # [ 57.774495] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.330Z level=INFO msg="Uploading 1 narinfos" lb # [ 57.734520] niks3-server[1013]: 2026/09/23 05:13:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign lb # [ 57.744349] niks3-server[1013]: 2026/09/23 05:13:34 INFO Signed narinfos id=12 count=1 lb # [ 57.752433] niks3-server[1013]: 2026/09/23 05:13:34 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv lb # [ 57.763181] niks3-server[1013]: 2026/09/23 05:13:34 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls lb # [ 57.775761] niks3-server[1013]: 2026/09/23 05:13:34 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete worker1 # [ 57.816912] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.373Z level=INFO msg="Upload complete. (287ms)" lb # [ 57.783813] niks3-server[1013]: 2026/09/23 05:13:34 INFO Completed upload id=12 lb # [ 57.787424] niks3-server[1013]: 2026/09/23 05:13:34 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo worker1 # [ 57.829436] nix-grpc-daemon[768]: ts=2026-09-23T05:13:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=1 worker2 # [ 57.838137] nix-grpc-daemon[771]: ts=2026-09-23T05:13:34Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=0 worker2 # [ 57.844730] nix-grpc-daemon[771]: ts=2026-09-23T05:13:34Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 57.851499] nix-daemon[835]: reaped child process 910, status = succeeded worker1 # [ 57.883123] nix-grpc-daemon[768]: ts=2026-09-23T05:13:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 57.900762] nix-grpc-daemon[768]: ts=2026-09-23T05:13:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 57.916634] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.472Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'... worker1 # [ 57.928531] nix-grpc-daemon[768]: ts=2026-09-23T05:13:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 57.942279] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 58.100414] nix-daemon[835]: accepted connection from pid 935, user nix-grpc-daemon (trusted) worker1 # [ 58.120886] nix-daemon[835]: reaped child process 938, status = succeeded lb # [ 58.083244] niks3-server[1013]: 2026/09/23 05:13:34 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 58.136090] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.692Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 58.137763] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.692Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)" worker1 # [ 58.172773] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.728Z level=INFO msg="Uploading 1 narinfos" lb # [ 58.123290] niks3-server[1013]: 2026/09/23 05:13:34 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls lb # [ 58.133989] niks3-server[1013]: 2026/09/23 05:13:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign lb # [ 58.141860] niks3-server[1013]: 2026/09/23 05:13:34 INFO Signed narinfos id=13 count=1 lb # [ 58.147142] niks3-server[1013]: 2026/09/23 05:13:34 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst worker1 # [ 58.202352] nix-grpc-daemon[862]: time=2026-09-23T05:13:34.758Z level=INFO msg="Upload complete. (242ms)" lb # [ 58.160569] niks3-server[1013]: 2026/09/23 05:13:34 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete lb # [ 58.170505] niks3-server[1013]: 2026/09/23 05:13:34 INFO Completed upload id=13 lb # [ 58.172770] niks3-server[1013]: 2026/09/23 05:13:34 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo worker1 # [ 58.213089] nix-grpc-daemon[768]: ts=2026-09-23T05:13:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 58.228341] nix-grpc-daemon[768]: ts=2026-09-23T05:13:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 58.234486] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 58.239458] nix-daemon[835]: reaped child process 933, status = succeeded client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051... worker1 # [ 58.251014] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv lb # [ 58.364822] envoy[900]: conn peer=[2001:db8:1::1]:58594 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9519 tx=4403 ms=5513 lb # [ 58.370078] envoy[900]: conn peer=[2001:db8:1::1]:58596 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=63 lb # [ 58.374841] envoy[900]: conn peer=[2001:db8:1::1]:58598 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1896 ms=1957 client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-sig worker1 # [ 59.767047] nix-daemon[835]: reaped child process 942, status = succeeded worker1 # [ 59.837426] nix-daemon[835]: accepted connection from pid 958, user nix-grpc-daemon (trusted) worker1 # [ 59.858760] nix-daemon[835]: reaped child process 961, status = succeeded lb # [ 59.815666] niks3-server[1013]: 2026/09/23 05:13:36 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 59.868497] nix-grpc-daemon[862]: time=2026-09-23T05:13:36.424Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 59.870026] nix-grpc-daemon[862]: time=2026-09-23T05:13:36.424Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)" lb # [ 59.854112] niks3-server[1013]: 2026/09/23 05:13:36 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst worker1 # [ 59.914486] nix-grpc-daemon[862]: time=2026-09-23T05:13:36.470Z level=INFO msg="Uploading 1 narinfos" lb # [ 59.872672] niks3-server[1013]: 2026/09/23 05:13:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign lb # [ 59.881521] niks3-server[1013]: 2026/09/23 05:13:36 INFO Signed narinfos id=14 count=1 lb # [ 59.888770] niks3-server[1013]: 2026/09/23 05:13:36 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls lb # [ 59.896627] niks3-server[1013]: 2026/09/23 05:13:36 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv lb # [ 59.906489] niks3-server[1013]: 2026/09/23 05:13:36 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete lb # [ 59.910570] niks3-server[1013]: 2026/09/23 05:13:36 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo lb # [ 59.915605] niks3-server[1013]: 2026/09/23 05:13:36 INFO Completed upload id=14 worker1 # [ 59.957655] nix-grpc-daemon[862]: time=2026-09-23T05:13:36.513Z level=INFO msg="Upload complete. (290ms)" worker1 # [ 59.969198] nix-grpc-daemon[768]: ts=2026-09-23T05:13:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=1 worker1 # [ 59.989834] nix-daemon[835]: reaped child process 940, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 6.99 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker1 # [ 60.343089] nix-grpc-daemon[768]: ts=2026-09-23T05:13:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:36766 worker1 # [ 60.386596] nix-grpc-daemon[768]: ts=2026-09-23T05:13:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:36766 client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.25 seconds) client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker2 # [ 60.638105] nix-grpc-daemon[771]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:55708 client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked): client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top worker2 # [ 60.685612] nix-grpc-daemon[771]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:55708 worker2 # [ 60.695296] nix-grpc-daemon[771]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'... worker2 # [ 60.713623] nix-grpc-daemon[771]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'... worker2 # [ 60.730297] nix-grpc-daemon[771]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'... client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.32 seconds) client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top client # finding garbage collector roots... client # removing stale temporary roots file "/nix/var/nix/temproots/906" client # removing stale temporary roots file "/nix/var/nix/temproots/898" client # removing stale temporary roots file "/nix/var/nix/temproots/849" client # removing stale temporary roots file "/nix/var/nix/temproots/926" client # removing stale temporary roots file "/nix/var/nix/temproots/918" client # removing stale temporary roots file "/nix/var/nix/temproots/886" client # removing stale temporary roots file "/nix/var/nix/temproots/941" client # removing stale temporary roots file "/nix/var/nix/temproots/865" client # deleting specified paths... client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' client # deleting unused links... client # note: hard linking is currently saving 0.0 KiB worker1 # [ 61.262294] nix-grpc-daemon[768]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 61.271922] nix-grpc-daemon[768]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 61.285778] nix-grpc-daemon[862]: time=2026-09-23T05:13:37.841Z level=INFO msg="All 1 paths already cached" worker1 # [ 61.294935] nix-grpc-daemon[768]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37640 worker2 # [ 61.304546] nix-grpc-daemon[771]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45480 client # copying 3 paths... worker1 # [ 61.322557] nix-grpc-daemon[768]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37640 client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'... worker2 # [ 61.358569] nix-grpc-daemon[771]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 61.367410] nix-grpc-daemon[771]: ts=2026-09-23T05:13:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_out=232 worker2 # [ 61.380583] nix-grpc-daemon[771]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 61.388333] nix-grpc-daemon[771]: ts=2026-09-23T05:13:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_out=176 worker1 # [ 61.398466] nix-grpc-daemon[768]: ts=2026-09-23T05:13:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 61.406082] nix-grpc-daemon[768]: ts=2026-09-23T05:13:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_out=128 client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'... client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'... client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.70 seconds) (finished: subtest: outputs read back through any worker carry the cache signature, in 8.45 seconds) subtest: two clients wanting the same drv build it once client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 61.528728] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup1.service; invocation ID: 8453cebab28a420cb6a4171e9dbba782 client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds) client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup client # [ 61.722915] nix[994]: don't know how to build these paths: client # [ 61.725043] nix[994]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker2 # [ 61.721634] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 61.737679] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker2 # [ 61.747179] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 61.782993] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 61.802388] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 client # [ 61.820207] nix[994]: copying 1 paths to 'lb:50051'... worker2 # [ 61.819323] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 client # [ 61.822824] nix[994]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'... worker2 # [ 61.837397] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 62.002063] nix-daemon[858]: accepted connection from pid 1051, user nix-grpc-daemon (trusted) worker2 # [ 62.025628] nix-daemon[858]: reaped child process 1054, status = succeeded lb # [ 61.991140] niks3-server[1013]: 2026/09/23 05:13:38 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 62.080494] nix-grpc-daemon[873]: time=2026-09-23T05:13:38.642Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 62.083711] nix-grpc-daemon[873]: time=2026-09-23T05:13:38.642Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)" worker2 # [ 62.125092] nix-grpc-daemon[873]: time=2026-09-23T05:13:38.688Z level=INFO msg="Uploading 1 narinfos" lb # [ 62.071727] niks3-server[1013]: 2026/09/23 05:13:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign lb # [ 62.087581] niks3-server[1013]: 2026/09/23 05:13:38 INFO Signed narinfos id=15 count=1 lb # [ 62.096525] niks3-server[1013]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls lb # [ 62.106700] niks3-server[1013]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst lb # [ 62.117329] niks3-server[1013]: 2026/09/23 05:13:38 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete lb # [ 62.127305] niks3-server[1013]: 2026/09/23 05:13:38 INFO Completed upload id=15 worker2 # [ 62.169721] nix-grpc-daemon[873]: time=2026-09-23T05:13:38.733Z level=INFO msg="Upload complete. (313ms)" lb # [ 62.130789] niks3-server[1013]: 2026/09/23 05:13:38 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo worker2 # [ 62.180463] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_in=432 worker2 # [ 62.200783] nix-grpc-daemon[771]: ts=2026-09-23T05:13:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 62.211462] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 62.218674] nix-daemon[858]: reaped child process 1049, status = succeeded client # [ 62.231334] nix[994]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... worker2 # [ 62.234628] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) client # [ 62.629405] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup. client # Running as unit: dedup2.service; invocation ID: c3302b0c7757426eb33bd07889e5d403 client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds) worker1 # [ 62.858300] nix-grpc-daemon[768]: ts=2026-09-23T05:13:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37640 client # [ 62.868941] nix[1010]: don't know how to build these paths: client # [ 62.869949] nix[1010]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker2 # [ 62.885238] nix-grpc-daemon[771]: ts=2026-09-23T05:13:39Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=0 worker2 # [ 62.892703] nix-grpc-daemon[771]: ts=2026-09-23T05:13:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 client # [ 62.900993] nix[1010]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051... client: must succeed: sleep 2 client: (finished: must succeed: sleep 2, in 2.06 seconds) client: waiting for success: ! systemctl is-active dedup1 dedup2 worker2 # [ 66.268348] nix-daemon[858]: reaped child process 1058, status = succeeded worker2 # [ 66.397647] nix-daemon[858]: accepted connection from pid 1090, user nix-grpc-daemon (trusted) worker2 # [ 66.429455] nix-daemon[858]: reaped child process 1093, status = succeeded lb # [ 66.385852] niks3-server[1013]: 2026/09/23 05:13:42 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 66.444526] nix-grpc-daemon[873]: time=2026-09-23T05:13:43.007Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 66.446096] nix-grpc-daemon[873]: time=2026-09-23T05:13:43.007Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)" lb # [ 66.480534] niks3-server[1013]: 2026/09/23 05:13:43 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst lb # [ 66.492617] niks3-server[1013]: 2026/09/23 05:13:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign worker2 # [ 66.540165] nix-grpc-daemon[873]: time=2026-09-23T05:13:43.102Z level=INFO msg="Uploading 1 narinfos" lb # [ 66.501097] niks3-server[1013]: 2026/09/23 05:13:43 INFO Signed narinfos id=16 count=1 lb # [ 66.507831] niks3-server[1013]: 2026/09/23 05:13:43 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls lb # [ 66.513990] niks3-server[1013]: 2026/09/23 05:13:43 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv lb # [ 66.525682] niks3-server[1013]: 2026/09/23 05:13:43 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete lb # [ 66.534401] niks3-server[1013]: 2026/09/23 05:13:43 INFO Completed upload id=16 worker2 # [ 66.577112] nix-grpc-daemon[873]: time=2026-09-23T05:13:43.141Z level=INFO msg="Upload complete. (328ms)" lb # [ 66.539662] niks3-server[1013]: 2026/09/23 05:13:43 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo worker2 # [ 66.597607] nix-grpc-daemon[771]: ts=2026-09-23T05:13:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker2 # [ 66.624456] nix-grpc-daemon[771]: ts=2026-09-23T05:13:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 worker2 # [ 66.632779] nix-daemon[858]: reaped child process 1056, status = succeeded client # [ 66.645801] systemd[1]: dedup1.service: Deactivated successfully. client # [ 66.653989] systemd[1]: dedup1.service: Consumed 189ms CPU time over 5.123s wall clock time, 41.5M memory peak, 7.4K incoming IP traffic, 8.5K outgoing IP traffic. client # [ 66.684694] systemd[1]: dedup2.service: Deactivated successfully. client # [ 66.687239] systemd[1]: dedup2.service: Consumed 208ms CPU time over 4.059s wall clock time, 33.2M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.18 seconds) client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; } client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds) worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds) worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: two clients wanting the same drv build it once, in 6.26 seconds) subtest: --timeout and --max-silent-time reach the builder worker1 # [ 67.985289] nix-grpc-daemon[768]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37640 worker2 # [ 68.007759] nix-grpc-daemon[771]: ts=2026-09-23T05:13:44Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=0 worker2 # [ 68.014887] nix-grpc-daemon[771]: ts=2026-09-23T05:13:44Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 68.020495] nix-grpc-daemon[768]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 68.042855] nix-grpc-daemon[768]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 68.061014] nix-grpc-daemon[768]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 68.078553] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 68.250451] nix-daemon[835]: accepted connection from pid 986, user nix-grpc-daemon (trusted) worker1 # [ 68.275398] nix-daemon[835]: reaped child process 989, status = succeeded lb # [ 68.238867] niks3-server[1013]: 2026/09/23 05:13:44 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 68.292965] nix-grpc-daemon[862]: time=2026-09-23T05:13:44.848Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 68.294578] nix-grpc-daemon[862]: time=2026-09-23T05:13:44.848Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)" worker1 # [ 68.332695] nix-grpc-daemon[862]: time=2026-09-23T05:13:44.888Z level=INFO msg="Uploading 1 narinfos" lb # [ 68.276540] niks3-server[1013]: 2026/09/23 05:13:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign lb # [ 68.294457] niks3-server[1013]: 2026/09/23 05:13:44 INFO Signed narinfos id=17 count=1 lb # [ 68.302828] niks3-server[1013]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst worker1 # [ 68.361699] nix-grpc-daemon[862]: time=2026-09-23T05:13:44.917Z level=INFO msg="Upload complete. (262ms)" lb # [ 68.319540] niks3-server[1013]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls lb # [ 68.329683] niks3-server[1013]: 2026/09/23 05:13:44 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete worker1 # [ 68.373020] nix-grpc-daemon[768]: ts=2026-09-23T05:13:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_in=424 lb # [ 68.334407] niks3-server[1013]: 2026/09/23 05:13:44 INFO Completed upload id=17 lb # [ 68.337014] niks3-server[1013]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo worker1 # [ 68.384985] nix-grpc-daemon[768]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 68.399792] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 68.406269] nix-daemon[835]: reaped child process 984, status = succeeded lb # [ 68.366595] envoy[900]: conn peer=[2001:db8:1::1]:57748 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9541 tx=4369 ms=6729 lb # [ 68.371928] envoy[900]: conn peer=[2001:db8:1::1]:34360 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2889 ms=199 worker1 # [ 68.421349] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) lb # [ 68.377838] envoy[900]: conn peer=[2001:db8:1::1]:34388 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=65 lb # [ 68.382794] envoy[900]: conn peer=[2001:db8:1::1]:34378 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=375 ms=86 lb # [ 68.387199] envoy[900]: conn peer=[2001:db8:1::1]:34370 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=409 ms=106 lb # [ 68.391176] envoy[900]: conn peer=[2001:db8:1::1]:34394 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1541 ms=4890 lb # [ 68.395000] envoy[900]: conn peer=[2001:db8:1::1]:34396 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1198 ms=3801 worker1 # [ 74.438971] nix-grpc-daemon[768]: ts=2026-09-23T05:13:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=0 worker1 # [ 74.477304] nix-daemon[835]: reaped child process 991, status = succeeded worker1 # [ 74.479955] nix-daemon[835]: reaped child process 993, status = succeeded worker1 # [ 74.686808] nix-grpc-daemon[768]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37640 worker2 # [ 74.705460] nix-grpc-daemon[771]: ts=2026-09-23T05:13:51Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=0 worker2 # [ 74.714113] nix-grpc-daemon[771]: ts=2026-09-23T05:13:51Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 74.724497] nix-grpc-daemon[771]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 74.742000] nix-grpc-daemon[771]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 74.759323] nix-grpc-daemon[771]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 74.776153] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 74.946590] nix-daemon[858]: accepted connection from pid 1110, user nix-grpc-daemon (trusted) worker2 # [ 74.970162] nix-daemon[858]: reaped child process 1113, status = succeeded lb # [ 74.926529] niks3-server[1013]: 2026/09/23 05:13:51 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 74.982503] nix-grpc-daemon[873]: time=2026-09-23T05:13:51.545Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 74.984166] nix-grpc-daemon[873]: time=2026-09-23T05:13:51.545Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)" worker2 # [ 75.019963] nix-grpc-daemon[873]: time=2026-09-23T05:13:51.583Z level=INFO msg="Uploading 1 narinfos" lb # [ 74.967300] niks3-server[1013]: 2026/09/23 05:13:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign lb # [ 74.983012] niks3-server[1013]: 2026/09/23 05:13:51 INFO Signed narinfos id=18 count=1 lb # [ 74.991932] niks3-server[1013]: 2026/09/23 05:13:51 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst lb # [ 75.001639] niks3-server[1013]: 2026/09/23 05:13:51 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls lb # [ 75.014225] niks3-server[1013]: 2026/09/23 05:13:51 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete worker2 # [ 75.062042] nix-grpc-daemon[873]: time=2026-09-23T05:13:51.625Z level=INFO msg="Upload complete. (266ms)" lb # [ 75.021726] niks3-server[1013]: 2026/09/23 05:13:51 INFO Completed upload id=18 lb # [ 75.024696] niks3-server[1013]: 2026/09/23 05:13:51 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo worker2 # [ 75.072406] nix-grpc-daemon[771]: ts=2026-09-23T05:13:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 75.093044] nix-grpc-daemon[771]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 75.104810] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 75.108244] nix-daemon[858]: reaped child process 1108, status = succeeded worker2 # [ 75.123659] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) lb # [ 78.366740] envoy[900]: conn peer=[2001:db8:1::1]:34406 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3014 tx=953 ms=6475 worker2 # [ 80.945955] nix-grpc-daemon[771]: ts=2026-09-23T05:13:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=5 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=0 worker2 # [ 80.986738] nix-daemon[858]: reaped child process 1115, status = succeeded worker2 # [ 80.993044] nix-daemon[858]: reaped child process 1117, status = succeeded (finished: subtest: --timeout and --max-silent-time reach the builder, in 13.44 seconds) subtest: the first client leaving does not fail the second client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 81.249947] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share1.service; invocation ID: bbf232037aa442619bdfc1253d113f3c client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds) client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share client # [ 81.474623] nix[1080]: don't know how to build these paths: client # [ 81.475740] nix[1080]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker1 # [ 81.473989] nix-grpc-daemon[768]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37640 worker2 # [ 81.486345] nix-grpc-daemon[771]: ts=2026-09-23T05:13:58Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker1 # [ 81.490207] nix-grpc-daemon[768]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 worker2 # [ 81.492930] nix-grpc-daemon[771]: ts=2026-09-23T05:13:58Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 81.507083] nix-grpc-daemon[768]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37640 client # [ 81.520690] nix[1080]: copying 1 paths to 'lb:50051'... client # [ 81.524136] nix[1080]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'... worker1 # [ 81.524312] nix-grpc-daemon[768]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 81.539912] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 81.704480] nix-daemon[835]: accepted connection from pid 1016, user nix-grpc-daemon (trusted) worker1 # [ 81.724284] nix-daemon[835]: reaped child process 1019, status = succeeded lb # [ 81.690156] niks3-server[1013]: 2026/09/23 05:13:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 81.743089] nix-grpc-daemon[862]: time=2026-09-23T05:13:58.298Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 81.745270] nix-grpc-daemon[862]: time=2026-09-23T05:13:58.298Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)" lb # [ 81.732906] niks3-server[1013]: 2026/09/23 05:13:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign worker1 # [ 81.788312] nix-grpc-daemon[862]: time=2026-09-23T05:13:58.344Z level=INFO msg="Uploading 1 narinfos" lb # [ 81.748089] niks3-server[1013]: 2026/09/23 05:13:58 INFO Signed narinfos id=19 count=1 lb # [ 81.750648] niks3-server[1013]: 2026/09/23 05:13:58 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst lb # [ 81.764787] niks3-server[1013]: 2026/09/23 05:13:58 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls lb # [ 81.767627] niks3-server[1013]: 2026/09/23 05:13:58 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete worker1 # [ 81.807296] nix-grpc-daemon[862]: time=2026-09-23T05:13:58.363Z level=INFO msg="Upload complete. (249ms)" lb # [ 81.774000] niks3-server[1013]: 2026/09/23 05:13:58 INFO Completed upload id=19 lb # [ 81.777726] niks3-server[1013]: 2026/09/23 05:13:58 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo worker1 # [ 81.816611] nix-grpc-daemon[768]: ts=2026-09-23T05:13:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_in=432 worker1 # [ 81.834847] nix-grpc-daemon[768]: ts=2026-09-23T05:13:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 81.841768] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 81.846938] nix-daemon[835]: reaped child process 1014, status = succeeded worker1 # [ 81.860182] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) client # [ 81.859309] nix[1080]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client # [ 82.364979] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share. client # Running as unit: share2.service; invocation ID: 3404ed2868e1421f9145ecc368877776 client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.12 seconds) worker2 # [ 82.587817] nix-grpc-daemon[771]: ts=2026-09-23T05:13:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:45480 client # [ 82.595992] nix[1096]: don't know how to build these paths: client # [ 82.598071] nix[1096]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker2 # [ 82.605854] nix-grpc-daemon[771]: ts=2026-09-23T05:13:59Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=0 worker1 # [ 82.611593] nix-grpc-daemon[768]: ts=2026-09-23T05:13:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 client # [ 82.620560] nix[1096]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051... client: must succeed: sleep 2; systemctl kill -s INT share1 client # [ 85.746066] systemd[1]: share1.service: Sent signal SIGINT to main process 1080 (nix) on client request. client # [ 85.757094] nix[1080]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 85.771259] nix[1080]: error: interrupted by the user client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds) client: waiting for success: ! systemctl is-active share1 client # [ 85.839854] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE client # [ 85.842733] systemd[1]: share1.service: Failed with result 'exit-code'. client # [ 85.845992] systemd[1]: share1.service: Consumed 189ms CPU time over 4.593s wall clock time, 39.4M memory peak, 6.2K incoming IP traffic, 8.2K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share1, in 1.12 seconds) client: waiting for success: ! systemctl is-active share2 worker1 # [ 87.386270] nix-daemon[835]: reaped child process 1023, status = succeeded worker1 # [ 87.461573] nix-daemon[835]: accepted connection from pid 1057, user nix-grpc-daemon (trusted) worker1 # [ 87.482089] nix-daemon[835]: reaped child process 1060, status = succeeded lb # [ 87.445906] niks3-server[1013]: 2026/09/23 05:14:04 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 87.502736] nix-grpc-daemon[862]: time=2026-09-23T05:14:04.055Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 87.504519] nix-grpc-daemon[862]: time=2026-09-23T05:14:04.056Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)" lb # [ 87.490588] niks3-server[1013]: 2026/09/23 05:14:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign worker1 # [ 87.542350] nix-grpc-daemon[862]: time=2026-09-23T05:14:04.098Z level=INFO msg="Uploading 1 narinfos" lb # [ 87.505740] niks3-server[1013]: 2026/09/23 05:14:04 INFO Signed narinfos id=20 count=1 lb # [ 87.515938] niks3-server[1013]: 2026/09/23 05:14:04 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst lb # [ 87.529677] niks3-server[1013]: 2026/09/23 05:14:04 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv lb # [ 87.538997] niks3-server[1013]: 2026/09/23 05:14:04 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls worker1 # [ 87.581336] nix-grpc-daemon[862]: time=2026-09-23T05:14:04.137Z level=INFO msg="Upload complete. (294ms)" lb # [ 87.544598] niks3-server[1013]: 2026/09/23 05:14:04 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete lb # [ 87.550082] niks3-server[1013]: 2026/09/23 05:14:04 INFO Completed upload id=20 lb # [ 87.551775] niks3-server[1013]: 2026/09/23 05:14:04 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo worker1 # [ 87.597164] nix-grpc-daemon[768]: ts=2026-09-23T05:14:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker1 # [ 87.611912] nix-grpc-daemon[768]: ts=2026-09-23T05:14:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 worker1 # [ 87.617753] nix-daemon[835]: reaped child process 1021, status = succeeded client # [ 87.670564] systemd[1]: share2.service: Deactivated successfully. client # [ 87.675730] systemd[1]: share2.service: Consumed 201ms CPU time over 5.307s wall clock time, 33.3M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic. client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds) client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; } client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.06 seconds) (finished: subtest: the first client leaving does not fail the second, in 7.16 seconds) subtest: build hook: nix-daemon with builders = grpc://lb client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log lb # [ 88.372441] envoy[900]: conn peer=[2001:db8:1::1]:47710 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3028 tx=953 ms=6320 lb # [ 88.372680] envoy[900]: conn peer=[2001:db8:1::1]:58040 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3003 tx=811 ms=4305 lb # [ 88.372846] envoy[900]: conn peer=[2001:db8:1::1]:58052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1198 ms=5062 client # [ 88.482364] systemd[1]: Started Nix Daemon. client # [ 88.605389] nix-daemon[1148]: accepted connection from pid 1145, user root (trusted) worker2 # [ 88.759528] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:45480 worker1 # [ 88.763900] nix-grpc-daemon[768]: ts=2026-09-23T05:14:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37640 worker2 # [ 88.782937] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=0 worker2 # [ 88.790574] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 88.801367] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 88.812697] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 88.826924] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 88.842109] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 88.999142] nix-daemon[858]: accepted connection from pid 1162, user nix-grpc-daemon (trusted) worker2 # [ 89.020388] nix-daemon[858]: reaped child process 1165, status = succeeded lb # [ 88.982358] niks3-server[1013]: 2026/09/23 05:14:05 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 89.034885] nix-grpc-daemon[873]: time=2026-09-23T05:14:05.598Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 89.036677] nix-grpc-daemon[873]: time=2026-09-23T05:14:05.598Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)" lb # [ 89.016172] niks3-server[1013]: 2026/09/23 05:14:05 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls worker2 # [ 89.067010] nix-grpc-daemon[873]: time=2026-09-23T05:14:05.630Z level=INFO msg="Uploading 1 narinfos" lb # [ 89.030185] niks3-server[1013]: 2026/09/23 05:14:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign lb # [ 89.040511] niks3-server[1013]: 2026/09/23 05:14:05 INFO Signed narinfos id=21 count=1 lb # [ 89.043079] niks3-server[1013]: 2026/09/23 05:14:05 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst worker2 # [ 89.098104] nix-grpc-daemon[873]: time=2026-09-23T05:14:05.662Z level=INFO msg="Upload complete. (239ms)" lb # [ 89.056571] niks3-server[1013]: 2026/09/23 05:14:05 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete lb # [ 89.060221] niks3-server[1013]: 2026/09/23 05:14:05 INFO Completed upload id=21 lb # [ 89.063012] niks3-server[1013]: 2026/09/23 05:14:05 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo worker2 # [ 89.108357] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 89.124350] nix-grpc-daemon[771]: ts=2026-09-23T05:14:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 89.131533] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 89.137846] nix-daemon[858]: reaped child process 1160, status = succeeded worker2 # [ 89.151145] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 90.422837] nix-daemon[858]: reaped child process 1169, status = succeeded worker2 # [ 90.552891] nix-daemon[858]: accepted connection from pid 1185, user nix-grpc-daemon (trusted) worker2 # [ 90.572630] nix-daemon[858]: reaped child process 1188, status = succeeded lb # [ 90.538137] niks3-server[1013]: 2026/09/23 05:14:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 90.591978] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.152Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 90.593997] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.152Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)" lb # [ 90.577083] niks3-server[1013]: 2026/09/23 05:14:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign worker2 # [ 90.626723] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.190Z level=INFO msg="Uploading 1 narinfos" lb # [ 90.591993] niks3-server[1013]: 2026/09/23 05:14:07 INFO Signed narinfos id=22 count=1 lb # [ 90.598619] niks3-server[1013]: 2026/09/23 05:14:07 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst lb # [ 90.611583] niks3-server[1013]: 2026/09/23 05:14:07 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv lb # [ 90.617350] niks3-server[1013]: 2026/09/23 05:14:07 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete lb # [ 90.620717] niks3-server[1013]: 2026/09/23 05:14:07 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls worker2 # [ 90.666119] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.229Z level=INFO msg="Upload complete. (298ms)" lb # [ 90.627900] niks3-server[1013]: 2026/09/23 05:14:07 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo lb # [ 90.631495] niks3-server[1013]: 2026/09/23 05:14:07 INFO Completed upload id=22 worker2 # [ 90.678177] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=1 worker2 # [ 90.697604] nix-daemon[858]: reaped child process 1167, status = succeeded worker1 # [ 90.725225] nix-grpc-daemon[768]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 90.741082] nix-grpc-daemon[768]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'... worker2 # [ 90.782807] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 90.790139] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_out=128 worker1 # [ 91.034342] nix-grpc-daemon[768]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 91.042770] nix-grpc-daemon[768]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 91.050624] nix-grpc-daemon[862]: time=2026-09-23T05:14:07.604Z level=INFO msg="All 1 paths already cached" worker2 # [ 91.062468] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=0 worker2 # [ 91.070128] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 91.079139] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 91.090861] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 91.105803] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.669Z level=INFO msg="All 1 paths already cached" worker2 # [ 91.115898] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 91.130318] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 91.284340] nix-daemon[858]: accepted connection from pid 1194, user nix-grpc-daemon (trusted) worker2 # [ 91.307624] nix-daemon[858]: reaped child process 1197, status = succeeded lb # [ 91.268751] niks3-server[1013]: 2026/09/23 05:14:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 91.320757] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.884Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 91.322368] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.884Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)" lb # [ 91.300384] niks3-server[1013]: 2026/09/23 05:14:07 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst worker2 # [ 91.354420] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.918Z level=INFO msg="Uploading 1 narinfos" lb # [ 91.313785] niks3-server[1013]: 2026/09/23 05:14:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign lb # [ 91.325062] niks3-server[1013]: 2026/09/23 05:14:07 INFO Signed narinfos id=23 count=1 lb # [ 91.327777] niks3-server[1013]: 2026/09/23 05:14:07 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls lb # [ 91.337368] niks3-server[1013]: 2026/09/23 05:14:07 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete lb # [ 91.347141] niks3-server[1013]: 2026/09/23 05:14:07 INFO Completed upload id=23 worker2 # [ 91.388917] nix-grpc-daemon[873]: time=2026-09-23T05:14:07.952Z level=INFO msg="Upload complete. (241ms)" lb # [ 91.350358] niks3-server[1013]: 2026/09/23 05:14:07 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo worker2 # [ 91.400117] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_in=624 worker2 # [ 91.408303] nix-grpc-daemon[771]: ts=2026-09-23T05:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 91.420966] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 91.425988] nix-daemon[858]: reaped child process 1192, status = succeeded worker2 # [ 91.439620] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 92.528581] nix-daemon[858]: reaped child process 1201, status = succeeded worker2 # [ 92.622243] nix-daemon[858]: accepted connection from pid 1218, user nix-grpc-daemon (trusted) worker2 # [ 92.643421] nix-daemon[858]: reaped child process 1221, status = succeeded lb # [ 92.599815] niks3-server[1013]: 2026/09/23 05:14:09 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 92.652489] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.215Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 92.654037] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.215Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)" lb # [ 92.648828] niks3-server[1013]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst worker2 # [ 92.704993] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.268Z level=INFO msg="Uploading 1 narinfos" lb # [ 92.664960] niks3-server[1013]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv lb # [ 92.674302] niks3-server[1013]: 2026/09/23 05:14:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign lb # [ 92.680778] niks3-server[1013]: 2026/09/23 05:14:09 INFO Signed narinfos id=24 count=1 worker2 # [ 92.731382] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.295Z level=INFO msg="Upload complete. (281ms)" lb # [ 92.689890] niks3-server[1013]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls lb # [ 92.698871] niks3-server[1013]: 2026/09/23 05:14:09 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete lb # [ 92.704070] niks3-server[1013]: 2026/09/23 05:14:09 INFO Completed upload id=24 worker2 # [ 92.745279] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=1 worker1 # [ 92.753727] nix-grpc-daemon[768]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37640 lb # [ 92.708310] niks3-server[1013]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo worker1 # [ 92.761861] nix-grpc-daemon[768]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'... worker2 # [ 92.766213] nix-daemon[858]: reaped child process 1199, status = succeeded worker2 # [ 92.799662] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 92.807664] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_out=176 worker1 # [ 93.044215] nix-grpc-daemon[768]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:37640 worker2 # [ 93.047851] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 93.058567] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.622Z level=INFO msg="All 2 paths already cached" worker2 # [ 93.080281] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=0 worker2 # [ 93.087920] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 93.096868] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 93.108312] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 93.122985] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.686Z level=INFO msg="All 2 paths already cached" worker2 # [ 93.133686] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 93.147969] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 93.320941] nix-daemon[858]: accepted connection from pid 1227, user nix-grpc-daemon (trusted) worker2 # [ 93.341690] nix-daemon[858]: reaped child process 1230, status = succeeded lb # [ 93.307044] niks3-server[1013]: 2026/09/23 05:14:09 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 93.359189] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.922Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 93.360967] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.922Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)" worker2 # [ 93.387415] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.951Z level=INFO msg="Uploading 1 narinfos" lb # [ 93.338307] niks3-server[1013]: 2026/09/23 05:14:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign lb # [ 93.355784] niks3-server[1013]: 2026/09/23 05:14:09 INFO Signed narinfos id=25 count=1 lb # [ 93.363099] niks3-server[1013]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst lb # [ 93.373214] niks3-server[1013]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls lb # [ 93.381344] niks3-server[1013]: 2026/09/23 05:14:09 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete worker2 # [ 93.431168] nix-grpc-daemon[873]: time=2026-09-23T05:14:09.995Z level=INFO msg="Upload complete. (264ms)" lb # [ 93.391051] niks3-server[1013]: 2026/09/23 05:14:09 INFO Completed upload id=25 lb # [ 93.396774] niks3-server[1013]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo worker2 # [ 93.440307] nix-grpc-daemon[771]: ts=2026-09-23T05:14:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_in=752 worker2 # [ 93.456424] nix-grpc-daemon[771]: ts=2026-09-23T05:14:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 93.466434] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 93.472341] nix-daemon[858]: reaped child process 1225, status = succeeded worker2 # [ 93.488849] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 94.720787] nix-daemon[858]: reaped child process 1234, status = succeeded worker2 # [ 94.817903] nix-daemon[858]: accepted connection from pid 1250, user nix-grpc-daemon (trusted) worker2 # [ 94.839934] nix-daemon[858]: reaped child process 1253, status = succeeded lb # [ 94.802330] niks3-server[1013]: 2026/09/23 05:14:11 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 94.856208] nix-grpc-daemon[873]: time=2026-09-23T05:14:11.417Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 94.857764] nix-grpc-daemon[873]: time=2026-09-23T05:14:11.418Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)" lb # [ 94.848662] niks3-server[1013]: 2026/09/23 05:14:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign worker2 # [ 94.899415] nix-grpc-daemon[873]: time=2026-09-23T05:14:11.463Z level=INFO msg="Uploading 1 narinfos" lb # [ 94.860568] niks3-server[1013]: 2026/09/23 05:14:11 INFO Signed narinfos id=26 count=1 lb # [ 94.868422] niks3-server[1013]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst lb # [ 94.879290] niks3-server[1013]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls worker2 # [ 94.931816] nix-grpc-daemon[873]: time=2026-09-23T05:14:11.495Z level=INFO msg="Upload complete. (232ms)" lb # [ 94.890860] niks3-server[1013]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv lb # [ 94.899221] niks3-server[1013]: 2026/09/23 05:14:11 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete lb # [ 94.903903] niks3-server[1013]: 2026/09/23 05:14:11 INFO Completed upload id=26 worker2 # [ 94.942628] nix-grpc-daemon[771]: ts=2026-09-23T05:14:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=1 lb # [ 94.908254] niks3-server[1013]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo worker2 # [ 94.963388] nix-grpc-daemon[771]: ts=2026-09-23T05:14:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 94.968001] nix-daemon[858]: reaped child process 1232, status = succeeded worker1 # [ 94.978971] nix-grpc-daemon[768]: ts=2026-09-23T05:14:11Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 94.993275] nix-grpc-daemon[768]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'... worker1 # [ 95.010183] nix-grpc-daemon[768]: ts=2026-09-23T05:14:11Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_out=232 client # [ 95.135161] nix-daemon[1148]: reaped child process 1150, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.82 seconds) client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds) client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds) client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; } client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds) (finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.94 seconds) subtest: an input only one worker has still reaches the builder client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only client # this derivation will be built: client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.30 seconds) client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.13 seconds) worker1: must succeed: nix-store --import < /tmp/shared/inp.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds) worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2 client # [ 96.111297] nix-daemon[1148]: accepted connection from pid 1230, user root (trusted) client # this derivation will be built: client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker1 # [ 96.263882] nix-grpc-daemon[768]: ts=2026-09-23T05:14:12Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:37640 worker2 # [ 96.267166] nix-grpc-daemon[771]: ts=2026-09-23T05:14:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 client # copying 1 paths... client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'... worker1 # [ 96.283367] nix-grpc-daemon[768]: ts=2026-09-23T05:14:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 96.298624] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 96.467905] nix-daemon[835]: accepted connection from pid 1072, user nix-grpc-daemon (trusted) worker1 # [ 96.491188] nix-daemon[835]: reaped child process 1075, status = succeeded lb # [ 96.450900] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 96.505132] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.060Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 96.506754] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.060Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)" lb # [ 96.488309] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst worker1 # [ 96.544276] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.100Z level=INFO msg="Uploading 1 narinfos" lb # [ 96.503018] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign lb # [ 96.512303] niks3-server[1013]: 2026/09/23 05:14:13 INFO Signed narinfos id=27 count=1 lb # [ 96.519005] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls lb # [ 96.526832] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete lb # [ 96.541194] niks3-server[1013]: 2026/09/23 05:14:13 INFO Completed upload id=27 worker1 # [ 96.583838] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.140Z level=INFO msg="Upload complete. (269ms)" lb # [ 96.546755] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo worker1 # [ 96.596358] nix-grpc-daemon[768]: ts=2026-09-23T05:14:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_in=120 worker2 # [ 96.606922] nix-grpc-daemon[771]: ts=2026-09-23T05:14:13Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=0 worker1 # [ 96.614577] nix-daemon[835]: reaped child process 1070, status = succeeded worker2 # [ 96.613550] nix-grpc-daemon[771]: ts=2026-09-23T05:14:13Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 96.655339] nix-grpc-daemon[768]: ts=2026-09-23T05:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 96.671261] nix-grpc-daemon[768]: ts=2026-09-23T05:14:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37640 client # copying 2 paths to 'lb:50051'... client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'... worker1 # [ 96.690337] nix-grpc-daemon[768]: ts=2026-09-23T05:14:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'... worker1 # [ 96.704147] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 96.861640] nix-daemon[835]: accepted connection from pid 1079, user nix-grpc-daemon (trusted) worker1 # [ 96.885766] nix-daemon[835]: reaped child process 1082, status = succeeded lb # [ 96.841579] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 96.850959] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 96.904111] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.458Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker1 # [ 96.907266] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.458Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)" worker1 # [ 96.909815] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.458Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)" lb # [ 96.884314] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst lb # [ 96.902749] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls lb # [ 96.914220] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst lb # [ 96.926574] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign worker1 # [ 96.968710] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.523Z level=INFO msg="Uploading 3 narinfos" lb # [ 96.933624] niks3-server[1013]: 2026/09/23 05:14:13 INFO Signed narinfos id=28 count=1 lb # [ 96.935746] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign lb # [ 96.944227] niks3-server[1013]: 2026/09/23 05:14:13 INFO Signed narinfos id=29 count=2 lb # [ 96.948921] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls lb # [ 96.961704] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo lb # [ 96.972553] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete lb # [ 96.981293] niks3-server[1013]: 2026/09/23 05:14:13 INFO Completed upload id=28 lb # [ 96.982926] niks3-server[1013]: 2026/09/23 05:14:13 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete worker1 # [ 97.023970] nix-grpc-daemon[862]: time=2026-09-23T05:14:13.580Z level=INFO msg="Upload complete. (300ms)" lb # [ 96.986078] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo lb # [ 96.989778] niks3-server[1013]: 2026/09/23 05:14:13 INFO Completed upload id=29 lb # [ 96.994031] niks3-server[1013]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo worker1 # [ 97.035761] nix-grpc-daemon[768]: ts=2026-09-23T05:14:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=2 nar_bytes_in=984 worker1 # [ 97.050867] nix-grpc-daemon[768]: ts=2026-09-23T05:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 97.065993] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 97.073716] nix-daemon[835]: reaped child process 1077, status = succeeded client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051... worker1 # [ 97.089437] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 98.388895] nix-daemon[835]: reaped child process 1086, status = succeeded lb # [ 98.373351] envoy[900]: conn peer=[2001:db8:1::1]:58066 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=29 worker1 # [ 98.482178] nix-daemon[835]: accepted connection from pid 1102, user nix-grpc-daemon (trusted) worker1 # [ 98.505814] nix-daemon[835]: reaped child process 1105, status = succeeded lb # [ 98.378692] envoy[900]: conn peer=[2001:db8:1::1]:58064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3414 tx=1944 ms=2050 lb # [ 98.378879] envoy[900]: conn peer=[2001:db8:1::1]:50684 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3888 tx=2158 ms=1791 lb # [ 98.379017] envoy[900]: conn peer=[2001:db8:1::1]:50686 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=29 lb # [ 98.379135] envoy[900]: conn peer=[2001:db8:1::1]:50694 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4378 tx=2390 ms=2023 lb # [ 98.379255] envoy[900]: conn peer=[2001:db8:1::1]:50696 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=407 ms=92 lb # [ 98.462735] niks3-server[1013]: 2026/09/23 05:14:15 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 98.540648] nix-grpc-daemon[862]: time=2026-09-23T05:14:15.094Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 98.525108] niks3-server[1013]: 2026/09/23 05:14:15 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls worker1 # [ 98.577578] nix-grpc-daemon[862]: time=2026-09-23T05:14:15.133Z level=INFO msg="Uploading 1 narinfos" lb # [ 98.535781] niks3-server[1013]: 2026/09/23 05:14:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign lb # [ 98.544252] niks3-server[1013]: 2026/09/23 05:14:15 INFO Signed narinfos id=30 count=1 lb # [ 98.549118] niks3-server[1013]: 2026/09/23 05:14:15 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker1 # [ 98.601187] nix-grpc-daemon[862]: time=2026-09-23T05:14:15.157Z level=INFO msg="Upload complete. (293ms)" lb # [ 98.560916] niks3-server[1013]: 2026/09/23 05:14:15 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete lb # [ 98.564147] niks3-server[1013]: 2026/09/23 05:14:15 INFO Completed upload id=30 lb # [ 98.567461] niks3-server[1013]: 2026/09/23 05:14:15 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo client # copying 1 paths... worker1 # [ 98.613183] nix-grpc-daemon[768]: ts=2026-09-23T05:14:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=1 worker1 # [ 98.634667] nix-grpc-daemon[768]: ts=2026-09-23T05:14:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 98.640575] nix-daemon[835]: reaped child process 1084, status = succeeded client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'... worker2 # [ 98.667087] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 98.679707] nix-grpc-daemon[771]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'... worker2 # [ 98.695345] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_out=120 client # [ 98.821430] nix-daemon[1148]: reaped child process 1233, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.91 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2 client # [ 98.990472] nix-daemon[1148]: accepted connection from pid 1252, user root (trusted) client # this derivation will be built: client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker1 # [ 99.140126] nix-grpc-daemon[768]: ts=2026-09-23T05:14:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:37640 client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 99.148803] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 99.173877] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=0 worker2 # [ 99.181680] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 99.191046] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 99.201951] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'... worker2 # [ 99.220930] nix-grpc-daemon[771]: ts=2026-09-23T05:14:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 99.236165] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 99.256113] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5) worker2 # [ 99.289410] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 2/5) worker2 # [ 99.354392] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 309 ms (attempt 3/5) worker2 # [ 99.665745] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 300 ms (attempt 4/5) worker2 # [ 99.970973] nix-grpc-daemon[771]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 99.980291] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 99.987855] nix-grpc-daemon[771]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'... worker2 # [ 100.154582] nix-daemon[858]: accepted connection from pid 1270, user nix-grpc-daemon (trusted) worker2 # [ 100.176293] nix-daemon[858]: reaped child process 1273, status = succeeded lb # [ 100.143408] niks3-server[1013]: 2026/09/23 05:14:16 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 100.196478] nix-grpc-daemon[873]: time=2026-09-23T05:14:16.759Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 100.198199] nix-grpc-daemon[873]: time=2026-09-23T05:14:16.759Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)" lb # [ 100.175512] niks3-server[1013]: 2026/09/23 05:14:16 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst lb # [ 100.190053] niks3-server[1013]: 2026/09/23 05:14:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign worker2 # [ 100.241520] nix-grpc-daemon[873]: time=2026-09-23T05:14:16.805Z level=INFO msg="Uploading 1 narinfos" lb # [ 100.203894] niks3-server[1013]: 2026/09/23 05:14:16 INFO Signed narinfos id=31 count=1 lb # [ 100.211527] niks3-server[1013]: 2026/09/23 05:14:16 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls worker2 # [ 100.268059] nix-grpc-daemon[873]: time=2026-09-23T05:14:16.832Z level=INFO msg="Upload complete. (266ms)" lb # [ 100.226927] niks3-server[1013]: 2026/09/23 05:14:16 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete lb # [ 100.235410] niks3-server[1013]: 2026/09/23 05:14:16 INFO Completed upload id=31 lb # [ 100.239478] niks3-server[1013]: 2026/09/23 05:14:16 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo worker2 # [ 100.280298] nix-grpc-daemon[771]: ts=2026-09-23T05:14:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=1 paths=1 nar_bytes_in=568 worker2 # [ 100.295774] nix-grpc-daemon[771]: ts=2026-09-23T05:14:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 100.308470] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) worker2 # [ 100.324002] nix-daemon[858]: reaped child process 1263, status = succeeded worker2 # [ 100.328402] nix-grpc-daemon[771]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'... client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051... worker2 # [ 100.344958] nix-daemon[858]: accepted connection from pid 771, user nix-grpc-daemon (trusted) client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 101.810804] nix-daemon[858]: reaped child process 1278, status = succeeded worker2 # [ 101.929389] nix-daemon[858]: accepted connection from pid 1294, user nix-grpc-daemon (trusted) worker2 # [ 101.951484] nix-daemon[858]: reaped child process 1297, status = succeeded lb # [ 101.913664] niks3-server[1013]: 2026/09/23 05:14:18 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 101.966042] nix-grpc-daemon[873]: time=2026-09-23T05:14:18.529Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 101.944078] niks3-server[1013]: 2026/09/23 05:14:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign worker2 # [ 102.002470] nix-grpc-daemon[873]: time=2026-09-23T05:14:18.566Z level=INFO msg="Uploading 1 narinfos" lb # [ 101.960230] niks3-server[1013]: 2026/09/23 05:14:18 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv lb # [ 101.975027] niks3-server[1013]: 2026/09/23 05:14:18 INFO Signed narinfos id=32 count=1 lb # [ 101.982967] niks3-server[1013]: 2026/09/23 05:14:18 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls worker2 # [ 102.037742] nix-grpc-daemon[873]: time=2026-09-23T05:14:18.601Z level=INFO msg="Upload complete. (280ms)" lb # [ 101.996428] niks3-server[1013]: 2026/09/23 05:14:18 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete lb # [ 102.005820] niks3-server[1013]: 2026/09/23 05:14:18 INFO Completed upload id=32 lb # [ 102.010021] niks3-server[1013]: 2026/09/23 05:14:18 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo worker2 # [ 102.053294] nix-grpc-daemon[771]: ts=2026-09-23T05:14:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=1 worker2 # [ 102.070479] nix-daemon[858]: reaped child process 1275, status = succeeded client # copying 1 paths... worker1 # [ 102.098518] nix-grpc-daemon[768]: ts=2026-09-23T05:14:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 102.112732] nix-grpc-daemon[768]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'... client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'... worker2 # [ 102.158829] nix-grpc-daemon[771]: ts=2026-09-23T05:14:18Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 102.166119] nix-grpc-daemon[771]: ts=2026-09-23T05:14:18Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_out=120 client # [ 102.252319] nix-daemon[1148]: reaped child process 1255, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.43 seconds) client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2 client # [ 102.427273] nix-daemon[1148]: accepted connection from pid 1273, user root (trusted) client # this derivation will be built: client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker1 # [ 102.583053] nix-grpc-daemon[768]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:37640 client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'... worker2 # [ 102.588409] nix-grpc-daemon[771]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 102.600615] nix-grpc-daemon[873]: time=2026-09-23T05:14:19.164Z level=INFO msg="All 1 paths already cached" worker2 # [ 102.617682] nix-grpc-daemon[771]: ts=2026-09-23T05:14:19Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=0 worker2 # [ 102.624610] nix-grpc-daemon[771]: ts=2026-09-23T05:14:19Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 102.627930] nix-grpc-daemon[768]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 102.644925] nix-grpc-daemon[768]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 102.662170] nix-grpc-daemon[862]: time=2026-09-23T05:14:19.217Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'... worker1 # [ 102.674506] nix-grpc-daemon[768]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 102.690770] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 102.854815] nix-daemon[835]: accepted connection from pid 1110, user nix-grpc-daemon (trusted) worker1 # [ 102.876429] nix-daemon[835]: reaped child process 1113, status = succeeded lb # [ 102.838467] niks3-server[1013]: 2026/09/23 05:14:19 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 102.893822] nix-grpc-daemon[862]: time=2026-09-23T05:14:19.449Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 102.895535] nix-grpc-daemon[862]: time=2026-09-23T05:14:19.449Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)" lb # [ 102.873472] niks3-server[1013]: 2026/09/23 05:14:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign worker1 # [ 102.921941] nix-grpc-daemon[862]: time=2026-09-23T05:14:19.478Z level=INFO msg="Uploading 1 narinfos" lb # [ 102.886998] niks3-server[1013]: 2026/09/23 05:14:19 INFO Signed narinfos id=33 count=1 lb # [ 102.891196] niks3-server[1013]: 2026/09/23 05:14:19 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls lb # [ 102.905326] niks3-server[1013]: 2026/09/23 05:14:19 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete worker1 # [ 102.960091] nix-grpc-daemon[862]: time=2026-09-23T05:14:19.516Z level=INFO msg="Upload complete. (249ms)" lb # [ 102.918845] niks3-server[1013]: 2026/09/23 05:14:19 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst lb # [ 102.925709] niks3-server[1013]: 2026/09/23 05:14:19 INFO Completed upload id=33 lb # [ 102.930167] niks3-server[1013]: 2026/09/23 05:14:19 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo worker1 # [ 102.972622] nix-grpc-daemon[768]: ts=2026-09-23T05:14:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=0 paths=1 nar_bytes_in=568 worker1 # [ 102.988349] nix-grpc-daemon[768]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 102.996000] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) worker1 # [ 103.001993] nix-daemon[835]: reaped child process 1108, status = succeeded worker1 # [ 103.014588] nix-daemon[835]: accepted connection from pid 768, user nix-grpc-daemon (trusted) client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051... client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 104.064624] nix-daemon[835]: reaped child process 1117, status = succeeded worker1 # [ 104.151598] nix-daemon[835]: accepted connection from pid 1133, user nix-grpc-daemon (trusted) worker1 # [ 104.170131] nix-daemon[835]: reaped child process 1136, status = succeeded lb # [ 104.135322] niks3-server[1013]: 2026/09/23 05:14:20 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 104.188399] nix-grpc-daemon[862]: time=2026-09-23T05:14:20.743Z level=INFO msg="Uploading 0 paths to lb (1 already cached)" lb # [ 104.164801] niks3-server[1013]: 2026/09/23 05:14:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign worker1 # [ 104.215700] nix-grpc-daemon[862]: time=2026-09-23T05:14:20.771Z level=INFO msg="Uploading 1 narinfos" lb # [ 104.177938] niks3-server[1013]: 2026/09/23 05:14:20 INFO Signed narinfos id=34 count=1 lb # [ 104.189326] niks3-server[1013]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls lb # [ 104.199467] niks3-server[1013]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv lb # [ 104.213395] niks3-server[1013]: 2026/09/23 05:14:20 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete worker1 # [ 104.262574] nix-grpc-daemon[862]: time=2026-09-23T05:14:20.818Z level=INFO msg="Upload complete. (234ms)" lb # [ 104.221784] niks3-server[1013]: 2026/09/23 05:14:20 INFO Completed upload id=34 lb # [ 104.224466] niks3-server[1013]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo client # copying 1 paths... worker1 # [ 104.273003] nix-grpc-daemon[768]: ts=2026-09-23T05:14:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37640 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=1 worker1 # [ 104.289091] nix-grpc-daemon[768]: ts=2026-09-23T05:14:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 104.294530] nix-daemon[835]: reaped child process 1115, status = succeeded client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'... worker2 # [ 104.323962] nix-grpc-daemon[771]: ts=2026-09-23T05:14:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 worker2 # [ 104.334521] nix-grpc-daemon[771]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'... worker2 # [ 104.353910] nix-grpc-daemon[771]: ts=2026-09-23T05:14:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:45480 duration_s=0 paths=1 nar_bytes_out=120 client # [ 104.480433] nix-daemon[1148]: reaped child process 1276, status = succeeded client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.23 seconds) (finished: subtest: an input only one worker has still reaches the builder, in 9.22 seconds) subtest: upload whose reference the worker lacks is completed from the cache client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache client # this derivation will be built: client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.40 seconds) client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache client # this derivation will be built: client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'... client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.36 seconds) client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.14 seconds) worker1: must succeed: nix-store --import < /tmp/shared/ref.closure worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.28 seconds) worker2 # [ 105.762997] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 105.818599] nix-grpc-daemon[771]: ts=2026-09-23T05:14:22Z level=info event=signal_exit lb # [ 106.762062] niks3-server[1013]: 2026/09/23 05:14:23 INFO lead: released remote=192.168.1.4:57726 worker2 # [ 107.803140] nix-grpc-daemon[771]: ts=2026-09-23T05:14:24Z level=info event=scheduler_restarting worker1 # [ 107.806195] nix-grpc-daemon[768]: ts=2026-09-23T05:14:24Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 107.821453] nix-grpc-daemon[771]: ts=2026-09-23T05:14:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:45476 worker2 # [ 107.827213] nix-grpc-daemon[771]: ts=2026-09-23T05:14:24Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 107.833791] nix-grpc-daemon[771]: ts=2026-09-23T05:14:24Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 107.838378] nix-grpc-daemon[771]: ts=2026-09-23T05:14:24Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:45476 worker2 # [ 107.846989] nix-grpc-daemon[771]: ts=2026-09-23T05:14:24Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 107.882105] nix-daemon[858]: reaped child process 860, status = succeeded worker2 # [ 107.993552] nix-grpc-daemon[771]: E0000 00:00:1790140464.556532 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true worker2 # [ 108.000517] nix-grpc-daemon[771]: E0000 00:00:1790140464.557346 817 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker1 # [ 108.006922] nix-grpc-daemon[768]: ts=2026-09-23T05:14:24Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 108.012816] nix-grpc-daemon[768]: ts=2026-09-23T05:14:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 108.048930] nix-grpc-daemon[771]: ts=2026-09-23T05:14:24Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 108.213884] nix-grpc-daemon[768]: ts=2026-09-23T05:14:24Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 108.219652] nix-grpc-daemon[768]: ts=2026-09-23T05:14:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 108.366799] envoy[900]: conn peer=[2001:db8:1::1]:50700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4257 tx=2071 ms=2492 lb # [ 108.371960] envoy[900]: conn peer=[2001:db8:1::1]:50704 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=91 worker1 # [ 108.420574] nix-grpc-daemon[768]: ts=2026-09-23T05:14:24Z level=info event=scheduler_connected addr=lb:50051 lb # [ 108.376831] envoy[900]: conn peer=[2001:db8:1::1]:34154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=27 lb # [ 108.383142] envoy[900]: conn peer=[2001:db8:1::1]:50720 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3753 tx=2124 ms=3042 worker1 # [ 108.425804] nix-grpc-daemon[768]: ts=2026-09-23T05:14:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 108.388136] envoy[900]: conn peer=[2001:db8:1::1]:34160 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3754 tx=2097 ms=1829 lb # [ 108.392836] envoy[900]: conn peer=[2001:db8:1::1]:34172 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=95 lb # [ 108.397478] envoy[900]: conn peer=[2001:db8:1::4]:46050 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=76994 tx=4394 ms=89196 worker1 # [ 108.626752] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 108.632068] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 108.797063] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=scheduler_take_over lb # [ 108.755104] niks3-server[1013]: 2026/09/23 05:14:25 INFO lead: acquired remote=192.168.1.3:41990 worker1 # [ 108.834749] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 108.838837] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 109.035221] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 109.055742] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:45670 worker1 # [ 109.061611] nix-grpc-daemon[768]: ts=2026-09-23T05:14:25Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 109.809729] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 109.811739] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 109.820675] systemd[1]: nix-grpc-daemon.service: Consumed 3.858s CPU time over 1min 31.249s wall clock time, 71.2M memory peak, 117.4K incoming IP traffic, 298.3K outgoing IP traffic. worker2 # [ 109.831297] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 109.842649] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 109.850069] systemd[1]: nix-grpc-daemon.socket: Consumed 678us CPU time over 1min 39.570s wall clock time, 256K memory peak, 125K incoming IP traffic, 102.1K outgoing IP traffic. lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds) client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2 worker1 # [ 110.170000] nix-grpc-daemon[768]: ts=2026-09-23T05:14:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37640 worker1 # [ 110.180276] nix-grpc-daemon[768]: ts=2026-09-23T05:14:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:37640 client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.27 seconds) worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds) worker2 # [ 110.307068] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 110.384468] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1 # [ 110.716521] nix-grpc-daemon[768]: ts=2026-09-23T05:14:27Z level=info event=signal_exit lb # [ 110.761316] niks3-server[1013]: 2026/09/23 05:14:27 INFO lead: released remote=192.168.1.3:41990 worker2 # [ 110.806872] systemd[1]: Starting scheduler-token.service... worker2 # [ 110.897410] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 110.905288] systemd[1]: Finished scheduler-token.service. worker2 # [ 110.907420] systemd[1]: scheduler-token.service: Consumed 59ms CPU time over 94ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 110.918583] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 111.067209] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 111.069579] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 111.073438] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 111.078901] nix-grpc-daemon[1324]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR lb # [ 111.052393] niks3-server[1013]: 2026/09/23 05:14:27 INFO lead: acquired remote=192.168.1.4:35022 worker2 # [ 111.096133] nix-grpc-daemon[1324]: E0000 00:00:1790140467.642324 1324 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 111.124139] nix-grpc-daemon[1324]: E0000 00:00:1790140467.646072 1324 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 111.126418] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 111.134134] nix-grpc-daemon[1324]: E0000 00:00:1790140467.658795 1366 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 111.137047] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=scheduler_take_over worker2 # [ 111.138321] nix-grpc-daemon[1324]: E0000 00:00:1790140467.678282 1366 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 111.140576] nix-grpc-daemon[1324]: E0000 00:00:1790140467.678415 1366 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 111.145787] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 111.148824] nix-grpc-daemon[1324]: E0000 00:00:1790140467.697406 1366 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 111.151086] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 111.152348] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 111.350123] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 111.355332] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 111.556410] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 111.561578] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 111.762547] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 111.767673] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 111.811389] nix-grpc-daemon[768]: ts=2026-09-23T05:14:28Z level=info event=scheduler_restarting worker1 # [ 111.815387] nix-grpc-daemon[768]: ts=2026-09-23T05:14:28Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 111.819952] nix-grpc-daemon[768]: ts=2026-09-23T05:14:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:45670 worker1 # [ 111.825153] nix-grpc-daemon[768]: ts=2026-09-23T05:14:28Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 111.856312] nix-daemon[835]: reaped child process 837, status = succeeded worker1 # [ 111.907535] nix-grpc-daemon[768]: E0000 00:00:1790140468.463548 814 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true worker1 # [ 111.920702] nix-grpc-daemon[768]: E0000 00:00:1790140468.476894 814 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker2 # [ 111.969072] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 111.974508] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 112.027805] nix-grpc-daemon[768]: ts=2026-09-23T05:14:28Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 112.178516] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 112.192133] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 112.197942] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:48356 worker2 # [ 112.203418] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:28Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker1 # [ 113.754621] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 113.759411] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 113.765190] systemd[1]: nix-grpc-daemon.service: Consumed 3.008s CPU time over 1min 35.340s wall clock time, 60.6M memory peak, 92.5K incoming IP traffic, 164.3K outgoing IP traffic. worker1 # [ 113.782929] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker1 # [ 113.797616] systemd[1]: Closed Nix worker-protocol over gRPC. lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy worker1 # [ 113.799553] systemd[1]: nix-grpc-daemon.socket: Consumed 684us CPU time over 1min 43.583s wall clock time, 256K memory peak, 102.6K incoming IP traffic, 90.8K outgoing IP traffic. lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds) client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2 worker2 # [ 114.098180] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48370 worker2 # [ 114.116435] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted) worker2 # [ 114.144983] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5) worker2 # [ 114.201905] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 153 ms (attempt 2/5) worker2 # [ 114.364343] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 3/5) worker2 # [ 114.451084] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 333 ms (attempt 4/5) worker2 # [ 114.786612] nix-grpc-daemon[1324]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 114.793645] nix-grpc-daemon[1324]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org client # copying 2 paths... client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'... worker2 # [ 114.820885] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48370 client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'... worker2 # [ 114.838942] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted) worker2 # [ 114.870608] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:31Z level=info event=niks3_push_started pid=1385 worker2 # [ 115.024561] nix-grpc-daemon[1385]: time=2026-09-23T05:14:31.588Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 115.136067] nix-daemon[858]: accepted connection from pid 1398, user nix-grpc-daemon (trusted) worker2 # [ 115.153884] nix-daemon[858]: reaped child process 1401, status = succeeded lb # [ 115.117950] niks3-server[1013]: 2026/09/23 05:14:31 INFO Received uploads request method=POST path=/api/pending_closures lb # [ 115.129190] niks3-server[1013]: 2026/09/23 05:14:31 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 115.182411] nix-grpc-daemon[1385]: time=2026-09-23T05:14:31.744Z level=INFO msg="Uploading 2 paths to lb (0 already cached)" worker2 # [ 115.183871] nix-grpc-daemon[1385]: time=2026-09-23T05:14:31.744Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)" worker2 # [ 115.187148] nix-grpc-daemon[1385]: time=2026-09-23T05:14:31.744Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)" lb # [ 115.194956] niks3-server[1013]: 2026/09/23 05:14:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign lb # [ 115.209174] niks3-server[1013]: 2026/09/23 05:14:31 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls lb # [ 115.211784] niks3-server[1013]: 2026/09/23 05:14:31 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst lb # [ 115.230472] niks3-server[1013]: 2026/09/23 05:14:31 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst lb # [ 115.236712] niks3-server[1013]: 2026/09/23 05:14:31 INFO Signed narinfos id=35 count=1 worker2 # [ 115.279706] nix-grpc-daemon[1385]: time=2026-09-23T05:14:31.840Z level=INFO msg="Uploading 3 narinfos" lb # [ 115.242636] niks3-server[1013]: 2026/09/23 05:14:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign lb # [ 115.247571] niks3-server[1013]: 2026/09/23 05:14:31 INFO Signed narinfos id=36 count=2 lb # [ 115.250353] niks3-server[1013]: 2026/09/23 05:14:31 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls lb # [ 115.266246] niks3-server[1013]: 2026/09/23 05:14:31 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete lb # [ 115.275428] niks3-server[1013]: 2026/09/23 05:14:31 INFO Completed upload id=35 lb # [ 115.278549] niks3-server[1013]: 2026/09/23 05:14:31 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete lb # [ 115.282148] niks3-server[1013]: 2026/09/23 05:14:31 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo lb # [ 115.289079] niks3-server[1013]: 2026/09/23 05:14:31 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 115.291100] niks3-server[1013]: 2026/09/23 05:14:31 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo lb # [ 115.294237] niks3-server[1013]: 2026/09/23 05:14:31 INFO Completed upload id=36 worker2 # [ 115.337048] nix-grpc-daemon[1385]: time=2026-09-23T05:14:31.900Z level=INFO msg="Upload complete. (308ms)" worker2 # [ 115.349228] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48370 duration_s=0 paths=2 nar_bytes_in=312 worker2 # [ 115.366624] nix-daemon[858]: reaped child process 1383, status = succeeded client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.51 seconds) worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds) worker1 # [ 115.489025] systemd[1]: Listening on Nix worker-protocol over gRPC. worker1 # [ 115.497612] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 115.654437] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:32Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 115.656791] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:32Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 115.660911] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:32Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 115.666332] nix-grpc-daemon[1159]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 115.667756] nix-grpc-daemon[1159]: E0000 00:00:1790140472.222075 1159 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 115.696168] nix-grpc-daemon[1159]: E0000 00:00:1790140472.228096 1159 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 115.698444] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:32Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 115.717249] nix-grpc-daemon[1159]: E0000 00:00:1790140472.238216 1202 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 115.720261] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 115.721084] nix-grpc-daemon[1159]: E0000 00:00:1790140472.258014 1202 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 115.726412] nix-grpc-daemon[1159]: E0000 00:00:1790140472.258177 1202 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 115.730728] nix-grpc-daemon[1159]: E0000 00:00:1790140472.272545 1202 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 115.733254] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:32Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 115.732932] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:32Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:48356 worker2 # [ 115.738840] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:32Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds) (finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.33 seconds) subtest: a niks3 restart does not cost the leader its role lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb # [ 116.911962] systemd[1]: Stopping niks3 server... lb # [ 116.917047] niks3-server[1013]: 2026/09/23 05:14:33 INFO Shutdown signal received, draining in-flight requests timeout=10s worker1 # [ 116.962107] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:33Z level=info event=scheduler_lock_lost status=200 worker2 # [ 116.961958] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:33Z level=info event=scheduler_lock_lost status=200 lb # [ 116.923457] niks3-server[1013]: 2026/09/23 05:14:33 INFO lead: released remote=192.168.1.4:35022 lb # [ 116.935067] systemd[1]: niks3.service: Deactivated successfully. lb # [ 116.938937] systemd[1]: Stopped niks3 server. lb # [ 116.939713] systemd[1]: niks3.service: Consumed 691ms CPU time over 1min 28.487s wall clock time, 27.7M memory peak, 98.4K incoming IP traffic, 142.1K outgoing IP traffic. lb # [ 116.950225] systemd[1]: Starting niks3 server... lb # [ 117.017093] niks3-server[1152]: 2026/09/23 05:14:33 goose: no migrations to run. current version: 20260920000000 lb # [ 117.024911] niks3-server[1152]: 2026/09/23 05:14:33 OK 1_commit_pending_closure.sql (5.99ms) lb # [ 117.028675] niks3-server[1152]: 2026/09/23 05:14:33 OK 2_object_stats_trigger.sql (3.38ms) lb # [ 117.030485] niks3-server[1152]: 2026/09/23 05:14:33 goose: up to current file version: 2 lb # [ 117.032079] niks3-server[1152]: 2026/09/23 05:14:33 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key lb # [ 117.042989] niks3-server[1152]: 2026/09/23 05:14:33 INFO Read proxy enabled — serving cache objects from S3 lb # [ 117.048535] niks3-server[1152]: 2026/09/23 05:14:33 INFO Using socket-activated listener address=0.0.0.0:5751 lb # [ 117.049952] niks3-server[1152]: 2026/09/23 05:14:33 INFO systemd watchdog enabled interval=15s lb # [ 117.051262] niks3-server[1152]: 2026/09/23 05:14:33 INFO Starting HTTP server address=0.0.0.0:5751 lb # [ 117.054010] systemd[1]: Started niks3 server. lb: waiting for unit niks3.service lb: (finished: waiting for unit niks3.service, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2 worker2 # [ 117.579267] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:48370 client # don't know how to build these paths: client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker1 # [ 117.595474] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 117.610612] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker1 # [ 117.636291] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5) worker1 # [ 117.717742] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 120 ms (attempt 2/5) worker1 # [ 117.839659] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 310 ms (attempt 3/5) lb # [ 117.924858] niks3-server[1152]: 2026/09/23 05:14:34 INFO lead: acquired remote=192.168.1.4:35072 worker1 # [ 118.152697] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 4/5) worker1 # [ 118.198700] nix-grpc-daemon[1159]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 118.205434] nix-grpc-daemon[1159]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 118.252396] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:34Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=0 worker2 # [ 118.257828] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:34Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 118.297311] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48370 worker2 # [ 118.315376] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48370 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'... worker2 # [ 118.331230] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48370 worker2 # [ 118.346314] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted) lb # [ 118.372636] envoy[900]: conn peer=[2001:db8:1::1]:34178 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=32 lb # [ 118.377647] envoy[900]: conn peer=[2001:db8:1::3]:45428 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=16090 tx=4442 ms=93284 lb # [ 118.382650] envoy[900]: conn peer=[2001:db8:1::1]:39538 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=282 ms=1267 worker2 # [ 118.499254] nix-daemon[858]: accepted connection from pid 1420, user nix-grpc-daemon (trusted) worker2 # [ 118.521427] nix-daemon[858]: reaped child process 1423, status = succeeded lb # [ 118.477947] niks3-server[1152]: 2026/09/23 05:14:35 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 118.535376] nix-grpc-daemon[1385]: time=2026-09-23T05:14:35.098Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 118.537858] nix-grpc-daemon[1385]: time=2026-09-23T05:14:35.098Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)" lb # [ 118.519836] niks3-server[1152]: 2026/09/23 05:14:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign lb # [ 118.552738] niks3-server[1152]: 2026/09/23 05:14:35 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls lb # [ 118.579964] niks3-server[1152]: 2026/09/23 05:14:35 INFO Signed narinfos id=37 count=1 lb # [ 118.582333] niks3-server[1152]: 2026/09/23 05:14:35 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst worker2 # [ 118.624141] nix-grpc-daemon[1385]: time=2026-09-23T05:14:35.186Z level=INFO msg="Uploading 1 narinfos" lb # [ 118.596105] niks3-server[1152]: 2026/09/23 05:14:35 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete lb # [ 118.609815] niks3-server[1152]: 2026/09/23 05:14:35 INFO Completed upload id=37 worker2 # [ 118.652363] nix-grpc-daemon[1385]: time=2026-09-23T05:14:35.216Z level=INFO msg="Upload complete. (288ms)" lb # [ 118.617354] niks3-server[1152]: 2026/09/23 05:14:35 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo worker2 # [ 118.665946] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48370 duration_s=0 paths=1 nar_bytes_in=544 worker2 # [ 118.685118] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48370 worker2 # [ 118.697651] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted) worker2 # [ 118.701735] nix-daemon[858]: reaped child process 1418, status = succeeded client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051... client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker2 # [ 118.717100] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-after-niks3-restart worker2 # [ 119.989170] nix-daemon[858]: reaped child process 1427, status = succeeded worker2 # [ 120.103653] nix-daemon[858]: accepted connection from pid 1443, user nix-grpc-daemon (trusted) worker2 # [ 120.121959] nix-daemon[858]: reaped child process 1446, status = succeeded lb # [ 120.085636] niks3-server[1152]: 2026/09/23 05:14:36 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 120.142618] nix-grpc-daemon[1385]: time=2026-09-23T05:14:36.704Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 120.144423] nix-grpc-daemon[1385]: time=2026-09-23T05:14:36.704Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)" lb # [ 120.139125] niks3-server[1152]: 2026/09/23 05:14:36 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker2 # [ 120.191203] nix-grpc-daemon[1385]: time=2026-09-23T05:14:36.754Z level=INFO msg="Uploading 1 narinfos" lb # [ 120.149565] niks3-server[1152]: 2026/09/23 05:14:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign lb # [ 120.159617] niks3-server[1152]: 2026/09/23 05:14:36 INFO Signed narinfos id=38 count=1 lb # [ 120.166343] niks3-server[1152]: 2026/09/23 05:14:36 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst lb # [ 120.176762] niks3-server[1152]: 2026/09/23 05:14:36 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls lb # [ 120.187981] niks3-server[1152]: 2026/09/23 05:14:36 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete worker2 # [ 120.230986] nix-grpc-daemon[1385]: time=2026-09-23T05:14:36.794Z level=INFO msg="Upload complete. (302ms)" lb # [ 120.191914] niks3-server[1152]: 2026/09/23 05:14:36 INFO Completed upload id=38 lb # [ 120.198961] niks3-server[1152]: 2026/09/23 05:14:36 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo worker2 # [ 120.252906] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48370 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 worker2 # [ 120.268474] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:36Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=0 worker2 # [ 120.275150] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:36Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 120.279420] nix-daemon[858]: reaped child process 1425, status = succeeded worker2 # [ 120.298789] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48370 worker2 # [ 120.315386] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48370 worker2 # [ 120.330715] nix-grpc-daemon[1385]: time=2026-09-23T05:14:36.894Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'... worker2 # [ 120.341501] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48370 worker2 # [ 120.355974] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted) worker2 # [ 120.512896] nix-daemon[858]: accepted connection from pid 1452, user nix-grpc-daemon (trusted) worker2 # [ 120.530598] nix-daemon[858]: reaped child process 1455, status = succeeded lb # [ 120.494775] niks3-server[1152]: 2026/09/23 05:14:37 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 120.549664] nix-grpc-daemon[1385]: time=2026-09-23T05:14:37.113Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 120.551356] nix-grpc-daemon[1385]: time=2026-09-23T05:14:37.113Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)" lb # [ 120.527846] niks3-server[1152]: 2026/09/23 05:14:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign worker2 # [ 120.577860] nix-grpc-daemon[1385]: time=2026-09-23T05:14:37.141Z level=INFO msg="Uploading 1 narinfos" lb # [ 120.543658] niks3-server[1152]: 2026/09/23 05:14:37 INFO Signed narinfos id=39 count=1 lb # [ 120.550511] niks3-server[1152]: 2026/09/23 05:14:37 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls lb # [ 120.563277] niks3-server[1152]: 2026/09/23 05:14:37 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst worker2 # [ 120.605377] nix-grpc-daemon[1385]: time=2026-09-23T05:14:37.169Z level=INFO msg="Upload complete. (233ms)" lb # [ 120.571419] niks3-server[1152]: 2026/09/23 05:14:37 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete lb # [ 120.576843] niks3-server[1152]: 2026/09/23 05:14:37 INFO Completed upload id=39 worker2 # [ 120.616538] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48370 duration_s=0 paths=1 nar_bytes_in=656 lb # [ 120.579066] niks3-server[1152]: 2026/09/23 05:14:37 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo worker2 # [ 120.631063] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48370 worker2 # [ 120.635240] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted) worker2 # [ 120.640779] nix-daemon[858]: reaped child process 1450, status = succeeded client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051... worker2 # [ 120.652472] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted) client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-after-niks3-restart worker2 # [ 121.892260] nix-daemon[858]: reaped child process 1459, status = succeeded worker2 # [ 122.002629] nix-daemon[858]: accepted connection from pid 1475, user nix-grpc-daemon (trusted) worker2 # [ 122.021600] nix-daemon[858]: reaped child process 1478, status = succeeded lb # [ 121.985368] niks3-server[1152]: 2026/09/23 05:14:38 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 122.037391] nix-grpc-daemon[1385]: time=2026-09-23T05:14:38.600Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 122.038950] nix-grpc-daemon[1385]: time=2026-09-23T05:14:38.600Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)" lb # [ 122.028703] niks3-server[1152]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv lb # [ 122.040254] niks3-server[1152]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst worker2 # [ 122.091336] nix-grpc-daemon[1385]: time=2026-09-23T05:14:38.655Z level=INFO msg="Uploading 1 narinfos" lb # [ 122.054359] niks3-server[1152]: 2026/09/23 05:14:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign lb # [ 122.061904] niks3-server[1152]: 2026/09/23 05:14:38 INFO Signed narinfos id=40 count=1 lb # [ 122.067305] niks3-server[1152]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls lb # [ 122.079999] niks3-server[1152]: 2026/09/23 05:14:38 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete worker2 # [ 122.128473] nix-grpc-daemon[1385]: time=2026-09-23T05:14:38.692Z level=INFO msg="Upload complete. (266ms)" lb # [ 122.085384] niks3-server[1152]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo lb # [ 122.094919] niks3-server[1152]: 2026/09/23 05:14:38 INFO Completed upload id=40 worker2 # [ 122.152310] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48370 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 worker2 # [ 122.164396] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:38Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=0 worker2 # [ 122.169325] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:38Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 122.173321] nix-daemon[858]: reaped child process 1457, status = succeeded worker2 # [ 122.194441] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48370 worker2 # [ 122.210018] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:48370 worker2 # [ 122.224916] nix-grpc-daemon[1385]: time=2026-09-23T05:14:38.788Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'... worker2 # [ 122.236119] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48370 worker2 # [ 122.250072] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted) worker2 # [ 122.396803] nix-daemon[858]: accepted connection from pid 1484, user nix-grpc-daemon (trusted) worker2 # [ 122.417196] nix-daemon[858]: reaped child process 1487, status = succeeded lb # [ 122.379353] niks3-server[1152]: 2026/09/23 05:14:38 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 122.430746] nix-grpc-daemon[1385]: time=2026-09-23T05:14:38.994Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 122.432524] nix-grpc-daemon[1385]: time=2026-09-23T05:14:38.994Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)" lb # [ 122.408646] niks3-server[1152]: 2026/09/23 05:14:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign worker2 # [ 122.462599] nix-grpc-daemon[1385]: time=2026-09-23T05:14:39.026Z level=INFO msg="Uploading 1 narinfos" lb # [ 122.425047] niks3-server[1152]: 2026/09/23 05:14:38 INFO Signed narinfos id=41 count=1 lb # [ 122.432269] niks3-server[1152]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls lb # [ 122.447493] niks3-server[1152]: 2026/09/23 05:14:38 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst lb # [ 122.457376] niks3-server[1152]: 2026/09/23 05:14:39 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete worker2 # [ 122.504732] nix-grpc-daemon[1385]: time=2026-09-23T05:14:39.068Z level=INFO msg="Upload complete. (239ms)" lb # [ 122.464201] niks3-server[1152]: 2026/09/23 05:14:39 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo lb # [ 122.471856] niks3-server[1152]: 2026/09/23 05:14:39 INFO Completed upload id=41 worker2 # [ 122.515193] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48370 duration_s=0 paths=1 nar_bytes_in=784 worker2 # [ 122.532450] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48370 worker2 # [ 122.541445] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted) worker2 # [ 122.544339] nix-daemon[858]: reaped child process 1482, status = succeeded client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051... worker2 # [ 122.556197] nix-daemon[858]: accepted connection from pid 1324, user nix-grpc-daemon (trusted) client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-after-niks3-restart worker2 # [ 123.436561] nix-daemon[858]: reaped child process 1491, status = succeeded worker2 # [ 123.529977] nix-daemon[858]: accepted connection from pid 1507, user nix-grpc-daemon (trusted) worker2 # [ 123.549829] nix-daemon[858]: reaped child process 1510, status = succeeded lb # [ 123.513861] niks3-server[1152]: 2026/09/23 05:14:40 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 123.567133] nix-grpc-daemon[1385]: time=2026-09-23T05:14:40.130Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 123.569264] nix-grpc-daemon[1385]: time=2026-09-23T05:14:40.130Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)" lb # [ 123.553874] niks3-server[1152]: 2026/09/23 05:14:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign worker2 # [ 123.606226] nix-grpc-daemon[1385]: time=2026-09-23T05:14:40.170Z level=INFO msg="Uploading 1 narinfos" lb # [ 123.569586] niks3-server[1152]: 2026/09/23 05:14:40 INFO Signed narinfos id=42 count=1 lb # [ 123.581533] niks3-server[1152]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv lb # [ 123.591215] niks3-server[1152]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst lb # [ 123.603877] niks3-server[1152]: 2026/09/23 05:14:40 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete lb # [ 123.608807] niks3-server[1152]: 2026/09/23 05:14:40 INFO Completed upload id=42 worker2 # [ 123.650667] nix-grpc-daemon[1385]: time=2026-09-23T05:14:40.214Z level=INFO msg="Upload complete. (283ms)" lb # [ 123.611909] niks3-server[1152]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo lb # [ 123.615034] niks3-server[1152]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls worker2 # [ 123.668233] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48370 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 worker2 # [ 123.684561] nix-daemon[858]: reaped child process 1489, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.35 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.17 seconds) (finished: subtest: a niks3 restart does not cost the leader its role, in 7.06 seconds) subtest: scheduler down: the other node takes the lock and keeps it lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2 # [ 124.125859] systemd[1]: Stopping Nix worker-protocol over gRPC... worker2 # [ 125.098981] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:41Z level=info event=signal_exit lb # [ 125.930081] niks3-server[1152]: 2026/09/23 05:14:42 INFO lead: released remote=192.168.1.4:35072 worker1 # [ 126.976062] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:43Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 126.979136] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:43Z level=info event=scheduler_restarting worker2 # [ 126.984720] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:43Z level=info event=scheduler_restarting addr=lb:50051 worker2 # [ 126.989245] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:48356 worker2 # [ 126.994513] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:43Z level=info event=worker_gone addr=192.168.1.3:50051 worker2 # [ 127.000266] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:48356 worker2 # [ 127.008476] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:43Z level=info event=worker_gone addr=192.168.1.4:50051 worker2 # [ 127.026966] nix-daemon[858]: reaped child process 1377, status = succeeded worker2 # [ 127.138958] nix-grpc-daemon[1324]: E0000 00:00:1790140483.702866 1370 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true worker2 # [ 127.146477] nix-grpc-daemon[1324]: E0000 00:00:1790140483.710263 1370 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true worker1 # [ 127.175709] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 127.181334] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker2 # [ 127.195903] nix-grpc-daemon[1324]: ts=2026-09-23T05:14:43Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 127.251821] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker2 # [ 127.256750] systemd[1]: Stopped Nix worker-protocol over gRPC. worker2 # [ 127.263970] systemd[1]: nix-grpc-daemon.service: Consumed 1.402s CPU time over 16.337s wall clock time, 38.9M memory peak, 47.4K incoming IP traffic, 83.4K outgoing IP traffic. worker2 # [ 127.276164] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully. worker2 # [ 127.282381] systemd[1]: Closed Nix worker-protocol over gRPC. worker2 # [ 127.285147] systemd[1]: nix-grpc-daemon.socket: Consumed 956us CPU time over 16.982s wall clock time, 256K memory peak, 38.7K incoming IP traffic, 29.4K outgoing IP traffic. worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 127.381587] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:43Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 127.387814] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds) worker1 # [ 127.586068] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 127.591379] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 127.792164] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 127.797168] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" lb # [ 127.926821] niks3-server[1152]: 2026/09/23 05:14:44 INFO lead: acquired remote=192.168.1.3:43296 worker1 # [ 127.971083] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_take_over worker1 # [ 127.999661] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 128.003849] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 128.200338] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 128.205379] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 128.406203] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 128.411472] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics lb # [ 128.376366] envoy[900]: conn peer=[2001:db8:1::1]:39550 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9753 tx=4467 ms=6097 lb # [ 128.381291] envoy[900]: conn peer=[2001:db8:1::4]:59078 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5513 tx=665 ms=16039 worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) worker1 # [ 128.612778] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 128.618083] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 128.818573] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:45Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 128.827606] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:36902 worker1 # [ 128.833308] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2 client # don't know how to build these paths: client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker1 # [ 129.774069] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 129.781671] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 129.808847] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=0 worker1 # [ 129.815767] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 129.854745] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 129.870334] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37174 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'... worker1 # [ 129.887177] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 129.902525] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker1 # [ 129.930748] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=info event=niks3_push_started pid=1239 worker1 # [ 130.077182] nix-grpc-daemon[1239]: time=2026-09-23T05:14:46.632Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 130.183576] nix-daemon[835]: accepted connection from pid 1251, user nix-grpc-daemon (trusted) worker1 # [ 130.201151] nix-daemon[835]: reaped child process 1254, status = succeeded lb # [ 130.164840] niks3-server[1152]: 2026/09/23 05:14:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 130.217697] nix-grpc-daemon[1239]: time=2026-09-23T05:14:46.773Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 130.219149] nix-grpc-daemon[1239]: time=2026-09-23T05:14:46.773Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)" lb # [ 130.203101] niks3-server[1152]: 2026/09/23 05:14:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign worker1 # [ 130.255666] nix-grpc-daemon[1239]: time=2026-09-23T05:14:46.811Z level=INFO msg="Uploading 1 narinfos" lb # [ 130.216915] niks3-server[1152]: 2026/09/23 05:14:46 INFO Signed narinfos id=43 count=1 lb # [ 130.224302] niks3-server[1152]: 2026/09/23 05:14:46 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls lb # [ 130.238799] niks3-server[1152]: 2026/09/23 05:14:46 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst lb # [ 130.253409] niks3-server[1152]: 2026/09/23 05:14:46 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete lb # [ 130.259820] niks3-server[1152]: 2026/09/23 05:14:46 INFO Completed upload id=43 worker1 # [ 130.301402] nix-grpc-daemon[1239]: time=2026-09-23T05:14:46.857Z level=INFO msg="Upload complete. (220ms)" lb # [ 130.267898] niks3-server[1152]: 2026/09/23 05:14:46 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo worker1 # [ 130.310831] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=0 paths=1 nar_bytes_in=520 worker1 # [ 130.327226] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 130.334429] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker1 # [ 130.339918] nix-daemon[835]: reaped child process 1237, status = succeeded client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051... client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker1 # [ 130.354495] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-on-standby worker1 # [ 131.254959] nix-daemon[835]: reaped child process 1259, status = succeeded worker1 # [ 131.329235] nix-daemon[835]: accepted connection from pid 1275, user nix-grpc-daemon (trusted) worker1 # [ 131.345387] nix-daemon[835]: reaped child process 1278, status = succeeded lb # [ 131.311120] niks3-server[1152]: 2026/09/23 05:14:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 131.364436] nix-grpc-daemon[1239]: time=2026-09-23T05:14:47.919Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 131.366243] nix-grpc-daemon[1239]: time=2026-09-23T05:14:47.919Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)" lb # [ 131.340712] niks3-server[1152]: 2026/09/23 05:14:47 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv lb # [ 131.360403] niks3-server[1152]: 2026/09/23 05:14:47 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst worker1 # [ 131.413978] nix-grpc-daemon[1239]: time=2026-09-23T05:14:47.969Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.371530] niks3-server[1152]: 2026/09/23 05:14:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign lb # [ 131.377721] niks3-server[1152]: 2026/09/23 05:14:47 INFO Signed narinfos id=44 count=1 lb # [ 131.382894] niks3-server[1152]: 2026/09/23 05:14:47 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls lb # [ 131.391354] niks3-server[1152]: 2026/09/23 05:14:47 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete lb # [ 131.398712] niks3-server[1152]: 2026/09/23 05:14:47 INFO Completed upload id=44 worker1 # [ 131.441453] nix-grpc-daemon[1239]: time=2026-09-23T05:14:47.997Z level=INFO msg="Upload complete. (277ms)" lb # [ 131.403654] niks3-server[1152]: 2026/09/23 05:14:47 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo worker1 # [ 131.465105] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 worker1 # [ 131.478710] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=0 worker1 # [ 131.482434] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 131.486274] nix-daemon[835]: reaped child process 1257, status = succeeded worker1 # [ 131.507370] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 131.522359] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 131.536045] nix-grpc-daemon[1239]: time=2026-09-23T05:14:48.092Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'... worker1 # [ 131.546824] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 131.560475] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker1 # [ 131.711396] nix-daemon[835]: accepted connection from pid 1284, user nix-grpc-daemon (trusted) worker1 # [ 131.728273] nix-daemon[835]: reaped child process 1287, status = succeeded lb # [ 131.692480] niks3-server[1152]: 2026/09/23 05:14:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 131.744091] nix-grpc-daemon[1239]: time=2026-09-23T05:14:48.299Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 131.745695] nix-grpc-daemon[1239]: time=2026-09-23T05:14:48.299Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)" lb # [ 131.720697] niks3-server[1152]: 2026/09/23 05:14:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign worker1 # [ 131.771056] nix-grpc-daemon[1239]: time=2026-09-23T05:14:48.326Z level=INFO msg="Uploading 1 narinfos" lb # [ 131.734332] niks3-server[1152]: 2026/09/23 05:14:48 INFO Signed narinfos id=45 count=1 lb # [ 131.743036] niks3-server[1152]: 2026/09/23 05:14:48 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls lb # [ 131.755721] niks3-server[1152]: 2026/09/23 05:14:48 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst worker1 # [ 131.806962] nix-grpc-daemon[1239]: time=2026-09-23T05:14:48.363Z level=INFO msg="Upload complete. (230ms)" lb # [ 131.767789] niks3-server[1152]: 2026/09/23 05:14:48 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete lb # [ 131.774477] niks3-server[1152]: 2026/09/23 05:14:48 INFO Completed upload id=45 worker1 # [ 131.816265] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=0 paths=1 nar_bytes_in=640 lb # [ 131.778691] niks3-server[1152]: 2026/09/23 05:14:48 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo worker1 # [ 131.832250] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 131.838619] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker1 # [ 131.843643] nix-daemon[835]: reaped child process 1282, status = succeeded client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051... worker1 # [ 131.854615] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-on-standby worker1 # [ 132.922100] nix-daemon[835]: reaped child process 1291, status = succeeded worker1 # [ 133.039260] nix-daemon[835]: accepted connection from pid 1307, user nix-grpc-daemon (trusted) worker1 # [ 133.057183] nix-daemon[835]: reaped child process 1310, status = succeeded lb # [ 133.021476] niks3-server[1152]: 2026/09/23 05:14:49 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 133.075956] nix-grpc-daemon[1239]: time=2026-09-23T05:14:49.631Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 133.077965] nix-grpc-daemon[1239]: time=2026-09-23T05:14:49.631Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)" lb # [ 133.062754] niks3-server[1152]: 2026/09/23 05:14:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign worker1 # [ 133.114795] nix-grpc-daemon[1239]: time=2026-09-23T05:14:49.670Z level=INFO msg="Uploading 1 narinfos" lb # [ 133.079785] niks3-server[1152]: 2026/09/23 05:14:49 INFO Signed narinfos id=46 count=1 lb # [ 133.088815] niks3-server[1152]: 2026/09/23 05:14:49 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls lb # [ 133.097670] niks3-server[1152]: 2026/09/23 05:14:49 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst lb # [ 133.103129] niks3-server[1152]: 2026/09/23 05:14:49 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv lb # [ 133.116413] niks3-server[1152]: 2026/09/23 05:14:49 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete worker1 # [ 133.158819] nix-grpc-daemon[1239]: time=2026-09-23T05:14:49.715Z level=INFO msg="Upload complete. (292ms)" lb # [ 133.120326] niks3-server[1152]: 2026/09/23 05:14:49 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo lb # [ 133.125932] niks3-server[1152]: 2026/09/23 05:14:49 INFO Completed upload id=46 worker1 # [ 133.176279] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 worker1 # [ 133.195860] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:49Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=0 worker1 # [ 133.201009] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:49Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 133.205658] nix-daemon[835]: reaped child process 1289, status = succeeded worker1 # [ 133.235570] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 133.252475] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 133.267431] nix-grpc-daemon[1239]: time=2026-09-23T05:14:49.823Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'... worker1 # [ 133.278987] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 133.292830] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker1 # [ 133.454390] nix-daemon[835]: accepted connection from pid 1316, user nix-grpc-daemon (trusted) worker1 # [ 133.474205] nix-daemon[835]: reaped child process 1319, status = succeeded lb # [ 133.434689] niks3-server[1152]: 2026/09/23 05:14:50 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 133.488064] nix-grpc-daemon[1239]: time=2026-09-23T05:14:50.044Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 133.489674] nix-grpc-daemon[1239]: time=2026-09-23T05:14:50.044Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)" lb # [ 133.471518] niks3-server[1152]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls worker1 # [ 133.522854] nix-grpc-daemon[1239]: time=2026-09-23T05:14:50.078Z level=INFO msg="Uploading 1 narinfos" lb # [ 133.483375] niks3-server[1152]: 2026/09/23 05:14:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign lb # [ 133.495260] niks3-server[1152]: 2026/09/23 05:14:50 INFO Signed narinfos id=47 count=1 lb # [ 133.501264] niks3-server[1152]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst worker1 # [ 133.555755] nix-grpc-daemon[1239]: time=2026-09-23T05:14:50.111Z level=INFO msg="Upload complete. (245ms)" lb # [ 133.514057] niks3-server[1152]: 2026/09/23 05:14:50 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete lb # [ 133.518093] niks3-server[1152]: 2026/09/23 05:14:50 INFO Completed upload id=47 lb # [ 133.521894] niks3-server[1152]: 2026/09/23 05:14:50 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo worker1 # [ 133.568332] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=0 paths=1 nar_bytes_in=768 worker1 # [ 133.585237] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 133.592730] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker1 # [ 133.598366] nix-daemon[835]: reaped child process 1314, status = succeeded client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051... worker1 # [ 133.610361] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-on-standby worker1 # [ 134.870758] nix-daemon[835]: reaped child process 1323, status = succeeded worker1 # [ 134.986934] nix-daemon[835]: accepted connection from pid 1339, user nix-grpc-daemon (trusted) worker1 # [ 135.009945] nix-daemon[835]: reaped child process 1342, status = succeeded lb # [ 134.967281] niks3-server[1152]: 2026/09/23 05:14:51 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 135.020419] nix-grpc-daemon[1239]: time=2026-09-23T05:14:51.575Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 135.021928] nix-grpc-daemon[1239]: time=2026-09-23T05:14:51.575Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)" lb # [ 135.010946] niks3-server[1152]: 2026/09/23 05:14:51 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls worker1 # [ 135.065527] nix-grpc-daemon[1239]: time=2026-09-23T05:14:51.621Z level=INFO msg="Uploading 1 narinfos" lb # [ 135.023766] niks3-server[1152]: 2026/09/23 05:14:51 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv lb # [ 135.033563] niks3-server[1152]: 2026/09/23 05:14:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign lb # [ 135.041688] niks3-server[1152]: 2026/09/23 05:14:51 INFO Signed narinfos id=48 count=1 lb # [ 135.049054] niks3-server[1152]: 2026/09/23 05:14:51 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst worker1 # [ 135.097492] nix-grpc-daemon[1239]: time=2026-09-23T05:14:51.653Z level=INFO msg="Upload complete. (273ms)" lb # [ 135.057032] niks3-server[1152]: 2026/09/23 05:14:51 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete lb # [ 135.060708] niks3-server[1152]: 2026/09/23 05:14:51 INFO Completed upload id=48 lb # [ 135.065823] niks3-server[1152]: 2026/09/23 05:14:51 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo worker1 # [ 135.113214] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 worker1 # [ 135.129521] nix-daemon[835]: reaped child process 1321, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.61 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds) worker2 # [ 135.414777] systemd[1]: Listening on Nix worker-protocol over gRPC. worker2 # [ 135.423594] systemd[1]: Starting scheduler-token.service... worker2 # [ 135.500295] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 135.508239] systemd[1]: Finished scheduler-token.service. worker2 # [ 135.509915] systemd[1]: scheduler-token.service: Consumed 44ms CPU time over 80ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 135.526389] systemd[1]: Starting Nix worker-protocol over gRPC... worker2 # [ 135.643716] nix-grpc-daemon[1531]: ts=2026-09-23T05:14:52Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 135.646211] nix-grpc-daemon[1531]: ts=2026-09-23T05:14:52Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 135.650179] nix-grpc-daemon[1531]: ts=2026-09-23T05:14:52Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 135.656069] nix-grpc-daemon[1531]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 135.657609] nix-grpc-daemon[1531]: E0000 00:00:1790140492.218880 1531 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. lb: must succeed: curl -sf localhost:9901/clusters worker2 # [ 135.702179] nix-grpc-daemon[1531]: E0000 00:00:1790140492.225229 1531 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 135.705233] nix-grpc-daemon[1531]: ts=2026-09-23T05:14:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 135.707130] nix-grpc-daemon[1531]: E0000 00:00:1790140492.234315 1572 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 135.709713] nix-grpc-daemon[1531]: E0000 00:00:1790140492.250866 1572 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 135.712091] nix-grpc-daemon[1531]: E0000 00:00:1790140492.251009 1572 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 135.717630] nix-grpc-daemon[1531]: E0000 00:00:1790140492.265445 1572 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 135.722237] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 135.723161] nix-grpc-daemon[1531]: ts=2026-09-23T05:14:52Z level=info event=scheduler_connected addr=lb:50051 worker1 # [ 135.732209] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=info event=oidc_jwks provider=mock keys=1 worker1 # [ 135.736813] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:36902 worker1 # [ 135.742078] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.10 seconds) subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs client # [ 136.050115] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs. client # Running as unit: rs.service; invocation ID: f1a868cf617c401ca5c0cac25df23f68 client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds) worker1 # [ 136.242629] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:37174 client # [ 136.250809] nix[1400]: don't know how to build these paths: client # [ 136.254113] nix[1400]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker1 # [ 136.262306] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=0 worker1 # [ 136.269608] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 136.307089] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 136.322445] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:37174 client # [ 136.336343] nix[1400]: copying 1 paths to 'lb:50051'... worker1 # [ 136.337288] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37174 client # [ 136.340184] nix[1400]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'... worker1 # [ 136.352708] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker1 # [ 136.511807] nix-daemon[835]: accepted connection from pid 1355, user nix-grpc-daemon (trusted) worker1 # [ 136.531407] nix-daemon[835]: reaped child process 1358, status = succeeded lb # [ 136.498372] niks3-server[1152]: 2026/09/23 05:14:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 136.550548] nix-grpc-daemon[1239]: time=2026-09-23T05:14:53.106Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 136.552289] nix-grpc-daemon[1239]: time=2026-09-23T05:14:53.106Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)" lb # [ 136.527537] niks3-server[1152]: 2026/09/23 05:14:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign worker1 # [ 136.578640] nix-grpc-daemon[1239]: time=2026-09-23T05:14:53.134Z level=INFO msg="Uploading 1 narinfos" lb # [ 136.542748] niks3-server[1152]: 2026/09/23 05:14:53 INFO Signed narinfos id=49 count=1 lb # [ 136.549235] niks3-server[1152]: 2026/09/23 05:14:53 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst lb # [ 136.560252] niks3-server[1152]: 2026/09/23 05:14:53 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls worker1 # [ 136.602613] nix-grpc-daemon[1239]: time=2026-09-23T05:14:53.158Z level=INFO msg="Upload complete. (233ms)" lb # [ 136.563236] niks3-server[1152]: 2026/09/23 05:14:53 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete lb # [ 136.569051] niks3-server[1152]: 2026/09/23 05:14:53 INFO Completed upload id=49 worker1 # [ 136.612304] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 136.574590] niks3-server[1152]: 2026/09/23 05:14:53 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo worker1 # [ 136.624767] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 worker1 # [ 136.634369] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) worker1 # [ 136.644347] nix-daemon[835]: reaped child process 1353, status = succeeded client # [ 136.649581] nix[1400]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051... worker1 # [ 136.652647] nix-daemon[835]: accepted connection from pid 1159, user nix-grpc-daemon (trusted) lb # [ 138.375857] envoy[900]: conn peer=[2001:db8:1::1]:42962 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9619 tx=4412 ms=5354 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds) worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true worker2 # [ 138.846360] systemd-journald[270]: Received client request to rotate journal, rotating. worker2 # [ 138.864909] systemd-journald[270]: Vacuuming done, freed 0B of archived journals from /var/log/journal/7c021b444f874723a0e9fb1639a0b9ab. worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds) worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service worker1 # [ 138.928998] systemd[1]: Stopping Nix worker-protocol over gRPC... worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds) client: must succeed: sleep 5.0 worker1 # [ 139.687092] nix-grpc-daemon[1159]: ts=2026-09-23T05:14:56Z level=info event=draining client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker1 # [ 144.286967] nix-daemon[835]: reaped child process 1362, status = succeeded worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=' worker1 # [ 144.413518] nix-daemon[835]: accepted connection from pid 1403, user nix-grpc-daemon (trusted) worker1 # [ 144.435302] nix-daemon[835]: reaped child process 1406, status = succeeded lb # [ 144.398608] niks3-server[1152]: 2026/09/23 05:15:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 144.456456] nix-grpc-daemon[1239]: time=2026-09-23T05:15:01.010Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 144.458028] nix-grpc-daemon[1239]: time=2026-09-23T05:15:01.010Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)" lb # [ 144.471323] niks3-server[1152]: 2026/09/23 05:15:01 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst lb # [ 144.492142] niks3-server[1152]: 2026/09/23 05:15:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign lb # [ 144.502881] niks3-server[1152]: 2026/09/23 05:15:01 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv lb # [ 144.519561] niks3-server[1152]: 2026/09/23 05:15:01 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls lb # [ 144.529784] niks3-server[1152]: 2026/09/23 05:15:01 INFO Signed narinfos id=50 count=1 worker1 # [ 144.576116] nix-grpc-daemon[1239]: time=2026-09-23T05:15:01.131Z level=INFO msg="Uploading 1 narinfos" lb # [ 144.544824] niks3-server[1152]: 2026/09/23 05:15:01 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete lb # [ 144.560379] niks3-server[1152]: 2026/09/23 05:15:01 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo lb # [ 144.571109] niks3-server[1152]: 2026/09/23 05:15:01 INFO Completed upload id=50 worker1 # [ 144.614123] nix-grpc-daemon[1239]: time=2026-09-23T05:15:01.170Z level=INFO msg="Upload complete. (344ms)" worker1 # [ 144.633838] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37174 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 worker1 # [ 144.656445] nix-daemon[835]: reaped child process 1360, status = succeeded client # [ 144.670967] systemd[1]: rs.service: Deactivated successfully. client # [ 144.676396] systemd[1]: rs.service: Consumed 194ms CPU time over 8.625s wall clock time, 39.4M memory peak, 7.4K incoming IP traffic, 8.5K outgoing IP traffic. worker1 # [ 144.688906] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:01Z level=info event=signal_exit lb # [ 144.932655] niks3-server[1152]: 2026/09/23 05:15:01 INFO lead: released remote=192.168.1.3:43296 lb # [ 145.631815] niks3-server[1152]: 2026/09/23 05:15:02 INFO lead: acquired remote=192.168.1.4:44912 worker2 # [ 145.674597] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=scheduler_take_over worker2 # [ 145.975188] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 145.982220] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:02Z level=info event=scheduler_restarting worker1 # [ 145.992383] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:02Z level=info event=scheduler_restarting addr=lb:50051 worker1 # [ 145.997571] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:36902 worker1 # [ 146.003551] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:02Z level=info event=worker_gone addr=192.168.1.4:50051 worker1 # [ 146.009512] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:36902 worker1 # [ 146.014572] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:02Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 146.044456] nix-daemon[835]: reaped child process 1212, status = succeeded worker2 # [ 146.179139] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 146.184484] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream" worker1 # [ 146.207655] nix-grpc-daemon[1159]: ts=2026-09-23T05:15:02Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker2 # [ 146.386005] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 146.408165] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 146.413871] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:43578 worker2 # [ 146.419540] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:02Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.23 seconds) client: waiting for success: ! systemctl is-active rs client: (finished: waiting for success: ! systemctl is-active rs, in 0.08 seconds) client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; } client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.06 seconds) client: must fail: journalctl -u rs | grep -q 'reconnecting' client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.11 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 146.844341] nix-grpc-daemon[1159]: E0000 00:00:1790140503.399790 1205 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true worker1 # [ 146.851280] nix-grpc-daemon[1159]: E0000 00:00:1790140503.400272 1205 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds) worker1 # [ 147.880389] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 147.886773] systemd[1]: Stopped Nix worker-protocol over gRPC. worker1 # [ 147.890410] systemd[1]: nix-grpc-daemon.service: Consumed 1.633s CPU time over 32.381s wall clock time, 39.2M memory peak, 48.2K incoming IP traffic, 99.6K outgoing IP traffic. lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 147.914666] systemd[1]: Starting Nix worker-protocol over gRPC... lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.13 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 148.083525] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 148.089786] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 148.112135] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds) worker1 # [ 148.136326] nix-grpc-daemon[1413]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 148.137822] nix-grpc-daemon[1413]: E0000 00:00:1790140504.651225 1413 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 148.149001] nix-grpc-daemon[1413]: E0000 00:00:1790140504.655207 1413 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 148.152911] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 148.154710] nix-grpc-daemon[1413]: E0000 00:00:1790140504.663271 1453 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 148.157138] nix-grpc-daemon[1413]: E0000 00:00:1790140504.682646 1453 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 148.163438] nix-grpc-daemon[1413]: E0000 00:00:1790140504.682845 1453 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 148.168127] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 148.169059] nix-grpc-daemon[1413]: E0000 00:00:1790140504.703955 1453 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 148.171558] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:04Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 148.175670] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:04Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:43578 worker2 # [ 148.180985] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:04Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb # [ 148.374533] envoy[900]: conn peer=[2001:db8:1::1]:39886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1540 ms=8399 lb # [ 148.380490] envoy[900]: conn peer=[2001:db8:1::3]:50484 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2608 tx=921 ms=30461 lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds) (finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.33 seconds) subtest: scheduler restart mid-build: build finishes, no double build client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid client # [ 149.397500] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid. client # Running as unit: mid.service; invocation ID: 77846b5a7d4949f9b6817840c77f1b8d client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.12 seconds) worker1 # [ 149.634237] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:38116 worker1 # [ 149.648424] nix-daemon[835]: accepted connection from pid 1413, user nix-grpc-daemon (trusted) client # [ 149.670751] nix[1435]: don't know how to build these paths: client # [ 149.673255] nix[1435]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker2 # [ 149.686669] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:06Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker2 # [ 149.692250] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:06Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 149.750150] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43594 worker2 # [ 149.767245] nix-daemon[858]: accepted connection from pid 1531, user nix-grpc-daemon (trusted) worker2 # [ 149.794291] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 1/5) worker2 # [ 149.851441] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 182 ms (attempt 2/5) worker2 # [ 150.035572] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 3/5) worker2 # [ 150.057328] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 515 ms (attempt 4/5) worker2 # [ 150.576232] nix-grpc-daemon[1531]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 150.582824] nix-grpc-daemon[1531]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 150.603187] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:43594 client # [ 150.620166] nix[1435]: copying 1 paths to 'lb:50051'... worker2 # [ 150.619122] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43594 client # [ 150.623658] nix[1435]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'... worker2 # [ 150.640583] nix-daemon[858]: accepted connection from pid 1531, user nix-grpc-daemon (trusted) worker2 # [ 150.674967] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:07Z level=info event=niks3_push_started pid=1660 worker2 # [ 150.845259] nix-grpc-daemon[1660]: time=2026-09-23T05:15:07.408Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 150.953400] nix-daemon[858]: accepted connection from pid 1674, user nix-grpc-daemon (trusted) worker2 # [ 150.972856] nix-daemon[858]: reaped child process 1677, status = succeeded lb # [ 150.935637] niks3-server[1152]: 2026/09/23 05:15:07 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 150.989790] nix-grpc-daemon[1660]: time=2026-09-23T05:15:07.552Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 150.991277] nix-grpc-daemon[1660]: time=2026-09-23T05:15:07.552Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)" lb # [ 150.985549] niks3-server[1152]: 2026/09/23 05:15:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign worker2 # [ 151.040912] nix-grpc-daemon[1660]: time=2026-09-23T05:15:07.603Z level=INFO msg="Uploading 1 narinfos" lb # [ 151.000288] niks3-server[1152]: 2026/09/23 05:15:07 INFO Signed narinfos id=51 count=1 lb # [ 151.002807] niks3-server[1152]: 2026/09/23 05:15:07 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls lb # [ 151.013596] niks3-server[1152]: 2026/09/23 05:15:07 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst lb # [ 151.023906] niks3-server[1152]: 2026/09/23 05:15:07 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete worker2 # [ 151.071789] nix-grpc-daemon[1660]: time=2026-09-23T05:15:07.635Z level=INFO msg="Upload complete. (222ms)" lb # [ 151.032247] niks3-server[1152]: 2026/09/23 05:15:07 INFO Completed upload id=51 lb # [ 151.035870] niks3-server[1152]: 2026/09/23 05:15:07 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo worker2 # [ 151.080905] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43594 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 151.097719] nix-grpc-daemon[1531]: ts=2026-09-23T05:15:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43594 worker2 # [ 151.103939] nix-daemon[858]: accepted connection from pid 1531, user nix-grpc-daemon (trusted) worker2 # [ 151.109331] nix-daemon[858]: reaped child process 1655, status = succeeded worker2 # [ 151.121532] nix-daemon[858]: accepted connection from pid 1531, user nix-grpc-daemon (trusted) client # [ 151.120311] nix[1435]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service worker2 # [ 152.291766] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1531 (nix-grpc-daemon) on client request. worker2 # [ 152.293946] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request. lb # [ 152.254607] niks3-server[1152]: 2026/09/23 05:15:08 INFO lead: released remote=192.168.1.4:44912 client # [ 152.304568] nix[1435]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting worker2 # [ 152.308891] systemd[1]: Starting scheduler-token.service... worker1 # [ 152.304708] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:08Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination" client # [ 152.309718] nix[1435]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again worker2 # [ 152.317841] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL worker2 # [ 152.319315] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'. worker2 # [ 152.326208] systemd[1]: nix-grpc-daemon.service: Consumed 541ms CPU time over 16.795s wall clock time, 34.6M memory peak, 15.8K incoming IP traffic, 28.5K outgoing IP traffic. worker2 # [ 152.367903] nix-daemon[858]: reaped child process 1679, status = succeeded worker2 # [ 152.371344] nix-daemon[858]: reaped child process 1649, status = succeeded worker2 # [ 152.380531] nix-daemon[1681]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 152.393166] nix-daemon[858]: reaped child process 1681, status = failed with exit code 1 worker2 # [ 152.410614] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 152.411572] systemd[1]: Finished scheduler-token.service. worker2 # [ 152.414176] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 98ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 152.419813] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1. worker2 # [ 152.421427] systemd[1]: Starting scheduler-token.service... worker2 # [ 152.460905] systemd[1]: scheduler-token.service: Deactivated successfully. worker2 # [ 152.464158] systemd[1]: Finished scheduler-token.service. worker2 # [ 152.464983] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 48ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic. worker2 # [ 152.468807] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 152.504869] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 152.570148] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker2 # [ 152.572393] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker2 # [ 152.576228] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=1 worker2 # [ 152.582626] nix-grpc-daemon[1720]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker2 # [ 152.600042] nix-grpc-daemon[1720]: E0000 00:00:1790140509.144545 1720 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 152.602448] nix-grpc-daemon[1720]: E0000 00:00:1790140509.146570 1720 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. lb # [ 152.561075] niks3-server[1152]: 2026/09/23 05:15:09 INFO lead: acquired remote=192.168.1.4:59880 worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.38 seconds) client: must succeed: sleep 5.0 worker2 # [ 152.638976] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker2 # [ 152.641192] nix-grpc-daemon[1720]: E0000 00:00:1790140509.162550 1763 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 152.643684] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=scheduler_take_over worker2 # [ 152.644887] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:40866 worker2 # [ 152.649263] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 152.653733] nix-grpc-daemon[1720]: E0000 00:00:1790140509.187564 1763 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 152.656775] nix-grpc-daemon[1720]: E0000 00:00:1790140509.187722 1763 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker2 # [ 152.659610] nix-grpc-daemon[1720]: E0000 00:00:1790140509.202340 1763 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker2 # [ 152.663961] systemd[1]: Started Nix worker-protocol over gRPC. worker2 # [ 152.664908] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 152.666146] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=oidc_jwks provider=mock keys=1 worker2 # [ 152.667380] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:40866 worker2 # [ 152.669136] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=0 worker2 # [ 152.812251] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=0 worker2 # [ 152.818149] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:09Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 152.863034] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38116 worker1 # [ 152.880228] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 78 ms (attempt 1/5) worker1 # [ 152.967425] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 2/5) worker1 # [ 153.054334] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 117 ms (attempt 3/5) worker1 # [ 153.173340] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 253 ms (attempt 4/5) worker1 # [ 153.429827] nix-grpc-daemon[1413]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 153.437676] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 153.452910] nix-grpc-daemon[1413]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'... worker1 # [ 153.478313] nix-daemon[835]: accepted connection from pid 1413, user nix-grpc-daemon (trusted) client # [ 153.500722] nix[1435]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051... worker1 # [ 153.504257] nix-daemon[835]: accepted connection from pid 1413, user nix-grpc-daemon (trusted) client: (finished: must succeed: sleep 5.0, in 5.05 seconds) worker1 # [ 157.924595] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:14Z level=info event=niks3_push_started pid=1516 client: waiting for success: ! systemctl is-active mid worker1 # [ 157.960532] nix-daemon[835]: reaped child process 1491, status = succeeded worker1 # [ 158.059406] nix-grpc-daemon[1516]: time=2026-09-23T05:15:14.615Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" worker1 # [ 158.175127] nix-daemon[835]: accepted connection from pid 1529, user nix-grpc-daemon (trusted) worker1 # [ 158.190505] nix-daemon[835]: reaped child process 1532, status = succeeded lb # [ 158.157558] niks3-server[1152]: 2026/09/23 05:15:14 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 158.212064] nix-grpc-daemon[1516]: time=2026-09-23T05:15:14.767Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 158.213528] nix-grpc-daemon[1516]: time=2026-09-23T05:15:14.767Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)" lb # [ 158.219067] niks3-server[1152]: 2026/09/23 05:15:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign lb # [ 158.232082] niks3-server[1152]: 2026/09/23 05:15:14 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls lb # [ 158.244879] niks3-server[1152]: 2026/09/23 05:15:14 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker1 # [ 158.303201] nix-grpc-daemon[1516]: time=2026-09-23T05:15:14.857Z level=INFO msg="Uploading 1 narinfos" lb # [ 158.258908] niks3-server[1152]: 2026/09/23 05:15:14 INFO Signed narinfos id=52 count=1 lb # [ 158.268909] niks3-server[1152]: 2026/09/23 05:15:14 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst lb # [ 158.276804] niks3-server[1152]: 2026/09/23 05:15:14 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete lb # [ 158.283040] niks3-server[1152]: 2026/09/23 05:15:14 INFO Completed upload id=52 worker1 # [ 158.325924] nix-grpc-daemon[1516]: time=2026-09-23T05:15:14.882Z level=INFO msg="Upload complete. (262ms)" lb # [ 158.286795] niks3-server[1152]: 2026/09/23 05:15:14 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo worker1 # [ 158.340388] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38116 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 worker1 # [ 158.357398] nix-daemon[835]: reaped child process 1489, status = succeeded client # [ 158.374024] systemd[1]: mid.service: Deactivated successfully. client # [ 158.379030] systemd[1]: mid.service: Consumed 236ms CPU time over 8.979s wall clock time, 43.5M memory peak, 8.7K incoming IP traffic, 10.3K outgoing IP traffic. lb # [ 158.375814] envoy[900]: conn peer=[2001:db8:1::4]:43202 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2709 tx=393 ms=16596 lb # [ 158.381233] envoy[900]: conn peer=[2001:db8:1::1]:39328 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3927 tx=1942 ms=8727 client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds) client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; } client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds) (finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.80 seconds) subtest: interrupting the client stops the build on the worker client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr client # [ 159.144529] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr. client # Running as unit: intr.service; invocation ID: d09c4e7be0b340bb930643bd0ea74810 client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds) worker2 # [ 159.325937] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 159.338743] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) client # [ 159.354954] nix[1471]: don't know how to build these paths: client # [ 159.356339] nix[1471]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker2 # [ 159.368592] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:15Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=0 worker2 # [ 159.375312] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:15Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 159.422872] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 159.438876] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5) worker2 # [ 159.463021] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 2/5) worker2 # [ 159.531812] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 388 ms (attempt 3/5) worker2 # [ 159.921758] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 278 ms (attempt 4/5) worker2 # [ 160.229851] nix-grpc-daemon[1720]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker2 # [ 160.236233] nix-grpc-daemon[1720]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker2 # [ 160.253530] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 client # [ 160.268404] nix[1471]: copying 1 paths to 'lb:50051'... worker2 # [ 160.268748] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 client # [ 160.268700] nix[1471]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'... worker2 # [ 160.285169] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 160.317165] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:16Z level=info event=niks3_push_started pid=1797 worker2 # [ 160.478632] nix-grpc-daemon[1797]: time=2026-09-23T05:15:17.040Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca="" worker2 # [ 160.586014] nix-daemon[858]: accepted connection from pid 1814, user nix-grpc-daemon (trusted) worker2 # [ 160.605976] nix-daemon[858]: reaped child process 1817, status = succeeded lb # [ 160.563145] niks3-server[1152]: 2026/09/23 05:15:17 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 160.618310] nix-grpc-daemon[1797]: time=2026-09-23T05:15:17.181Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 160.622087] nix-grpc-daemon[1797]: time=2026-09-23T05:15:17.181Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)" lb # [ 160.606979] niks3-server[1152]: 2026/09/23 05:15:17 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst worker2 # [ 160.660063] nix-grpc-daemon[1797]: time=2026-09-23T05:15:17.223Z level=INFO msg="Uploading 1 narinfos" lb # [ 160.623161] niks3-server[1152]: 2026/09/23 05:15:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign lb # [ 160.631934] niks3-server[1152]: 2026/09/23 05:15:17 INFO Signed narinfos id=53 count=1 lb # [ 160.640542] niks3-server[1152]: 2026/09/23 05:15:17 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls lb # [ 160.643918] niks3-server[1152]: 2026/09/23 05:15:17 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete worker2 # [ 160.692983] nix-grpc-daemon[1797]: time=2026-09-23T05:15:17.256Z level=INFO msg="Upload complete. (215ms)" lb # [ 160.652119] niks3-server[1152]: 2026/09/23 05:15:17 INFO Completed upload id=53 lb # [ 160.654623] niks3-server[1152]: 2026/09/23 05:15:17 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo worker2 # [ 160.700846] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=424 worker2 # [ 160.714038] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 160.723028] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 160.725515] nix-daemon[858]: reaped child process 1795, status = succeeded worker2 # [ 160.737711] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) client # [ 160.736595] nix[1471]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051... client: must succeed: systemctl kill -s INT intr worker2 # [ 162.796463] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:19Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv client # [ 162.795036] systemd[1]: intr.service: Sent signal SIGINT to main process 1471 (nix) on client request. client # [ 162.801170] nix[1471]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED) client # [ 162.813927] nix[1471]: error: interrupted by the user client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds) client # [ 162.839944] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE client # [ 162.845025] systemd[1]: intr.service: Failed with result 'exit-code'. client # [ 162.847326] systemd[1]: intr.service: Consumed 171ms CPU time over 3.698s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic. worker2 # [ 162.963364] nix-daemon[1821]: unexpected Nix daemon error: error: interrupted by the user worker2 # [ 162.968687] nix-daemon[858]: reaped child process 1819, status = succeeded worker2 # [ 162.973786] nix-daemon[858]: reaped child process 1821, status = failed with exit code 1 (finished: subtest: interrupting the client stops the build on the worker, in 4.99 seconds) subtest: a deploy mid-build drains: the build finishes, the new generation takes over client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw client # [ 164.158618] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw. client # Running as unit: sw.service; invocation ID: 6f7747b2456d46d9bfe51f4489d710df client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds) client # [ 164.367447] nix[1492]: don't know how to build these paths: client # [ 164.368548] nix[1492]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker1 # [ 164.366886] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:38116 worker2 # [ 164.380757] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:20Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=0 worker2 # [ 164.386140] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:20Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 164.426940] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38116 worker1 # [ 164.443342] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:38116 client # [ 164.458325] nix[1492]: copying 1 paths to 'lb:50051'... client # [ 164.463823] nix[1492]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'... worker1 # [ 164.463135] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38116 worker1 # [ 164.478157] nix-daemon[835]: accepted connection from pid 1413, user nix-grpc-daemon (trusted) worker1 # [ 164.644367] nix-daemon[835]: accepted connection from pid 1564, user nix-grpc-daemon (trusted) worker1 # [ 164.665229] nix-daemon[835]: reaped child process 1567, status = succeeded lb # [ 164.627974] niks3-server[1152]: 2026/09/23 05:15:21 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 164.683294] nix-grpc-daemon[1516]: time=2026-09-23T05:15:21.238Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 164.685220] nix-grpc-daemon[1516]: time=2026-09-23T05:15:21.238Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)" worker1 # [ 164.703269] nix-grpc-daemon[1516]: time=2026-09-23T05:15:21.258Z level=INFO msg="Uploading 1 narinfos" lb # [ 164.660428] niks3-server[1152]: 2026/09/23 05:15:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign lb # [ 164.675931] niks3-server[1152]: 2026/09/23 05:15:21 INFO Signed narinfos id=54 count=1 lb # [ 164.690363] niks3-server[1152]: 2026/09/23 05:15:21 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst lb # [ 164.704610] niks3-server[1152]: 2026/09/23 05:15:21 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls lb # [ 164.717587] niks3-server[1152]: 2026/09/23 05:15:21 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete worker1 # [ 164.756474] nix-grpc-daemon[1516]: time=2026-09-23T05:15:21.312Z level=INFO msg="Upload complete. (261ms)" lb # [ 164.726161] niks3-server[1152]: 2026/09/23 05:15:21 INFO Completed upload id=54 worker1 # [ 164.769199] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38116 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 164.729730] niks3-server[1152]: 2026/09/23 05:15:21 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo worker1 # [ 164.784762] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38116 worker1 # [ 164.794488] nix-daemon[835]: accepted connection from pid 1413, user nix-grpc-daemon (trusted) worker1 # [ 164.800777] nix-daemon[835]: reaped child process 1562, status = succeeded worker1 # [ 164.814668] nix-daemon[835]: accepted connection from pid 1413, user nix-grpc-daemon (trusted) client # [ 164.814081] nix[1492]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051... worker1: must succeed: systemctl reload nix-grpc-daemon.service worker1 # [ 166.780221] systemd[1]: Reloading Nix worker-protocol over gRPC... worker1 # [ 166.816983] systemd[1]: Reloaded Nix worker-protocol over gRPC. worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds) worker1: must succeed: pgrep -f 'read -t [9]0 x' worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2 client # don't know how to build these paths: client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker1 # [ 167.097156] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:38116 worker2 # [ 167.094423] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40882 worker1 # [ 167.111117] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:23Z level=info event=draining worker2 # [ 167.119912] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:23Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=0 worker2 # [ 167.126552] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:23Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 167.171055] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 167.186183] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 client # copying 1 paths to 'lb:50051'... worker2 # [ 167.202156] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'... worker2 # [ 167.225650] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 167.383826] nix-daemon[858]: accepted connection from pid 1876, user nix-grpc-daemon (trusted) worker2 # [ 167.402369] nix-daemon[858]: reaped child process 1879, status = succeeded lb # [ 167.367383] niks3-server[1152]: 2026/09/23 05:15:23 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 167.419436] nix-grpc-daemon[1797]: time=2026-09-23T05:15:23.982Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 167.421234] nix-grpc-daemon[1797]: time=2026-09-23T05:15:23.982Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)" lb # [ 167.396627] niks3-server[1152]: 2026/09/23 05:15:23 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst worker2 # [ 167.452265] nix-grpc-daemon[1797]: time=2026-09-23T05:15:24.016Z level=INFO msg="Uploading 1 narinfos" lb # [ 167.409981] niks3-server[1152]: 2026/09/23 05:15:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign lb # [ 167.422763] niks3-server[1152]: 2026/09/23 05:15:23 INFO Signed narinfos id=55 count=1 lb # [ 167.428956] niks3-server[1152]: 2026/09/23 05:15:23 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls worker2 # [ 167.483580] nix-grpc-daemon[1797]: time=2026-09-23T05:15:24.047Z level=INFO msg="Upload complete. (240ms)" lb # [ 167.443203] niks3-server[1152]: 2026/09/23 05:15:24 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete lb # [ 167.450277] niks3-server[1152]: 2026/09/23 05:15:24 INFO Completed upload id=55 lb # [ 167.453445] niks3-server[1152]: 2026/09/23 05:15:24 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo worker2 # [ 167.496595] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=528 worker2 # [ 167.511315] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 167.517632] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 167.522615] nix-daemon[858]: reaped child process 1874, status = succeeded client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051... worker2 # [ 167.534946] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv lb # [ 168.370152] envoy[900]: conn peer=[2001:db8:1::1]:41662 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=853 ms=3503 client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-during-drain worker2 # [ 168.782535] nix-daemon[858]: reaped child process 1883, status = succeeded worker2 # [ 168.910712] nix-daemon[858]: accepted connection from pid 1899, user nix-grpc-daemon (trusted) worker2 # [ 168.929865] nix-daemon[858]: reaped child process 1902, status = succeeded lb # [ 168.893465] niks3-server[1152]: 2026/09/23 05:15:25 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 168.950772] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.512Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 168.953402] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.512Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)" lb # [ 168.943934] niks3-server[1152]: 2026/09/23 05:15:25 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls worker2 # [ 168.995379] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.559Z level=INFO msg="Uploading 1 narinfos" lb # [ 168.956410] niks3-server[1152]: 2026/09/23 05:15:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign lb # [ 168.965592] niks3-server[1152]: 2026/09/23 05:15:25 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv lb # [ 168.975888] niks3-server[1152]: 2026/09/23 05:15:25 INFO Signed narinfos id=56 count=1 worker2 # [ 169.018094] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.582Z level=INFO msg="Upload complete. (272ms)" lb # [ 168.979843] niks3-server[1152]: 2026/09/23 05:15:25 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst lb # [ 168.984797] niks3-server[1152]: 2026/09/23 05:15:25 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete lb # [ 168.990195] niks3-server[1152]: 2026/09/23 05:15:25 INFO Completed upload id=56 lb # [ 168.994353] niks3-server[1152]: 2026/09/23 05:15:25 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo worker2 # [ 169.034358] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=1 worker2 # [ 169.051939] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=0 worker2 # [ 169.055646] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 169.059498] nix-daemon[858]: reaped child process 1881, status = succeeded worker2 # [ 169.090782] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 169.105975] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 client # copying 1 paths to 'lb:50051'... worker2 # [ 169.120653] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.684Z level=INFO msg="All 1 paths already cached" worker2 # [ 169.127714] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'... worker2 # [ 169.149887] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 169.303655] nix-daemon[858]: accepted connection from pid 1908, user nix-grpc-daemon (trusted) worker2 # [ 169.320803] nix-daemon[858]: reaped child process 1911, status = succeeded lb # [ 169.284114] niks3-server[1152]: 2026/09/23 05:15:25 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 169.335713] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.899Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 169.337500] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.899Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)" worker2 # [ 169.361950] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.925Z level=INFO msg="Uploading 1 narinfos" lb # [ 169.312900] niks3-server[1152]: 2026/09/23 05:15:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign lb # [ 169.325155] niks3-server[1152]: 2026/09/23 05:15:25 INFO Signed narinfos id=57 count=1 lb # [ 169.335213] niks3-server[1152]: 2026/09/23 05:15:25 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls lb # [ 169.349770] niks3-server[1152]: 2026/09/23 05:15:25 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst worker2 # [ 169.405242] nix-grpc-daemon[1797]: time=2026-09-23T05:15:25.969Z level=INFO msg="Upload complete. (238ms)" lb # [ 169.363388] niks3-server[1152]: 2026/09/23 05:15:25 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete lb # [ 169.368826] niks3-server[1152]: 2026/09/23 05:15:25 INFO Completed upload id=57 lb # [ 169.372005] niks3-server[1152]: 2026/09/23 05:15:25 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo worker2 # [ 169.414839] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=640 worker2 # [ 169.429703] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 169.436614] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 169.441554] nix-daemon[858]: reaped child process 1906, status = succeeded client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051... worker2 # [ 169.454651] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-during-drain worker2 # [ 170.712439] nix-daemon[858]: reaped child process 1915, status = succeeded worker2 # [ 170.800968] nix-daemon[858]: accepted connection from pid 1931, user nix-grpc-daemon (trusted) worker2 # [ 170.818454] nix-daemon[858]: reaped child process 1934, status = succeeded lb # [ 170.781384] niks3-server[1152]: 2026/09/23 05:15:27 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 170.834740] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.397Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 170.836407] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.397Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)" lb # [ 170.816852] niks3-server[1152]: 2026/09/23 05:15:27 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst lb # [ 170.831305] niks3-server[1152]: 2026/09/23 05:15:27 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls worker2 # [ 170.882273] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.446Z level=INFO msg="Uploading 1 narinfos" lb # [ 170.842828] niks3-server[1152]: 2026/09/23 05:15:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign lb # [ 170.853718] niks3-server[1152]: 2026/09/23 05:15:27 INFO Signed narinfos id=58 count=1 lb # [ 170.862636] niks3-server[1152]: 2026/09/23 05:15:27 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker2 # [ 170.911998] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.476Z level=INFO msg="Upload complete. (247ms)" lb # [ 170.873275] niks3-server[1152]: 2026/09/23 05:15:27 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete lb # [ 170.879142] niks3-server[1152]: 2026/09/23 05:15:27 INFO Completed upload id=58 lb # [ 170.882752] niks3-server[1152]: 2026/09/23 05:15:27 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo worker2 # [ 170.927731] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=1 worker2 # [ 170.948576] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=0 worker2 # [ 170.954391] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 170.959379] nix-daemon[858]: reaped child process 1913, status = succeeded worker2 # [ 170.990800] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 171.005848] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 171.019757] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.583Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'... worker2 # [ 171.031269] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 171.048975] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 171.206213] nix-daemon[858]: accepted connection from pid 1940, user nix-grpc-daemon (trusted) worker2 # [ 171.224526] nix-daemon[858]: reaped child process 1943, status = succeeded lb # [ 171.185975] niks3-server[1152]: 2026/09/23 05:15:27 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 171.237022] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.800Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 171.238660] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.800Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)" lb # [ 171.215592] niks3-server[1152]: 2026/09/23 05:15:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign worker2 # [ 171.266309] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.830Z level=INFO msg="Uploading 1 narinfos" lb # [ 171.231161] niks3-server[1152]: 2026/09/23 05:15:27 INFO Signed narinfos id=59 count=1 lb # [ 171.238371] niks3-server[1152]: 2026/09/23 05:15:27 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls lb # [ 171.250082] niks3-server[1152]: 2026/09/23 05:15:27 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst worker2 # [ 171.300606] nix-grpc-daemon[1797]: time=2026-09-23T05:15:27.864Z level=INFO msg="Upload complete. (234ms)" lb # [ 171.261043] niks3-server[1152]: 2026/09/23 05:15:27 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete lb # [ 171.266648] niks3-server[1152]: 2026/09/23 05:15:27 INFO Completed upload id=59 worker2 # [ 171.308318] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=768 lb # [ 171.271348] niks3-server[1152]: 2026/09/23 05:15:27 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo worker2 # [ 171.322286] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 171.331192] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 171.335040] nix-daemon[858]: reaped child process 1938, status = succeeded client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051... worker2 # [ 171.345631] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-during-drain worker2 # [ 172.581544] nix-daemon[858]: reaped child process 1947, status = succeeded worker2 # [ 172.702177] nix-daemon[858]: accepted connection from pid 1963, user nix-grpc-daemon (trusted) worker2 # [ 172.717911] nix-daemon[858]: reaped child process 1966, status = succeeded lb # [ 172.683435] niks3-server[1152]: 2026/09/23 05:15:29 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 172.736409] nix-grpc-daemon[1797]: time=2026-09-23T05:15:29.299Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 172.737945] nix-grpc-daemon[1797]: time=2026-09-23T05:15:29.299Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)" worker2 # [ 172.774747] nix-grpc-daemon[1797]: time=2026-09-23T05:15:29.338Z level=INFO msg="Uploading 1 narinfos" lb # [ 172.723208] niks3-server[1152]: 2026/09/23 05:15:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign lb # [ 172.738779] niks3-server[1152]: 2026/09/23 05:15:29 INFO Signed narinfos id=60 count=1 lb # [ 172.749247] niks3-server[1152]: 2026/09/23 05:15:29 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst lb # [ 172.764155] niks3-server[1152]: 2026/09/23 05:15:29 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv lb # [ 172.767517] niks3-server[1152]: 2026/09/23 05:15:29 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls lb # [ 172.773396] niks3-server[1152]: 2026/09/23 05:15:29 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete worker2 # [ 172.814896] nix-grpc-daemon[1797]: time=2026-09-23T05:15:29.378Z level=INFO msg="Upload complete. (253ms)" lb # [ 172.776593] niks3-server[1152]: 2026/09/23 05:15:29 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo lb # [ 172.780756] niks3-server[1152]: 2026/09/23 05:15:29 INFO Completed upload id=60 worker2 # [ 172.827422] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=1 worker2 # [ 172.846099] nix-daemon[858]: reaped child process 1945, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.00 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds) worker1: must succeed: pkill -f 'read -t [9]0 x' worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds) client: waiting for success: ! systemctl is-active sw worker1 # [ 173.327984] nix-daemon[835]: reaped child process 1571, status = succeeded worker1 # [ 173.396780] nix-daemon[835]: accepted connection from pid 1613, user nix-grpc-daemon (trusted) worker1 # [ 173.414766] nix-daemon[835]: reaped child process 1616, status = succeeded lb # [ 173.376376] niks3-server[1152]: 2026/09/23 05:15:29 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 173.428367] nix-grpc-daemon[1516]: time=2026-09-23T05:15:29.983Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 173.429985] nix-grpc-daemon[1516]: time=2026-09-23T05:15:29.984Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)" lb # [ 173.419300] niks3-server[1152]: 2026/09/23 05:15:29 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls worker1 # [ 173.470801] nix-grpc-daemon[1516]: time=2026-09-23T05:15:30.026Z level=INFO msg="Uploading 1 narinfos" lb # [ 173.433734] niks3-server[1152]: 2026/09/23 05:15:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign lb # [ 173.444541] niks3-server[1152]: 2026/09/23 05:15:29 INFO Signed narinfos id=61 count=1 lb # [ 173.452239] niks3-server[1152]: 2026/09/23 05:15:30 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst worker1 # [ 173.501221] nix-grpc-daemon[1516]: time=2026-09-23T05:15:30.057Z level=INFO msg="Upload complete. (264ms)" lb # [ 173.461688] niks3-server[1152]: 2026/09/23 05:15:30 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv lb # [ 173.467041] niks3-server[1152]: 2026/09/23 05:15:30 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete lb # [ 173.472801] niks3-server[1152]: 2026/09/23 05:15:30 INFO Completed upload id=61 lb # [ 173.475772] niks3-server[1152]: 2026/09/23 05:15:30 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo worker1 # [ 173.517149] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38116 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=1 worker1 # [ 173.532784] nix-daemon[835]: reaped child process 1569, status = succeeded client # [ 173.550287] systemd[1]: sw.service: Deactivated successfully. client # [ 173.555543] systemd[1]: sw.service: Consumed 197ms CPU time over 9.392s wall clock time, 43.5M memory peak, 7.4K incoming IP traffic, 8.4K outgoing IP traffic. worker1 # [ 174.114515] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:30Z level=info event=signal_exit client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds) client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; } client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds) worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service worker1 # [ 176.120951] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:32Z level=info event=scheduler_restarting worker2 # [ 176.120325] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:40866 worker2 # [ 176.120637] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:32Z level=info event=worker_gone addr=192.168.1.3:50051 worker1 # [ 176.146553] nix-grpc-daemon[1413]: ts=2026-09-23T05:15:32Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited" worker1 # [ 176.170304] nix-daemon[835]: reaped child process 1470, status = succeeded worker1 # [ 176.955039] nix-grpc-daemon[1413]: E0000 00:00:1790140533.511022 1459 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true worker1 # [ 176.962852] nix-grpc-daemon[1413]: E0000 00:00:1790140533.518916 1459 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true worker1 # [ 178.142782] systemd[1]: nix-grpc-daemon.service: Deactivated successfully. worker1 # [ 178.148548] systemd[1]: nix-grpc-daemon.service: Consumed 872ms CPU time over 30.232s wall clock time, 36M memory peak, 21.6K incoming IP traffic, 28.5K outgoing IP traffic. worker1 # [ 178.173052] systemd[1]: Starting Nix worker-protocol over gRPC... worker1 # [ 178.316720] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:34Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear" worker1 # [ 178.318737] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:34Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc worker1 # [ 178.322790] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:34Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=1 worker1 # [ 178.328049] nix-grpc-daemon[1630]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR worker1 # [ 178.348179] nix-grpc-daemon[1630]: E0000 00:00:1790140534.883894 1630 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 178.350522] nix-grpc-daemon[1630]: E0000 00:00:1790140534.887460 1630 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 4.11 seconds) lb: waiting until exactly no endpoints of aarch64-linux are unhealthy lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 178.377671] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:34Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket worker1 # [ 178.379834] nix-grpc-daemon[1630]: E0000 00:00:1790140534.894801 1673 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 178.382561] systemd[1]: Started Nix worker-protocol over gRPC. worker1 # [ 178.383374] nix-grpc-daemon[1630]: E0000 00:00:1790140534.916489 1673 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 178.385780] nix-grpc-daemon[1630]: E0000 00:00:1790140534.917748 1673 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates. worker1 # [ 178.388097] nix-grpc-daemon[1630]: E0000 00:00:1790140534.933184 1673 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates. worker1 # [ 178.396102] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:34Z level=info event=scheduler_connected addr=lb:50051 worker2 # [ 178.398700] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:34Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:40866 worker2 # [ 178.404602] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:34Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=0 lb # [ 178.377285] envoy[900]: conn peer=[2001:db8:1::1]:53242 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9608 tx=4375 ms=5758 lb # [ 178.377612] envoy[900]: conn peer=[2001:db8:1::1]:53230 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1537 ms=9163 lb # [ 178.377734] envoy[900]: conn peer=[2001:db8:1::3]:57540 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1020 tx=482 ms=27966 lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.54 seconds) subtest: queue gauges show a burst while every build blocks lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]' client # [ 179.856803] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]". client # Running as unit: burst.service; invocation ID: c3bec4e0958d42ccb08dbf9526dc5b11 client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics client # [ 180.049720] nix[1548]: don't know how to build these paths: client # [ 180.050694] nix[1548]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv client # [ 180.051728] nix[1548]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker2 # [ 180.049026] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40882 client # [ 180.053377] nix[1548]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv client # [ 180.054376] nix[1548]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker2 # [ 180.064078] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=want client=5 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=0 worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) worker2 # [ 180.071301] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 180.076502] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=want client=5 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=0 worker2 # [ 180.080860] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 180.085751] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=want client=5 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=0 worker2 # [ 180.089821] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=want client=5 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=0 worker2 # [ 180.115402] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker1 # [ 180.122262] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker2 # [ 180.131718] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker1 # [ 180.135987] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) client # [ 180.147317] nix[1548]: copying 1 paths to 'lb:50051'... worker2 # [ 180.146591] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 client # [ 180.149872] nix[1548]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'... worker2 # [ 180.161408] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker1 # [ 180.161786] nix-grpc-daemon[1630]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 1/5) worker1 # [ 180.234706] nix-grpc-daemon[1630]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 2/5) worker1 # [ 180.259268] nix-grpc-daemon[1630]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 363 ms (attempt 3/5) worker2 # [ 180.307194] nix-daemon[858]: accepted connection from pid 1981, user nix-grpc-daemon (trusted) worker2 # [ 180.327125] nix-daemon[858]: reaped child process 1984, status = succeeded lb # [ 180.291467] niks3-server[1152]: 2026/09/23 05:15:36 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 180.344332] nix-grpc-daemon[1797]: time=2026-09-23T05:15:36.907Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 180.345977] nix-grpc-daemon[1797]: time=2026-09-23T05:15:36.907Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)" lb # [ 180.329051] niks3-server[1152]: 2026/09/23 05:15:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign worker2 # [ 180.379817] nix-grpc-daemon[1797]: time=2026-09-23T05:15:36.943Z level=INFO msg="Uploading 1 narinfos" lb # [ 180.340366] niks3-server[1152]: 2026/09/23 05:15:36 INFO Signed narinfos id=62 count=1 lb # [ 180.352214] niks3-server[1152]: 2026/09/23 05:15:36 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst lb # [ 180.362379] niks3-server[1152]: 2026/09/23 05:15:36 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls lb # [ 180.374163] niks3-server[1152]: 2026/09/23 05:15:36 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete worker2 # [ 180.423169] nix-grpc-daemon[1797]: time=2026-09-23T05:15:36.987Z level=INFO msg="Upload complete. (246ms)" lb # [ 180.382720] niks3-server[1152]: 2026/09/23 05:15:36 INFO Completed upload id=62 lb # [ 180.386347] niks3-server[1152]: 2026/09/23 05:15:36 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo worker2 # [ 180.437052] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=416 worker2 # [ 180.452936] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 180.463293] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 180.466672] nix-daemon[858]: reaped child process 1979, status = succeeded client # [ 180.479178] nix[1548]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051... worker2 # [ 180.482897] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker1 # [ 180.631976] nix-grpc-daemon[1630]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 185 ms (attempt 4/5) worker1 # [ 180.819816] nix-grpc-daemon[1630]: disabling binary cache 'https://cache.nixos.org' for 60 seconds worker1 # [ 180.825787] nix-grpc-daemon[1630]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org worker1 # [ 180.844613] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41406 client # [ 180.860169] nix[1548]: copying 1 paths to 'lb:50051'... worker1 # [ 180.860888] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 client # [ 180.862508] nix[1548]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'... worker1 # [ 180.876442] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 180.911191] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:37Z level=info event=niks3_push_started pid=1691 worker1 # [ 181.051879] nix-grpc-daemon[1691]: time=2026-09-23T05:15:37.607Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca="" lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters worker1 # [ 181.162954] nix-daemon[835]: accepted connection from pid 1704, user nix-grpc-daemon (trusted) worker1 # [ 181.183522] nix-daemon[835]: reaped child process 1707, status = succeeded lb # [ 181.145544] niks3-server[1152]: 2026/09/23 05:15:37 INFO Received uploads request method=POST path=/api/pending_closures lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1 # [ 181.196250] nix-grpc-daemon[1691]: time=2026-09-23T05:15:37.752Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 181.197681] nix-grpc-daemon[1691]: time=2026-09-23T05:15:37.752Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)" lb # [ 181.179639] niks3-server[1152]: 2026/09/23 05:15:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign worker1 # [ 181.226746] nix-grpc-daemon[1691]: time=2026-09-23T05:15:37.782Z level=INFO msg="Uploading 1 narinfos" lb # [ 181.184126] niks3-server[1152]: 2026/09/23 05:15:37 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls lb # [ 181.188705] niks3-server[1152]: 2026/09/23 05:15:37 INFO Signed narinfos id=63 count=1 lb # [ 181.192521] niks3-server[1152]: 2026/09/23 05:15:37 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst lb # [ 181.202150] niks3-server[1152]: 2026/09/23 05:15:37 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete worker1 # [ 181.247791] nix-grpc-daemon[1691]: time=2026-09-23T05:15:37.804Z level=INFO msg="Upload complete. (192ms)" lb # [ 181.206611] niks3-server[1152]: 2026/09/23 05:15:37 INFO Completed upload id=63 lb # [ 181.210378] niks3-server[1152]: 2026/09/23 05:15:37 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo worker1 # [ 181.256716] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=416 worker1 # [ 181.260291] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 181.269494] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds) worker1 # [ 181.273478] nix-daemon[835]: reaped child process 1689, status = succeeded worker1 # [ 181.288224] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) client # [ 181.286663] nix[1548]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051... worker2 # [ 182.718580] nix-daemon[858]: reaped child process 1988, status = succeeded worker2 # [ 182.820558] nix-daemon[858]: accepted connection from pid 2016, user nix-grpc-daemon (trusted) worker2 # [ 182.843117] nix-daemon[858]: reaped child process 2019, status = succeeded lb # [ 182.800610] niks3-server[1152]: 2026/09/23 05:15:39 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 182.852526] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.415Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 182.854002] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.415Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)" lb # [ 182.845738] niks3-server[1152]: 2026/09/23 05:15:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign worker2 # [ 182.896869] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.460Z level=INFO msg="Uploading 1 narinfos" lb # [ 182.859599] niks3-server[1152]: 2026/09/23 05:15:39 INFO Signed narinfos id=64 count=1 lb # [ 182.867448] niks3-server[1152]: 2026/09/23 05:15:39 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv lb # [ 182.878035] niks3-server[1152]: 2026/09/23 05:15:39 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls worker2 # [ 182.923607] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.487Z level=INFO msg="Upload complete. (227ms)" lb # [ 182.884564] niks3-server[1152]: 2026/09/23 05:15:39 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst lb # [ 182.890397] niks3-server[1152]: 2026/09/23 05:15:39 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete lb # [ 182.894098] niks3-server[1152]: 2026/09/23 05:15:39 INFO Completed upload id=64 lb # [ 182.895927] niks3-server[1152]: 2026/09/23 05:15:39 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo worker2 # [ 182.937034] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=2 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=7 outputs=1 worker2 # [ 182.950515] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:39Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 182.954558] nix-daemon[858]: reaped child process 1986, status = succeeded worker2 # [ 182.979009] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 182.994782] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 client # [ 183.009411] nix[1548]: copying 1 paths to 'lb:50051'... worker2 # [ 183.009774] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 client # [ 183.014249] nix[1548]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'... worker2 # [ 183.023815] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 183.170654] nix-daemon[858]: accepted connection from pid 2023, user nix-grpc-daemon (trusted) worker2 # [ 183.191158] nix-daemon[858]: reaped child process 2026, status = succeeded lb # [ 183.147413] niks3-server[1152]: 2026/09/23 05:15:39 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 183.198824] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.762Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 183.200550] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.762Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)" worker2 # [ 183.221639] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.785Z level=INFO msg="Uploading 1 narinfos" lb # [ 183.179538] niks3-server[1152]: 2026/09/23 05:15:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign lb # [ 183.191371] niks3-server[1152]: 2026/09/23 05:15:39 INFO Signed narinfos id=65 count=1 lb # [ 183.201304] niks3-server[1152]: 2026/09/23 05:15:39 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls lb # [ 183.216785] niks3-server[1152]: 2026/09/23 05:15:39 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst worker2 # [ 183.266942] nix-grpc-daemon[1797]: time=2026-09-23T05:15:39.830Z level=INFO msg="Upload complete. (228ms)" lb # [ 183.226140] niks3-server[1152]: 2026/09/23 05:15:39 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete lb # [ 183.231766] niks3-server[1152]: 2026/09/23 05:15:39 INFO Completed upload id=65 worker2 # [ 183.276304] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 183.236733] niks3-server[1152]: 2026/09/23 05:15:39 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo worker2 # [ 183.294146] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 183.301243] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 183.306399] nix-daemon[858]: reaped child process 2021, status = succeeded worker2 # [ 183.317725] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) client # [ 183.317195] nix[1548]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051... worker1 # [ 183.999877] nix-daemon[835]: reaped child process 1714, status = succeeded worker1 # [ 184.077504] nix-daemon[835]: accepted connection from pid 1740, user nix-grpc-daemon (trusted) worker1 # [ 184.096461] nix-daemon[835]: reaped child process 1743, status = succeeded lb # [ 184.058340] niks3-server[1152]: 2026/09/23 05:15:40 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 184.112003] nix-grpc-daemon[1691]: time=2026-09-23T05:15:40.665Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 184.114377] nix-grpc-daemon[1691]: time=2026-09-23T05:15:40.666Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)" lb # [ 184.096697] niks3-server[1152]: 2026/09/23 05:15:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign worker1 # [ 184.147839] nix-grpc-daemon[1691]: time=2026-09-23T05:15:40.703Z level=INFO msg="Uploading 1 narinfos" lb # [ 184.112722] niks3-server[1152]: 2026/09/23 05:15:40 INFO Signed narinfos id=66 count=1 lb # [ 184.121748] niks3-server[1152]: 2026/09/23 05:15:40 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls lb # [ 184.134076] niks3-server[1152]: 2026/09/23 05:15:40 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst worker1 # [ 184.188137] nix-grpc-daemon[1691]: time=2026-09-23T05:15:40.744Z level=INFO msg="Upload complete. (282ms)" lb # [ 184.145968] niks3-server[1152]: 2026/09/23 05:15:40 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv lb # [ 184.150480] niks3-server[1152]: 2026/09/23 05:15:40 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete lb # [ 184.155410] niks3-server[1152]: 2026/09/23 05:15:40 INFO Completed upload id=66 lb # [ 184.158349] niks3-server[1152]: 2026/09/23 05:15:40 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo worker2 # [ 184.199553] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:40Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 184.201126] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=2 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=8 outputs=1 worker1 # [ 184.218226] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 184.223864] nix-daemon[835]: reaped child process 1709, status = succeeded worker1 # [ 184.231440] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41406 client # [ 184.245326] nix[1548]: copying 1 paths to 'lb:50051'... worker1 # [ 184.243793] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 client # [ 184.249312] nix[1548]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'... worker1 # [ 184.256289] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 184.393408] nix-daemon[835]: accepted connection from pid 1747, user nix-grpc-daemon (trusted) worker1 # [ 184.410446] nix-daemon[835]: reaped child process 1750, status = succeeded lb # [ 184.373613] niks3-server[1152]: 2026/09/23 05:15:40 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 184.424064] nix-grpc-daemon[1691]: time=2026-09-23T05:15:40.980Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 184.425514] nix-grpc-daemon[1691]: time=2026-09-23T05:15:40.980Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)" lb # [ 184.404176] niks3-server[1152]: 2026/09/23 05:15:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign worker1 # [ 184.454118] nix-grpc-daemon[1691]: time=2026-09-23T05:15:41.010Z level=INFO msg="Uploading 1 narinfos" lb # [ 184.419561] niks3-server[1152]: 2026/09/23 05:15:40 INFO Signed narinfos id=67 count=1 lb # [ 184.426324] niks3-server[1152]: 2026/09/23 05:15:40 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls lb # [ 184.436918] niks3-server[1152]: 2026/09/23 05:15:40 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst worker1 # [ 184.485138] nix-grpc-daemon[1691]: time=2026-09-23T05:15:41.041Z level=INFO msg="Upload complete. (216ms)" lb # [ 184.443706] niks3-server[1152]: 2026/09/23 05:15:41 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete lb # [ 184.448681] niks3-server[1152]: 2026/09/23 05:15:41 INFO Completed upload id=67 worker1 # [ 184.495288] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=416 lb # [ 184.454639] niks3-server[1152]: 2026/09/23 05:15:41 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo worker1 # [ 184.507476] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 184.517365] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 184.520478] nix-daemon[835]: reaped child process 1745, status = succeeded client # [ 184.530919] nix[1548]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051... worker1 # [ 184.533649] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker2 # [ 185.185528] nix-daemon[858]: reaped child process 2030, status = succeeded worker2 # [ 185.309429] nix-daemon[858]: accepted connection from pid 2055, user nix-grpc-daemon (trusted) worker2 # [ 185.325502] nix-daemon[858]: reaped child process 2058, status = succeeded lb # [ 185.290693] niks3-server[1152]: 2026/09/23 05:15:41 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 185.344456] nix-grpc-daemon[1797]: time=2026-09-23T05:15:41.905Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 185.346302] nix-grpc-daemon[1797]: time=2026-09-23T05:15:41.906Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)" lb # [ 185.339755] niks3-server[1152]: 2026/09/23 05:15:41 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls worker2 # [ 185.393173] nix-grpc-daemon[1797]: time=2026-09-23T05:15:41.956Z level=INFO msg="Uploading 1 narinfos" lb # [ 185.352224] niks3-server[1152]: 2026/09/23 05:15:41 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst lb # [ 185.366299] niks3-server[1152]: 2026/09/23 05:15:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign lb # [ 185.378565] niks3-server[1152]: 2026/09/23 05:15:41 INFO Signed narinfos id=68 count=1 lb # [ 185.384618] niks3-server[1152]: 2026/09/23 05:15:41 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker2 # [ 185.424757] nix-grpc-daemon[1797]: time=2026-09-23T05:15:41.988Z level=INFO msg="Upload complete. (291ms)" lb # [ 185.390757] niks3-server[1152]: 2026/09/23 05:15:41 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete lb # [ 185.394636] niks3-server[1152]: 2026/09/23 05:15:41 INFO Completed upload id=68 lb # [ 185.397239] niks3-server[1152]: 2026/09/23 05:15:41 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo worker2 # [ 185.437317] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=2 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=9 outputs=1 worker2 # [ 185.455087] nix-daemon[858]: reaped child process 2028, status = succeeded worker1 # [ 186.157992] nix-daemon[835]: reaped child process 1754, status = succeeded worker1 # [ 186.262730] nix-daemon[835]: accepted connection from pid 1779, user nix-grpc-daemon (trusted) worker1 # [ 186.281807] nix-daemon[835]: reaped child process 1782, status = succeeded lb # [ 186.244155] niks3-server[1152]: 2026/09/23 05:15:42 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 186.296408] nix-grpc-daemon[1691]: time=2026-09-23T05:15:42.851Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 186.297887] nix-grpc-daemon[1691]: time=2026-09-23T05:15:42.851Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)" lb # [ 186.284376] niks3-server[1152]: 2026/09/23 05:15:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign worker1 # [ 186.332652] nix-grpc-daemon[1691]: time=2026-09-23T05:15:42.888Z level=INFO msg="Uploading 1 narinfos" lb # [ 186.300816] niks3-server[1152]: 2026/09/23 05:15:42 INFO Signed narinfos id=69 count=1 lb # [ 186.303384] niks3-server[1152]: 2026/09/23 05:15:42 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst worker1 # [ 186.361606] nix-grpc-daemon[1691]: time=2026-09-23T05:15:42.917Z level=INFO msg="Upload complete. (218ms)" lb # [ 186.320485] niks3-server[1152]: 2026/09/23 05:15:42 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls lb # [ 186.323199] niks3-server[1152]: 2026/09/23 05:15:42 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv lb # [ 186.328272] niks3-server[1152]: 2026/09/23 05:15:42 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete lb # [ 186.331606] niks3-server[1152]: 2026/09/23 05:15:42 INFO Completed upload id=69 worker1 # [ 186.371263] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=10 outputs=1 lb # [ 186.334343] niks3-server[1152]: 2026/09/23 05:15:42 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo worker1 # [ 186.385545] nix-daemon[835]: reaped child process 1752, status = succeeded client # [ 186.411514] systemd[1]: burst.service: Deactivated successfully. client # [ 186.415831] systemd[1]: burst.service: Consumed 219ms CPU time over 6.558s wall clock time, 51.6M memory peak, 14K incoming IP traffic, 18.9K outgoing IP traffic. client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; } client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) (finished: subtest: queue gauges show a burst while every build blocks, in 8.06 seconds) subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1 worker1 # [ 187.909616] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 187.919781] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41406 worker2 # [ 187.964696] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:44Z level=debug event=want client=6 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=0 worker2 # [ 187.970131] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:44Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 187.976207] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:44Z level=debug event=want client=6 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=0 worker2 # [ 187.981621] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:44Z level=debug event=want client=6 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=0 worker1 # [ 188.011009] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 188.026602] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 188.041934] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 188.057104] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 188.216827] nix-daemon[835]: accepted connection from pid 1790, user nix-grpc-daemon (trusted) worker1 # [ 188.239353] nix-daemon[835]: reaped child process 1793, status = succeeded lb # [ 188.197504] niks3-server[1152]: 2026/09/23 05:15:44 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 188.250488] nix-grpc-daemon[1691]: time=2026-09-23T05:15:44.806Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 188.252082] nix-grpc-daemon[1691]: time=2026-09-23T05:15:44.806Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)" lb # [ 188.230714] niks3-server[1152]: 2026/09/23 05:15:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign lb # [ 188.247897] niks3-server[1152]: 2026/09/23 05:15:44 INFO Signed narinfos id=70 count=1 worker1 # [ 188.290828] nix-grpc-daemon[1691]: time=2026-09-23T05:15:44.846Z level=INFO msg="Uploading 1 narinfos" lb # [ 188.259413] niks3-server[1152]: 2026/09/23 05:15:44 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls lb # [ 188.276375] niks3-server[1152]: 2026/09/23 05:15:44 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst lb # [ 188.290580] niks3-server[1152]: 2026/09/23 05:15:44 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete lb # [ 188.297675] niks3-server[1152]: 2026/09/23 05:15:44 INFO Completed upload id=70 worker1 # [ 188.338310] nix-grpc-daemon[1691]: time=2026-09-23T05:15:44.894Z level=INFO msg="Upload complete. (264ms)" lb # [ 188.300413] niks3-server[1152]: 2026/09/23 05:15:44 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo worker1 # [ 188.349833] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 188.368079] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 188.375531] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 188.381932] nix-daemon[835]: reaped child process 1788, status = succeeded worker1 # [ 188.395256] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) lb # [ 188.373323] envoy[900]: conn peer=[2001:db8:1::1]:58694 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9688 tx=5208 ms=6345 worker1 # [ 189.439259] nix-daemon[835]: reaped child process 1797, status = succeeded worker1 # [ 189.529119] nix-daemon[835]: accepted connection from pid 1813, user nix-grpc-daemon (trusted) worker1 # [ 189.549938] nix-daemon[835]: reaped child process 1816, status = succeeded lb # [ 189.506434] niks3-server[1152]: 2026/09/23 05:15:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 189.558988] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.113Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 189.560683] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.113Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)" lb # [ 189.543869] niks3-server[1152]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls worker1 # [ 189.598485] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.154Z level=INFO msg="Uploading 1 narinfos" lb # [ 189.557461] niks3-server[1152]: 2026/09/23 05:15:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign lb # [ 189.567060] niks3-server[1152]: 2026/09/23 05:15:46 INFO Signed narinfos id=71 count=1 lb # [ 189.572500] niks3-server[1152]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker1 # [ 189.628102] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.184Z level=INFO msg="Upload complete. (217ms)" lb # [ 189.586236] niks3-server[1152]: 2026/09/23 05:15:46 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete lb # [ 189.594292] niks3-server[1152]: 2026/09/23 05:15:46 INFO Completed upload id=71 worker1 # [ 189.637207] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=11 outputs=1 lb # [ 189.597764] niks3-server[1152]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst worker2 # [ 189.638809] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:46Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=1 lb # [ 189.602353] niks3-server[1152]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo worker1 # [ 189.652523] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker2 # [ 189.650860] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:46Z level=debug event=want client=6 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=0 worker1 # [ 189.658503] nix-daemon[835]: reaped child process 1795, status = succeeded worker1 # [ 189.665452] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 189.677025] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 189.686615] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 189.815344] nix-daemon[835]: accepted connection from pid 1820, user nix-grpc-daemon (trusted) worker1 # [ 189.836473] nix-daemon[835]: reaped child process 1823, status = succeeded lb # [ 189.796564] niks3-server[1152]: 2026/09/23 05:15:46 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 189.847933] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.403Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 189.849649] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.403Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)" lb # [ 189.828985] niks3-server[1152]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst worker1 # [ 189.878563] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.434Z level=INFO msg="Uploading 1 narinfos" lb # [ 189.843846] niks3-server[1152]: 2026/09/23 05:15:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign worker1 # [ 189.899097] nix-grpc-daemon[1691]: time=2026-09-23T05:15:46.455Z level=INFO msg="Upload complete. (202ms)" lb # [ 189.857859] niks3-server[1152]: 2026/09/23 05:15:46 INFO Signed narinfos id=72 count=1 lb # [ 189.864225] niks3-server[1152]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls worker1 # [ 189.906909] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=512 lb # [ 189.867550] niks3-server[1152]: 2026/09/23 05:15:46 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete lb # [ 189.872460] niks3-server[1152]: 2026/09/23 05:15:46 INFO Completed upload id=72 lb # [ 189.875124] niks3-server[1152]: 2026/09/23 05:15:46 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo worker1 # [ 189.919037] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 189.924654] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 189.928959] nix-daemon[835]: reaped child process 1818, status = succeeded worker1 # [ 189.937912] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 191.173342] nix-daemon[835]: reaped child process 1827, status = succeeded worker1 # [ 191.285184] nix-daemon[835]: accepted connection from pid 1843, user nix-grpc-daemon (trusted) worker1 # [ 191.304839] nix-daemon[835]: reaped child process 1846, status = succeeded lb # [ 191.267599] niks3-server[1152]: 2026/09/23 05:15:47 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 191.320374] nix-grpc-daemon[1691]: time=2026-09-23T05:15:47.875Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 191.321821] nix-grpc-daemon[1691]: time=2026-09-23T05:15:47.875Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)" lb # [ 191.308916] niks3-server[1152]: 2026/09/23 05:15:47 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst worker1 # [ 191.364929] nix-grpc-daemon[1691]: time=2026-09-23T05:15:47.921Z level=INFO msg="Uploading 1 narinfos" lb # [ 191.324004] niks3-server[1152]: 2026/09/23 05:15:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign lb # [ 191.339009] niks3-server[1152]: 2026/09/23 05:15:47 INFO Signed narinfos id=73 count=1 lb # [ 191.346396] niks3-server[1152]: 2026/09/23 05:15:47 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls worker1 # [ 191.400727] nix-grpc-daemon[1691]: time=2026-09-23T05:15:47.956Z level=INFO msg="Upload complete. (252ms)" lb # [ 191.354283] niks3-server[1152]: 2026/09/23 05:15:47 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv lb # [ 191.364566] niks3-server[1152]: 2026/09/23 05:15:47 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete lb # [ 191.369472] niks3-server[1152]: 2026/09/23 05:15:47 INFO Completed upload id=73 worker2 # [ 191.408805] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:47Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=1 lb # [ 191.372910] niks3-server[1152]: 2026/09/23 05:15:47 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo worker1 # [ 191.413248] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=12 outputs=1 worker2 # [ 191.422669] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:47Z level=debug event=want client=6 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=0 worker1 # [ 191.429796] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 191.435897] nix-daemon[835]: reaped child process 1825, status = succeeded worker1 # [ 191.443530] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 191.455202] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 191.467237] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 191.600211] nix-daemon[835]: accepted connection from pid 1850, user nix-grpc-daemon (trusted) worker1 # [ 191.618982] nix-daemon[835]: reaped child process 1853, status = succeeded lb # [ 191.582900] niks3-server[1152]: 2026/09/23 05:15:48 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 191.633760] nix-grpc-daemon[1691]: time=2026-09-23T05:15:48.189Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 191.635440] nix-grpc-daemon[1691]: time=2026-09-23T05:15:48.189Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)" worker1 # [ 191.657079] nix-grpc-daemon[1691]: time=2026-09-23T05:15:48.213Z level=INFO msg="Uploading 1 narinfos" lb # [ 191.615346] niks3-server[1152]: 2026/09/23 05:15:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign lb # [ 191.628736] niks3-server[1152]: 2026/09/23 05:15:48 INFO Signed narinfos id=74 count=1 lb # [ 191.631833] niks3-server[1152]: 2026/09/23 05:15:48 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls lb # [ 191.641555] niks3-server[1152]: 2026/09/23 05:15:48 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst lb # [ 191.652743] niks3-server[1152]: 2026/09/23 05:15:48 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete lb # [ 191.657369] niks3-server[1152]: 2026/09/23 05:15:48 INFO Completed upload id=74 worker1 # [ 191.693546] nix-grpc-daemon[1691]: time=2026-09-23T05:15:48.249Z level=INFO msg="Upload complete. (214ms)" lb # [ 191.661209] niks3-server[1152]: 2026/09/23 05:15:48 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo worker1 # [ 191.703926] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=512 worker1 # [ 191.712440] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 191.722301] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 191.727105] nix-daemon[835]: reaped child process 1848, status = succeeded worker1 # [ 191.738990] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 193.210250] nix-daemon[835]: reaped child process 1857, status = succeeded worker1 # [ 193.341322] nix-daemon[835]: accepted connection from pid 1873, user nix-grpc-daemon (trusted) worker1 # [ 193.361432] nix-daemon[835]: reaped child process 1876, status = succeeded lb # [ 193.317964] niks3-server[1152]: 2026/09/23 05:15:49 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 193.371566] nix-grpc-daemon[1691]: time=2026-09-23T05:15:49.926Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker1 # [ 193.373444] nix-grpc-daemon[1691]: time=2026-09-23T05:15:49.926Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)" lb # [ 193.359416] niks3-server[1152]: 2026/09/23 05:15:49 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker1 # [ 193.413658] nix-grpc-daemon[1691]: time=2026-09-23T05:15:49.969Z level=INFO msg="Uploading 1 narinfos" lb # [ 193.372930] niks3-server[1152]: 2026/09/23 05:15:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign lb # [ 193.382538] niks3-server[1152]: 2026/09/23 05:15:49 INFO Signed narinfos id=75 count=1 lb # [ 193.390061] niks3-server[1152]: 2026/09/23 05:15:49 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst lb # [ 193.397457] niks3-server[1152]: 2026/09/23 05:15:49 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls worker1 # [ 193.439028] nix-grpc-daemon[1691]: time=2026-09-23T05:15:49.995Z level=INFO msg="Upload complete. (285ms)" lb # [ 193.400182] niks3-server[1152]: 2026/09/23 05:15:49 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete lb # [ 193.404363] niks3-server[1152]: 2026/09/23 05:15:49 INFO Completed upload id=75 lb # [ 193.407128] niks3-server[1152]: 2026/09/23 05:15:49 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo worker2 # [ 193.448580] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:50Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 193.451107] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=13 outputs=1 worker2 # [ 193.461178] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:50Z level=debug event=want client=6 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=0 worker1 # [ 193.472479] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 193.479031] nix-daemon[835]: reaped child process 1855, status = succeeded worker1 # [ 193.486701] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 193.498838] nix-grpc-daemon[1691]: time=2026-09-23T05:15:50.055Z level=INFO msg="All 1 paths already cached" worker1 # [ 193.507989] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 193.522944] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 193.661565] nix-daemon[835]: accepted connection from pid 1880, user nix-grpc-daemon (trusted) worker1 # [ 193.681578] nix-daemon[835]: reaped child process 1883, status = succeeded lb # [ 193.644333] niks3-server[1152]: 2026/09/23 05:15:50 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 193.695322] nix-grpc-daemon[1691]: time=2026-09-23T05:15:50.251Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 193.696965] nix-grpc-daemon[1691]: time=2026-09-23T05:15:50.251Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)" lb # [ 193.674215] niks3-server[1152]: 2026/09/23 05:15:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign worker1 # [ 193.722421] nix-grpc-daemon[1691]: time=2026-09-23T05:15:50.278Z level=INFO msg="Uploading 1 narinfos" lb # [ 193.688894] niks3-server[1152]: 2026/09/23 05:15:50 INFO Signed narinfos id=76 count=1 lb # [ 193.696470] niks3-server[1152]: 2026/09/23 05:15:50 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls lb # [ 193.702704] niks3-server[1152]: 2026/09/23 05:15:50 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst worker1 # [ 193.754500] nix-grpc-daemon[1691]: time=2026-09-23T05:15:50.310Z level=INFO msg="Upload complete. (221ms)" lb # [ 193.712892] niks3-server[1152]: 2026/09/23 05:15:50 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete lb # [ 193.718239] niks3-server[1152]: 2026/09/23 05:15:50 INFO Completed upload id=76 lb # [ 193.721741] niks3-server[1152]: 2026/09/23 05:15:50 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo worker1 # [ 193.764802] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=624 worker1 # [ 193.775442] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 193.782133] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 193.786608] nix-daemon[835]: reaped child process 1878, status = succeeded worker1 # [ 193.797954] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 194.634515] nix-daemon[835]: reaped child process 1887, status = succeeded worker1 # [ 194.744446] nix-daemon[835]: accepted connection from pid 1903, user nix-grpc-daemon (trusted) worker1 # [ 194.762484] nix-daemon[835]: reaped child process 1906, status = succeeded lb # [ 194.727861] niks3-server[1152]: 2026/09/23 05:15:51 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 194.780498] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.336Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 194.782008] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.336Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)" lb # [ 194.770180] niks3-server[1152]: 2026/09/23 05:15:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign worker1 # [ 194.822337] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.378Z level=INFO msg="Uploading 1 narinfos" lb # [ 194.786383] niks3-server[1152]: 2026/09/23 05:15:51 INFO Signed narinfos id=77 count=1 lb # [ 194.793045] niks3-server[1152]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv lb # [ 194.801347] niks3-server[1152]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst lb # [ 194.809001] niks3-server[1152]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls lb # [ 194.812336] niks3-server[1152]: 2026/09/23 05:15:51 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete lb # [ 194.817363] niks3-server[1152]: 2026/09/23 05:15:51 INFO Completed upload id=77 worker1 # [ 194.858628] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.414Z level=INFO msg="Upload complete. (249ms)" lb # [ 194.819907] niks3-server[1152]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo worker2 # [ 194.871565] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:51Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 194.873726] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=14 outputs=1 worker2 # [ 194.881638] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:51Z level=debug event=want client=6 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=0 worker1 # [ 194.888454] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 194.899810] nix-daemon[835]: reaped child process 1885, status = succeeded worker1 # [ 194.904555] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 194.913174] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.469Z level=INFO msg="All 1 paths already cached" worker1 # [ 194.921628] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 194.935697] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 195.072436] nix-daemon[835]: accepted connection from pid 1910, user nix-grpc-daemon (trusted) worker1 # [ 195.091681] nix-daemon[835]: reaped child process 1913, status = succeeded lb # [ 195.054473] niks3-server[1152]: 2026/09/23 05:15:51 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 195.105118] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.661Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 195.107097] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.663Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)" lb # [ 195.086150] niks3-server[1152]: 2026/09/23 05:15:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign worker1 # [ 195.135503] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.691Z level=INFO msg="Uploading 1 narinfos" lb # [ 195.100888] niks3-server[1152]: 2026/09/23 05:15:51 INFO Signed narinfos id=78 count=1 lb # [ 195.108490] niks3-server[1152]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst lb # [ 195.121797] niks3-server[1152]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls worker1 # [ 195.169674] nix-grpc-daemon[1691]: time=2026-09-23T05:15:51.725Z level=INFO msg="Upload complete. (223ms)" lb # [ 195.129659] niks3-server[1152]: 2026/09/23 05:15:51 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete lb # [ 195.135155] niks3-server[1152]: 2026/09/23 05:15:51 INFO Completed upload id=78 worker1 # [ 195.180303] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 195.139784] niks3-server[1152]: 2026/09/23 05:15:51 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo worker1 # [ 195.196743] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 195.203282] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 195.209013] nix-daemon[835]: reaped child process 1908, status = succeeded worker1 # [ 195.224469] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 196.292186] nix-daemon[835]: reaped child process 1917, status = succeeded worker1 # [ 196.386672] nix-daemon[835]: accepted connection from pid 1933, user nix-grpc-daemon (trusted) worker1 # [ 196.405329] nix-daemon[835]: reaped child process 1936, status = succeeded lb # [ 196.369417] niks3-server[1152]: 2026/09/23 05:15:52 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 196.424448] nix-grpc-daemon[1691]: time=2026-09-23T05:15:52.978Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 196.426165] nix-grpc-daemon[1691]: time=2026-09-23T05:15:52.978Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)" lb # [ 196.419373] niks3-server[1152]: 2026/09/23 05:15:52 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls worker1 # [ 196.472634] nix-grpc-daemon[1691]: time=2026-09-23T05:15:53.028Z level=INFO msg="Uploading 1 narinfos" lb # [ 196.432003] niks3-server[1152]: 2026/09/23 05:15:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign lb # [ 196.444713] niks3-server[1152]: 2026/09/23 05:15:52 INFO Signed narinfos id=79 count=1 lb # [ 196.452466] niks3-server[1152]: 2026/09/23 05:15:53 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst lb # [ 196.464945] niks3-server[1152]: 2026/09/23 05:15:53 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker1 # [ 196.507459] nix-grpc-daemon[1691]: time=2026-09-23T05:15:53.063Z level=INFO msg="Upload complete. (272ms)" lb # [ 196.470713] niks3-server[1152]: 2026/09/23 05:15:53 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete lb # [ 196.476112] niks3-server[1152]: 2026/09/23 05:15:53 INFO Completed upload id=79 worker2 # [ 196.519568] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:53Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=1 lb # [ 196.480815] niks3-server[1152]: 2026/09/23 05:15:53 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo worker1 # [ 196.522332] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=15 outputs=1 worker2 # [ 196.532920] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:53Z level=debug event=want client=6 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=0 worker1 # [ 196.541862] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 196.548890] nix-daemon[835]: reaped child process 1915, status = succeeded worker1 # [ 196.556957] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 196.569960] nix-grpc-daemon[1691]: time=2026-09-23T05:15:53.126Z level=INFO msg="All 1 paths already cached" worker1 # [ 196.579461] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 196.594909] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 196.732710] nix-daemon[835]: accepted connection from pid 1940, user nix-grpc-daemon (trusted) worker1 # [ 196.753959] nix-daemon[835]: reaped child process 1943, status = succeeded lb # [ 196.715024] niks3-server[1152]: 2026/09/23 05:15:53 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 196.766187] nix-grpc-daemon[1691]: time=2026-09-23T05:15:53.322Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 196.767831] nix-grpc-daemon[1691]: time=2026-09-23T05:15:53.322Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)" worker1 # [ 196.793960] nix-grpc-daemon[1691]: time=2026-09-23T05:15:53.349Z level=INFO msg="Uploading 1 narinfos" lb # [ 196.744654] niks3-server[1152]: 2026/09/23 05:15:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign lb # [ 196.760203] niks3-server[1152]: 2026/09/23 05:15:53 INFO Signed narinfos id=80 count=1 lb # [ 196.763376] niks3-server[1152]: 2026/09/23 05:15:53 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst lb # [ 196.775269] niks3-server[1152]: 2026/09/23 05:15:53 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls worker1 # [ 196.827984] nix-grpc-daemon[1691]: time=2026-09-23T05:15:53.384Z level=INFO msg="Upload complete. (222ms)" lb # [ 196.784813] niks3-server[1152]: 2026/09/23 05:15:53 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete lb # [ 196.791312] niks3-server[1152]: 2026/09/23 05:15:53 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo worker1 # [ 196.835010] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=624 lb # [ 196.798898] niks3-server[1152]: 2026/09/23 05:15:53 INFO Completed upload id=80 worker1 # [ 196.849769] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 196.856451] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 196.861151] nix-daemon[835]: reaped child process 1938, status = succeeded worker1 # [ 196.874645] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 198.381789] nix-daemon[835]: reaped child process 1947, status = succeeded worker1 # [ 198.456113] nix-daemon[835]: accepted connection from pid 1963, user nix-grpc-daemon (trusted) worker1 # [ 198.474538] nix-daemon[835]: reaped child process 1966, status = succeeded lb # [ 198.437601] niks3-server[1152]: 2026/09/23 05:15:55 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 198.492456] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.047Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker1 # [ 198.494224] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.047Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)" lb # [ 198.491222] niks3-server[1152]: 2026/09/23 05:15:55 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst worker1 # [ 198.539622] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.095Z level=INFO msg="Uploading 1 narinfos" lb # [ 198.503256] niks3-server[1152]: 2026/09/23 05:15:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign lb # [ 198.512661] niks3-server[1152]: 2026/09/23 05:15:55 INFO Signed narinfos id=81 count=1 lb # [ 198.514492] niks3-server[1152]: 2026/09/23 05:15:55 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv lb # [ 198.519602] niks3-server[1152]: 2026/09/23 05:15:55 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls lb # [ 198.528356] niks3-server[1152]: 2026/09/23 05:15:55 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete worker1 # [ 198.569816] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.126Z level=INFO msg="Upload complete. (283ms)" lb # [ 198.531941] niks3-server[1152]: 2026/09/23 05:15:55 INFO Completed upload id=81 lb # [ 198.533973] niks3-server[1152]: 2026/09/23 05:15:55 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo worker1 # [ 198.581337] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=16 outputs=1 worker2 # [ 198.585927] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:55Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=1 worker2 # [ 198.594344] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:55Z level=debug event=want client=6 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=0 worker1 # [ 198.603947] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 198.611011] nix-daemon[835]: reaped child process 1945, status = succeeded worker1 # [ 198.619171] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 198.633122] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.189Z level=INFO msg="All 2 paths already cached" worker1 # [ 198.642675] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 198.657392] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 198.801255] nix-daemon[835]: accepted connection from pid 1970, user nix-grpc-daemon (trusted) worker1 # [ 198.822156] nix-daemon[835]: reaped child process 1973, status = succeeded lb # [ 198.778413] niks3-server[1152]: 2026/09/23 05:15:55 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 198.830226] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.386Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 198.831932] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.386Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)" lb # [ 198.810799] niks3-server[1152]: 2026/09/23 05:15:55 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst worker1 # [ 198.863825] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.419Z level=INFO msg="Uploading 1 narinfos" lb # [ 198.824696] niks3-server[1152]: 2026/09/23 05:15:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign lb # [ 198.833804] niks3-server[1152]: 2026/09/23 05:15:55 INFO Signed narinfos id=82 count=1 lb # [ 198.841156] niks3-server[1152]: 2026/09/23 05:15:55 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls worker1 # [ 198.888207] nix-grpc-daemon[1691]: time=2026-09-23T05:15:55.444Z level=INFO msg="Upload complete. (218ms)" lb # [ 198.848137] niks3-server[1152]: 2026/09/23 05:15:55 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete lb # [ 198.852269] niks3-server[1152]: 2026/09/23 05:15:55 INFO Completed upload id=82 lb # [ 198.855096] niks3-server[1152]: 2026/09/23 05:15:55 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo worker1 # [ 198.896122] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 198.911388] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 198.917400] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 198.922211] nix-daemon[835]: reaped child process 1968, status = succeeded worker1 # [ 198.934598] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 200.008254] nix-daemon[835]: reaped child process 1977, status = succeeded worker1 # [ 200.114078] nix-daemon[835]: accepted connection from pid 1993, user nix-grpc-daemon (trusted) worker1 # [ 200.129803] nix-daemon[835]: reaped child process 1996, status = succeeded lb # [ 200.095655] niks3-server[1152]: 2026/09/23 05:15:56 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 200.151655] nix-grpc-daemon[1691]: time=2026-09-23T05:15:56.704Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 200.153542] nix-grpc-daemon[1691]: time=2026-09-23T05:15:56.704Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)" lb # [ 200.135826] niks3-server[1152]: 2026/09/23 05:15:56 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv lb # [ 200.148595] niks3-server[1152]: 2026/09/23 05:15:56 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst worker1 # [ 200.201133] nix-grpc-daemon[1691]: time=2026-09-23T05:15:56.757Z level=INFO msg="Uploading 1 narinfos" lb # [ 200.162644] niks3-server[1152]: 2026/09/23 05:15:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign lb # [ 200.170234] niks3-server[1152]: 2026/09/23 05:15:56 INFO Signed narinfos id=83 count=1 lb # [ 200.177254] niks3-server[1152]: 2026/09/23 05:15:56 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls worker1 # [ 200.218509] nix-grpc-daemon[1691]: time=2026-09-23T05:15:56.774Z level=INFO msg="Upload complete. (273ms)" lb # [ 200.181845] niks3-server[1152]: 2026/09/23 05:15:56 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete lb # [ 200.186234] niks3-server[1152]: 2026/09/23 05:15:56 INFO Completed upload id=83 lb # [ 200.188783] niks3-server[1152]: 2026/09/23 05:15:56 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo worker2 # [ 200.230194] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:56Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=1 worker1 # [ 200.232126] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=17 outputs=1 worker1 # [ 200.247539] nix-daemon[835]: reaped child process 1975, status = succeeded worker1 # [ 200.285094] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 200.300139] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 200.314025] nix-grpc-daemon[1691]: time=2026-09-23T05:15:56.870Z level=INFO msg="All 2 paths already cached" worker1 # [ 200.324119] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 200.336489] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 200.486362] nix-daemon[835]: accepted connection from pid 2000, user nix-grpc-daemon (trusted) worker1 # [ 200.504719] nix-daemon[835]: reaped child process 2003, status = succeeded lb # [ 200.467794] niks3-server[1152]: 2026/09/23 05:15:57 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 200.519063] nix-grpc-daemon[1691]: time=2026-09-23T05:15:57.075Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 200.520739] nix-grpc-daemon[1691]: time=2026-09-23T05:15:57.075Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)" lb # [ 200.496288] niks3-server[1152]: 2026/09/23 05:15:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign worker1 # [ 200.544913] nix-grpc-daemon[1691]: time=2026-09-23T05:15:57.101Z level=INFO msg="Uploading 1 narinfos" lb # [ 200.510598] niks3-server[1152]: 2026/09/23 05:15:57 INFO Signed narinfos id=84 count=1 lb # [ 200.517637] niks3-server[1152]: 2026/09/23 05:15:57 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls worker1 # [ 200.573271] nix-grpc-daemon[1691]: time=2026-09-23T05:15:57.129Z level=INFO msg="Upload complete. (217ms)" lb # [ 200.530121] niks3-server[1152]: 2026/09/23 05:15:57 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst worker1 # [ 200.580155] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=752 lb # [ 200.543316] niks3-server[1152]: 2026/09/23 05:15:57 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete lb # [ 200.548795] niks3-server[1152]: 2026/09/23 05:15:57 INFO Completed upload id=84 worker1 # [ 200.591211] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 lb # [ 200.551269] niks3-server[1152]: 2026/09/23 05:15:57 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo worker1 # [ 200.601342] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 200.607884] nix-daemon[835]: reaped child process 1998, status = succeeded worker1 # [ 200.620343] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 201.927725] nix-daemon[835]: reaped child process 2007, status = succeeded worker1 # [ 202.000610] nix-daemon[835]: accepted connection from pid 2023, user nix-grpc-daemon (trusted) worker1 # [ 202.018240] nix-daemon[835]: reaped child process 2026, status = succeeded lb # [ 201.981424] niks3-server[1152]: 2026/09/23 05:15:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 202.034719] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.589Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 202.036546] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.589Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)" lb # [ 202.026339] niks3-server[1152]: 2026/09/23 05:15:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign worker1 # [ 202.077983] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.633Z level=INFO msg="Uploading 1 narinfos" lb # [ 202.041254] niks3-server[1152]: 2026/09/23 05:15:58 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv lb # [ 202.049222] niks3-server[1152]: 2026/09/23 05:15:58 INFO Signed narinfos id=85 count=1 lb # [ 202.053904] niks3-server[1152]: 2026/09/23 05:15:58 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls worker1 # [ 202.105038] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.661Z level=INFO msg="Upload complete. (274ms)" lb # [ 202.062788] niks3-server[1152]: 2026/09/23 05:15:58 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst lb # [ 202.066788] niks3-server[1152]: 2026/09/23 05:15:58 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete lb # [ 202.074117] niks3-server[1152]: 2026/09/23 05:15:58 INFO Completed upload id=85 worker2 # [ 202.115684] nix-grpc-daemon[1720]: ts=2026-09-23T05:15:58Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=1 lb # [ 202.077214] niks3-server[1152]: 2026/09/23 05:15:58 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo worker1 # [ 202.117497] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=18 outputs=1 worker1 # [ 202.134844] nix-daemon[835]: reaped child process 2005, status = succeeded worker1 # [ 202.163352] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 202.178640] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 202.193423] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.749Z level=INFO msg="All 2 paths already cached" worker1 # [ 202.203065] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 202.219542] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 202.367285] nix-daemon[835]: accepted connection from pid 2030, user nix-grpc-daemon (trusted) worker1 # [ 202.386528] nix-daemon[835]: reaped child process 2033, status = succeeded lb # [ 202.342082] niks3-server[1152]: 2026/09/23 05:15:58 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 202.393265] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.949Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 202.394888] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.949Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)" lb # [ 202.371610] niks3-server[1152]: 2026/09/23 05:15:58 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst worker1 # [ 202.427450] nix-grpc-daemon[1691]: time=2026-09-23T05:15:58.983Z level=INFO msg="Uploading 1 narinfos" lb # [ 202.388425] niks3-server[1152]: 2026/09/23 05:15:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign lb # [ 202.399639] niks3-server[1152]: 2026/09/23 05:15:58 INFO Signed narinfos id=86 count=1 lb # [ 202.406305] niks3-server[1152]: 2026/09/23 05:15:58 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls lb # [ 202.415650] niks3-server[1152]: 2026/09/23 05:15:58 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete lb # [ 202.421614] niks3-server[1152]: 2026/09/23 05:15:58 INFO Completed upload id=86 lb # [ 202.425106] niks3-server[1152]: 2026/09/23 05:15:58 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo worker1 # [ 202.465446] nix-grpc-daemon[1691]: time=2026-09-23T05:15:59.019Z level=INFO msg="Upload complete. (230ms)" worker1 # [ 202.473099] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=0 paths=1 nar_bytes_in=752 worker1 # [ 202.486214] nix-grpc-daemon[1630]: ts=2026-09-23T05:15:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 worker1 # [ 202.495222] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 202.498612] nix-daemon[835]: reaped child process 2028, status = succeeded worker1 # [ 202.509420] nix-daemon[835]: accepted connection from pid 1630, user nix-grpc-daemon (trusted) worker1 # [ 203.821942] nix-daemon[835]: reaped child process 2037, status = succeeded worker1 # [ 203.892886] nix-daemon[835]: accepted connection from pid 2053, user nix-grpc-daemon (trusted) worker1 # [ 203.912196] nix-daemon[835]: reaped child process 2056, status = succeeded lb # [ 203.873728] niks3-server[1152]: 2026/09/23 05:16:00 INFO Received uploads request method=POST path=/api/pending_closures worker1 # [ 203.927677] nix-grpc-daemon[1691]: time=2026-09-23T05:16:00.481Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker1 # [ 203.929768] nix-grpc-daemon[1691]: time=2026-09-23T05:16:00.481Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)" lb # [ 203.911059] niks3-server[1152]: 2026/09/23 05:16:00 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst worker1 # [ 203.966797] nix-grpc-daemon[1691]: time=2026-09-23T05:16:00.522Z level=INFO msg="Uploading 1 narinfos" lb # [ 203.925793] niks3-server[1152]: 2026/09/23 05:16:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign lb # [ 203.937154] niks3-server[1152]: 2026/09/23 05:16:00 INFO Signed narinfos id=87 count=1 lb # [ 203.939527] niks3-server[1152]: 2026/09/23 05:16:00 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv lb # [ 203.951055] niks3-server[1152]: 2026/09/23 05:16:00 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls worker1 # [ 203.997484] nix-grpc-daemon[1691]: time=2026-09-23T05:16:00.553Z level=INFO msg="Upload complete. (274ms)" lb # [ 203.956572] niks3-server[1152]: 2026/09/23 05:16:00 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete lb # [ 203.959863] niks3-server[1152]: 2026/09/23 05:16:00 INFO Completed upload id=87 lb # [ 203.961952] niks3-server[1152]: 2026/09/23 05:16:00 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo worker1 # [ 204.013333] nix-grpc-daemon[1630]: ts=2026-09-23T05:16:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41406 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=19 outputs=1 worker1 # [ 204.029033] nix-daemon[835]: reaped child process 2035, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.43 seconds) worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]' client # [ 204.248813] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]". client # Running as unit: gpu.service; invocation ID: 0e084282888e44c496ca3174652c87a1 client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) worker1 # [ 204.447889] nix-grpc-daemon[1630]: ts=2026-09-23T05:16:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:41406 client # [ 204.455980] nix[1622]: don't know how to build these paths: client # [ 204.456993] nix[1622]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv worker2 # [ 204.461876] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 204.484134] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:01Z level=debug event=want client=7 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=0 client # [ 204.490524] nix[1622]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds) client: waiting for success: systemctl show -p Result gpu | grep -q exit-code lb # [ 208.377280] envoy[900]: conn peer=[2001:db8:1::1]:50668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27183 tx=12381 ms=16162 client # [ 234.497480] nix[1622]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip} client # [ 234.512445] nix[1622]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed client # [ 234.558053] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE client # [ 234.566778] systemd[1]: gpu.service: Failed with result 'exit-code'. client # [ 234.570323] systemd[1]: gpu.service: Consumed 203ms CPU time over 30.314s wall clock time, 37.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic. client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.18 seconds) client: must succeed: journalctl -u gpu -o cat client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds) lb: must succeed: curl -sf localhost:9901/clusters lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds) worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds) (finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 47.36 seconds) subtest: low disk drains a worker and builds go to the other worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.20 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free' worker1 # [ 236.361981] nix-grpc-daemon[1630]: ts=2026-09-23T05:16:32Z level=info event=unhealthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.11 seconds) client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2 client # don't know how to build these paths: client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker2 # [ 237.590419] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:40882 worker1 # [ 237.595856] nix-grpc-daemon[1630]: ts=2026-09-23T05:16:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:41406 worker2 # [ 237.628260] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=debug event=want client=8 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=0 worker2 # [ 237.636718] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 237.682998] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 237.699797] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'... worker2 # [ 237.715963] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 237.732928] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 237.901135] nix-daemon[858]: accepted connection from pid 2100, user nix-grpc-daemon (trusted) worker2 # [ 237.922576] nix-daemon[858]: reaped child process 2103, status = succeeded lb # [ 237.886360] niks3-server[1152]: 2026/09/23 05:16:34 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 237.941062] nix-grpc-daemon[1797]: time=2026-09-23T05:16:34.504Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 237.942836] nix-grpc-daemon[1797]: time=2026-09-23T05:16:34.504Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)" lb # [ 237.919902] niks3-server[1152]: 2026/09/23 05:16:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign worker2 # [ 237.972822] nix-grpc-daemon[1797]: time=2026-09-23T05:16:34.536Z level=INFO msg="Uploading 1 narinfos" lb # [ 237.935390] niks3-server[1152]: 2026/09/23 05:16:34 INFO Signed narinfos id=88 count=1 lb # [ 237.942708] niks3-server[1152]: 2026/09/23 05:16:34 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls lb # [ 237.952810] niks3-server[1152]: 2026/09/23 05:16:34 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst lb # [ 237.967965] niks3-server[1152]: 2026/09/23 05:16:34 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete lb # [ 237.975731] niks3-server[1152]: 2026/09/23 05:16:34 INFO Completed upload id=88 worker2 # [ 238.018212] nix-grpc-daemon[1797]: time=2026-09-23T05:16:34.582Z level=INFO msg="Upload complete. (266ms)" lb # [ 237.983064] niks3-server[1152]: 2026/09/23 05:16:34 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo worker2 # [ 238.028322] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=512 worker2 # [ 238.051962] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 238.056725] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 238.062511] nix-daemon[858]: reaped child process 2098, status = succeeded client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051... client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker2 # [ 238.076110] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) lb # [ 238.387991] envoy[900]: conn peer=[2001:db8:1::1]:40962 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=30080 client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-a> LOG-farm-a-drain worker2 # [ 239.563673] nix-daemon[858]: reaped child process 2107, status = succeeded worker2 # [ 239.657294] nix-daemon[858]: accepted connection from pid 2123, user nix-grpc-daemon (trusted) worker2 # [ 239.675782] nix-daemon[858]: reaped child process 2126, status = succeeded lb # [ 239.639420] niks3-server[1152]: 2026/09/23 05:16:36 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 239.693817] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.255Z level=INFO msg="Uploading 1 paths to lb (0 already cached)" worker2 # [ 239.695548] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.255Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)" lb # [ 239.680131] niks3-server[1152]: 2026/09/23 05:16:36 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker2 # [ 239.734483] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.298Z level=INFO msg="Uploading 1 narinfos" lb # [ 239.693811] niks3-server[1152]: 2026/09/23 05:16:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign lb # [ 239.702972] niks3-server[1152]: 2026/09/23 05:16:36 INFO Signed narinfos id=89 count=1 lb # [ 239.708455] niks3-server[1152]: 2026/09/23 05:16:36 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst lb # [ 239.720301] niks3-server[1152]: 2026/09/23 05:16:36 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls lb # [ 239.723623] niks3-server[1152]: 2026/09/23 05:16:36 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete worker2 # [ 239.771937] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.335Z level=INFO msg="Upload complete. (284ms)" lb # [ 239.730225] niks3-server[1152]: 2026/09/23 05:16:36 INFO Completed upload id=89 lb # [ 239.734481] niks3-server[1152]: 2026/09/23 05:16:36 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo worker2 # [ 239.787991] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=20 outputs=1 worker2 # [ 239.808214] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=debug event=want client=8 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=0 worker2 # [ 239.813511] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 239.818973] nix-daemon[858]: reaped child process 2105, status = succeeded worker2 # [ 239.834673] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 239.850000] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 239.863721] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.427Z level=INFO msg="All 1 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'... worker2 # [ 239.874996] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 239.890725] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 240.036932] nix-daemon[858]: accepted connection from pid 2132, user nix-grpc-daemon (trusted) worker2 # [ 240.057501] nix-daemon[858]: reaped child process 2135, status = succeeded lb # [ 240.013152] niks3-server[1152]: 2026/09/23 05:16:36 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 240.064206] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.627Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 240.065660] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.627Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)" lb # [ 240.041491] niks3-server[1152]: 2026/09/23 05:16:36 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst worker2 # [ 240.095169] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.659Z level=INFO msg="Uploading 1 narinfos" lb # [ 240.054930] niks3-server[1152]: 2026/09/23 05:16:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign lb # [ 240.062696] niks3-server[1152]: 2026/09/23 05:16:36 INFO Signed narinfos id=90 count=1 worker2 # [ 240.110686] nix-grpc-daemon[1797]: time=2026-09-23T05:16:36.674Z level=INFO msg="Upload complete. (207ms)" lb # [ 240.070816] niks3-server[1152]: 2026/09/23 05:16:36 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls lb # [ 240.077325] niks3-server[1152]: 2026/09/23 05:16:36 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete worker2 # [ 240.119801] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=632 lb # [ 240.080389] niks3-server[1152]: 2026/09/23 05:16:36 INFO Completed upload id=90 lb # [ 240.082617] niks3-server[1152]: 2026/09/23 05:16:36 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo worker2 # [ 240.128858] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 240.134390] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 240.138280] nix-daemon[858]: reaped child process 2130, status = succeeded client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051... worker2 # [ 240.149304] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-b> LOG-farm-b-drain worker2 # [ 241.832919] nix-daemon[858]: reaped child process 2139, status = succeeded worker2 # [ 241.941131] nix-daemon[858]: accepted connection from pid 2155, user nix-grpc-daemon (trusted) worker2 # [ 241.960946] nix-daemon[858]: reaped child process 2158, status = succeeded lb # [ 241.925955] niks3-server[1152]: 2026/09/23 05:16:38 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 241.984352] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.544Z level=INFO msg="Uploading 1 paths to lb (1 already cached)" worker2 # [ 241.986293] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.544Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)" lb # [ 241.970018] niks3-server[1152]: 2026/09/23 05:16:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign worker2 # [ 242.023139] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.586Z level=INFO msg="Uploading 1 narinfos" lb # [ 241.986422] niks3-server[1152]: 2026/09/23 05:16:38 INFO Signed narinfos id=91 count=1 lb # [ 241.994301] niks3-server[1152]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst lb # [ 242.008344] niks3-server[1152]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls lb # [ 242.011924] niks3-server[1152]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker2 # [ 242.063191] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.627Z level=INFO msg="Upload complete. (302ms)" lb # [ 242.024005] niks3-server[1152]: 2026/09/23 05:16:38 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete lb # [ 242.030057] niks3-server[1152]: 2026/09/23 05:16:38 INFO Completed upload id=91 lb # [ 242.032465] niks3-server[1152]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo worker2 # [ 242.079368] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=21 outputs=1 worker2 # [ 242.102326] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:38Z level=debug event=want client=8 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=0 worker2 # [ 242.107797] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:38Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=1 worker2 # [ 242.113254] nix-daemon[858]: reaped child process 2137, status = succeeded worker2 # [ 242.138923] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 242.154603] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 242.169178] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.733Z level=INFO msg="All 2 paths already cached" client # copying 1 paths to 'lb:50051'... client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'... worker2 # [ 242.180494] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 242.194348] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 242.350961] nix-daemon[858]: accepted connection from pid 2164, user nix-grpc-daemon (trusted) worker2 # [ 242.371855] nix-daemon[858]: reaped child process 2167, status = succeeded lb # [ 242.334638] niks3-server[1152]: 2026/09/23 05:16:38 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 242.385680] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.949Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 242.387542] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.949Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)" lb # [ 242.364663] niks3-server[1152]: 2026/09/23 05:16:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign worker2 # [ 242.415293] nix-grpc-daemon[1797]: time=2026-09-23T05:16:38.979Z level=INFO msg="Uploading 1 narinfos" lb # [ 242.380171] niks3-server[1152]: 2026/09/23 05:16:38 INFO Signed narinfos id=92 count=1 lb # [ 242.383320] niks3-server[1152]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls lb # [ 242.393342] niks3-server[1152]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst worker2 # [ 242.447556] nix-grpc-daemon[1797]: time=2026-09-23T05:16:39.011Z level=INFO msg="Upload complete. (236ms)" lb # [ 242.405759] niks3-server[1152]: 2026/09/23 05:16:38 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete lb # [ 242.414475] niks3-server[1152]: 2026/09/23 05:16:38 INFO Completed upload id=92 worker2 # [ 242.456493] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=0 paths=1 nar_bytes_in=760 lb # [ 242.418564] niks3-server[1152]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo worker2 # [ 242.469155] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 worker2 # [ 242.479401] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) worker2 # [ 242.484697] nix-daemon[858]: reaped child process 2162, status = succeeded client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051... client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker2 # [ 242.495514] nix-daemon[858]: accepted connection from pid 1720, user nix-grpc-daemon (trusted) client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds client # farm-top> LOG-farm-top-drain worker2 # [ 243.607535] nix-daemon[858]: reaped child process 2171, status = succeeded worker2 # [ 243.677332] nix-daemon[858]: accepted connection from pid 2187, user nix-grpc-daemon (trusted) worker2 # [ 243.693844] nix-daemon[858]: reaped child process 2190, status = succeeded lb # [ 243.659701] niks3-server[1152]: 2026/09/23 05:16:40 INFO Received uploads request method=POST path=/api/pending_closures worker2 # [ 243.714633] nix-grpc-daemon[1797]: time=2026-09-23T05:16:40.277Z level=INFO msg="Uploading 1 paths to lb (2 already cached)" worker2 # [ 243.716604] nix-grpc-daemon[1797]: time=2026-09-23T05:16:40.277Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)" lb # [ 243.702684] niks3-server[1152]: 2026/09/23 05:16:40 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls worker2 # [ 243.756348] nix-grpc-daemon[1797]: time=2026-09-23T05:16:40.320Z level=INFO msg="Uploading 1 narinfos" lb # [ 243.717710] niks3-server[1152]: 2026/09/23 05:16:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign lb # [ 243.726069] niks3-server[1152]: 2026/09/23 05:16:40 INFO Signed narinfos id=93 count=1 lb # [ 243.731336] niks3-server[1152]: 2026/09/23 05:16:40 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst worker2 # [ 243.785935] nix-grpc-daemon[1797]: time=2026-09-23T05:16:40.349Z level=INFO msg="Upload complete. (275ms)" lb # [ 243.744436] niks3-server[1152]: 2026/09/23 05:16:40 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv lb # [ 243.749043] niks3-server[1152]: 2026/09/23 05:16:40 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete lb # [ 243.753507] niks3-server[1152]: 2026/09/23 05:16:40 INFO Completed upload id=93 lb # [ 243.755662] niks3-server[1152]: 2026/09/23 05:16:40 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo worker2 # [ 243.801189] nix-grpc-daemon[1720]: ts=2026-09-23T05:16:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40882 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=22 outputs=1 worker2 # [ 243.815989] nix-daemon[858]: reaped child process 2169, status = succeeded client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 6.52 seconds) client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds) worker1: must succeed: rm /nix/.rw-store/fill worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds) worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free' worker1 # [ 244.369556] nix-grpc-daemon[1630]: ts=2026-09-23T05:16:40Z level=info event=healthy reason=min_free worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds) (finished: subtest: low disk drains a worker and builds go to the other, in 10.36 seconds) (finished: run the VM test script, in 246.31 seconds) test script finished in 246.56s cleanup kill QemuMachine (pid 49) client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) client # [2026-09-23T05:16:41Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T05:16:41Z INFO virtiofsd] Client disconnected, shutting down client # [2026-09-23T05:16:41Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 50) lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) lb # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down lb # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 52) worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker1 # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down worker1 # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down kill QemuMachine (pid 51) worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14) worker2 # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down worker2 # [2026-09-23T05:16:42Z INFO virtiofsd] Client disconnected, shutting down (finished: cleanup, in 1.24 seconds) additionally exposed symbols: client, lb, worker1, worker2, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh