vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #518
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2: QEMU running (pid 52)14worker2 # Formatting '/build/vm-state-worker2/tmp.fK3UmTUHCK', fmt=raw size=107374182415client # Disk image does not exist, creating the virtualisation disk image...16worker1: QEMU running (pid 51)17worker2 # mke2fs 1.47.4 (6-Mar-2025)18worker2 # Discarding device blocks: 0/262144 done19client # Formatting '/build/vm-state-client/tmp.Q15N65xSjo', fmt=raw size=107374182420worker1 # Disk image does not exist, creating the virtualisation disk image...21lb # Disk image does not exist, creating the virtualisation disk image...22client # mke2fs 1.47.4 (6-Mar-2025)23worker1 # Formatting '/build/vm-state-worker1/tmp.Pj2lpEIY6y', fmt=raw size=107374182424lb: QEMU running (pid 50)25client: QEMU running (pid 49)26worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes27lb # Formatting '/build/vm-state-lb/tmp.N9FHIroyDW', fmt=raw size=107374182428worker1 # mke2fs 1.47.4 (6-Mar-2025)29client # Discarding device blocks: 0/262144 done30worker2 # Filesystem UUID: 203ff707-c759-4c3a-a3f9-d5fbb101f69b31(finished: start all VMs, in 0.56 seconds)32client # Creating filesystem with 262144 4k blocks and 65536 inodes33worker2 # Superblock backups stored on blocks:34client: waiting for unit network-addresses-eth1.service35client # Filesystem UUID: 17878813-a2c4-493a-b0e7-fd3fa6804a9736worker2 # 32768, 98304, 163840, 22937637client: waiting for the VM to finish booting38client # Superblock backups stored on blocks:39client # 32768, 98304, 163840, 22937640worker1 # Discarding device blocks: 0/262144 done41lb # mke2fs 1.47.4 (6-Mar-2025)42worker2 # 43client # 44worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes45lb # Discarding device blocks: 0/262144 done46worker2 # Allocating group tables: 0/8 done47client # Allocating group tables: 0/8 done48worker1 # Filesystem UUID: 6fdfa4c4-3465-40d7-a9c6-192ad8c81d9849lb # Creating filesystem with 262144 4k blocks and 65536 inodes50worker2 # Writing inode tables: 0/8 done51client # Writing inode tables: 0/8 done52worker1 # Superblock backups stored on blocks:53lb # Filesystem UUID: 98fd74e6-f0d8-4597-b85e-8a4553334efc54worker2 # Creating journal (8192 blocks): done55client # Creating journal (8192 blocks): done56worker1 # 32768, 98304, 163840, 22937657lb # Superblock backups stored on blocks:58worker2 # Writing superblocks and filesystem accounting information: 0/8 done59client # Writing superblocks and filesystem accounting information: 0/8 done60worker1 # 61lb # 32768, 98304, 163840, 22937662worker2 # 63client # 64worker1 # Allocating group tables: 0/8 done65lb # 66worker2 # Virtualisation disk image created.67client # Virtualisation disk image created.68worker1 # Writing inode tables: 0/8 done69lb # Allocating group tables: 0/8 done70worker2 # Starting virtiofs daemons...71client # Starting virtiofs daemons...72worker1 # Creating journal (8192 blocks): done73lb # Writing inode tables: 0/8 done74worker2 # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)75client # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76worker1 # Writing superblocks and filesystem accounting information: 0/8 done77lb # Creating journal (8192 blocks): done78worker2 # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79client # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether80worker1 # 81lb # Writing superblocks and filesystem accounting information: 0/8 done82worker2 # [2026-09-23T05:12:11Z INFO virtiofsd] Waiting for vhost-user socket connection...83client # [2026-09-23T05:12:11Z INFO virtiofsd] Waiting for vhost-user socket connection...84worker1 # Virtualisation disk image created.85lb # 86worker2 # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)87client # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)88worker1 # Starting virtiofs daemons...89lb # Virtualisation disk image created.90worker2 # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether91client # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92worker1 # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)93lb # Starting virtiofs daemons...94worker2 # [2026-09-23T05:12:11Z INFO virtiofsd] Waiting for vhost-user socket connection...95client # [2026-09-23T05:12:11Z INFO virtiofsd] Waiting for vhost-user socket connection...96worker1 # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether97lb # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)98worker2 # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)99client # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)100worker1 # [2026-09-23T05:12:11Z INFO virtiofsd] Waiting for vhost-user socket connection...101lb # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether102worker2 # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether103client # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether104worker1 # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)105lb # [2026-09-23T05:12:11Z INFO virtiofsd] Waiting for vhost-user socket connection...106worker2 # [2026-09-23T05:12:11Z INFO virtiofsd] Waiting for vhost-user socket connection...107client # [2026-09-23T05:12:11Z INFO virtiofsd] Waiting for vhost-user socket connection...108worker1 # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether109lb # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)110worker2 # [2026-09-23T05:12:11Z INFO virtiofsd] Client connected, servicing requests111client # [2026-09-23T05:12:11Z INFO virtiofsd] Client connected, servicing requests112worker1 # [2026-09-23T05:12:11Z INFO virtiofsd] Waiting for vhost-user socket connection...113lb # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether114worker2 # [2026-09-23T05:12:11Z INFO virtiofsd] Client connected, servicing requests115client # [2026-09-23T05:12:11Z INFO virtiofsd] Client connected, servicing requests116worker1 # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)117lb # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)118worker2 # [2026-09-23T05:12:11Z INFO virtiofsd] Client connected, servicing requests119client # [2026-09-23T05:12:11Z INFO virtiofsd] Client connected, servicing requests120worker1 # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121lb # [2026-09-23T05:12:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether122worker1 # [2026-09-23T05:12:11Z INFO virtiofsd] Waiting for vhost-user socket connection...123lb # [2026-09-23T05:12:11Z INFO virtiofsd] Waiting for vhost-user socket connection...124worker1 # [2026-09-23T05:12:11Z INFO virtiofsd] Client connected, servicing requests125lb # [2026-09-23T05:12:11Z INFO virtiofsd] Waiting for vhost-user socket connection...126worker1 # [2026-09-23T05:12:11Z INFO virtiofsd] Client connected, servicing requests127lb # [2026-09-23T05:12:11Z INFO virtiofsd] Client connected, servicing requests128worker1 # [2026-09-23T05:12:11Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-23T05:12:11Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-23T05:12:11Z INFO virtiofsd] Client connected, servicing requests131worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133worker2 # [ 0.000000] KASLR enabled134worker2 # [ 0.000000] random: crng init done135worker2 # [ 0.000000] Machine model: linux,dummy-virt136worker2 # [ 0.000000] efi: UEFI not found.137worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140worker2 # [ 0.000000] Zone ranges:141worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142worker2 # [ 0.000000] DMA32 empty143worker2 # [ 0.000000] Normal empty144worker2 # [ 0.000000] Device empty145worker2 # [ 0.000000] Movable zone start for each node146worker2 # [ 0.000000] Early memory node ranges147worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150worker2 # [ 0.000000] psci: probing for conduit method from DT.151worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.152worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153worker2 # [ 0.000000] psci: Trusted OS migration not required154worker2 # [ 0.000000] psci: SMC Calling Convention v1.1155worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157worker2 # [ 0.000000] Detected PIPT I-cache on CPU0158worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)159worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface160worker2 # [ 0.000000] CPU features: detected: Spectre-v4161worker2 # [ 0.000000] CPU features: detected: Spectre-BHB162worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38163worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23164worker2 # [ 0.000000] alternatives: applying boot alternatives165worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zmhsivlz18fkqv0yqzlasss6r4v9smgp-nixos-system-worker2-test/init regInfo=/nix/store/3d1c8rkg6cnhhvdrq0ra4awlc1dyyzs4-closure-info/registration console=ttyAMA0,115200n8 console=tty0166worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/3d1c8rkg6cnhhvdrq0ra4awlc1dyyzs4-closure-info/registration", will be passed to user space.167worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes168worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)169worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)170worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB171worker2 # [ 0.000000] software IO TLB: area num 1.172worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB173worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)174worker2 # [ 0.000000] Fallback order for Node 0: 0175worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216176worker2 # [ 0.000000] Policy zone: DMA177worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off178worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1179worker2 # [ 0.000000] allocated 3145728 bytes of page_ext180worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages181worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups182worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.183worker2 # [ 0.000000] rcu: RCU event tracing is enabled.184worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.185worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.186worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.187worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.188worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.189worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1190worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.191worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.192client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]193worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.194worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0195worker2 # [ 0.000000] GICv3: 256 SPIs implemented196client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026197worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented198client # [ 0.000000] KASLR enabled199client # [ 0.000000] random: crng init done200worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq201client # [ 0.000000] Machine model: linux,dummy-virt202worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI203client # [ 0.000000] efi: UEFI not found.204worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0205client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT206worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000207worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]208client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]209client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]210worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)211client # [ 0.000000] Zone ranges:212client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]213worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)214client # [ 0.000000] DMA32 empty215client # [ 0.000000] Normal empty216worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a20000217client # [ 0.000000] Device empty218client # [ 0.000000] Movable zone start for each node219worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a30000220client # [ 0.000000] Early memory node ranges221worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.222client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]223client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]224worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns225client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000226worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).227client # [ 0.000000] psci: probing for conduit method from DT.228client # [ 0.000000] psci: PSCIv1.3 detected in firmware.229worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns230client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs231client # [ 0.000000] psci: Trusted OS migration not required232worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns233client # [ 0.000000] psci: SMC Calling Convention v1.1234worker2 # [ 0.000038] arm-pv: using stolen time PV235client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)236worker2 # [ 0.000817] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)237worker2 # [ 0.001026] Console: colour dummy device 80x25238client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296239worker2 # [ 0.001034] printk: legacy console [tty0] enabled240client # [ 0.000000] Detected PIPT I-cache on CPU0241client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)242worker2 # [ 0.001252] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)243client # [ 0.000000] CPU features: detected: GICv3 CPU interface244worker2 # [ 0.001259] pid_max: default: 32768 minimum: 301245client # [ 0.000000] CPU features: detected: Spectre-v4246worker2 # [ 0.001337] LSM: initializing lsm=capability,landlock,yama,bpf,ima247client # [ 0.000000] CPU features: detected: Spectre-BHB248worker2 # [ 0.001572] landlock: Up and running.249worker2 # [ 0.001575] Yama: becoming mindful.250client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38251worker2 # [ 0.002318] LSM support for eBPF active252client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23253worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]254worker2 # [ 0.002465] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)255client # [ 0.000000] alternatives: applying boot alternatives256worker2 # [ 0.002505] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)257worker2 # [ 0.004538] rcu: Hierarchical SRCU implementation.258worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026259worker1 # [ 0.000000] KASLR enabled260worker2 # [ 0.004545] rcu: Max phase no-delay instances is 1000.261worker1 # [ 0.000000] random: crng init done262worker2 # [ 0.005870] fsl-mc MSI: its@8080000 domain created263worker1 # [ 0.000000] Machine model: linux,dummy-virt264worker1 # [ 0.000000] efi: UEFI not found.265worker2 # [ 0.005964] EFI services will not be available.266worker2 # [ 0.006052] smp: Bringing up secondary CPUs ...267client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3zbq1wdsr2hpk2a2bj4l4l0k357dbi4h-nixos-system-client-test/init regInfo=/nix/store/9a83v96a2qvmk2ylwmy17pwh52xpkrhr-closure-info/registration console=ttyAMA0,115200n8 console=tty0268worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT269worker2 # [ 0.006062] smp: Brought up 1 node, 1 CPU270worker2 # [ 0.006065] SMP: Total of 1 processors activated.271worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]272worker2 # [ 0.006072] CPU: All CPU(s) started at EL1273client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/9a83v96a2qvmk2ylwmy17pwh52xpkrhr-closure-info/registration", will be passed to user space.274worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]275worker1 # [ 0.000000] Zone ranges:276worker2 # [ 0.006084] CPU features: detected: Branch Target Identification277client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes278worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]279worker2 # [ 0.006088] CPU features: detected: ARMv8.4 Translation Table Level280worker1 # [ 0.000000] DMA32 empty281client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)282worker1 # [ 0.000000] Normal empty283worker1 # [ 0.000000] Device empty284worker2 # [ 0.006091] CPU features: detected: Instruction cache invalidation not required for I/D coherence285worker1 # [ 0.000000] Movable zone start for each node286client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)287worker1 # [ 0.000000] Early memory node ranges288worker2 # [ 0.006095] CPU features: detected: Data cache clean to the PoU not required for I/D coherence289client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB290worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]291client # [ 0.000000] software IO TLB: area num 1.292worker2 # [ 0.006099] CPU features: detected: Common not Private translations293worker2 # [ 0.006102] CPU features: detected: CRC32 instructions294worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]295client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB296worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000297worker2 # [ 0.006105] CPU features: detected: Data cache clean to Point of Deep Persistence298client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)299worker1 # [ 0.000000] psci: probing for conduit method from DT.300client # [ 0.000000] Fallback order for Node 0: 0301worker2 # [ 0.006108] CPU features: detected: Data cache clean to Point of Persistence302worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.303client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216304worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs305worker2 # [ 0.006111] CPU features: detected: Data independent timing control (DIT)306client # [ 0.000000] Policy zone: DMA307worker1 # [ 0.000000] psci: Trusted OS migration not required308worker2 # [ 0.006115] CPU features: detected: E0PD309client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off310worker1 # [ 0.000000] psci: SMC Calling Convention v1.1311worker2 # [ 0.006117] CPU features: detected: Enhanced Counter Virtualization312client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1313worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)314worker2 # [ 0.006120] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)315client # [ 0.000000] allocated 3145728 bytes of page_ext316worker2 # [ 0.006124] CPU features: detected: Enhanced Virtualization Traps317client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages318worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296319worker1 # [ 0.000000] Detected PIPT I-cache on CPU0320worker2 # [ 0.006127] CPU features: detected: Fine Grained Traps321client # [ 0.000000] ftrace: allocated 294 pages with 4 groups322client # [ 0.000000] rcu: Hierarchical RCU implementation.323worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)324worker2 # [ 0.006131] CPU features: detected: Generic authentication (architected QARMA5 algorithm)325client # [ 0.000000] rcu: RCU event tracing is enabled.326worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface327worker2 # [ 0.006136] CPU features: detected: RCpc load-acquire (LDAPR)328worker1 # [ 0.000000] CPU features: detected: Spectre-v4329client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.330worker2 # [ 0.006139] CPU features: detected: LSE atomic instructions331worker1 # [ 0.000000] CPU features: detected: Spectre-BHB332client # [ 0.000000] Trampoline variant of Tasks RCU enabled.333worker2 # [ 0.006142] CPU features: detected: Privileged Access Never334client # [ 0.000000] Rude variant of Tasks RCU enabled.335worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38336worker2 # [ 0.006144] CPU features: detected: PMUv3337client # [ 0.000000] Tracing variant of Tasks RCU enabled.338worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23339worker2 # [ 0.006148] CPU features: detected: RAS Extension Support340worker1 # [ 0.000000] alternatives: applying boot alternatives341client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.342worker2 # [ 0.006151] CPU features: detected: RASv1p1 Extension Support343client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1344worker2 # [ 0.006153] CPU features: detected: Random Number Generator345worker2 # [ 0.006156] CPU features: detected: Speculation barrier (SB)346client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.347worker2 # [ 0.006159] CPU features: detected: Stage-2 Force Write-Back348client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.349worker2 # [ 0.006162] CPU features: detected: TLB range maintenance instructions350worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/6c2avm42395n27rpri261y0pclhhiysh-nixos-system-worker1-test/init regInfo=/nix/store/3d6a9ard07ivq4f654pnmjwkx5sfygsp-closure-info/registration console=ttyAMA0,115200n8 console=tty0351worker2 # [ 0.006166] CPU features: detected: Speculative Store Bypassing Safe (SSBS)352client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.353worker2 # [ 0.006208] alternatives: applying system-wide alternatives354client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0355client # [ 0.000000] GICv3: 256 SPIs implemented356worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/3d6a9ard07ivq4f654pnmjwkx5sfygsp-closure-info/registration", will be passed to user space.357worker2 # [ 0.009282] CPU features: detected: BBM Level 2 without TLB conflict abort358client # [ 0.000000] GICv3: 0 Extended SPIs implemented359worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes360client # [ 0.000000] Root IRQ handler: gic_handle_irq361client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI362worker2 # [ 0.009528] Memory: 1405616K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)363worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)364client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0365worker2 # [ 0.009974] devtmpfs: initialized366worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)367client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000368worker2 # [ 0.011964] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)369lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]370client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]371worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB372worker1 # [ 0.000000] software IO TLB: area num 1.373worker2 # [ 0.011989] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).374client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)375worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB376worker2 # [ 0.012192] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL377lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026378lb # [ 0.000000] KASLR enabled379worker2 # [ 0.012197] 0 pages in range for non-PLT usage380lb # [ 0.000000] random: crng init done381worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)382worker2 # [ 0.012198] 508272 pages in range for PLT usage383worker1 # [ 0.000000] Fallback order for Node 0: 0384worker2 # [ 0.012322] pinctrl core: initialized pinctrl subsystem385lb # [ 0.000000] Machine model: linux,dummy-virt386client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)387worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216388worker2 # [ 0.014189] DMI not present or invalid.389lb # [ 0.000000] efi: UEFI not found.390client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000391worker1 # [ 0.000000] Policy zone: DMA392client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000393lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT394worker2 # [ 0.018218] NET: Registered PF_NETLINK/PF_ROUTE protocol family395worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off396lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]397client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.398worker2 # [ 0.020659] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations399lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]400worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1401lb # [ 0.000000] Zone ranges:402client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns403worker2 # [ 0.020854] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations404worker1 # [ 0.000000] allocated 3145728 bytes of page_ext405lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]406client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).407worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages408lb # [ 0.000000] DMA32 empty409worker2 # [ 0.021068] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations410lb # [ 0.000000] Normal empty411worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups412lb # [ 0.000000] Device empty413worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.414worker2 # [ 0.021120] audit: initializing netlink subsys (disabled)415client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns416lb # [ 0.000000] Movable zone start for each node417worker1 # [ 0.000000] rcu: RCU event tracing is enabled.418lb # [ 0.000000] Early memory node ranges419worker2 # [ 0.021408] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1420client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns421worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.422lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]423client # [ 0.000040] arm-pv: using stolen time PV424worker2 # [ 0.021779] thermal_sys: Registered thermal governor 'fair_share'425worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.426lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]427worker2 # [ 0.021782] thermal_sys: Registered thermal governor 'bang_bang'428worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.429client # [ 0.000635] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)430lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000431worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.432worker2 # [ 0.021785] thermal_sys: Registered thermal governor 'step_wise'433client # [ 0.000844] Console: colour dummy device 80x25434lb # [ 0.000000] psci: probing for conduit method from DT.435worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.436worker2 # [ 0.021791] thermal_sys: Registered thermal governor 'user_space'437client # [ 0.000853] printk: legacy console [tty0] enabled438lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.439worker2 # [ 0.021793] thermal_sys: Registered thermal governor 'power_allocator'440lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs441worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1442worker2 # [ 0.021821] cpuidle: using governor ladder443client # [ 0.001054] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)444lb # [ 0.000000] psci: Trusted OS migration not required445worker2 # [ 0.021829] cpuidle: using governor menu446worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.447client # [ 0.001061] pid_max: default: 32768 minimum: 301448lb # [ 0.000000] psci: SMC Calling Convention v1.1449worker2 # [ 0.022038] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.450client # [ 0.001150] LSM: initializing lsm=capability,landlock,yama,bpf,ima451worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.452lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)453client # [ 0.001372] landlock: Up and running.454worker2 # [ 0.022058] ASID allocator initialised with 65536 entries455client # [ 0.001375] Yama: becoming mindful.456worker2 # [ 0.023371] Serial: AMBA PL011 UART driver457lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296458worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.459client # [ 0.002056] LSM support for eBPF active460lb # [ 0.000000] Detected PIPT I-cache on CPU0461worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0462worker2 # [ 0.029241] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1463client # [ 0.002233] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)464worker1 # [ 0.000000] GICv3: 256 SPIs implemented465lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)466worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented467client # [ 0.002267] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)468lb # [ 0.000000] CPU features: detected: GICv3 CPU interface469worker2 # [ 0.029408] printk: console [ttyAMA0] enabled470worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq471lb # [ 0.000000] CPU features: detected: Spectre-v4472client # [ 0.004301] rcu: Hierarchical SRCU implementation.473worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI474lb # [ 0.000000] CPU features: detected: Spectre-BHB475client # [ 0.004307] rcu: Max phase no-delay instances is 1000.476worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0477client # [ 0.005613] fsl-mc MSI: its@8080000 domain created478lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38479worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000480client # [ 0.005710] EFI services will not be available.481lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23482worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]483client # [ 0.005798] smp: Bringing up secondary CPUs ...484lb # [ 0.000000] alternatives: applying boot alternatives485client # [ 0.005808] smp: Brought up 1 node, 1 CPU486worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)487client # [ 0.005811] SMP: Total of 1 processors activated.488client # [ 0.005817] CPU: All CPU(s) started at EL1489worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)490client # [ 0.005828] CPU features: detected: Branch Target Identification491worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000492client # [ 0.005832] CPU features: detected: ARMv8.4 Translation Table Level493worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000494lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kgjzg78npi7z07am2szqxkr41f06jp5w-nixos-system-lb-test/init regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration console=ttyAMA0,115200n8 console=tty0495client # [ 0.005836] CPU features: detected: Instruction cache invalidation not required for I/D coherence496worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.497client # [ 0.005839] CPU features: detected: Data cache clean to the PoU not required for I/D coherence498worker2 # [ 0.154082] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages499worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns500lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration", will be passed to user space.501client # [ 0.005843] CPU features: detected: Common not Private translations502worker2 # [ 0.154107] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page503worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).504client # [ 0.005847] CPU features: detected: CRC32 instructions505lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes506worker2 # [ 0.154113] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages507client # [ 0.005850] CPU features: detected: Data cache clean to Point of Deep Persistence508lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)509worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns510worker2 # [ 0.154118] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page511client # [ 0.005853] CPU features: detected: Data cache clean to Point of Persistence512lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)513worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns514worker2 # [ 0.154122] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages515client # [ 0.005856] CPU features: detected: Data independent timing control (DIT)516worker1 # [ 0.000037] arm-pv: using stolen time PV517lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB518client # [ 0.005859] CPU features: detected: E0PD519worker2 # [ 0.154126] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page520lb # [ 0.000000] software IO TLB: area num 1.521client # [ 0.005862] CPU features: detected: Enhanced Counter Virtualization522worker1 # [ 0.000486] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)523worker2 # [ 0.154131] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages524lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB525worker1 # [ 0.000698] Console: colour dummy device 80x25526worker2 # [ 0.154135] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page527client # [ 0.005865] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)528worker1 # [ 0.000706] printk: legacy console [tty0] enabled529lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)530client # [ 0.005869] CPU features: detected: Enhanced Virtualization Traps531lb # [ 0.000000] Fallback order for Node 0: 0532worker1 # [ 0.000900] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)533client # [ 0.005872] CPU features: detected: Fine Grained Traps534lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216535lb # [ 0.000000] Policy zone: DMA536worker1 # [ 0.000907] pid_max: default: 32768 minimum: 301537client # [ 0.005876] CPU features: detected: Generic authentication (architected QARMA5 algorithm)538worker1 # [ 0.000986] LSM: initializing lsm=capability,landlock,yama,bpf,ima539lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off540worker2 # [ 0.161933] fbcon: Taking over console541client # [ 0.005881] CPU features: detected: RCpc load-acquire (LDAPR)542worker1 # [ 0.001139] landlock: Up and running.543worker2 # [ 0.161956] ACPI: Interpreter disabled.544lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1545worker1 # [ 0.001142] Yama: becoming mindful.546client # [ 0.005884] CPU features: detected: LSE atomic instructions547worker1 # [ 0.001694] LSM support for eBPF active548lb # [ 0.000000] allocated 3145728 bytes of page_ext549client # [ 0.005887] CPU features: detected: Privileged Access Never550lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages551worker1 # [ 0.001843] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)552client # [ 0.005889] CPU features: detected: PMUv3553lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups554client # [ 0.005892] CPU features: detected: RAS Extension Support555lb # [ 0.000000] rcu: Hierarchical RCU implementation.556worker1 # [ 0.001884] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)557client # [ 0.005895] CPU features: detected: RASv1p1 Extension Support558lb # [ 0.000000] rcu: RCU event tracing is enabled.559worker1 # [ 0.003795] rcu: Hierarchical SRCU implementation.560client # [ 0.005898] CPU features: detected: Random Number Generator561worker1 # [ 0.003802] rcu: Max phase no-delay instances is 1000.562lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.563worker1 # [ 0.005107] fsl-mc MSI: its@8080000 domain created564client # [ 0.005900] CPU features: detected: Speculation barrier (SB)565lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.566worker1 # [ 0.005231] EFI services will not be available.567client # [ 0.005903] CPU features: detected: Stage-2 Force Write-Back568lb # [ 0.000000] Rude variant of Tasks RCU enabled.569worker1 # [ 0.005321] smp: Bringing up secondary CPUs ...570lb # [ 0.000000] Tracing variant of Tasks RCU enabled.571client # [ 0.005906] CPU features: detected: TLB range maintenance instructions572worker1 # [ 0.005330] smp: Brought up 1 node, 1 CPU573worker1 # [ 0.005333] SMP: Total of 1 processors activated.574lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.575client # [ 0.005910] CPU features: detected: Speculative Store Bypassing Safe (SSBS)576worker1 # [ 0.005339] CPU: All CPU(s) started at EL1577client # [ 0.005951] alternatives: applying system-wide alternatives578lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1579worker2 # [ 0.170526] iommu: Default domain type: Translated580worker1 # [ 0.005350] CPU features: detected: Branch Target Identification581worker2 # [ 0.170539] iommu: DMA domain TLB invalidation policy: strict mode582client # [ 0.009163] CPU features: detected: BBM Level 2 without TLB conflict abort583lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.584worker1 # [ 0.005354] CPU features: detected: ARMv8.4 Translation Table Level585worker2 # [ 0.172341] SCSI subsystem initialized586worker1 # [ 0.005357] CPU features: detected: Instruction cache invalidation not required for I/D coherence587lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.588client # [ 0.009383] Memory: 1405616K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)589client # [ 0.009802] devtmpfs: initialized590worker2 # [ 0.173079] usbcore: registered new interface driver usbfs591worker1 # [ 0.005361] CPU features: detected: Data cache clean to the PoU not required for I/D coherence592lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.593worker2 # [ 0.173144] usbcore: registered new interface driver hub594client # [ 0.011720] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)595worker1 # [ 0.005364] CPU features: detected: Common not Private translations596lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0597worker2 # [ 0.173161] usbcore: registered new device driver usb598lb # [ 0.000000] GICv3: 256 SPIs implemented599worker1 # [ 0.005368] CPU features: detected: CRC32 instructions600client # [ 0.011750] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).601lb # [ 0.000000] GICv3: 0 Extended SPIs implemented602worker2 # [ 0.173446] pps_core: LinuxPPS API ver. 1 registered603worker1 # [ 0.005370] CPU features: detected: Data cache clean to Point of Deep Persistence604lb # [ 0.000000] Root IRQ handler: gic_handle_irq605client # [ 0.011954] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL606lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI607client # [ 0.011959] 0 pages in range for non-PLT usage608worker2 # [ 0.173453] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>609worker1 # [ 0.005374] CPU features: detected: Data cache clean to Point of Persistence610worker2 # [ 0.173464] PTP clock support registered611lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0612client # [ 0.011960] 508272 pages in range for PLT usage613worker2 # [ 0.173519] EDAC MC: Ver: 3.0.0614worker1 # [ 0.005377] CPU features: detected: Data independent timing control (DIT)615client # [ 0.012081] pinctrl core: initialized pinctrl subsystem616lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000617worker1 # [ 0.005380] CPU features: detected: E0PD618client # [ 0.012897] DMI not present or invalid.619lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]620worker2 # [ 0.178230] scmi_core: SCMI protocol bus registered621worker1 # [ 0.005383] CPU features: detected: Enhanced Counter Virtualization622client # [ 0.016617] NET: Registered PF_NETLINK/PF_ROUTE protocol family623lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)624worker2 # [ 0.179298] FPGA manager framework625worker1 # [ 0.005386] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)626client # [ 0.018961] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations627worker2 # [ 0.180349] vgaarb: loaded628worker1 # [ 0.005390] CPU features: detected: Enhanced Virtualization Traps629client # [ 0.019166] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations630lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)631worker1 # [ 0.005393] CPU features: detected: Fine Grained Traps632worker2 # [ 0.181060] clocksource: Switched to clocksource arch_sys_counter633lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000634client # [ 0.019391] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations635worker1 # [ 0.005396] CPU features: detected: Generic authentication (architected QARMA5 algorithm)636lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000637client # [ 0.019416] audit: initializing netlink subsys (disabled)638worker1 # [ 0.005401] CPU features: detected: RCpc load-acquire (LDAPR)639client # [ 0.020043] thermal_sys: Registered thermal governor 'fair_share'640lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.641worker1 # [ 0.005404] CPU features: detected: LSE atomic instructions642client # [ 0.020046] thermal_sys: Registered thermal governor 'bang_bang'643worker1 # [ 0.005407] CPU features: detected: Privileged Access Never644lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns645worker1 # [ 0.005410] CPU features: detected: PMUv3646client # [ 0.020050] thermal_sys: Registered thermal governor 'step_wise'647lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).648worker2 # [ 0.184807] VFS: Disk quotas dquot_6.6.0649client # [ 0.020053] thermal_sys: Registered thermal governor 'user_space'650worker1 # [ 0.005413] CPU features: detected: RAS Extension Support651worker1 # [ 0.005416] CPU features: detected: RASv1p1 Extension Support652worker2 # [ 0.184858] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)653client # [ 0.020058] thermal_sys: Registered thermal governor 'power_allocator'654lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns655worker1 # [ 0.005418] CPU features: detected: Random Number Generator656client # [ 0.020090] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1657worker1 # [ 0.005421] CPU features: detected: Speculation barrier (SB)658lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns659client # [ 0.020099] cpuidle: using governor ladder660lb # [ 0.000030] arm-pv: using stolen time PV661worker1 # [ 0.005424] CPU features: detected: Stage-2 Force Write-Back662client # [ 0.020105] cpuidle: using governor menu663worker1 # [ 0.005426] CPU features: detected: TLB range maintenance instructions664client # [ 0.020336] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.665lb # [ 0.000467] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)666worker2 # [ 0.189054] netfs: FS-Cache loaded667client # [ 0.020355] ASID allocator initialised with 65536 entries668worker1 # [ 0.005432] CPU features: detected: Speculative Store Bypassing Safe (SSBS)669lb # [ 0.000663] Console: colour dummy device 80x25670client # [ 0.021624] Serial: AMBA PL011 UART driver671worker2 # [ 0.189716] pnp: PnP ACPI: disabled672worker1 # [ 0.005473] alternatives: applying system-wide alternatives673lb # [ 0.000674] printk: legacy console [tty0] enabled674worker1 # [ 0.008469] CPU features: detected: BBM Level 2 without TLB conflict abort675client # [ 0.027126] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1676lb # [ 0.000867] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)677client # [ 0.027313] printk: console [ttyAMA0] enabled678lb # [ 0.000874] pid_max: default: 32768 minimum: 301679worker1 # [ 0.008674] Memory: 1405616K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)680lb # [ 0.000950] LSM: initializing lsm=capability,landlock,yama,bpf,ima681worker1 # [ 0.009093] devtmpfs: initialized682lb # [ 0.001134] landlock: Up and running.683lb # [ 0.001137] Yama: becoming mindful.684worker1 # [ 0.011003] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)685lb # [ 0.001657] LSM support for eBPF active686worker1 # [ 0.011027] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).687lb # [ 0.001808] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)688worker1 # [ 0.011208] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL689lb # [ 0.001842] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)690worker1 # [ 0.011212] 0 pages in range for non-PLT usage691lb # [ 0.003780] rcu: Hierarchical SRCU implementation.692worker1 # [ 0.011214] 508272 pages in range for PLT usage693lb # [ 0.003786] rcu: Max phase no-delay instances is 1000.694worker2 # [ 0.193678] NET: Registered PF_INET protocol family695worker1 # [ 0.011331] pinctrl core: initialized pinctrl subsystem696lb # [ 0.005089] fsl-mc MSI: its@8080000 domain created697worker1 # [ 0.012130] DMI not present or invalid.698worker2 # [ 0.193981] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)699lb # [ 0.005190] EFI services will not be available.700worker1 # [ 0.016042] NET: Registered PF_NETLINK/PF_ROUTE protocol family701lb # [ 0.005274] smp: Bringing up secondary CPUs ...702lb # [ 0.005283] smp: Brought up 1 node, 1 CPU703worker1 # [ 0.018363] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations704lb # [ 0.005286] SMP: Total of 1 processors activated.705worker1 # [ 0.018552] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations706client # [ 0.150905] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages707lb # [ 0.005291] CPU: All CPU(s) started at EL1708lb # [ 0.005301] CPU features: detected: Branch Target Identification709client # [ 0.150927] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page710worker1 # [ 0.018764] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations711lb # [ 0.005306] CPU features: detected: ARMv8.4 Translation Table Level712worker1 # [ 0.018789] audit: initializing netlink subsys (disabled)713client # [ 0.150932] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages714client # [ 0.150937] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page715lb # [ 0.005309] CPU features: detected: Instruction cache invalidation not required for I/D coherence716worker1 # [ 0.019378] thermal_sys: Registered thermal governor 'fair_share'717client # [ 0.150941] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages718lb # [ 0.005313] CPU features: detected: Data cache clean to the PoU not required for I/D coherence719worker1 # [ 0.019381] thermal_sys: Registered thermal governor 'bang_bang'720client # [ 0.150946] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page721lb # [ 0.005316] CPU features: detected: Common not Private translations722worker1 # [ 0.019385] thermal_sys: Registered thermal governor 'step_wise'723client # [ 0.150950] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages724lb # [ 0.005319] CPU features: detected: CRC32 instructions725worker1 # [ 0.019388] thermal_sys: Registered thermal governor 'user_space'726client # [ 0.150954] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page727lb # [ 0.005322] CPU features: detected: Data cache clean to Point of Deep Persistence728worker1 # [ 0.019393] thermal_sys: Registered thermal governor 'power_allocator'729lb # [ 0.005326] CPU features: detected: Data cache clean to Point of Persistence730worker1 # [ 0.019429] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1731worker1 # [ 0.019438] cpuidle: using governor ladder732lb # [ 0.005329] CPU features: detected: Data independent timing control (DIT)733client # [ 0.158651] fbcon: Taking over console734worker1 # [ 0.019444] cpuidle: using governor menu735lb # [ 0.005332] CPU features: detected: E0PD736client # [ 0.158668] ACPI: Interpreter disabled.737worker1 # [ 0.019631] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.738lb # [ 0.005334] CPU features: detected: Enhanced Counter Virtualization739worker1 # [ 0.019648] ASID allocator initialised with 65536 entries740lb # [ 0.005337] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)741worker1 # [ 0.020893] Serial: AMBA PL011 UART driver742lb # [ 0.005341] CPU features: detected: Enhanced Virtualization Traps743worker1 # [ 0.026417] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1744lb # [ 0.005344] CPU features: detected: Fine Grained Traps745worker1 # [ 0.026588] printk: console [ttyAMA0] enabled746lb # [ 0.005348] CPU features: detected: Generic authentication (architected QARMA5 algorithm)747lb # [ 0.005353] CPU features: detected: RCpc load-acquire (LDAPR)748lb # [ 0.005356] CPU features: detected: LSE atomic instructions749lb # [ 0.005359] CPU features: detected: Privileged Access Never750lb # [ 0.005362] CPU features: detected: PMUv3751lb # [ 0.005365] CPU features: detected: RAS Extension Support752lb # [ 0.005367] CPU features: detected: RASv1p1 Extension Support753client # [ 0.167285] iommu: Default domain type: Translated754lb # [ 0.005370] CPU features: detected: Random Number Generator755client # [ 0.167298] iommu: DMA domain TLB invalidation policy: strict mode756lb # [ 0.005372] CPU features: detected: Speculation barrier (SB)757client # [ 0.167706] SCSI subsystem initialized758lb # [ 0.005375] CPU features: detected: Stage-2 Force Write-Back759client # [ 0.169859] usbcore: registered new interface driver usbfs760lb # [ 0.005378] CPU features: detected: TLB range maintenance instructions761client # [ 0.169902] usbcore: registered new interface driver hub762lb # [ 0.005384] CPU features: detected: Speculative Store Bypassing Safe (SSBS)763worker1 # [ 0.148376] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages764client # [ 0.169918] usbcore: registered new device driver usb765lb # [ 0.005423] alternatives: applying system-wide alternatives766client # [ 0.170241] pps_core: LinuxPPS API ver. 1 registered767worker1 # [ 0.148396] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page768lb # [ 0.008424] CPU features: detected: BBM Level 2 without TLB conflict abort769worker1 # [ 0.148402] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages770client # [ 0.170252] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>771client # [ 0.170262] PTP clock support registered772worker1 # [ 0.148406] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page773client # [ 0.170327] EDAC MC: Ver: 3.0.0774lb # [ 0.008589] Memory: 1405632K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)775worker1 # [ 0.148411] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages776lb # [ 0.009019] devtmpfs: initialized777client # [ 0.175187] scmi_core: SCMI protocol bus registered778worker1 # [ 0.148415] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page779lb # [ 0.010920] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)780client # [ 0.176237] FPGA manager framework781worker1 # [ 0.148419] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages782client # [ 0.177292] vgaarb: loaded783lb # [ 0.010944] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).784worker1 # [ 0.148424] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page785lb # [ 0.011154] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL786client # [ 0.177968] clocksource: Switched to clocksource arch_sys_counter787lb # [ 0.011158] 0 pages in range for non-PLT usage788lb # [ 0.011160] 508272 pages in range for PLT usage789worker1 # [ 0.156107] fbcon: Taking over console790lb # [ 0.011291] pinctrl core: initialized pinctrl subsystem791worker1 # [ 0.156127] ACPI: Interpreter disabled.792lb # [ 0.012116] DMI not present or invalid.793lb # [ 0.015589] NET: Registered PF_NETLINK/PF_ROUTE protocol family794client # [ 0.181455] VFS: Disk quotas dquot_6.6.0795lb # [ 0.018085] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations796client # [ 0.181495] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)797lb # [ 0.018288] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations798lb # [ 0.018495] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations799worker2 # [ 0.226661] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)800lb # [ 0.018523] audit: initializing netlink subsys (disabled)801client # [ 0.185618] netfs: FS-Cache loaded802worker2 # [ 0.226719] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)803lb # [ 0.019131] thermal_sys: Registered thermal governor 'fair_share'804client # [ 0.185762] pnp: PnP ACPI: disabled805lb # [ 0.019133] thermal_sys: Registered thermal governor 'bang_bang'806worker2 # [ 0.226774] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)807lb # [ 0.019137] thermal_sys: Registered thermal governor 'step_wise'808worker2 # [ 0.226866] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)809lb # [ 0.019139] thermal_sys: Registered thermal governor 'user_space'810worker1 # [ 0.164701] iommu: Default domain type: Translated811worker2 # [ 0.227016] TCP: Hash tables configured (established 16384 bind 16384)812worker1 # [ 0.164713] iommu: DMA domain TLB invalidation policy: strict mode813lb # [ 0.019144] thermal_sys: Registered thermal governor 'power_allocator'814worker1 # [ 0.165164] SCSI subsystem initialized815worker2 # [ 0.227122] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)816lb # [ 0.019170] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1817worker2 # [ 0.227168] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)818lb # [ 0.019179] cpuidle: using governor ladder819worker1 # [ 0.167171] usbcore: registered new interface driver usbfs820lb # [ 0.019184] cpuidle: using governor menu821worker2 # [ 0.227208] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)822worker1 # [ 0.167204] usbcore: registered new interface driver hub823worker2 # [ 0.227326] NET: Registered PF_UNIX/PF_LOCAL protocol family824lb # [ 0.019371] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.825worker1 # [ 0.167220] usbcore: registered new device driver usb826worker2 # [ 0.227350] NET: Registered PF_XDP protocol family827lb # [ 0.019387] ASID allocator initialised with 65536 entries828worker1 # [ 0.167510] pps_core: LinuxPPS API ver. 1 registered829worker2 # [ 0.227371] PCI: CLS 0 bytes, default 64830lb # [ 0.020635] Serial: AMBA PL011 UART driver831worker2 # [ 0.227628] Trying to unpack rootfs image as initramfs...832worker1 # [ 0.167517] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>833client # [ 0.190061] NET: Registered PF_INET protocol family834worker1 # [ 0.167527] PTP clock support registered835lb # [ 0.026087] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1836worker1 # [ 0.167578] EDAC MC: Ver: 3.0.0837lb # [ 0.026257] printk: console [ttyAMA0] enabled838client # [ 0.190352] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)839worker1 # [ 0.172238] scmi_core: SCMI protocol bus registered840worker1 # [ 0.173248] FPGA manager framework841worker1 # [ 0.174231] vgaarb: loaded842worker1 # [ 0.174875] clocksource: Switched to clocksource arch_sys_counter843lb # [ 0.149140] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages844worker1 # [ 0.179794] VFS: Disk quotas dquot_6.6.0845lb # [ 0.149168] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page846worker1 # [ 0.179846] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)847worker2 # [ 0.249148] kvm [1]: HYP mode not available848lb # [ 0.149173] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages849worker1 # [ 0.181632] netfs: FS-Cache loaded850worker1 # [ 0.181759] pnp: PnP ACPI: disabled851lb # [ 0.149178] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page852lb # [ 0.149182] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages853lb # [ 0.149186] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page854lb # [ 0.149191] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages855lb # [ 0.149195] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page856worker1 # [ 0.185759] NET: Registered PF_INET protocol family857lb # [ 0.157161] fbcon: Taking over console858worker1 # [ 0.186038] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)859lb # [ 0.157182] ACPI: Interpreter disabled.860lb # [ 0.166263] iommu: Default domain type: Translated861lb # [ 0.166275] iommu: DMA domain TLB invalidation policy: strict mode862lb # [ 0.166669] SCSI subsystem initialized863lb # [ 0.168754] usbcore: registered new interface driver usbfs864client # [ 0.221761] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)865lb # [ 0.168788] usbcore: registered new interface driver hub866client # [ 0.221821] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)867lb # [ 0.168804] usbcore: registered new device driver usb868lb # [ 0.169102] pps_core: LinuxPPS API ver. 1 registered869client # [ 0.221855] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)870client # [ 0.221935] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)871lb # [ 0.169109] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>872lb # [ 0.169120] PTP clock support registered873lb # [ 0.169171] EDAC MC: Ver: 3.0.0874lb # [ 0.173997] scmi_core: SCMI protocol bus registered875lb # [ 0.175044] FPGA manager framework876lb # [ 0.176075] vgaarb: loaded877lb # [ 0.176718] clocksource: Switched to clocksource arch_sys_counter878lb # [ 0.181100] VFS: Disk quotas dquot_6.6.0879client # [ 0.226241] TCP: Hash tables configured (established 16384 bind 16384)880lb # [ 0.181140] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)881client # [ 0.226345] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)882client # [ 0.226387] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)883client # [ 0.226425] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)884client # [ 0.226546] NET: Registered PF_UNIX/PF_LOCAL protocol family885client # [ 0.226588] NET: Registered PF_XDP protocol family886lb # [ 0.185127] netfs: FS-Cache loaded887lb # [ 0.185282] pnp: PnP ACPI: disabled888client # [ 0.226610] PCI: CLS 0 bytes, default 64889client # [ 0.226906] Trying to unpack rootfs image as initramfs...890lb # [ 0.189390] NET: Registered PF_INET protocol family891worker1 # [ 0.216090] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)892lb # [ 0.189673] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)893worker1 # [ 0.216141] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)894worker1 # [ 0.216172] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)895worker1 # [ 0.216249] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)896worker1 # [ 0.216395] TCP: Hash tables configured (established 16384 bind 16384)897worker1 # [ 0.216499] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)898worker1 # [ 0.216535] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)899worker1 # [ 0.216572] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)900worker1 # [ 0.216677] NET: Registered PF_UNIX/PF_LOCAL protocol family901client # [ 0.246019] kvm [1]: HYP mode not available902worker1 # [ 0.216700] NET: Registered PF_XDP protocol family903worker1 # [ 0.216719] PCI: CLS 0 bytes, default 64904worker1 # [ 0.216972] Trying to unpack rootfs image as initramfs...905worker1 # [ 0.234537] kvm [1]: HYP mode not available906lb # [ 0.220098] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)907lb # [ 0.220153] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)908lb # [ 0.220188] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)909lb # [ 0.220267] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)910lb # [ 0.220411] TCP: Hash tables configured (established 16384 bind 16384)911lb # [ 0.220507] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)912lb # [ 0.220546] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)913lb # [ 0.220582] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)914lb # [ 0.220692] NET: Registered PF_UNIX/PF_LOCAL protocol family915lb # [ 0.229115] NET: Registered PF_XDP protocol family916lb # [ 0.229143] PCI: CLS 0 bytes, default 64917lb # [ 0.229412] Trying to unpack rootfs image as initramfs...918lb # [ 0.238577] kvm [1]: HYP mode not available919worker2 # [ 0.389774] Initialise system trusted keyrings920worker2 # [ 0.390568] workingset: timestamp_bits=42 max_order=19 bucket_order=0921worker2 # [ 0.391998] squashfs: version 4.0 (2009/01/31) Phillip Lougher922worker2 # [ 0.392816] 9p: Installing v9fs 9p2000 file system support923worker1 # [ 0.348629] Initialise system trusted keyrings924worker1 # [ 0.349493] workingset: timestamp_bits=42 max_order=19 bucket_order=0925worker2 # [ 0.421773] Key type asymmetric registered926worker2 # [ 0.421802] Asymmetric key parser 'x509' registered927worker1 # [ 0.350863] squashfs: version 4.0 (2009/01/31) Phillip Lougher928worker2 # [ 0.421890] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)929worker2 # [ 0.424150] io scheduler mq-deadline registered930worker2 # [ 0.424164] io scheduler kyber registered931worker1 # [ 0.351699] 9p: Installing v9fs 9p2000 file system support932lb # [ 0.339383] Initialise system trusted keyrings933worker2 # [ 0.437918] pl061_gpio 9030000.pl061: PL061 GPIO chip registered934worker2 # [ 0.438577] ledtrig-cpu: registered to indicate activity on CPUs935worker2 # [ 0.438991] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:936lb # [ 0.344831] workingset: timestamp_bits=42 max_order=19 bucket_order=0937worker2 # [ 0.439013] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000938lb # [ 0.346345] squashfs: version 4.0 (2009/01/31) Phillip Lougher939worker2 # [ 0.439027] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000940worker2 # [ 0.439035] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000941lb # [ 0.347210] 9p: Installing v9fs 9p2000 file system support942worker2 # [ 0.439059] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits943worker1 # [ 0.379834] Key type asymmetric registered944worker2 # [ 0.439086] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]945worker1 # [ 0.379865] Asymmetric key parser 'x509' registered946worker2 # [ 0.439168] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00947worker1 # [ 0.379954] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)948worker2 # [ 0.439179] pci_bus 0000:00: root bus resource [bus 00-ff]949worker1 # [ 0.382291] io scheduler mq-deadline registered950worker1 # [ 0.382305] io scheduler kyber registered951worker2 # [ 0.439185] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]952worker2 # [ 0.439191] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]953worker2 # [ 0.439196] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]954worker2 # [ 0.439268] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint955worker2 # [ 0.439742] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint956worker2 # [ 0.439935] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]957worker2 # [ 0.439955] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]958worker2 # [ 0.439989] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]959worker2 # [ 0.440009] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]960worker2 # [ 0.440468] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint961worker2 # [ 0.440651] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]962worker2 # [ 0.440668] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]963worker2 # [ 0.440699] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]964worker2 # [ 0.441258] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint965worker2 # [ 0.441461] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]966worker2 # [ 0.441480] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]967worker2 # [ 0.441515] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]968worker2 # [ 0.442003] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint969worker2 # [ 0.442209] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]970lb # [ 0.368003] Key type asymmetric registered971worker2 # [ 0.442227] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]972lb # [ 0.368035] Asymmetric key parser 'x509' registered973worker2 # [ 0.442258] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]974lb # [ 0.368112] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)975worker2 # [ 0.442280] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]976client # [ 0.423680] Initialise system trusted keyrings977worker2 # [ 0.442769] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint978client # [ 0.424548] workingset: timestamp_bits=42 max_order=19 bucket_order=0979worker2 # [ 0.442965] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]980worker1 # [ 0.395993] pl061_gpio 9030000.pl061: PL061 GPIO chip registered981client # [ 0.425945] squashfs: version 4.0 (2009/01/31) Phillip Lougher982worker2 # [ 0.442998] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]983worker1 # [ 0.396615] ledtrig-cpu: registered to indicate activity on CPUs984worker2 # [ 0.443464] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint985worker1 # [ 0.397028] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:986worker2 # [ 0.443649] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]987worker2 # [ 0.443680] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]988worker1 # [ 0.397053] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000989worker2 # [ 0.444076] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint990worker1 # [ 0.397070] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000991lb # [ 0.376799] io scheduler mq-deadline registered992client # [ 0.426839] 9p: Installing v9fs 9p2000 file system support993lb # [ 0.376824] io scheduler kyber registered994worker2 # [ 0.444273] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]995worker1 # [ 0.397080] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000996worker2 # [ 0.444571] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint997worker1 # [ 0.397104] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits998worker2 # [ 0.444761] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]999worker2 # [ 0.444792] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1000worker1 # [ 0.397129] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1001worker1 # [ 0.397245] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001002worker1 # [ 0.397256] pci_bus 0000:00: root bus resource [bus 00-ff]1003worker1 # [ 0.397263] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1004worker1 # [ 0.397268] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1005worker1 # [ 0.397273] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1006worker1 # [ 0.397339] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1007worker1 # [ 0.397804] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1008worker1 # [ 0.397997] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1009worker1 # [ 0.398014] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1010worker1 # [ 0.398043] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1011worker1 # [ 0.398060] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1012lb # [ 0.388830] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1013worker1 # [ 0.398533] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1014worker1 # [ 0.398725] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1015worker1 # [ 0.398741] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1016worker1 # [ 0.398770] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1017worker1 # [ 0.399280] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1018worker1 # [ 0.399473] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1019worker1 # [ 0.399491] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1020worker2 # [ 0.489464] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1021worker1 # [ 0.399522] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1022worker2 # [ 0.489694] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1023lb # [ 0.390356] ledtrig-cpu: registered to indicate activity on CPUs1024worker2 # [ 0.489733] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025worker1 # [ 0.400001] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1026lb # [ 0.390805] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1027worker1 # [ 0.400191] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1028worker2 # [ 0.490231] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1029worker1 # [ 0.400207] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1030worker2 # [ 0.490429] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1031lb # [ 0.390827] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001032worker1 # [ 0.400237] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1033worker2 # [ 0.490461] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1034lb # [ 0.390840] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001035worker1 # [ 0.400256] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1036worker2 # [ 0.490947] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1037lb # [ 0.390849] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001038worker1 # [ 0.400720] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1039worker2 # [ 0.491262] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1040worker1 # [ 0.400915] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1041worker2 # [ 0.491286] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1042lb # [ 0.390872] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1043client # [ 0.455042] Key type asymmetric registered1044worker1 # [ 0.400945] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1045worker2 # [ 0.491319] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1046client # [ 0.455080] Asymmetric key parser 'x509' registered1047lb # [ 0.390903] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1048client # [ 0.455174] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1049worker1 # [ 0.401438] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1050worker2 # [ 0.491815] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1051lb # [ 0.391000] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001052client # [ 0.457374] io scheduler mq-deadline registered1053worker2 # [ 0.492003] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1054worker1 # [ 0.401634] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1055lb # [ 0.391014] pci_bus 0000:00: root bus resource [bus 00-ff]1056client # [ 0.457387] io scheduler kyber registered1057worker2 # [ 0.492021] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1058worker1 # [ 0.401665] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1059lb # [ 0.391020] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1060worker2 # [ 0.492053] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1061lb # [ 0.391025] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1062worker1 # [ 0.402118] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1063worker2 # [ 0.492683] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1064worker1 # [ 0.402311] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1065lb # [ 0.391030] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1066worker2 # [ 0.492697] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1067worker1 # [ 0.402568] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1068lb # [ 0.391110] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1069worker1 # [ 0.402758] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1070worker2 # [ 0.492703] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1071lb # [ 0.391562] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1072worker1 # [ 0.402788] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1073worker2 # [ 0.492752] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1074lb # [ 0.391750] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1075lb # [ 0.391766] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1076worker2 # [ 0.492805] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1077lb # [ 0.391796] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1078worker2 # [ 0.492855] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1079lb # [ 0.391812] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1080worker2 # [ 0.492910] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1081lb # [ 0.392282] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1082worker2 # [ 0.492959] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1083lb # [ 0.392468] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1084worker2 # [ 0.493012] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1085lb # [ 0.392484] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1086worker2 # [ 0.493117] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1087lb # [ 0.392517] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1088worker2 # [ 0.493164] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1089lb # [ 0.393012] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1090lb # [ 0.393200] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1091worker2 # [ 0.493218] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1092lb # [ 0.393217] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1093worker2 # [ 0.493304] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1094lb # [ 0.393249] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1095worker2 # [ 0.493356] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1096lb # [ 0.393714] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1097worker2 # [ 0.493379] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1098lb # [ 0.393912] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1099worker2 # [ 0.493403] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1100lb # [ 0.393929] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1101worker2 # [ 0.493429] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1102lb # [ 0.393960] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1103worker2 # [ 0.493452] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1104lb # [ 0.393979] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1105worker2 # [ 0.493475] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1106lb # [ 0.394457] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1107worker2 # [ 0.493499] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1108client # [ 0.471175] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1109lb # [ 0.394647] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1110worker1 # [ 0.447460] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1111worker2 # [ 0.493528] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1112client # [ 0.471807] ledtrig-cpu: registered to indicate activity on CPUs1113worker1 # [ 0.447686] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1114lb # [ 0.394679] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1115worker2 # [ 0.493553] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1116client # [ 0.472257] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1117worker1 # [ 0.447720] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1118lb # [ 0.395199] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1119worker2 # [ 0.493578] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1120client # [ 0.472280] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001121lb # [ 0.395393] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1122worker1 # [ 0.448196] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1123worker2 # [ 0.493602] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1124lb # [ 0.395423] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1125client # [ 0.472295] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001126worker1 # [ 0.448388] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1127worker2 # [ 0.493626] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1128worker1 # [ 0.448418] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1129worker2 # [ 0.493648] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1130lb # [ 0.395824] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1131client # [ 0.472305] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001132worker2 # [ 0.493671] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1133lb # [ 0.396012] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1134worker1 # [ 0.448893] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1135client # [ 0.472336] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1136worker2 # [ 0.493694] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1137lb # [ 0.396265] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1138worker1 # [ 0.449245] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1139client # [ 0.472359] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1140worker1 # [ 0.449268] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1141worker2 # [ 0.493716] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1142lb # [ 0.396456] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1143worker2 # [ 0.493739] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1144worker1 # [ 0.449301] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1145client # [ 0.472467] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001146lb # [ 0.396486] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1147worker2 # [ 0.493762] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1148client # [ 0.472478] pci_bus 0000:00: root bus resource [bus 00-ff]1149worker1 # [ 0.449803] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1150worker2 # [ 0.493792] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1151client # [ 0.472485] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1152worker1 # [ 0.449999] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1153worker2 # [ 0.493803] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1154client # [ 0.472490] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1155worker1 # [ 0.450016] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1156worker2 # [ 0.493807] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1157client # [ 0.472495] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1158worker1 # [ 0.450046] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1159worker2 # [ 0.494679] pci 0000:00:07.0: enabling device (0000 -> 0002)1160worker1 # [ 0.450642] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1161client # [ 0.472570] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1162worker1 # [ 0.450656] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1163client # [ 0.473062] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1164client # [ 0.473254] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1165worker1 # [ 0.450662] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1166client # [ 0.473273] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1167worker1 # [ 0.450710] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1168client # [ 0.473304] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1169worker1 # [ 0.450759] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1170lb # [ 0.437122] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1171client # [ 0.473320] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1172worker1 # [ 0.450811] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1173lb # [ 0.437343] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1174client # [ 0.473810] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1175lb # [ 0.437373] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1176client # [ 0.474039] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1177client # [ 0.474057] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1178lb # [ 0.437847] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1179client # [ 0.474087] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1180lb # [ 0.438037] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1181worker1 # [ 0.450861] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1182lb # [ 0.438068] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1183client # [ 0.474554] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1184worker1 # [ 0.450938] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1185client # [ 0.474743] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1186lb # [ 0.438546] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1187worker1 # [ 0.450991] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1188client # [ 0.474760] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1189lb # [ 0.438853] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1190lb # [ 0.438871] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1191client # [ 0.474790] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1192worker1 # [ 0.451039] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1193lb # [ 0.438901] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1194client # [ 0.475287] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1195worker1 # [ 0.451086] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1196client # [ 0.475480] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1197lb # [ 0.439422] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1198client # [ 0.475496] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1199worker1 # [ 0.451136] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1200lb # [ 0.439617] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1201lb # [ 0.439634] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1202client # [ 0.475526] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1203worker1 # [ 0.451211] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1204lb # [ 0.439664] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1205client # [ 0.475546] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1206worker1 # [ 0.451260] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1207lb # [ 0.440259] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1208client # [ 0.476022] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1209worker1 # [ 0.451281] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1210lb # [ 0.440270] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1211client # [ 0.476243] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1212worker1 # [ 0.451304] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1213lb # [ 0.440275] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1214client # [ 0.476276] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1215worker1 # [ 0.451325] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1216lb # [ 0.440323] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1217client # [ 0.476800] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1218worker1 # [ 0.451347] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1219client # [ 0.476996] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1220lb # [ 0.440371] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1221worker1 # [ 0.451369] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1222client # [ 0.477027] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1223lb # [ 0.440418] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1224worker1 # [ 0.451390] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1225client # [ 0.477463] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1226lb # [ 0.440469] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1227worker1 # [ 0.451418] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1228client # [ 0.477652] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1229lb # [ 0.440517] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1230worker1 # [ 0.451440] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1231client # [ 0.477926] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1232worker1 # [ 0.451462] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1233lb # [ 0.440565] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1234worker1 # [ 0.451484] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1235lb # [ 0.440613] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1236worker1 # [ 0.451506] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1237lb # [ 0.440661] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1238worker1 # [ 0.451528] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1239lb # [ 0.440707] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1240worker2 # [ 0.564418] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1241worker1 # [ 0.451549] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1242worker1 # [ 0.451571] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1243worker1 # [ 0.451592] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1244worker1 # [ 0.451613] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1245worker1 # [ 0.451637] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1246worker1 # [ 0.451664] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1247worker1 # [ 0.451673] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1248client # [ 0.518219] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1249worker1 # [ 0.451678] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1250client # [ 0.518276] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1251worker1 # [ 0.452533] pci 0000:00:07.0: enabling device (0000 -> 0002)1252client # [ 0.518831] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1253client # [ 0.519031] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1254client # [ 0.519066] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1255lb # [ 0.468941] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1256client # [ 0.519549] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1257lb # [ 0.469014] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1258client # [ 0.519750] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1259lb # [ 0.469038] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1260client # [ 0.519782] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1261lb # [ 0.469061] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1262worker2 # [ 0.574728] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1263lb # [ 0.469084] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1264client # [ 0.520284] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1265client # [ 0.520583] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1266lb # [ 0.469107] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1267client # [ 0.520603] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1268lb # [ 0.469132] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1269client # [ 0.520633] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1270lb # [ 0.469155] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1271lb # [ 0.469183] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1272client # [ 0.521133] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1273client # [ 0.521328] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1274lb # [ 0.469205] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1275client # [ 0.521345] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1276lb # [ 0.469229] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1277client # [ 0.521375] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1278lb # [ 0.469253] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1279worker2 # [ 0.579271] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1280lb # [ 0.469276] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1281lb # [ 0.469298] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1282worker2 # [ 0.581745] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1283lb # [ 0.469318] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1284worker2 # [ 0.584137] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1285lb # [ 0.469341] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1286lb # [ 0.469363] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1287lb # [ 0.469384] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1288lb # [ 0.469406] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1289lb # [ 0.469438] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1290lb # [ 0.469448] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1291lb # [ 0.469453] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1292lb # [ 0.470299] pci 0000:00:07.0: enabling device (0000 -> 0002)1293client # [ 0.542097] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1294client # [ 0.542127] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1295client # [ 0.542132] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1296client # [ 0.542229] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1297worker2 # [ 0.594788] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1298client # [ 0.542286] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1299worker2 # [ 0.596910] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1300worker1 # [ 0.526276] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1301client # [ 0.542335] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1302client # [ 0.542387] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1303client # [ 0.542436] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1304client # [ 0.542489] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1305client # [ 0.542537] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1306client # [ 0.542585] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1307client # [ 0.542636] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1308client # [ 0.542733] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1309client # [ 0.542822] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1310client # [ 0.542846] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1311worker2 # [ 0.607260] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1312client # [ 0.542868] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1313client # [ 0.542891] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1314client # [ 0.542918] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1315worker1 # [ 0.536604] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1316client # [ 0.542940] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1317client # [ 0.542963] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1318client # [ 0.542992] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1319client # [ 0.543016] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1320client # [ 0.543039] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1321client # [ 0.543062] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1322client # [ 0.543087] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1323client # [ 0.543110] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1324lb # [ 0.515963] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1325client # [ 0.543136] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1326client # [ 0.543158] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1327client # [ 0.543180] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1328client # [ 0.543203] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1329client # [ 0.543226] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1330lb # [ 0.519472] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1331worker1 # [ 0.547624] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1332worker1 # [ 0.549946] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1333client # [ 0.543255] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1334client # [ 0.543266] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1335client # [ 0.543271] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1336client # [ 0.544136] pci 0000:00:07.0: enabling device (0000 -> 0002)1337worker2 # [ 0.617440] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1338worker2 # [ 0.619825] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1339worker1 # [ 0.554219] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1340worker2 # [ 0.623448] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1341lb # [ 0.525841] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1342worker1 # [ 0.562389] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1343lb # [ 0.537408] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1344lb # [ 0.539640] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1345worker2 # [ 0.644869] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1346client # [ 0.596276] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1347worker1 # [ 0.572434] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1348worker2 # [ 0.647824] msm_serial: driver initialized1349worker2 # [ 0.647972] SuperH (H)SCI(F) driver initialized1350worker2 # [ 0.648029] STM32 USART driver initialized1351worker1 # [ 0.576695] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1352worker1 # [ 0.578713] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1353lb # [ 0.549963] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1354lb # [ 0.552529] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1355client # [ 0.606735] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1356lb # [ 0.563680] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1357worker1 # [ 0.592437] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1358lb # [ 0.566586] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1359worker1 # [ 0.595895] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1360client # [ 0.618312] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1361client # [ 0.620485] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1362client # [ 0.622685] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1363client # [ 0.624995] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1364worker1 # [ 0.609137] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1365worker2 # [ 0.683205] loop: module loaded1366lb # [ 0.576891] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1367worker2 # [ 0.683429] virtio_blk virtio2: 1/0/0 default/read/poll queues1368lb # [ 0.579439] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1369worker1 # [ 0.612002] msm_serial: driver initialized1370worker1 # [ 0.612149] SuperH (H)SCI(F) driver initialized1371worker2 # [ 0.684498] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1372client # [ 0.635328] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1373worker1 # [ 0.612204] STM32 USART driver initialized1374client # [ 0.637439] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1375lb # [ 0.590061] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1376lb # [ 0.592872] msm_serial: driver initialized1377client # [ 0.648832] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1378lb # [ 0.593110] SuperH (H)SCI(F) driver initialized1379lb # [ 0.593172] STM32 USART driver initialized1380worker2 # [ 0.697987] megasas: 07.734.00.00-rc11381worker2 # [ 0.698897] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1382client # [ 0.655143] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1383client # [ 0.657730] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1384worker2 # [ 0.710493] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001385worker2 # [ 0.710554] Intel/Sharp Extended Query Table at 0x00311386worker2 # [ 0.712494] Using buffer write method1387worker2 # [ 0.712621] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1388worker1 # [ 0.647881] loop: module loaded1389worker1 # [ 0.648108] virtio_blk virtio2: 1/0/0 default/read/poll queues1390worker1 # [ 0.648941] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1391client # [ 0.675482] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1392worker2 # [ 0.726516] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001393worker2 # [ 0.726574] Intel/Sharp Extended Query Table at 0x00311394worker2 # [ 0.728476] Using buffer write method1395worker1 # [ 0.655649] megasas: 07.734.00.00-rc11396worker2 # [ 0.728522] Concatenating MTD devices:1397worker2 # [ 0.728526] (0): "0.flash"1398worker1 # [ 0.656444] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1399worker2 # [ 0.728530] (1): "0.flash"1400worker2 # [ 0.728543] into device "0.flash"1401lb # [ 0.630085] loop: module loaded1402lb # [ 0.630286] virtio_blk virtio2: 1/0/0 default/read/poll queues1403client # [ 0.683520] msm_serial: driver initialized1404lb # [ 0.631185] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1405client # [ 0.683715] SuperH (H)SCI(F) driver initialized1406client # [ 0.683784] STM32 USART driver initialized1407worker1 # [ 0.667713] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001408worker1 # [ 0.667755] Intel/Sharp Extended Query Table at 0x00311409worker1 # [ 0.670316] Using buffer write method1410worker1 # [ 0.670390] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1411lb # [ 0.641516] megasas: 07.734.00.00-rc11412lb # [ 0.642313] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1413lb # [ 0.644691] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001414worker1 # [ 0.675624] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001415worker1 # [ 0.675653] Intel/Sharp Extended Query Table at 0x00311416lb # [ 0.647577] Intel/Sharp Extended Query Table at 0x00311417worker1 # [ 0.677303] Using buffer write method1418worker1 # [ 0.677332] Concatenating MTD devices:1419worker1 # [ 0.677336] (0): "0.flash"1420worker1 # [ 0.677340] (1): "0.flash"1421worker1 # [ 0.677351] into device "0.flash"1422lb # [ 0.653396] Using buffer write method1423lb # [ 0.653496] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1424lb # [ 0.656676] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001425lb # [ 0.656705] Intel/Sharp Extended Query Table at 0x00311426client # [ 0.721420] loop: module loaded1427client # [ 0.721666] virtio_blk virtio2: 1/0/0 default/read/poll queues1428lb # [ 0.670460] Using buffer write method1429lb # [ 0.670526] Concatenating MTD devices:1430lb # [ 0.670531] (0): "0.flash"1431lb # [ 0.670535] (1): "0.flash"1432lb # [ 0.670541] into device "0.flash"1433client # [ 0.730882] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1434client # [ 0.735892] megasas: 07.734.00.00-rc11435client # [ 0.736925] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1436client # [ 0.740898] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001437client # [ 0.740935] Intel/Sharp Extended Query Table at 0x00311438client # [ 0.751564] Using buffer write method1439client # [ 0.751678] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1440client # [ 0.753700] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001441client # [ 0.753746] Intel/Sharp Extended Query Table at 0x00311442client # [ 0.767170] Using buffer write method1443client # [ 0.767224] Concatenating MTD devices:1444client # [ 0.767228] (0): "0.flash"1445client # [ 0.767232] (1): "0.flash"1446client # [ 0.767244] into device "0.flash"1447worker2 # [ 1.030072] Freeing initrd memory: 26944K1448worker2 # [ 1.036467] tun: Universal TUN/TAP device driver, 1.61449worker2 # [ 1.041565] thunder_xcv, ver 1.01450worker2 # [ 1.041620] thunder_bgx, ver 1.01451worker2 # [ 1.041642] nicpf, ver 1.01452worker2 # [ 1.042252] e1000: Intel(R) PRO/1000 Network Driver1453worker2 # [ 1.042265] e1000: Copyright (c) 1999-2006 Intel Corporation.1454worker1 # [ 0.976861] Freeing initrd memory: 26976K1455worker2 # [ 1.042290] e1000e: Intel(R) PRO/1000 Network Driver1456worker2 # [ 1.042299] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1457worker2 # [ 1.042331] igb: Intel(R) Gigabit Ethernet Network Driver1458worker2 # [ 1.042337] igb: Copyright (c) 2007-2014 Intel Corporation.1459worker2 # [ 1.042386] igbvf: Intel(R) Gigabit Virtual Function Network Driver1460worker2 # [ 1.042392] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1461worker2 # [ 1.042533] sky2: driver version 1.301462worker2 # [ 1.044151] usbcore: registered new interface driver usb-storage1463worker2 # [ 1.044216] usbcore: registered new interface driver usbserial_generic1464worker1 # [ 0.983258] tun: Universal TUN/TAP device driver, 1.61465worker2 # [ 1.044230] usbserial: USB Serial support registered for generic1466worker2 # [ 1.044846] hv_vmbus: registering driver hyperv_keyboard1467worker2 # [ 1.045910] ehci-pci 0000:00:07.0: EHCI Host Controller1468worker2 # [ 1.045946] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11469worker2 # [ 1.046241] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001470worker2 # [ 1.059467] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001471worker2 # [ 1.059899] hub 1-0:1.0: USB hub found1472worker2 # [ 1.059929] hub 1-0:1.0: 6 ports detected1473worker1 # [ 0.987420] thunder_xcv, ver 1.01474worker1 # [ 0.987466] thunder_bgx, ver 1.01475worker1 # [ 0.987492] nicpf, ver 1.01476worker1 # [ 0.988087] e1000: Intel(R) PRO/1000 Network Driver1477worker2 # [ 1.063731] rtc-pl031 9010000.pl031: registered as rtc01478worker1 # [ 0.988095] e1000: Copyright (c) 1999-2006 Intel Corporation.1479worker2 # [ 1.063774] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:12:13 UTC (1790140333)1480worker1 # [ 0.988120] e1000e: Intel(R) PRO/1000 Network Driver1481worker2 # [ 1.064120] i2c_dev: i2c /dev entries driver1482worker1 # [ 0.988128] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1483worker1 # [ 0.988157] igb: Intel(R) Gigabit Ethernet Network Driver1484worker1 # [ 0.988163] igb: Copyright (c) 2007-2014 Intel Corporation.1485worker1 # [ 0.988186] igbvf: Intel(R) Gigabit Virtual Function Network Driver1486worker1 # [ 0.988192] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1487worker1 # [ 0.988324] sky2: driver version 1.301488worker1 # [ 0.989962] usbcore: registered new interface driver usb-storage1489worker1 # [ 0.990025] usbcore: registered new interface driver usbserial_generic1490worker1 # [ 0.990039] usbserial: USB Serial support registered for generic1491worker2 # [ 1.069973] sdhci: Secure Digital Host Controller Interface driver1492worker1 # [ 0.990655] hv_vmbus: registering driver hyperv_keyboard1493worker2 # [ 1.069992] sdhci: Copyright(c) Pierre Ossman1494worker1 # [ 0.991611] ehci-pci 0000:00:07.0: EHCI Host Controller1495worker2 # [ 1.070266] Synopsys Designware Multimedia Card Interface Driver1496worker1 # [ 0.991639] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11497worker2 # [ 1.070644] sdhci-pltfm: SDHCI platform and OF driver helper1498worker1 # [ 0.991814] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001499worker2 # [ 1.072334] hid: raw HID events driver (C) Jiri Kosina1500worker2 # [ 1.072585] usbcore: registered new interface driver usbhid1501worker2 # [ 1.072592] usbhid: USB HID core driver1502worker1 # [ 1.004681] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001503worker1 # [ 1.005059] hub 1-0:1.0: USB hub found1504worker1 # [ 1.005080] hub 1-0:1.0: 6 ports detected1505lb # [ 0.977407] Freeing initrd memory: 26948K1506worker1 # [ 1.008124] rtc-pl031 9010000.pl031: registered as rtc01507worker2 # [ 1.078678] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1508worker2 # [ 1.080330] drop_monitor: Initializing network drop monitor service1509worker1 # [ 1.008159] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:12:13 UTC (1790140333)1510worker2 # [ 1.080513] NET: Registered PF_INET6 protocol family1511worker1 # [ 1.008468] i2c_dev: i2c /dev entries driver1512worker2 # [ 1.084130] Segment Routing with IPv61513worker2 # [ 1.084175] In-situ OAM (IOAM) with IPv61514worker2 # [ 1.084206] NET: Registered PF_PACKET protocol family1515lb # [ 0.984024] tun: Universal TUN/TAP device driver, 1.61516worker1 # [ 1.014056] sdhci: Secure Digital Host Controller Interface driver1517worker1 # [ 1.014073] sdhci: Copyright(c) Pierre Ossman1518worker1 # [ 1.014340] Synopsys Designware Multimedia Card Interface Driver1519worker1 # [ 1.014721] sdhci-pltfm: SDHCI platform and OF driver helper1520worker2 # [ 1.086021] 9pnet: Installing 9P2000 support1521worker2 # [ 1.086128] Key type dns_resolver registered1522lb # [ 0.987903] thunder_xcv, ver 1.01523lb # [ 0.987952] thunder_bgx, ver 1.01524lb # [ 0.987976] nicpf, ver 1.01525lb # [ 0.988609] e1000: Intel(R) PRO/1000 Network Driver1526worker1 # [ 1.019230] hid: raw HID events driver (C) Jiri Kosina1527lb # [ 0.988619] e1000: Copyright (c) 1999-2006 Intel Corporation.1528worker1 # [ 1.019484] usbcore: registered new interface driver usbhid1529lb # [ 0.988661] e1000e: Intel(R) PRO/1000 Network Driver1530worker1 # [ 1.019491] usbhid: USB HID core driver1531lb # [ 0.988668] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1532lb # [ 0.988701] igb: Intel(R) Gigabit Ethernet Network Driver1533worker1 # [ 1.022509] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1534lb # [ 0.988707] igb: Copyright (c) 2007-2014 Intel Corporation.1535worker2 # [ 1.094937] registered taskstats version 11536worker2 # [ 1.095109] Loading compiled-in X.509 certificates1537worker1 # [ 1.025246] drop_monitor: Initializing network drop monitor service1538client # [ 1.047511] Freeing initrd memory: 26972K1539worker1 # [ 1.025466] NET: Registered PF_INET6 protocol family1540worker1 # [ 1.027755] Segment Routing with IPv61541lb # [ 0.994129] igbvf: Intel(R) Gigabit Virtual Function Network Driver1542worker1 # [ 1.027779] In-situ OAM (IOAM) with IPv61543lb # [ 0.994139] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1544worker1 # [ 1.027830] NET: Registered PF_PACKET protocol family1545lb # [ 0.994292] sky2: driver version 1.301546lb # [ 0.996109] usbcore: registered new interface driver usb-storage1547lb # [ 0.996208] usbcore: registered new interface driver usbserial_generic1548worker1 # [ 1.029599] 9pnet: Installing 9P2000 support1549worker1 # [ 1.029650] Key type dns_resolver registered1550client # [ 1.054283] tun: Universal TUN/TAP device driver, 1.61551lb # [ 0.996225] usbserial: USB Serial support registered for generic1552lb # [ 0.997503] ehci-pci 0000:00:07.0: EHCI Host Controller1553worker2 # [ 1.104270] Demotion targets for Node 0: null1554lb # [ 0.997530] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11555worker2 # [ 1.104432] Key type .fscrypt registered1556lb # [ 0.997704] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001557worker2 # [ 1.104448] Key type fscrypt-provisioning registered1558lb # [ 1.004439] hv_vmbus: registering driver hyperv_keyboard1559worker2 # [ 1.104570] ima: No TPM chip found, activating TPM-bypass!1560worker2 # [ 1.104592] ima: Allocated hash algorithm: sha11561worker2 # [ 1.104618] ima: No architecture policies found1562lb # [ 1.006169] rtc-pl031 9010000.pl031: registered as rtc01563worker2 # [ 1.109233] input: gpio-keys as /devices/platform/gpio-keys/input/input01564lb # [ 1.006201] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:12:13 UTC (1790140333)1565worker1 # [ 1.036597] registered taskstats version 11566lb # [ 1.006523] i2c_dev: i2c /dev entries driver1567worker1 # [ 1.036767] Loading compiled-in X.509 certificates1568client # [ 1.058575] thunder_xcv, ver 1.01569lb # [ 1.009769] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001570client # [ 1.058621] thunder_bgx, ver 1.01571client # [ 1.058648] nicpf, ver 1.01572lb # [ 1.010814] hub 1-0:1.0: USB hub found1573lb # [ 1.011359] hub 1-0:1.0: 6 ports detected1574client # [ 1.059233] e1000: Intel(R) PRO/1000 Network Driver1575client # [ 1.059242] e1000: Copyright (c) 1999-2006 Intel Corporation.1576client # [ 1.059266] e1000e: Intel(R) PRO/1000 Network Driver1577client # [ 1.059276] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1578client # [ 1.059310] igb: Intel(R) Gigabit Ethernet Network Driver1579client # [ 1.059317] igb: Copyright (c) 2007-2014 Intel Corporation.1580client # [ 1.059341] igbvf: Intel(R) Gigabit Virtual Function Network Driver1581worker1 # [ 1.045703] Demotion targets for Node 0: null1582client # [ 1.059347] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1583worker1 # [ 1.045819] Key type .fscrypt registered1584lb # [ 1.014813] sdhci: Secure Digital Host Controller Interface driver1585client # [ 1.059488] sky2: driver version 1.301586worker1 # [ 1.045826] Key type fscrypt-provisioning registered1587lb # [ 1.014829] sdhci: Copyright(c) Pierre Ossman1588client # [ 1.061152] usbcore: registered new interface driver usb-storage1589worker1 # [ 1.045924] ima: No TPM chip found, activating TPM-bypass!1590lb # [ 1.015152] Synopsys Designware Multimedia Card Interface Driver1591client # [ 1.061219] usbcore: registered new interface driver usbserial_generic1592worker1 # [ 1.045944] ima: Allocated hash algorithm: sha11593lb # [ 1.015541] sdhci-pltfm: SDHCI platform and OF driver helper1594worker1 # [ 1.045967] ima: No architecture policies found1595client # [ 1.061234] usbserial: USB Serial support registered for generic1596client # [ 1.061866] hv_vmbus: registering driver hyperv_keyboard1597worker1 # [ 1.050368] input: gpio-keys as /devices/platform/gpio-keys/input/input01598lb # [ 1.019967] hid: raw HID events driver (C) Jiri Kosina1599client # [ 1.062831] ehci-pci 0000:00:07.0: EHCI Host Controller1600lb # [ 1.020247] usbcore: registered new interface driver usbhid1601client # [ 1.062860] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11602lb # [ 1.020254] usbhid: USB HID core driver1603client # [ 1.063100] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001604client # [ 1.076216] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001605client # [ 1.076599] hub 1-0:1.0: USB hub found1606client # [ 1.076622] hub 1-0:1.0: 6 ports detected1607lb # [ 1.023280] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1608client # [ 1.079828] rtc-pl031 9010000.pl031: registered as rtc01609lb # [ 1.027529] drop_monitor: Initializing network drop monitor service1610lb # [ 1.027728] NET: Registered PF_INET6 protocol family1611client # [ 1.079866] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T05:12:13 UTC (1790140333)1612lb # [ 1.029804] Segment Routing with IPv61613client # [ 1.080254] i2c_dev: i2c /dev entries driver1614lb # [ 1.029824] In-situ OAM (IOAM) with IPv61615lb # [ 1.029854] NET: Registered PF_PACKET protocol family1616lb # [ 1.031532] 9pnet: Installing 9P2000 support1617lb # [ 1.031586] Key type dns_resolver registered1618client # [ 1.086124] sdhci: Secure Digital Host Controller Interface driver1619client # [ 1.086135] sdhci: Copyright(c) Pierre Ossman1620client # [ 1.086406] Synopsys Designware Multimedia Card Interface Driver1621client # [ 1.086788] sdhci-pltfm: SDHCI platform and OF driver helper1622client # [ 1.088449] hid: raw HID events driver (C) Jiri Kosina1623client # [ 1.088729] usbcore: registered new interface driver usbhid1624worker1 # [ 1.070253] clk: Disabling unused clocks1625lb # [ 1.038636] registered taskstats version 11626client # [ 1.088737] usbhid: USB HID core driver1627lb # [ 1.038785] Loading compiled-in X.509 certificates1628worker1 # [ 1.070292] PM: genpd: Disabling unused power domains1629worker2 # [ 1.141964] clk: Disabling unused clocks1630worker2 # [ 1.142006] PM: genpd: Disabling unused power domains1631client # [ 1.094672] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1632worker1 # [ 1.074637] Freeing unused kernel memory: 4736K1633client # [ 1.096354] drop_monitor: Initializing network drop monitor service1634worker2 # [ 1.146449] Freeing unused kernel memory: 4736K1635worker1 # [ 1.074846] Run /init as init process1636worker2 # [ 1.146693] Run /init as init process1637client # [ 1.096526] NET: Registered PF_INET6 protocol family1638client # [ 1.099984] Segment Routing with IPv61639client # [ 1.100034] In-situ OAM (IOAM) with IPv61640client # [ 1.100070] NET: Registered PF_PACKET protocol family1641lb # [ 1.047847] Demotion targets for Node 0: null1642client # [ 1.101764] 9pnet: Installing 9P2000 support1643lb # [ 1.047979] Key type .fscrypt registered1644client # [ 1.101828] Key type dns_resolver registered1645lb # [ 1.047986] Key type fscrypt-provisioning registered1646lb # [ 1.048099] ima: No TPM chip found, activating TPM-bypass!1647lb # [ 1.048123] ima: Allocated hash algorithm: sha11648lb # [ 1.048146] ima: No architecture policies found1649lb # [ 1.052423] input: gpio-keys as /devices/platform/gpio-keys/input/input01650client # [ 1.108822] registered taskstats version 11651client # [ 1.108997] Loading compiled-in X.509 certificates1652worker2 # [ 1.164236] systemd[1]: Successfully made /usr/ read-only.1653worker1 # [ 1.095325] systemd[1]: Successfully made /usr/ read-only.1654client # [ 1.118195] Demotion targets for Node 0: null1655client # [ 1.118339] Key type .fscrypt registered1656client # [ 1.118348] Key type fscrypt-provisioning registered1657client # [ 1.118469] ima: No TPM chip found, activating TPM-bypass!1658client # [ 1.118491] ima: Allocated hash algorithm: sha11659client # [ 1.118518] ima: No architecture policies found1660client # [ 1.122900] input: gpio-keys as /devices/platform/gpio-keys/input/input01661lb # [ 1.070785] clk: Disabling unused clocks1662lb # [ 1.071154] PM: genpd: Disabling unused power domains1663lb # [ 1.075680] Freeing unused kernel memory: 4736K1664lb # [ 1.075872] Run /init as init process1665lb # [ 1.091973] systemd[1]: Successfully made /usr/ read-only.1666client # [ 1.156509] clk: Disabling unused clocks1667client # [ 1.156545] PM: genpd: Disabling unused power domains1668client # [ 1.160895] Freeing unused kernel memory: 4736K1669client # [ 1.161103] Run /init as init process1670client # [ 1.177114] systemd[1]: Successfully made /usr/ read-only.1671worker2 # [ 1.305190] usb 1-1: new high-speed USB device number 2 using ehci-pci1672worker1 # [ 1.250973] usb 1-1: new high-speed USB device number 2 using ehci-pci1673lb # [ 1.257654] usb 1-1: new high-speed USB device number 2 using ehci-pci1674client # [ 1.322074] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker2 # [ 1.458336] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11676worker1 # [ 1.409813] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677worker2 # [ 1.499340] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1678worker2 # [ 1.503481] systemd[1]: Detected virtualization qemu.1679worker2 # [ 1.504181] systemd[1]: Detected architecture arm64.1680worker2 # [ 1.504855] systemd[1]: Running in initrd.1681worker2 # [ 1.505922] systemd[1]: Initializing machine ID from random generator.1682worker2 # [ 1.506937] systemd[1]: Hostname set to <worker2>.1683worker1 # [ 1.430498] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1684worker1 # [ 1.443418] systemd[1]: Detected virtualization qemu.1685worker1 # [ 1.445594] systemd[1]: Detected architecture arm64.1686worker1 # [ 1.447690] systemd[1]: Running in initrd.1687lb # [ 1.413633] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11688worker1 # [ 1.450741] systemd[1]: Initializing machine ID from random generator.1689worker1 # [ 1.454032] systemd[1]: Hostname set to <worker1>.1690client # [ 1.474782] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11691lb # [ 1.427387] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1692lb # [ 1.440301] systemd[1]: Detected virtualization qemu.1693lb # [ 1.440410] systemd[1]: Detected architecture arm64.1694lb # [ 1.440435] systemd[1]: Running in initrd.1695lb # [ 1.447516] systemd[1]: Initializing machine ID from random generator.1696lb # [ 1.450650] systemd[1]: Hostname set to <lb>.1697worker2 # [ 1.556802] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01698client # [ 1.512254] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1699client # [ 1.525006] systemd[1]: Detected virtualization qemu.1700client # [ 1.527285] systemd[1]: Detected architecture arm64.1701client # [ 1.529315] systemd[1]: Running in initrd.1702worker1 # [ 1.507235] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01703client # [ 1.534344] systemd[1]: Initializing machine ID from random generator.1704client # [ 1.537679] systemd[1]: Hostname set to <client>.1705client # [ 1.554448] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706lb # [ 1.509065] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker2 # [ 1.677105] usb 1-2: new high-speed USB device number 3 using ehci-pci1708worker1 # [ 1.630940] usb 1-2: new high-speed USB device number 3 using ehci-pci1709client # [ 1.686086] usb 1-2: new high-speed USB device number 3 using ehci-pci1710lb # [ 1.632786] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker2 # [ 1.777550] systemd[1]: bpf-restrict-fs: LSM BPF program attached1712worker2 # [ 1.829579] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21713worker2 # [ 1.833339] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01714worker1 # [ 1.789694] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715worker1 # [ 1.793704] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21716worker2 # [ 1.874311] systemd[1]: Queued start job for default target Initrd Default Target.1717worker1 # [ 1.799450] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01718worker2 # [ 1.883526] systemd[1]: Created slice Slice /system/modprobe.1719worker2 # [ 1.884659] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1720worker2 # [ 1.885972] systemd[1]: Expecting device /dev/disk/by-label/nixos...1721worker2 # [ 1.887029] systemd[1]: Reached target Path Units.1722worker2 # [ 1.887772] systemd[1]: Reached target Slice Units.1723worker2 # [ 1.888533] systemd[1]: Reached target Swaps.1724worker2 # [ 1.889274] systemd[1]: Reached target Timer Units.1725worker2 # [ 1.890218] systemd[1]: Listening on D-Bus System Message Bus Socket.1726worker2 # [ 1.891345] systemd[1]: Listening on Journal Socket (/dev/log).1727lb # [ 1.785145] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21728worker2 # [ 1.892417] systemd[1]: Listening on Journal Sockets.1729client # [ 1.838725] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21730worker2 # [ 1.893374] systemd[1]: Listening on udev Control Socket.1731worker2 # [ 1.894306] systemd[1]: Listening on udev Kernel Socket.1732worker2 # [ 1.895225] systemd[1]: Reached target Socket Units.1733lb # [ 1.790844] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01734worker2 # [ 1.897648] systemd[1]: Starting Create List of Static Device Nodes...1735client # [ 1.846289] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01736worker2 # [ 1.898746] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1737worker2 # [ 1.907260] systemd[1]: Mounting Kernel Configuration File System...1738worker2 # [ 1.917291] systemd[1]: Starting Journal Service...1739client # [ 1.877375] systemd[1]: bpf-restrict-fs: LSM BPF program attached1740worker2 # [ 1.926287] systemd[1]: Starting Load Kernel Modules...1741worker2 # [ 1.928757] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1742worker2 # [ 1.957347] systemd[1]: Starting Coldplug All udev Devices...1743worker1 # [ 1.895701] systemd[1]: Queued start job for default target Initrd Default Target.1744worker2 # [ 1.973619] systemd[1]: Finished Create List of Static Device Nodes.1745worker1 # [ 1.905406] systemd[1]: Created slice Slice /system/modprobe.1746worker1 # [ 1.906708] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1747worker1 # [ 1.908185] systemd[1]: Expecting device /dev/disk/by-label/nixos...1748lb # [ 1.880558] systemd[1]: bpf-restrict-fs: LSM BPF program attached1749worker1 # [ 1.908223] systemd[1]: Reached target Path Units.1750worker1 # [ 1.908244] systemd[1]: Reached target Slice Units.1751worker1 # [ 1.908266] systemd[1]: Reached target Swaps.1752worker2 # [ 1.985502] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1753worker1 # [ 1.908286] systemd[1]: Reached target Timer Units.1754worker2 # [ 1.986903] systemd[1]: Mounted Kernel Configuration File System.1755worker1 # [ 1.908482] systemd[1]: Listening on D-Bus System Message Bus Socket.1756worker1 # [ 1.908680] systemd[1]: Listening on Journal Socket (/dev/log).1757worker1 # [ 1.908874] systemd[1]: Listening on Journal Sockets.1758worker1 # [ 1.909018] systemd[1]: Listening on udev Control Socket.1759worker1 # [ 1.909156] systemd[1]: Listening on udev Kernel Socket.1760worker1 # [ 1.909181] systemd[1]: Reached target Socket Units.1761worker1 # [ 1.921642] systemd[1]: Starting Create List of Static Device Nodes...1762worker2 # [ 1.995083] systemd-journald[72]: Collecting audit messages is disabled.1763worker1 # [ 1.922829] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1764worker1 # [ 1.934646] systemd[1]: Mounting Kernel Configuration File System...1765client # [ 1.977268] systemd[1]: Queued start job for default target Initrd Default Target.1766worker1 # [ 1.963433] systemd[1]: Starting Journal Service...1767client # [ 1.988721] systemd[1]: Created slice Slice /system/modprobe.1768worker1 # [ 1.966187] systemd[1]: Starting Load Kernel Modules...1769client # [ 1.989971] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1770worker1 # [ 1.967286] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1771client # [ 1.991391] systemd[1]: Expecting device /dev/disk/by-label/nixos...1772client # [ 1.992537] systemd[1]: Reached target Path Units.1773client # [ 1.993363] systemd[1]: Reached target Slice Units.1774client # [ 1.994234] systemd[1]: Reached target Swaps.1775client # [ 1.994976] systemd[1]: Reached target Timer Units.1776client # [ 1.996012] systemd[1]: Listening on D-Bus System Message Bus Socket.1777client # [ 1.997270] systemd[1]: Listening on Journal Socket (/dev/log).1778client # [ 1.998510] systemd[1]: Listening on Journal Sockets.1779client # [ 1.998682] systemd[1]: Listening on udev Control Socket.1780worker1 # [ 1.978609] systemd[1]: Starting Coldplug All udev Devices...1781client # [ 1.998811] systemd[1]: Listening on udev Kernel Socket.1782client # [ 1.998837] systemd[1]: Reached target Socket Units.1783client # [ 2.004188] systemd[1]: Starting Create List of Static Device Nodes...1784worker2 # [ 2.050740] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1785client # [ 2.005383] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1786worker2 # [ 2.051716] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1787worker2 # [ 2.061534] systemd[1]: Starting Create Static Device Nodes in /dev...1788client # [ 2.014174] systemd[1]: Mounting Kernel Configuration File System...1789worker1 # [ 1.992418] systemd[1]: Finished Create List of Static Device Nodes.1790worker1 # [ 1.993198] systemd[1]: Mounted Kernel Configuration File System.1791worker2 # [ 2.069166] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1792client # [ 2.022336] systemd[1]: Starting Journal Service...1793worker1 # [ 2.011155] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1794worker2 # [ 2.082534] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01795worker2 # [ 2.082779] [drm] features: -virgl +edid -resource_blob -host_visible1796worker2 # [ 2.082790] [drm] features: -context_init1797worker2 # [ 2.083547] [drm] number of scanouts: 11798worker2 # [ 2.083569] [drm] number of cap sets: 01799client # [ 2.046643] systemd[1]: Starting Load Kernel Modules...1800lb # [ 1.994431] systemd[1]: Queued start job for default target Initrd Default Target.1801client # [ 2.046775] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1802worker2 # [ 2.101843] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1803lb # [ 2.003690] systemd[1]: Created slice Slice /system/modprobe.1804worker2 # [ 2.101870] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01805lb # [ 2.005142] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1806lb # [ 2.006647] systemd[1]: Expecting device /dev/disk/by-label/nixos...1807lb # [ 2.007878] systemd[1]: Reached target Path Units.1808lb # [ 2.008883] systemd[1]: Reached target Slice Units.1809lb # [ 2.009845] systemd[1]: Reached target Swaps.1810lb # [ 2.010713] systemd[1]: Reached target Timer Units.1811worker1 # [ 2.039793] systemd-journald[72]: Collecting audit messages is disabled.1812lb # [ 2.011917] systemd[1]: Listening on D-Bus System Message Bus Socket.1813lb # [ 2.013439] systemd[1]: Listening on Journal Socket (/dev/log).1814lb # [ 2.014741] systemd[1]: Listening on Journal Sockets.1815lb # [ 2.015965] systemd[1]: Listening on udev Control Socket.1816client # [ 2.069745] systemd-journald[72]: Collecting audit messages is disabled.1817lb # [ 2.017205] systemd[1]: Listening on udev Kernel Socket.1818lb # [ 2.017243] systemd[1]: Reached target Socket Units.1819lb # [ 2.021048] systemd[1]: Starting Create List of Static Device Nodes...1820worker1 # [ 2.048223] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1821client # [ 2.074187] systemd[1]: Starting Coldplug All udev Devices...1822lb # [ 2.022337] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1823worker2 # [ 2.125362] systemd[1]: Finished Create Static Device Nodes in /dev.1824worker2 # [ 2.125600] systemd[1]: Reached target Preparation for Local File Systems.1825client # [ 2.078997] systemd[1]: Finished Create List of Static Device Nodes.1826worker2 # [ 2.125627] systemd[1]: Reached target Local File Systems.1827client # [ 2.080629] systemd[1]: Mounted Kernel Configuration File System.1828lb # [ 2.033016] systemd[1]: Mounting Kernel Configuration File System...1829worker1 # [ 2.063667] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1830worker1 # [ 2.066468] systemd[1]: Starting Create Static Device Nodes in /dev...1831worker2 # [ 2.137600] systemd[1]: Starting Rule-based Manager for Device Events and Files...1832lb # [ 2.040099] systemd[1]: Starting Journal Service...1833client # [ 2.095263] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1834worker1 # [ 2.071020] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1835worker2 # [ 2.142381] Console: switching to colour frame buffer device 160x501836worker1 # [ 2.077589] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01837worker1 # [ 2.077842] [drm] features: -virgl +edid -resource_blob -host_visible1838worker1 # [ 2.077853] [drm] features: -context_init1839worker1 # [ 2.078619] [drm] number of scanouts: 11840worker1 # [ 2.078638] [drm] number of cap sets: 01841lb # [ 2.058078] systemd[1]: Starting Load Kernel Modules...1842lb # [ 2.058230] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1843worker2 # [ 2.163291] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1844lb # [ 2.073903] systemd[1]: Starting Coldplug All udev Devices...1845worker2 # [ 2.181810] systemd[1]: Finished Load Kernel Modules.1846worker1 # [ 2.107654] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1847worker1 # [ 2.107681] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01848worker2 # [ 2.189587] systemd[1]: Starting Apply Kernel Variables...1849lb # [ 2.096940] systemd[1]: Finished Create List of Static Device Nodes.1850lb # [ 2.097808] systemd[1]: Mounted Kernel Configuration File System.1851worker2 # [ 2.185199] systemd-modules-load[73]: Inserted module 'dm_mod'1852worker2 # [ 2.186287] systemd-modules-load[73]: Module 'virtio_balloon' is built in1853worker2 # [ 2.187384] systemd-modules-load[73]: Module 'virtio_console' is built in1854client # [ 2.154773] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1855worker1 # [ 2.131690] systemd[1]: Finished Create Static Device Nodes in /dev.1856worker1 # [ 2.132091] systemd[1]: Reached target Preparation for Local File Systems.1857client # [ 2.157565] systemd[1]: Starting Create Static Device Nodes in /dev...1858worker1 # [ 2.132119] systemd[1]: Reached target Local File Systems.1859lb # [ 2.105167] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1860worker2 # [ 2.209302] systemd[1]: Started Journal Service.1861worker2 # [ 2.196476] systemd-modules-load[73]: Inserted module 'virtio_gpu'1862client # [ 2.167948] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1863worker1 # [ 2.139193] Console: switching to colour frame buffer device 160x501864worker2 # [ 2.204354] systemd-modules-load[73]: Module 'virtio_rng' is built in1865worker1 # [ 2.149919] systemd[1]: Starting Rule-based Manager for Device Events and Files...1866lb # [ 2.136009] systemd-journald[72]: Collecting audit messages is disabled.1867worker1 # [ 2.171466] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1868client # [ 2.186046] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1869worker2 # [ 2.230608] systemd[1]: Starting Create System Files and Directories...1870client # [ 2.192687] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01871client # [ 2.192955] [drm] features: -virgl +edid -resource_blob -host_visible1872client # [ 2.192964] [drm] features: -context_init1873client # [ 2.193936] [drm] number of scanouts: 11874client # [ 2.193955] [drm] number of cap sets: 01875lb # [ 2.154773] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1876worker1 # [ 2.187528] systemd[1]: Finished Load Kernel Modules.1877worker2 # [ 2.248523] systemd-udevd[79]: Using default interface naming scheme 'v261'.1878worker1 # [ 2.195452] systemd[1]: Starting Apply Kernel Variables...1879client # [ 2.223366] systemd[1]: Finished Create Static Device Nodes in /dev.1880client # [ 2.223761] systemd[1]: Reached target Preparation for Local File Systems.1881client # [ 2.223793] systemd[1]: Reached target Local File Systems.1882worker2 # [ 2.261982] systemd[1]: Finished Apply Kernel Variables.1883client # [ 2.228088] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1884client # [ 2.228108] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01885lb # [ 2.176875] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1886client # [ 2.236266] systemd[1]: Starting Rule-based Manager for Device Events and Files...1887lb # [ 2.183618] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01888lb # [ 2.183902] [drm] features: -virgl +edid -resource_blob -host_visible1889lb # [ 2.183918] [drm] features: -context_init1890worker2 # [ 2.278110] systemd[1]: Finished Create System Files and Directories.1891lb # [ 2.200802] [drm] number of scanouts: 11892lb # [ 2.200840] [drm] number of cap sets: 01893lb # [ 2.202781] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1894worker1 # [ 2.237242] systemd[1]: Started Journal Service.1895client # [ 2.255043] Console: switching to colour frame buffer device 160x501896lb # [ 2.205829] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1897worker2 # [ 2.297173] systemd[1]: Started Rule-based Manager for Device Events and Files.1898lb # [ 2.205849] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01899worker1 # [ 2.232358] systemd-modules-load[73]: Inserted module 'dm_mod'1900client # [ 2.252656] systemd-modules-load[74]: Inserted module 'dm_mod'1901client # [ 2.256263] systemd-modules-load[74]: Module 'virtio_balloon' is built in1902worker1 # [ 2.236537] systemd-modules-load[73]: Module 'virtio_balloon' is built in1903worker1 # [ 2.237664] systemd-modules-load[73]: Module 'virtio_console' is built in1904client # [ 2.257606] systemd-modules-load[74]: Module 'virtio_console' is built in1905worker1 # [ 2.238776] systemd-modules-load[73]: Inserted module 'virtio_gpu'1906worker1 # [ 2.239780] systemd-modules-load[73]: Module 'virtio_rng' is built in1907client # [ 2.277306] systemd[1]: Started Journal Service.1908lb # [ 2.214752] systemd[1]: Starting Create Static Device Nodes in /dev...1909client # [ 2.290061] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1910worker1 # [ 2.256595] systemd[1]: Starting Create System Files and Directories...1911worker1 # [ 2.257946] systemd-udevd[78]: Using default interface naming scheme 'v261'.1912lb # [ 2.237054] Console: switching to colour frame buffer device 160x501913client # [ 2.280867] systemd-modules-load[74]: Inserted module 'virtio_gpu'1914lb # [ 2.244003] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1915worker1 # [ 2.264153] systemd[1]: Finished Apply Kernel Variables.1916client # [ 2.288264] systemd-modules-load[74]: Module 'virtio_rng' is built in1917client # [ 2.292190] systemd[1]: Starting Create System Files and Directories...1918client # [ 2.300794] systemd[1]: Finished Load Kernel Modules.1919lb # [ 2.266051] systemd[1]: Finished Create Static Device Nodes in /dev.1920lb # [ 2.266570] systemd[1]: Reached target Preparation for Local File Systems.1921lb # [ 2.266617] systemd[1]: Reached target Local File Systems.1922worker2 # [ 2.360988] systemd[1]: Starting Virtual Console Setup...1923worker1 # [ 2.293239] systemd[1]: Finished Create System Files and Directories.1924client # [ 2.320461] systemd-udevd[78]: Using default interface naming scheme 'v261'.1925client # [ 2.324198] systemd[1]: Starting Apply Kernel Variables...1926worker1 # [ 2.309086] systemd[1]: Started Rule-based Manager for Device Events and Files.1927lb # [ 2.292898] systemd[1]: Starting Rule-based Manager for Device Events and Files...1928lb # [ 2.295912] systemd[1]: Finished Load Kernel Modules.1929lb # [ 2.299821] systemd[1]: Starting Apply Kernel Variables...1930worker2 # [ 2.412542] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1931worker2 # [ 2.416460] systemd[1]: Finished Virtual Console Setup.1932client # [ 2.369886] systemd[1]: Finished Create System Files and Directories.1933client # [ 2.376319] systemd[1]: Finished Apply Kernel Variables.1934lb # [ 2.340530] systemd[1]: Started Journal Service.1935client # [ 2.390569] systemd[1]: Started Rule-based Manager for Device Events and Files.1936lb # [ 2.341055] systemd-modules-load[73]: Inserted module 'dm_mod'1937worker1 # [ 2.373267] systemd[1]: Starting Virtual Console Setup...1938lb # [ 2.345023] systemd-modules-load[73]: Module 'virtio_balloon' is built in1939lb # [ 2.346246] systemd-modules-load[73]: Module 'virtio_console' is built in1940lb # [ 2.347396] systemd-modules-load[73]: Inserted module 'virtio_gpu'1941lb # [ 2.362727] systemd-modules-load[73]: Module 'virtio_rng' is built in1942lb # [ 2.372595] systemd[1]: Starting Create System Files and Directories...1943lb # [ 2.376679] systemd[1]: Finished Apply Kernel Variables.1944worker1 # [ 2.436508] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1945worker1 # [ 2.440243] systemd[1]: Finished Virtual Console Setup.1946client # [ 2.460510] systemd[1]: Starting Virtual Console Setup...1947lb # [ 2.420487] systemd-udevd[78]: Using default interface naming scheme 'v261'.1948lb # [ 2.424401] systemd[1]: Finished Create System Files and Directories.1949lb # [ 2.453300] systemd[1]: Started Rule-based Manager for Device Events and Files.1950client # [ 2.512533] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1951client # [ 2.516407] systemd[1]: Finished Virtual Console Setup.1952lb # [ 2.516642] systemd[1]: Starting Virtual Console Setup...1953lb # [ 2.572558] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1954lb # [ 2.576138] systemd[1]: Finished Virtual Console Setup.1955worker2 # [ 3.035366] systemd[1]: Finished Coldplug All udev Devices.1956worker2 # [ 3.037079] systemd[1]: Reached target System Initialization.1957worker2 # [ 3.040116] systemd[1]: Reached target Basic System.1958worker1 # [ 3.061161] systemd[1]: Finished Coldplug All udev Devices.1959worker1 # [ 3.062150] systemd[1]: Reached target System Initialization.1960worker1 # [ 3.064101] systemd[1]: Reached target Basic System.1961worker2 # [ 3.172119] (udev-worker)[91]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1962client # [ 3.133062] systemd[1]: Finished Coldplug All udev Devices.1963client # [ 3.134715] systemd[1]: Reached target System Initialization.1964client # [ 3.140151] systemd[1]: Reached target Basic System.1965worker2 # [ 3.213618] (udev-worker)[104]: Network interface NamePolicy= disabled on kernel command line.1966worker2 # [ 3.216567] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1967lb # [ 3.162301] systemd[1]: Finished Coldplug All udev Devices.1968lb # [ 3.163295] systemd[1]: Reached target System Initialization.1969lb # [ 3.164203] systemd[1]: Reached target Basic System.1970worker2 # [ 3.288622] systemd[1]: Found device /dev/disk/by-label/nixos.1971worker2 # [ 3.301675] systemd[1]: Reached target Initrd Root Device.1972worker2 # [ 3.305763] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1973worker1 # [ 3.245282] (udev-worker)[103]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1974worker1 # [ 3.249846] (udev-worker)[103]: Network interface NamePolicy= disabled on kernel command line.1975worker1 # [ 3.257959] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1976worker2 # [ 3.353635] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1977client # [ 3.305738] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1978worker2 # [ 3.364155] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1979worker2 # [ 3.366328] systemd[1]: Mounting /sysroot...1980client # [ 3.322314] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1981client # [ 3.326227] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1982worker2 # [ 3.418945] EXT4-fs (vda): mounted filesystem 203ff707-c759-4c3a-a3f9-d5fbb101f69b r/w with ordered data mode. Quota mode: none.1983worker2 # [ 3.407641] systemd[1]: Mounted /sysroot.1984worker2 # [ 3.410021] systemd[1]: Reached target Initrd Root File System.1985worker1 # [ 3.341041] systemd[1]: Found device /dev/disk/by-label/nixos.1986worker1 # [ 3.342962] systemd[1]: Reached target Initrd Root Device.1987worker2 # [ 3.415359] systemd[1]: Starting Mountpoints Configured in the Real Root...1988worker1 # [ 3.349378] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1989lb # [ 3.317506] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1990lb # [ 3.324418] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1991client # [ 3.393957] systemd[1]: Found device /dev/disk/by-label/nixos.1992worker2 # [ 3.443778] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload.1993client # [ 3.400190] systemd[1]: Reached target Initrd Root Device.1994worker2 # [ 3.450851] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)...1995lb # [ 3.352823] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1996worker2 # [ 3.455753] systemd[1]: Reloading...1997client # [ 3.405501] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1998worker1 # [ 3.397312] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks1999worker1 # [ 3.404186] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.2000worker1 # [ 3.412321] systemd[1]: Mounting /sysroot...
loading 2,377 hidden lines…
4378lb # [ 54.377640] niks3-server[967]: 2026/09/23 05:13:06 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4379lb # [ 54.391226] niks3-server[967]: 2026/09/23 05:13:06 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4380worker1 # [ 54.429312] nix-grpc-daemon[860]: time=2026-09-23T05:13:06.935Z level=INFO msg="Upload complete. (272ms)"4381lb # [ 54.400945] niks3-server[967]: 2026/09/23 05:13:06 INFO Completed upload id=104382lb # [ 54.407735] niks3-server[967]: 2026/09/23 05:13:06 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4383worker1 # [ 54.441351] nix-grpc-daemon[766]: ts=2026-09-23T05:13:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14384worker2 # [ 54.519623] nix-grpc-daemon[769]: ts=2026-09-23T05:13:06Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04385worker2 # [ 54.525755] nix-grpc-daemon[769]: ts=2026-09-23T05:13:06Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=14386worker1 # [ 54.464638] nix-daemon[832]: reaped child process 878, status = succeeded4387worker1 # [ 54.497015] nix-grpc-daemon[766]: ts=2026-09-23T05:13:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490904388worker1 # [ 54.513897] nix-grpc-daemon[766]: ts=2026-09-23T05:13:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490904389worker1 # [ 54.530230] nix-grpc-daemon[860]: time=2026-09-23T05:13:07.036Z level=INFO msg="All 1 paths already cached"4390client # copying 1 paths to 'lb:50051'...4391client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4392worker1 # [ 54.541430] nix-grpc-daemon[766]: ts=2026-09-23T05:13:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:490904393worker1 # [ 54.555212] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4394worker1 # [ 54.719598] nix-daemon[832]: accepted connection from pid 903, user nix-grpc-daemon (trusted)4395worker1 # [ 54.742634] nix-daemon[832]: reaped child process 906, status = succeeded4396lb # [ 54.708390] niks3-server[967]: 2026/09/23 05:13:07 INFO Received uploads request method=POST path=/api/pending_closures4397worker1 # [ 54.749753] nix-grpc-daemon[860]: time=2026-09-23T05:13:07.255Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4398worker1 # [ 54.751408] nix-grpc-daemon[860]: time=2026-09-23T05:13:07.256Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4399lb # [ 54.739913] niks3-server[967]: 2026/09/23 05:13:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4400worker1 # [ 54.779006] nix-grpc-daemon[860]: time=2026-09-23T05:13:07.285Z level=INFO msg="Uploading 1 narinfos"4401lb # [ 54.753389] niks3-server[967]: 2026/09/23 05:13:07 INFO Signed narinfos id=11 count=14402lb # [ 54.760736] niks3-server[967]: 2026/09/23 05:13:07 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4403lb # [ 54.777060] niks3-server[967]: 2026/09/23 05:13:07 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4404worker1 # [ 54.815494] nix-grpc-daemon[860]: time=2026-09-23T05:13:07.322Z level=INFO msg="Upload complete. (243ms)"4405lb # [ 54.787620] niks3-server[967]: 2026/09/23 05:13:07 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4406lb # [ 54.793170] niks3-server[967]: 2026/09/23 05:13:07 INFO Completed upload id=114407worker1 # [ 54.824740] nix-grpc-daemon[766]: ts=2026-09-23T05:13:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=0 paths=1 nar_bytes_in=6244408lb # [ 54.796550] niks3-server[967]: 2026/09/23 05:13:07 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4409worker1 # [ 54.842223] nix-grpc-daemon[766]: ts=2026-09-23T05:13:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490904410worker1 # [ 54.849242] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4411worker1 # [ 54.854926] nix-daemon[832]: reaped child process 901, status = succeeded4412client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4413worker1 # [ 54.867816] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4414client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4415client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4416client # farm-b> LOG-farm-b-sig4417worker1 # [ 55.960157] nix-daemon[832]: reaped child process 910, status = succeeded4418worker1 # [ 56.059922] nix-daemon[832]: accepted connection from pid 926, user nix-grpc-daemon (trusted)4419worker1 # [ 56.081701] nix-daemon[832]: reaped child process 929, status = succeeded4420lb # [ 56.051922] niks3-server[967]: 2026/09/23 05:13:08 INFO Received uploads request method=POST path=/api/pending_closures4421worker1 # [ 56.095967] nix-grpc-daemon[860]: time=2026-09-23T05:13:08.601Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4422worker1 # [ 56.097875] nix-grpc-daemon[860]: time=2026-09-23T05:13:08.601Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4423lb # [ 56.093362] niks3-server[967]: 2026/09/23 05:13:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4424worker1 # [ 56.136549] nix-grpc-daemon[860]: time=2026-09-23T05:13:08.642Z level=INFO msg="Uploading 1 narinfos"4425lb # [ 56.108326] niks3-server[967]: 2026/09/23 05:13:08 INFO Signed narinfos id=12 count=14426lb # [ 56.114417] niks3-server[967]: 2026/09/23 05:13:08 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4427lb # [ 56.124113] niks3-server[967]: 2026/09/23 05:13:08 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4428lb # [ 56.131961] niks3-server[967]: 2026/09/23 05:13:08 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4429lb # [ 56.142589] niks3-server[967]: 2026/09/23 05:13:08 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4430lb # [ 56.146912] niks3-server[967]: 2026/09/23 05:13:08 INFO Completed upload id=124431worker1 # [ 56.172648] nix-grpc-daemon[860]: time=2026-09-23T05:13:08.679Z level=INFO msg="Upload complete. (287ms)"4432lb # [ 56.151108] niks3-server[967]: 2026/09/23 05:13:08 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4433worker1 # [ 56.185585] nix-grpc-daemon[766]: ts=2026-09-23T05:13:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14434worker2 # [ 56.265996] nix-grpc-daemon[769]: ts=2026-09-23T05:13:08Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04435worker2 # [ 56.266303] nix-grpc-daemon[769]: ts=2026-09-23T05:13:08Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=14436worker1 # [ 56.207687] nix-daemon[832]: reaped child process 908, status = succeeded4437worker1 # [ 56.240637] nix-grpc-daemon[766]: ts=2026-09-23T05:13:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490904438worker1 # [ 56.256657] nix-grpc-daemon[766]: ts=2026-09-23T05:13:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490904439worker1 # [ 56.271664] nix-grpc-daemon[860]: time=2026-09-23T05:13:08.778Z level=INFO msg="All 2 paths already cached"4440client # copying 1 paths to 'lb:50051'...4441client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4442worker1 # [ 56.283590] nix-grpc-daemon[766]: ts=2026-09-23T05:13:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:490904443worker1 # [ 56.298087] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4444worker1 # [ 56.456123] nix-daemon[832]: accepted connection from pid 933, user nix-grpc-daemon (trusted)4445worker1 # [ 56.478161] nix-daemon[832]: reaped child process 936, status = succeeded4446lb # [ 56.449511] niks3-server[967]: 2026/09/23 05:13:08 INFO Received uploads request method=POST path=/api/pending_closures4447worker1 # [ 56.491066] nix-grpc-daemon[860]: time=2026-09-23T05:13:08.997Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4448worker1 # [ 56.493252] nix-grpc-daemon[860]: time=2026-09-23T05:13:08.997Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4449lb # [ 56.481374] niks3-server[967]: 2026/09/23 05:13:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4450worker1 # [ 56.520353] nix-grpc-daemon[860]: time=2026-09-23T05:13:09.026Z level=INFO msg="Uploading 1 narinfos"4451lb # [ 56.495582] niks3-server[967]: 2026/09/23 05:13:08 INFO Signed narinfos id=13 count=14452lb # [ 56.506492] niks3-server[967]: 2026/09/23 05:13:09 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4453worker1 # [ 56.550582] nix-grpc-daemon[860]: time=2026-09-23T05:13:09.057Z level=INFO msg="Upload complete. (235ms)"4454lb # [ 56.517639] niks3-server[967]: 2026/09/23 05:13:09 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4455lb # [ 56.526198] niks3-server[967]: 2026/09/23 05:13:09 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4456worker1 # [ 56.559988] nix-grpc-daemon[766]: ts=2026-09-23T05:13:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=0 paths=1 nar_bytes_in=7524457lb # [ 56.533249] niks3-server[967]: 2026/09/23 05:13:09 INFO Completed upload id=134458lb # [ 56.538134] niks3-server[967]: 2026/09/23 05:13:09 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4459worker1 # [ 56.576841] nix-grpc-daemon[766]: ts=2026-09-23T05:13:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490904460worker1 # [ 56.583106] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4461worker1 # [ 56.588872] nix-daemon[832]: reaped child process 931, status = succeeded4462client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4463client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4464worker1 # [ 56.602950] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4465client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4466client # farm-top> LOG-farm-top-sig4467worker1 # [ 57.470124] nix-daemon[832]: reaped child process 940, status = succeeded4468worker1 # [ 57.605385] nix-daemon[832]: accepted connection from pid 956, user nix-grpc-daemon (trusted)4469worker1 # [ 57.629065] nix-daemon[832]: reaped child process 959, status = succeeded4470lb # [ 57.595343] niks3-server[967]: 2026/09/23 05:13:10 INFO Received uploads request method=POST path=/api/pending_closures4471worker1 # [ 57.639988] nix-grpc-daemon[860]: time=2026-09-23T05:13:10.144Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4472worker1 # [ 57.641753] nix-grpc-daemon[860]: time=2026-09-23T05:13:10.144Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4473lb # [ 57.642862] niks3-server[967]: 2026/09/23 05:13:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4474worker1 # [ 57.683439] nix-grpc-daemon[860]: time=2026-09-23T05:13:10.189Z level=INFO msg="Uploading 1 narinfos"4475lb # [ 57.656621] niks3-server[967]: 2026/09/23 05:13:10 INFO Signed narinfos id=14 count=14476lb # [ 57.663712] niks3-server[967]: 2026/09/23 05:13:10 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4477lb # [ 57.672990] niks3-server[967]: 2026/09/23 05:13:10 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4478lb # [ 57.683137] niks3-server[967]: 2026/09/23 05:13:10 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4479lb # [ 57.690990] niks3-server[967]: 2026/09/23 05:13:10 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4480worker1 # [ 57.722942] nix-grpc-daemon[860]: time=2026-09-23T05:13:10.229Z level=INFO msg="Upload complete. (308ms)"4481lb # [ 57.696237] niks3-server[967]: 2026/09/23 05:13:10 INFO Completed upload id=144482lb # [ 57.699439] niks3-server[967]: 2026/09/23 05:13:10 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4483worker1 # [ 57.737228] nix-grpc-daemon[766]: ts=2026-09-23T05:13:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14484worker1 # [ 57.755854] nix-daemon[832]: reaped child process 938, status = succeeded4485client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.97 seconds)4486client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4487client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds)4488client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4489worker1 # [ 58.107836] nix-grpc-daemon[766]: ts=2026-09-23T05:13:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:539284490worker1 # [ 58.150632] nix-grpc-daemon[766]: ts=2026-09-23T05:13:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:539284491client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.25 seconds)4492client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4493worker2 # [ 58.473239] nix-grpc-daemon[769]: ts=2026-09-23T05:13:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:343604494client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4495client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4496client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4497client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4498worker2 # [ 58.517029] nix-grpc-daemon[769]: ts=2026-09-23T05:13:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:343604499worker2 # [ 58.526335] nix-grpc-daemon[769]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4500worker2 # [ 58.546054] nix-grpc-daemon[769]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4501worker2 # [ 58.561060] nix-grpc-daemon[769]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4502client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.32 seconds)4503client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4504client # finding garbage collector roots...4505client # removing stale temporary roots file "/nix/var/nix/temproots/919"4506client # removing stale temporary roots file "/nix/var/nix/temproots/942"4507client # removing stale temporary roots file "/nix/var/nix/temproots/906"4508client # removing stale temporary roots file "/nix/var/nix/temproots/927"4509client # removing stale temporary roots file "/nix/var/nix/temproots/866"4510client # removing stale temporary roots file "/nix/var/nix/temproots/887"4511client # removing stale temporary roots file "/nix/var/nix/temproots/849"4512client # removing stale temporary roots file "/nix/var/nix/temproots/898"4513client # deleting specified paths...4514client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4515client # deleting unused links...4516client # note: hard linking is currently saving 0.0 KiB4517worker2 # [ 58.973380] nix-grpc-daemon[769]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587264518worker1 # [ 58.908963] nix-grpc-daemon[766]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490904519worker1 # [ 58.921876] nix-grpc-daemon[860]: time=2026-09-23T05:13:11.427Z level=INFO msg="All 1 paths already cached"4520worker1 # [ 58.931152] nix-grpc-daemon[766]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:490904521worker1 # [ 58.940750] nix-grpc-daemon[766]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:490904522worker1 # [ 58.950096] nix-grpc-daemon[766]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:490904523client # copying 3 paths...4524worker1 # [ 58.964998] nix-grpc-daemon[766]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:490904525client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4526worker2 # [ 59.063949] nix-grpc-daemon[769]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:587264527worker2 # [ 59.071776] nix-grpc-daemon[769]: ts=2026-09-23T05:13:11Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=0 paths=1 nar_bytes_out=2324528worker2 # [ 59.081607] nix-grpc-daemon[769]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:587264529worker2 # [ 59.088701] nix-grpc-daemon[769]: ts=2026-09-23T05:13:11Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=0 paths=1 nar_bytes_out=1764530worker1 # [ 59.028262] nix-grpc-daemon[766]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:490904531worker1 # [ 59.035923] nix-grpc-daemon[766]: ts=2026-09-23T05:13:11Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=0 paths=1 nar_bytes_out=1284532client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4533client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4534client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.57 seconds)4535(finished: subtest: outputs read back through any worker carry the cache signature, in 7.30 seconds)4536subtest: two clients wanting the same drv build it once4537client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4538client # [ 59.184971] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4539client # Running as unit: dedup1.service; invocation ID: c839abcf7c834d46ae1352b88ca67d2a4540client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds)4541client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4542lb # [ 59.186708] envoy[895]: conn peer=[2001:db8:1::1]:34888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4185 tx=1920 ms=20304543lb # [ 59.191725] envoy[895]: conn peer=[2001:db8:1::1]:36950 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9528 tx=4368 ms=57164544lb # [ 59.197004] envoy[895]: conn peer=[2001:db8:1::1]:36964 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2928 ms=1864545lb # [ 59.201890] envoy[895]: conn peer=[2001:db8:1::1]:36990 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=644546lb # [ 59.206740] envoy[895]: conn peer=[2001:db8:1::1]:36986 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=375 ms=814547lb # [ 59.212455] envoy[895]: conn peer=[2001:db8:1::1]:36980 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=409 ms=984548worker2 # [ 59.435690] nix-grpc-daemon[769]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587264549client # [ 59.394816] nix[995]: don't know how to build these paths:4550client # [ 59.395972] nix[995]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4551worker2 # [ 59.453557] nix-grpc-daemon[769]: ts=2026-09-23T05:13:11Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04552worker2 # [ 59.461106] nix-grpc-daemon[769]: ts=2026-09-23T05:13:11Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14553worker2 # [ 59.499024] nix-grpc-daemon[769]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587264554worker2 # [ 59.514484] nix-grpc-daemon[769]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587264555client # [ 59.480175] nix[995]: copying 1 paths to 'lb:50051'...4556worker2 # [ 59.530148] nix-grpc-daemon[769]: ts=2026-09-23T05:13:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587264557client # [ 59.482616] nix[995]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4558worker2 # [ 59.548159] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4559worker2 # [ 59.708456] nix-daemon[855]: accepted connection from pid 1050, user nix-grpc-daemon (trusted)4560worker2 # [ 59.730473] nix-daemon[855]: reaped child process 1053, status = succeeded4561lb # [ 59.638133] niks3-server[967]: 2026/09/23 05:13:12 INFO Received uploads request method=POST path=/api/pending_closures4562worker2 # [ 59.749949] nix-grpc-daemon[870]: time=2026-09-23T05:13:12.202Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4563worker2 # [ 59.752000] nix-grpc-daemon[870]: time=2026-09-23T05:13:12.202Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4564lb # [ 59.670924] niks3-server[967]: 2026/09/23 05:13:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4565worker2 # [ 59.780473] nix-grpc-daemon[870]: time=2026-09-23T05:13:12.233Z level=INFO msg="Uploading 1 narinfos"4566lb # [ 59.683579] niks3-server[967]: 2026/09/23 05:13:12 INFO Signed narinfos id=15 count=14567lb # [ 59.693817] niks3-server[967]: 2026/09/23 05:13:12 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4568lb # [ 59.704569] niks3-server[967]: 2026/09/23 05:13:12 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4569lb # [ 59.722434] niks3-server[967]: 2026/09/23 05:13:12 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4570lb # [ 59.731623] niks3-server[967]: 2026/09/23 05:13:12 INFO Completed upload id=154571worker2 # [ 59.831573] nix-grpc-daemon[870]: time=2026-09-23T05:13:12.284Z level=INFO msg="Upload complete. (266ms)"4572lb # [ 59.735031] niks3-server[967]: 2026/09/23 05:13:12 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4573worker2 # [ 59.843586] nix-grpc-daemon[769]: ts=2026-09-23T05:13:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=0 paths=1 nar_bytes_in=4324574worker2 # [ 59.865238] nix-grpc-daemon[769]: ts=2026-09-23T05:13:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587264575worker2 # [ 59.873841] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4576worker2 # [ 59.880226] nix-daemon[855]: reaped child process 1048, status = succeeded4577client # [ 59.844066] nix[995]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4578worker2 # [ 59.897968] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4579client # [ 60.283740] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4580client # Running as unit: dedup2.service; invocation ID: 3b7b4e4a75144222baa020fc3930a6594581client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.12 seconds)4582client # [ 60.535263] nix[1011]: don't know how to build these paths:4583client # [ 60.536535] nix[1011]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4584worker1 # [ 60.513212] nix-grpc-daemon[766]: ts=2026-09-23T05:13:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:490904585worker2 # [ 60.591535] nix-grpc-daemon[769]: ts=2026-09-23T05:13:13Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04586worker2 # [ 60.596715] nix-grpc-daemon[769]: ts=2026-09-23T05:13:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587264587client # [ 60.550043] nix[1011]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4588client: must succeed: sleep 24589client: (finished: must succeed: sleep 2, in 2.06 seconds)4590client: waiting for success: ! systemctl is-active dedup1 dedup24591worker2 # [ 63.927416] nix-daemon[855]: reaped child process 1057, status = succeeded4592worker2 # [ 64.025289] nix-daemon[855]: accepted connection from pid 1089, user nix-grpc-daemon (trusted)4593worker2 # [ 64.047155] nix-daemon[855]: reaped child process 1092, status = succeeded4594lb # [ 63.951167] niks3-server[967]: 2026/09/23 05:13:16 INFO Received uploads request method=POST path=/api/pending_closures4595worker2 # [ 64.064425] nix-grpc-daemon[870]: time=2026-09-23T05:13:16.516Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4596worker2 # [ 64.065944] nix-grpc-daemon[870]: time=2026-09-23T05:13:16.516Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4597lb # [ 63.998835] niks3-server[967]: 2026/09/23 05:13:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4598worker2 # [ 64.109734] nix-grpc-daemon[870]: time=2026-09-23T05:13:16.562Z level=INFO msg="Uploading 1 narinfos"4599lb # [ 64.012594] niks3-server[967]: 2026/09/23 05:13:16 INFO Signed narinfos id=16 count=14600lb # [ 64.015115] niks3-server[967]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4601lb # [ 64.034205] niks3-server[967]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4602lb # [ 64.037564] niks3-server[967]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4603worker2 # [ 64.142957] nix-grpc-daemon[870]: time=2026-09-23T05:13:16.596Z level=INFO msg="Upload complete. (230ms)"4604lb # [ 64.041848] niks3-server[967]: 2026/09/23 05:13:16 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4605lb # [ 64.046334] niks3-server[967]: 2026/09/23 05:13:16 INFO Completed upload id=164606lb # [ 64.051296] niks3-server[967]: 2026/09/23 05:13:16 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4607worker2 # [ 64.154099] nix-grpc-daemon[769]: ts=2026-09-23T05:13:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14608worker2 # [ 64.174574] nix-grpc-daemon[769]: ts=2026-09-23T05:13:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14609worker2 # [ 64.180607] nix-daemon[855]: reaped child process 1055, status = succeeded4610client # [ 64.152763] systemd[1]: dedup1.service: Deactivated successfully.4611client # [ 64.158413] systemd[1]: dedup1.service: Consumed 182ms CPU time over 4.966s wall clock time, 41.5M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.4612client # [ 64.201370] systemd[1]: dedup2.service: Deactivated successfully.4613client # [ 64.205256] systemd[1]: dedup2.service: Consumed 218ms CPU time over 3.921s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.5K outgoing IP traffic.4614client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds)4615client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4616client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4617worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4618worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4619worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4620worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4621worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4622worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4623worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4624worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4625(finished: subtest: two clients wanting the same drv build it once, in 6.16 seconds)4626subtest: --timeout and --max-silent-time reach the builder4627worker1 # [ 65.510687] nix-grpc-daemon[766]: ts=2026-09-23T05:13:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:490904628worker2 # [ 65.600875] nix-grpc-daemon[769]: ts=2026-09-23T05:13:18Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04629worker1 # [ 65.535421] nix-grpc-daemon[766]: ts=2026-09-23T05:13:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490904630worker1 # [ 65.555243] nix-grpc-daemon[766]: ts=2026-09-23T05:13:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490904631worker1 # [ 65.573042] nix-grpc-daemon[766]: ts=2026-09-23T05:13:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:490904632worker1 # [ 65.589695] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4633worker1 # [ 65.749980] nix-daemon[832]: accepted connection from pid 984, user nix-grpc-daemon (trusted)4634worker1 # [ 65.773254] nix-daemon[832]: reaped child process 987, status = succeeded4635lb # [ 65.739548] niks3-server[967]: 2026/09/23 05:13:18 INFO Received uploads request method=POST path=/api/pending_closures4636worker1 # [ 65.816277] nix-grpc-daemon[860]: time=2026-09-23T05:13:18.321Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4637worker1 # [ 65.819517] nix-grpc-daemon[860]: time=2026-09-23T05:13:18.321Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4638lb # [ 65.808471] niks3-server[967]: 2026/09/23 05:13:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4639worker1 # [ 65.849914] nix-grpc-daemon[860]: time=2026-09-23T05:13:18.356Z level=INFO msg="Uploading 1 narinfos"4640lb # [ 65.823580] niks3-server[967]: 2026/09/23 05:13:18 INFO Signed narinfos id=17 count=14641lb # [ 65.831126] niks3-server[967]: 2026/09/23 05:13:18 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4642lb # [ 65.843599] niks3-server[967]: 2026/09/23 05:13:18 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4643worker1 # [ 65.890570] nix-grpc-daemon[860]: time=2026-09-23T05:13:18.397Z level=INFO msg="Upload complete. (284ms)"4644lb # [ 65.856916] niks3-server[967]: 2026/09/23 05:13:18 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4645lb # [ 65.866926] niks3-server[967]: 2026/09/23 05:13:18 INFO Completed upload id=174646worker1 # [ 65.900427] nix-grpc-daemon[766]: ts=2026-09-23T05:13:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=0 paths=1 nar_bytes_in=4244647lb # [ 65.873350] niks3-server[967]: 2026/09/23 05:13:18 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4648worker1 # [ 65.921058] nix-grpc-daemon[766]: ts=2026-09-23T05:13:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490904649worker1 # [ 65.932429] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4650worker1 # [ 65.938954] nix-daemon[832]: reaped child process 982, status = succeeded4651worker1 # [ 65.952785] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4652worker2 # [ 65.606666] nix-grpc-daemon[769]: ts=2026-09-23T05:13:18Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=14653lb # [ 69.186403] envoy[895]: conn peer=[2001:db8:1::1]:36996 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1564 ms=47434654lb # [ 69.191471] envoy[895]: conn peer=[2001:db8:1::1]:36998 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1194 ms=36504655worker1 # [ 71.774462] nix-grpc-daemon[766]: ts=2026-09-23T05:13:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=5 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04656worker1 # [ 71.804816] nix-daemon[832]: reaped child process 989, status = succeeded4657worker1 # [ 71.806980] nix-daemon[832]: reaped child process 991, status = succeeded4658worker2 # [ 72.086508] nix-grpc-daemon[769]: ts=2026-09-23T05:13:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587264659worker2 # [ 72.103955] nix-grpc-daemon[769]: ts=2026-09-23T05:13:24Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04660worker2 # [ 72.112771] nix-grpc-daemon[769]: ts=2026-09-23T05:13:24Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=14661worker2 # [ 72.121229] nix-grpc-daemon[769]: ts=2026-09-23T05:13:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587264662worker2 # [ 72.133261] nix-grpc-daemon[769]: ts=2026-09-23T05:13:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587264663worker2 # [ 72.148476] nix-grpc-daemon[769]: ts=2026-09-23T05:13:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587264664worker2 # [ 72.165073] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4665worker2 # [ 72.326223] nix-daemon[855]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)4666worker2 # [ 72.351254] nix-daemon[855]: reaped child process 1112, status = succeeded4667lb # [ 72.250110] niks3-server[967]: 2026/09/23 05:13:24 INFO Received uploads request method=POST path=/api/pending_closures4668worker2 # [ 72.363976] nix-grpc-daemon[870]: time=2026-09-23T05:13:24.816Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4669worker2 # [ 72.365657] nix-grpc-daemon[870]: time=2026-09-23T05:13:24.816Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4670lb # [ 72.293491] niks3-server[967]: 2026/09/23 05:13:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4671worker2 # [ 72.403099] nix-grpc-daemon[870]: time=2026-09-23T05:13:24.856Z level=INFO msg="Uploading 1 narinfos"4672lb # [ 72.306920] niks3-server[967]: 2026/09/23 05:13:24 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4673lb # [ 72.316249] niks3-server[967]: 2026/09/23 05:13:24 INFO Signed narinfos id=18 count=14674lb # [ 72.324355] niks3-server[967]: 2026/09/23 05:13:24 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4675worker2 # [ 72.432780] nix-grpc-daemon[870]: time=2026-09-23T05:13:24.885Z level=INFO msg="Upload complete. (251ms)"4676lb # [ 72.332497] niks3-server[967]: 2026/09/23 05:13:24 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4677lb # [ 72.337245] niks3-server[967]: 2026/09/23 05:13:24 INFO Completed upload id=184678lb # [ 72.340527] niks3-server[967]: 2026/09/23 05:13:24 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4679worker2 # [ 72.443227] nix-grpc-daemon[769]: ts=2026-09-23T05:13:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=0 paths=1 nar_bytes_in=4244680worker2 # [ 72.460382] nix-grpc-daemon[769]: ts=2026-09-23T05:13:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587264681worker2 # [ 72.467417] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4682worker2 # [ 72.474071] nix-daemon[855]: reaped child process 1107, status = succeeded4683worker2 # [ 72.487563] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4684worker2 # [ 78.312541] nix-grpc-daemon[769]: ts=2026-09-23T05:13:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=5 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04685worker2 # [ 78.351703] nix-daemon[855]: reaped child process 1114, status = succeeded4686worker2 # [ 78.356740] nix-daemon[855]: reaped child process 1116, status = succeeded4687(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.20 seconds)4688subtest: the first client leaving does not fail the second4689client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4690client # [ 78.567078] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4691client # Running as unit: share1.service; invocation ID: 48840f2fa3af4164b2cbec7ccc1252784692client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4693client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4694worker2 # [ 78.827693] nix-grpc-daemon[769]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587264695client # [ 78.785252] nix[1084]: don't know how to build these paths:4696client # [ 78.786212] nix[1084]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4697worker2 # [ 78.848707] nix-grpc-daemon[769]: ts=2026-09-23T05:13:31Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04698worker1 # [ 78.783563] nix-grpc-daemon[766]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490904699worker2 # [ 78.855820] nix-grpc-daemon[769]: ts=2026-09-23T05:13:31Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14700worker1 # [ 78.799990] nix-grpc-daemon[766]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490904701client # [ 78.831939] nix[1084]: copying 1 paths to 'lb:50051'...4702client # [ 78.834705] nix[1084]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4703worker1 # [ 78.816193] nix-grpc-daemon[766]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:490904704worker1 # [ 78.831641] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4705worker1 # [ 78.990888] nix-daemon[832]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)4706worker1 # [ 79.013822] nix-daemon[832]: reaped child process 1017, status = succeeded4707lb # [ 78.981087] niks3-server[967]: 2026/09/23 05:13:31 INFO Received uploads request method=POST path=/api/pending_closures4708worker1 # [ 79.023527] nix-grpc-daemon[860]: time=2026-09-23T05:13:31.529Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4709worker1 # [ 79.025780] nix-grpc-daemon[860]: time=2026-09-23T05:13:31.529Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4710lb # [ 79.014678] niks3-server[967]: 2026/09/23 05:13:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4711worker1 # [ 79.056055] nix-grpc-daemon[860]: time=2026-09-23T05:13:31.562Z level=INFO msg="Uploading 1 narinfos"4712lb # [ 79.026818] niks3-server[967]: 2026/09/23 05:13:31 INFO Signed narinfos id=19 count=14713lb # [ 79.036777] niks3-server[967]: 2026/09/23 05:13:31 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4714lb # [ 79.046606] niks3-server[967]: 2026/09/23 05:13:31 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4715lb # [ 79.056387] niks3-server[967]: 2026/09/23 05:13:31 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4716lb # [ 79.061825] niks3-server[967]: 2026/09/23 05:13:31 INFO Completed upload id=194717worker1 # [ 79.092501] nix-grpc-daemon[860]: time=2026-09-23T05:13:31.599Z level=INFO msg="Upload complete. (243ms)"4718lb # [ 79.066898] niks3-server[967]: 2026/09/23 05:13:31 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4719worker1 # [ 79.104004] nix-grpc-daemon[766]: ts=2026-09-23T05:13:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=0 paths=1 nar_bytes_in=4324720worker1 # [ 79.116641] nix-grpc-daemon[766]: ts=2026-09-23T05:13:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490904721worker1 # [ 79.130131] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4722worker1 # [ 79.138423] nix-daemon[832]: reaped child process 1012, status = succeeded4723worker1 # [ 79.147743] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4724client # [ 79.166844] nix[1084]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4725lb # [ 79.188790] envoy[895]: conn peer=[2001:db8:1::1]:44346 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2975 tx=977 ms=62894726lb # [ 79.194180] envoy[895]: conn peer=[2001:db8:1::1]:48532 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2972 tx=936 ms=62904727client # [ 79.671908] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4728client # Running as unit: share2.service; invocation ID: 14f6376761fd4fb795a4dca747ca819b4729client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.12 seconds)4730worker2 # [ 79.914576] nix-grpc-daemon[769]: ts=2026-09-23T05:13:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:587264731client # [ 79.871590] nix[1100]: don't know how to build these paths:4732client # [ 79.873667] nix[1100]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4733worker2 # [ 79.933424] nix-grpc-daemon[769]: ts=2026-09-23T05:13:32Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04734worker1 # [ 79.868654] nix-grpc-daemon[766]: ts=2026-09-23T05:13:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490904735client # [ 79.895621] nix[1100]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4736client: must succeed: sleep 2; systemctl kill -s INT share14737client # [ 83.051567] systemd[1]: share1.service: Sent signal SIGINT to main process 1084 (nix) on client request.4738client # [ 83.061356] nix[1084]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4739client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4740client: waiting for success: ! systemctl is-active share14741client # [ 83.070907] nix[1084]: [mutex.cc : 451] RAW: Unlock 0xf489edbfc788 @ 0xf48df22359a4 0xf48df330d40c 0xf48e127757bc 0xf48e12777414 0xf48e1276e1d4 0xf48e117b4f7c 0xf48e11542d804742client # [ 83.100992] systemd-coredump[1115]: Process 1084 (nix) of user 0 terminated abnormally with signal 11/SEGV, processing...4743client # [ 83.128175] systemd[1]: Created slice Slice /system/systemd-coredump.4744client # [ 83.141496] systemd[1]: Started Process Core Dump (PID 1115/UID 0).4745client # [ 85.602993] systemd-coredump[1120]: Cannot store coredump of 1084 (nix): No space left on device4746client # [ 85.743890] systemd-coredump[1120]: Process 1084 (nix) of user 0 terminated abnormally without generating a coredump.4747client # [ 85.762416] systemd[1]: systemd-coredump@0-1-1115_1116-0.service: Deactivated successfully.4748client # [ 85.766289] systemd[1]: systemd-coredump@0-1-1115_1116-0.service: Consumed 2.081s CPU time over 2.615s wall clock time, 968.8M memory peak, 926.3M written to disk.4749client # [ 85.775216] systemd[1]: share1.service: Main process exited, code=dumped, status=11/SEGV4750client # [ 85.778064] systemd[1]: share1.service: Failed with result 'core-dump'.4751client # [ 85.779311] systemd[1]: share1.service: Consumed 374ms CPU time over 7.208s wall clock time, 41.5M memory peak, 4K written to disk, 6.3K incoming IP traffic, 8.1K outgoing IP traffic.4752client: (finished: waiting for success: ! systemctl is-active share1, in 3.31 seconds)4753client: waiting for success: ! systemctl is-active share24754worker1 # [ 86.845069] nix-daemon[832]: reaped child process 1021, status = succeeded4755worker1 # [ 86.964211] nix-daemon[832]: accepted connection from pid 1055, user nix-grpc-daemon (trusted)4756worker1 # [ 86.989161] nix-daemon[832]: reaped child process 1058, status = succeeded4757lb # [ 86.963764] niks3-server[967]: 2026/09/23 05:13:39 INFO Received uploads request method=POST path=/api/pending_closures4758worker1 # [ 87.008483] nix-grpc-daemon[860]: time=2026-09-23T05:13:39.511Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4759worker1 # [ 87.010214] nix-grpc-daemon[860]: time=2026-09-23T05:13:39.511Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4760lb # [ 87.017263] niks3-server[967]: 2026/09/23 05:13:39 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4761lb # [ 87.028718] niks3-server[967]: 2026/09/23 05:13:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4762worker1 # [ 87.068142] nix-grpc-daemon[860]: time=2026-09-23T05:13:39.573Z level=INFO msg="Uploading 1 narinfos"4763lb # [ 87.039353] niks3-server[967]: 2026/09/23 05:13:39 INFO Signed narinfos id=20 count=14764lb # [ 87.048393] niks3-server[967]: 2026/09/23 05:13:39 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4765lb # [ 87.056772] niks3-server[967]: 2026/09/23 05:13:39 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4766worker1 # [ 87.100750] nix-grpc-daemon[860]: time=2026-09-23T05:13:39.607Z level=INFO msg="Upload complete. (326ms)"4767lb # [ 87.068711] niks3-server[967]: 2026/09/23 05:13:39 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4768lb # [ 87.073282] niks3-server[967]: 2026/09/23 05:13:39 INFO Completed upload id=204769lb # [ 87.075190] niks3-server[967]: 2026/09/23 05:13:39 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4770worker1 # [ 87.113406] nix-grpc-daemon[766]: ts=2026-09-23T05:13:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=7 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14771worker1 # [ 87.136337] nix-grpc-daemon[766]: ts=2026-09-23T05:13:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=8 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14772worker1 # [ 87.143437] nix-daemon[832]: reaped child process 1019, status = succeeded4773client # [ 87.220809] systemd[1]: share2.service: Deactivated successfully.4774client # [ 87.225809] systemd[1]: share2.service: Consumed 184ms CPU time over 7.553s wall clock time, 35.3M memory peak, 4K written to disk, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.4775client: (finished: waiting for success: ! systemctl is-active share2, in 1.17 seconds)4776client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4777client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.06 seconds)4778(finished: subtest: the first client leaving does not fail the second, in 9.39 seconds)4779subtest: build hook: nix-daemon with builders = grpc://lb4780client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4781client # [ 88.057706] systemd[1]: Started Nix Daemon.4782client # [ 88.212173] nix-daemon[1165]: accepted connection from pid 1162, user root (trusted)4783worker1 # [ 88.391587] nix-grpc-daemon[766]: ts=2026-09-23T05:13:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:490904784worker1 # [ 88.399200] nix-grpc-daemon[766]: ts=2026-09-23T05:13:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490904785worker2 # [ 88.489726] nix-grpc-daemon[769]: ts=2026-09-23T05:13:40Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04786worker2 # [ 88.490327] nix-grpc-daemon[769]: ts=2026-09-23T05:13:40Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14787worker2 # [ 88.503920] nix-grpc-daemon[769]: ts=2026-09-23T05:13:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587264788worker2 # [ 88.525418] nix-grpc-daemon[769]: ts=2026-09-23T05:13:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587264789worker2 # [ 88.546311] nix-grpc-daemon[769]: ts=2026-09-23T05:13:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587264790worker2 # [ 88.566487] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4791worker2 # [ 88.757645] nix-daemon[855]: accepted connection from pid 1161, user nix-grpc-daemon (trusted)4792worker2 # [ 88.784699] nix-daemon[855]: reaped child process 1164, status = succeeded4793lb # [ 88.682252] niks3-server[967]: 2026/09/23 05:13:41 INFO Received uploads request method=POST path=/api/pending_closures4794worker2 # [ 88.797354] nix-grpc-daemon[870]: time=2026-09-23T05:13:41.247Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4795worker2 # [ 88.799041] nix-grpc-daemon[870]: time=2026-09-23T05:13:41.247Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4796lb # [ 88.725664] niks3-server[967]: 2026/09/23 05:13:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4797worker2 # [ 88.840389] nix-grpc-daemon[870]: time=2026-09-23T05:13:41.292Z level=INFO msg="Uploading 1 narinfos"4798lb # [ 88.744664] niks3-server[967]: 2026/09/23 05:13:41 INFO Signed narinfos id=21 count=14799lb # [ 88.750641] niks3-server[967]: 2026/09/23 05:13:41 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4800lb # [ 88.769081] niks3-server[967]: 2026/09/23 05:13:41 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4801worker2 # [ 88.880737] nix-grpc-daemon[870]: time=2026-09-23T05:13:41.333Z level=INFO msg="Upload complete. (292ms)"4802lb # [ 88.781969] niks3-server[967]: 2026/09/23 05:13:41 INFO Completed upload id=214803lb # [ 88.785284] niks3-server[967]: 2026/09/23 05:13:41 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4804worker2 # [ 88.892436] nix-grpc-daemon[769]: ts=2026-09-23T05:13:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=0 paths=1 nar_bytes_in=5124805lb # [ 88.792982] niks3-server[967]: 2026/09/23 05:13:41 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4806worker2 # [ 88.904492] nix-grpc-daemon[769]: ts=2026-09-23T05:13:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587264807worker2 # [ 88.919995] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4808worker2 # [ 88.932397] nix-daemon[855]: reaped child process 1159, status = succeeded4809worker2 # [ 88.939979] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4810lb # [ 89.186809] envoy[895]: conn peer=[2001:db8:1::1]:48538 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2977 tx=812 ms=69874811lb # [ 89.192111] envoy[895]: conn peer=[2001:db8:1::1]:48544 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1197 ms=73284812worker2 # [ 90.643859] nix-daemon[855]: reaped child process 1168, status = succeeded4813worker2 # [ 90.736866] nix-daemon[855]: accepted connection from pid 1184, user nix-grpc-daemon (trusted)4814worker2 # [ 90.763248] nix-daemon[855]: reaped child process 1187, status = succeeded4815lb # [ 90.666302] niks3-server[967]: 2026/09/23 05:13:43 INFO Received uploads request method=POST path=/api/pending_closures4816worker2 # [ 90.784000] nix-grpc-daemon[870]: time=2026-09-23T05:13:43.233Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4817worker2 # [ 90.785897] nix-grpc-daemon[870]: time=2026-09-23T05:13:43.233Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4818lb # [ 90.725941] niks3-server[967]: 2026/09/23 05:13:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4819worker2 # [ 90.839310] nix-grpc-daemon[870]: time=2026-09-23T05:13:43.291Z level=INFO msg="Uploading 1 narinfos"4820lb # [ 90.739076] niks3-server[967]: 2026/09/23 05:13:43 INFO Signed narinfos id=22 count=14821lb # [ 90.745183] niks3-server[967]: 2026/09/23 05:13:43 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4822lb # [ 90.753544] niks3-server[967]: 2026/09/23 05:13:43 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4823lb # [ 90.759365] niks3-server[967]: 2026/09/23 05:13:43 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4824lb # [ 90.769315] niks3-server[967]: 2026/09/23 05:13:43 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4825lb # [ 90.771819] niks3-server[967]: 2026/09/23 05:13:43 INFO Completed upload id=224826worker2 # [ 90.871435] nix-grpc-daemon[870]: time=2026-09-23T05:13:43.324Z level=INFO msg="Upload complete. (315ms)"4827lb # [ 90.776493] niks3-server[967]: 2026/09/23 05:13:43 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4828worker2 # [ 90.892526] nix-grpc-daemon[769]: ts=2026-09-23T05:13:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14829worker2 # [ 90.912653] nix-grpc-daemon[769]: ts=2026-09-23T05:13:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:587264830worker2 # [ 90.918703] nix-daemon[855]: reaped child process 1166, status = succeeded4831worker1 # [ 90.883648] nix-grpc-daemon[766]: ts=2026-09-23T05:13:43Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:490904832worker1 # [ 90.900184] nix-grpc-daemon[766]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4833worker1 # [ 90.923366] nix-grpc-daemon[766]: ts=2026-09-23T05:13:43Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=0 paths=1 nar_bytes_out=1284834worker2 # [ 91.291207] nix-grpc-daemon[769]: ts=2026-09-23T05:13:43Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:587264835worker1 # [ 91.226087] nix-grpc-daemon[766]: ts=2026-09-23T05:13:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490904836worker1 # [ 91.239026] nix-grpc-daemon[860]: time=2026-09-23T05:13:43.744Z level=INFO msg="All 1 paths already cached"4837worker2 # [ 91.327699] nix-grpc-daemon[769]: ts=2026-09-23T05:13:43Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04838worker2 # [ 91.334995] nix-grpc-daemon[769]: ts=2026-09-23T05:13:43Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14839worker2 # [ 91.375353] nix-grpc-daemon[769]: ts=2026-09-23T05:13:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587264840worker2 # [ 91.392495] nix-grpc-daemon[769]: ts=2026-09-23T05:13:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587264841worker2 # [ 91.408235] nix-grpc-daemon[870]: time=2026-09-23T05:13:43.861Z level=INFO msg="All 1 paths already cached"4842worker2 # [ 91.418943] nix-grpc-daemon[769]: ts=2026-09-23T05:13:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587264843worker2 # [ 91.433570] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4844worker2 # [ 91.601024] nix-daemon[855]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)4845worker2 # [ 91.624345] nix-daemon[855]: reaped child process 1196, status = succeeded4846lb # [ 91.528686] niks3-server[967]: 2026/09/23 05:13:44 INFO Received uploads request method=POST path=/api/pending_closures4847worker2 # [ 91.640094] nix-grpc-daemon[870]: time=2026-09-23T05:13:44.093Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4848worker2 # [ 91.642128] nix-grpc-daemon[870]: time=2026-09-23T05:13:44.093Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4849lb # [ 91.571783] niks3-server[967]: 2026/09/23 05:13:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4850worker2 # [ 91.681555] nix-grpc-daemon[870]: time=2026-09-23T05:13:44.134Z level=INFO msg="Uploading 1 narinfos"4851lb # [ 91.586485] niks3-server[967]: 2026/09/23 05:13:44 INFO Signed narinfos id=23 count=14852lb # [ 91.594356] niks3-server[967]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4853lb # [ 91.608146] niks3-server[967]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4854worker2 # [ 91.718449] nix-grpc-daemon[870]: time=2026-09-23T05:13:44.171Z level=INFO msg="Upload complete. (266ms)"4855lb # [ 91.617127] niks3-server[967]: 2026/09/23 05:13:44 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4856lb # [ 91.626323] niks3-server[967]: 2026/09/23 05:13:44 INFO Completed upload id=234857lb # [ 91.629952] niks3-server[967]: 2026/09/23 05:13:44 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4858worker2 # [ 91.728458] nix-grpc-daemon[769]: ts=2026-09-23T05:13:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=0 paths=1 nar_bytes_in=6244859worker2 # [ 91.749000] nix-grpc-daemon[769]: ts=2026-09-23T05:13:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587264860worker2 # [ 91.761006] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4861worker2 # [ 91.765237] nix-daemon[855]: reaped child process 1191, status = succeeded4862worker2 # [ 91.780170] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4863worker2 # [ 92.657409] nix-daemon[855]: reaped child process 1200, status = succeeded4864worker2 # [ 92.722143] nix-daemon[855]: accepted connection from pid 1216, user nix-grpc-daemon (trusted)4865worker2 # [ 92.745434] nix-daemon[855]: reaped child process 1219, status = succeeded4866lb # [ 92.650405] niks3-server[967]: 2026/09/23 05:13:45 INFO Received uploads request method=POST path=/api/pending_closures4867worker2 # [ 92.764450] nix-grpc-daemon[870]: time=2026-09-23T05:13:45.215Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4868worker2 # [ 92.765958] nix-grpc-daemon[870]: time=2026-09-23T05:13:45.216Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4869lb # [ 92.702791] niks3-server[967]: 2026/09/23 05:13:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4870worker2 # [ 92.812947] nix-grpc-daemon[870]: time=2026-09-23T05:13:45.265Z level=INFO msg="Uploading 1 narinfos"4871lb # [ 92.715961] niks3-server[967]: 2026/09/23 05:13:45 INFO Signed narinfos id=24 count=14872lb # [ 92.721355] niks3-server[967]: 2026/09/23 05:13:45 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4873lb # [ 92.730690] niks3-server[967]: 2026/09/23 05:13:45 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4874lb # [ 92.740242] niks3-server[967]: 2026/09/23 05:13:45 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4875worker2 # [ 92.850146] nix-grpc-daemon[870]: time=2026-09-23T05:13:45.303Z level=INFO msg="Upload complete. (259ms)"4876lb # [ 92.748771] niks3-server[967]: 2026/09/23 05:13:45 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4877lb # [ 92.753409] niks3-server[967]: 2026/09/23 05:13:45 INFO Completed upload id=244878lb # [ 92.758088] niks3-server[967]: 2026/09/23 05:13:45 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4879worker2 # [ 92.866272] nix-grpc-daemon[769]: ts=2026-09-23T05:13:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14880worker1 # [ 92.802804] nix-grpc-daemon[766]: ts=2026-09-23T05:13:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:490904881worker2 # [ 92.882805] nix-daemon[855]: reaped child process 1198, status = succeeded4882worker1 # [ 92.816781] nix-grpc-daemon[766]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4883worker1 # [ 92.858610] nix-grpc-daemon[766]: ts=2026-09-23T05:13:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:490904884worker1 # [ 92.865886] nix-grpc-daemon[766]: ts=2026-09-23T05:13:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=0 paths=1 nar_bytes_out=1764885worker2 # [ 93.220093] nix-grpc-daemon[769]: ts=2026-09-23T05:13:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:587264886worker1 # [ 93.155625] nix-grpc-daemon[766]: ts=2026-09-23T05:13:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490904887worker1 # [ 93.167105] nix-grpc-daemon[860]: time=2026-09-23T05:13:45.672Z level=INFO msg="All 2 paths already cached"4888worker2 # [ 93.255853] nix-grpc-daemon[769]: ts=2026-09-23T05:13:45Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04889worker2 # [ 93.264348] nix-grpc-daemon[769]: ts=2026-09-23T05:13:45Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14890worker2 # [ 93.271853] nix-grpc-daemon[769]: ts=2026-09-23T05:13:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587264891worker2 # [ 93.284097] nix-grpc-daemon[769]: ts=2026-09-23T05:13:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587264892worker2 # [ 93.298656] nix-grpc-daemon[870]: time=2026-09-23T05:13:45.751Z level=INFO msg="All 2 paths already cached"4893worker2 # [ 93.309959] nix-grpc-daemon[769]: ts=2026-09-23T05:13:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587264894worker2 # [ 93.324457] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4895worker2 # [ 93.488886] nix-daemon[855]: accepted connection from pid 1225, user nix-grpc-daemon (trusted)4896worker2 # [ 93.515715] nix-daemon[855]: reaped child process 1228, status = succeeded4897lb # [ 93.411206] niks3-server[967]: 2026/09/23 05:13:45 INFO Received uploads request method=POST path=/api/pending_closures4898worker2 # [ 93.523364] nix-grpc-daemon[870]: time=2026-09-23T05:13:45.976Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4899worker2 # [ 93.525381] nix-grpc-daemon[870]: time=2026-09-23T05:13:45.976Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4900lb # [ 93.444314] niks3-server[967]: 2026/09/23 05:13:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4901worker2 # [ 93.551988] nix-grpc-daemon[870]: time=2026-09-23T05:13:46.004Z level=INFO msg="Uploading 1 narinfos"4902lb # [ 93.458853] niks3-server[967]: 2026/09/23 05:13:45 INFO Signed narinfos id=25 count=14903lb # [ 93.466904] niks3-server[967]: 2026/09/23 05:13:45 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4904lb # [ 93.478764] niks3-server[967]: 2026/09/23 05:13:45 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4905worker2 # [ 93.588650] nix-grpc-daemon[870]: time=2026-09-23T05:13:46.041Z level=INFO msg="Upload complete. (246ms)"4906lb # [ 93.488723] niks3-server[967]: 2026/09/23 05:13:45 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4907lb # [ 93.495025] niks3-server[967]: 2026/09/23 05:13:45 INFO Completed upload id=254908worker2 # [ 93.596412] nix-grpc-daemon[769]: ts=2026-09-23T05:13:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=0 paths=1 nar_bytes_in=7524909lb # [ 93.498358] niks3-server[967]: 2026/09/23 05:13:45 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4910worker2 # [ 93.608614] nix-grpc-daemon[769]: ts=2026-09-23T05:13:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587264911worker2 # [ 93.618865] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4912worker2 # [ 93.626435] nix-daemon[855]: reaped child process 1223, status = succeeded4913worker2 # [ 93.639337] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4914worker2 # [ 94.881843] nix-daemon[855]: reaped child process 1232, status = succeeded4915worker2 # [ 95.000144] nix-daemon[855]: accepted connection from pid 1248, user nix-grpc-daemon (trusted)4916worker2 # [ 95.024219] nix-daemon[855]: reaped child process 1251, status = succeeded4917lb # [ 94.921695] niks3-server[967]: 2026/09/23 05:13:47 INFO Received uploads request method=POST path=/api/pending_closures4918worker2 # [ 95.034931] nix-grpc-daemon[870]: time=2026-09-23T05:13:47.486Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4919worker2 # [ 95.036589] nix-grpc-daemon[870]: time=2026-09-23T05:13:47.486Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4920lb # [ 94.963742] niks3-server[967]: 2026/09/23 05:13:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4921worker2 # [ 95.073430] nix-grpc-daemon[870]: time=2026-09-23T05:13:47.526Z level=INFO msg="Uploading 1 narinfos"4922lb # [ 94.978040] niks3-server[967]: 2026/09/23 05:13:47 INFO Signed narinfos id=26 count=14923lb # [ 94.984835] niks3-server[967]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4924lb # [ 94.994082] niks3-server[967]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4925lb # [ 95.003923] niks3-server[967]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4926worker2 # [ 95.111094] nix-grpc-daemon[870]: time=2026-09-23T05:13:47.564Z level=INFO msg="Upload complete. (257ms)"4927lb # [ 95.012170] niks3-server[967]: 2026/09/23 05:13:47 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4928lb # [ 95.015569] niks3-server[967]: 2026/09/23 05:13:47 INFO Completed upload id=264929lb # [ 95.019717] niks3-server[967]: 2026/09/23 05:13:47 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4930worker2 # [ 95.123289] nix-grpc-daemon[769]: ts=2026-09-23T05:13:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=14931worker1 # [ 95.062500] nix-grpc-daemon[766]: ts=2026-09-23T05:13:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:490904932worker2 # [ 95.143688] nix-daemon[855]: reaped child process 1230, status = succeeded4933worker1 # [ 95.075770] nix-grpc-daemon[766]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4934worker1 # [ 95.117266] nix-grpc-daemon[766]: ts=2026-09-23T05:13:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:490904935worker1 # [ 95.124293] nix-grpc-daemon[766]: ts=2026-09-23T05:13:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=0 paths=1 nar_bytes_out=2324936client # [ 95.279222] nix-daemon[1165]: reaped child process 1167, status = succeeded4937client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.41 seconds)4938client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4939client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4940client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4941client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4942client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4943client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4944(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.53 seconds)4945subtest: an input only one worker has still reaches the builder4946client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4947client # this derivation will be built:4948client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4949client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4950client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.33 seconds)4951client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4952client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.13 seconds)4953worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4954worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.20 seconds)4955worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4956worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)4957client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24958client # [ 96.311575] nix-daemon[1165]: accepted connection from pid 1248, user root (trusted)4959client # this derivation will be built:4960client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4961worker2 # [ 96.523557] nix-grpc-daemon[769]: ts=2026-09-23T05:13:48Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:587264962client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4963worker1 # [ 96.457944] nix-grpc-daemon[766]: ts=2026-09-23T05:13:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490904964worker1 # [ 96.633462] nix-daemon[832]: accepted connection from pid 1068, user nix-grpc-daemon (trusted)4965worker1 # [ 96.657376] nix-daemon[832]: reaped child process 1071, status = succeeded4966lb # [ 96.626930] niks3-server[967]: 2026/09/23 05:13:49 INFO Received uploads request method=POST path=/api/pending_closures4967worker1 # [ 96.672566] nix-grpc-daemon[860]: time=2026-09-23T05:13:49.177Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4968worker1 # [ 96.674582] nix-grpc-daemon[860]: time=2026-09-23T05:13:49.178Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4969lb # [ 96.667628] niks3-server[967]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4970worker1 # [ 96.714817] nix-grpc-daemon[860]: time=2026-09-23T05:13:49.220Z level=INFO msg="Uploading 1 narinfos"4971lb # [ 96.684397] niks3-server[967]: 2026/09/23 05:13:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign4972lb # [ 96.694797] niks3-server[967]: 2026/09/23 05:13:49 INFO Signed narinfos id=27 count=14973lb # [ 96.706242] niks3-server[967]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4974worker1 # [ 96.747694] nix-grpc-daemon[860]: time=2026-09-23T05:13:49.254Z level=INFO msg="Upload complete. (284ms)"4975lb # [ 96.718195] niks3-server[967]: 2026/09/23 05:13:49 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete4976lb # [ 96.725702] niks3-server[967]: 2026/09/23 05:13:49 INFO Completed upload id=274977lb # [ 96.729270] niks3-server[967]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4978worker2 # [ 96.844200] nix-grpc-daemon[769]: ts=2026-09-23T05:13:49Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04979worker2 # [ 96.851306] nix-grpc-daemon[769]: ts=2026-09-23T05:13:49Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14980worker1 # [ 96.831534] nix-grpc-daemon[766]: ts=2026-09-23T05:13:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490904981worker1 # [ 96.848626] nix-grpc-daemon[766]: ts=2026-09-23T05:13:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490904982client # copying 2 paths to 'lb:50051'...4983client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4984worker1 # [ 96.869275] nix-grpc-daemon[766]: ts=2026-09-23T05:13:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:490904985client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4986worker1 # [ 96.883286] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4987worker1 # [ 97.027214] nix-daemon[832]: accepted connection from pid 1075, user nix-grpc-daemon (trusted)4988lb # [ 97.018615] niks3-server[967]: 2026/09/23 05:13:49 INFO Received uploads request method=POST path=/api/pending_closures4989worker1 # [ 97.054265] nix-daemon[832]: reaped child process 1078, status = succeeded4990lb # [ 97.029272] niks3-server[967]: 2026/09/23 05:13:49 INFO Received uploads request method=POST path=/api/pending_closures4991worker1 # [ 97.072122] nix-grpc-daemon[860]: time=2026-09-23T05:13:49.577Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4992worker1 # [ 97.073827] nix-grpc-daemon[860]: time=2026-09-23T05:13:49.577Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4993worker1 # [ 97.075558] nix-grpc-daemon[860]: time=2026-09-23T05:13:49.577Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4994lb # [ 97.064056] niks3-server[967]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4995lb # [ 97.097296] niks3-server[967]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4996lb # [ 97.105854] niks3-server[967]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4997lb # [ 97.117295] niks3-server[967]: 2026/09/23 05:13:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign4998worker1 # [ 97.155498] nix-grpc-daemon[860]: time=2026-09-23T05:13:49.661Z level=INFO msg="Uploading 3 narinfos"4999lb # [ 97.125598] niks3-server[967]: 2026/09/23 05:13:49 INFO Signed narinfos id=28 count=15000lb # [ 97.130186] niks3-server[967]: 2026/09/23 05:13:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5001lb # [ 97.137578] niks3-server[967]: 2026/09/23 05:13:49 INFO Signed narinfos id=29 count=25002lb # [ 97.141533] niks3-server[967]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls5003lb # [ 97.156703] niks3-server[967]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo5004lb # [ 97.164446] niks3-server[967]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5005lb # [ 97.170578] niks3-server[967]: 2026/09/23 05:13:49 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5006lb # [ 97.174582] niks3-server[967]: 2026/09/23 05:13:49 INFO Completed upload id=285007lb # [ 97.179717] niks3-server[967]: 2026/09/23 05:13:49 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5008lb # [ 97.183297] niks3-server[967]: 2026/09/23 05:13:49 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5009worker1 # [ 97.214885] nix-grpc-daemon[860]: time=2026-09-23T05:13:49.721Z level=INFO msg="Upload complete. (311ms)"5010lb # [ 97.185961] niks3-server[967]: 2026/09/23 05:13:49 INFO Completed upload id=295011worker1 # [ 97.226039] nix-grpc-daemon[766]: ts=2026-09-23T05:13:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=0 paths=2 nar_bytes_in=9845012worker1 # [ 97.239665] nix-grpc-daemon[766]: ts=2026-09-23T05:13:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490905013worker1 # [ 97.256812] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5014worker1 # [ 97.266307] nix-daemon[832]: reaped child process 1073, status = succeeded5015client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...5016client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5017worker1 # [ 97.285755] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5018client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5019worker1 # [ 98.603964] nix-daemon[832]: reaped child process 1082, status = succeeded5020worker1 # [ 98.677697] nix-daemon[832]: accepted connection from pid 1098, user nix-grpc-daemon (trusted)5021worker1 # [ 98.699929] nix-daemon[832]: reaped child process 1101, status = succeeded5022lb # [ 98.673992] niks3-server[967]: 2026/09/23 05:13:51 INFO Received uploads request method=POST path=/api/pending_closures5023worker1 # [ 98.719314] nix-grpc-daemon[860]: time=2026-09-23T05:13:51.223Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5024lb # [ 98.707501] niks3-server[967]: 2026/09/23 05:13:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5025worker1 # [ 98.752744] nix-grpc-daemon[860]: time=2026-09-23T05:13:51.259Z level=INFO msg="Uploading 1 narinfos"5026lb # [ 98.725090] niks3-server[967]: 2026/09/23 05:13:51 INFO Signed narinfos id=30 count=15027lb # [ 98.735511] niks3-server[967]: 2026/09/23 05:13:51 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5028lb # [ 98.746520] niks3-server[967]: 2026/09/23 05:13:51 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls5029worker1 # [ 98.791228] nix-grpc-daemon[860]: time=2026-09-23T05:13:51.297Z level=INFO msg="Upload complete. (291ms)"5030lb # [ 98.761132] niks3-server[967]: 2026/09/23 05:13:51 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5031lb # [ 98.769669] niks3-server[967]: 2026/09/23 05:13:51 INFO Completed upload id=305032worker1 # [ 98.801133] nix-grpc-daemon[766]: ts=2026-09-23T05:13:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=15033client # copying 1 paths...5034lb # [ 98.774144] niks3-server[967]: 2026/09/23 05:13:51 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo5035worker1 # [ 98.819839] nix-grpc-daemon[766]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:490905036worker1 # [ 98.825390] nix-daemon[832]: reaped child process 1080, status = succeeded5037client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...5038worker1 # [ 98.851928] nix-grpc-daemon[766]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:490905039worker1 # [ 98.854971] nix-grpc-daemon[766]: ts=2026-09-23T05:13:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=0 paths=1 nar_bytes_out=1205040client # [ 99.009192] nix-daemon[1165]: reaped child process 1251, status = succeeded5041client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.91 seconds)5042client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&25043client # [ 99.192593] nix-daemon[1165]: accepted connection from pid 1270, user root (trusted)5044client # this derivation will be built:5045client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5046lb # [ 99.186138] envoy[895]: conn peer=[2001:db8:1::1]:47220 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=334 ms=1095047lb # [ 99.191201] envoy[895]: conn peer=[2001:db8:1::1]:46398 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3427 tx=1968 ms=25985048lb # [ 99.196505] envoy[895]: conn peer=[2001:db8:1::1]:47228 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3887 tx=2158 ms=17055049lb # [ 99.202173] envoy[895]: conn peer=[2001:db8:1::1]:47236 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=735050lb # [ 99.207510] envoy[895]: conn peer=[2001:db8:1::1]:47246 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4352 tx=2392 ms=20345051lb # [ 99.212984] envoy[895]: conn peer=[2001:db8:1::1]:47260 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=695052lb # [ 99.218111] envoy[895]: conn peer=[2001:db8:1::1]:47268 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=655053lb # [ 99.224303] envoy[895]: conn peer=[2001:db8:1::1]:47266 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3862 tx=2065 ms=24625054worker1 # [ 99.338046] nix-grpc-daemon[766]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:490905055client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5056worker2 # [ 99.415910] nix-grpc-daemon[769]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587265057worker2 # [ 99.445855] nix-grpc-daemon[769]: ts=2026-09-23T05:13:51Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=05058worker2 # [ 99.454601] nix-grpc-daemon[769]: ts=2026-09-23T05:13:51Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15059worker2 # [ 99.466735] nix-grpc-daemon[769]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587265060worker2 # [ 99.481263] nix-grpc-daemon[769]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:587265061client # copying 1 paths to 'lb:50051'...5062client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...5063worker2 # [ 99.511490] nix-grpc-daemon[769]: ts=2026-09-23T05:13:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:587265064worker2 # [ 99.530128] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5065worker2 # [ 99.554292] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 1/5)5066worker2 # [ 99.599934] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 100 ms (attempt 2/5)5067worker2 # [ 99.702220] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 153 ms (attempt 3/5)5068worker2 # [ 99.865015] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 4/5)5069worker2 # [ 99.923505] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5070worker2 # [ 99.931795] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5071worker2 # [ 99.938959] nix-grpc-daemon[769]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...5072worker2 # [ 100.108942] nix-daemon[855]: accepted connection from pid 1267, user nix-grpc-daemon (trusted)5073worker2 # [ 100.134153] nix-daemon[855]: reaped child process 1270, status = succeeded5074lb # [ 100.039358] niks3-server[967]: 2026/09/23 05:13:52 INFO Received uploads request method=POST path=/api/pending_closures5075worker2 # [ 100.153471] nix-grpc-daemon[870]: time=2026-09-23T05:13:52.605Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5076worker2 # [ 100.155626] nix-grpc-daemon[870]: time=2026-09-23T05:13:52.605Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"5077lb # [ 100.074119] niks3-server[967]: 2026/09/23 05:13:52 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst5078worker2 # [ 100.187671] nix-grpc-daemon[870]: time=2026-09-23T05:13:52.640Z level=INFO msg="Uploading 1 narinfos"5079lb # [ 100.090207] niks3-server[967]: 2026/09/23 05:13:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5080lb # [ 100.100518] niks3-server[967]: 2026/09/23 05:13:52 INFO Signed narinfos id=31 count=15081lb # [ 100.106283] niks3-server[967]: 2026/09/23 05:13:52 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls5082worker2 # [ 100.215899] nix-grpc-daemon[870]: time=2026-09-23T05:13:52.669Z level=INFO msg="Upload complete. (257ms)"5083lb # [ 100.115978] niks3-server[967]: 2026/09/23 05:13:52 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5084lb # [ 100.121164] niks3-server[967]: 2026/09/23 05:13:52 INFO Completed upload id=315085lb # [ 100.126931] niks3-server[967]: 2026/09/23 05:13:52 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo5086worker2 # [ 100.228429] nix-grpc-daemon[769]: ts=2026-09-23T05:13:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=0 paths=1 nar_bytes_in=5685087worker2 # [ 100.238978] nix-grpc-daemon[769]: ts=2026-09-23T05:13:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:587265088worker2 # [ 100.250922] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5089worker2 # [ 100.257225] nix-daemon[855]: reaped child process 1260, status = succeeded5090worker2 # [ 100.265621] nix-grpc-daemon[769]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...5091worker2 # [ 100.280076] nix-daemon[855]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5092client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...5093client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5094client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5095worker2 # [ 101.176374] nix-daemon[855]: reaped child process 1275, status = succeeded5096worker2 # [ 101.267264] nix-daemon[855]: accepted connection from pid 1291, user nix-grpc-daemon (trusted)5097worker2 # [ 101.290206] nix-daemon[855]: reaped child process 1294, status = succeeded5098lb # [ 101.186661] niks3-server[967]: 2026/09/23 05:13:53 INFO Received uploads request method=POST path=/api/pending_closures5099worker2 # [ 101.299630] nix-grpc-daemon[870]: time=2026-09-23T05:13:53.751Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5100lb # [ 101.219810] niks3-server[967]: 2026/09/23 05:13:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5101worker2 # [ 101.333253] nix-grpc-daemon[870]: time=2026-09-23T05:13:53.786Z level=INFO msg="Uploading 1 narinfos"5102lb # [ 101.235881] niks3-server[967]: 2026/09/23 05:13:53 INFO Signed narinfos id=32 count=15103lb # [ 101.243288] niks3-server[967]: 2026/09/23 05:13:53 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5104lb # [ 101.254462] niks3-server[967]: 2026/09/23 05:13:53 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls5105worker2 # [ 101.360610] nix-grpc-daemon[870]: time=2026-09-23T05:13:53.813Z level=INFO msg="Upload complete. (267ms)"5106lb # [ 101.261595] niks3-server[967]: 2026/09/23 05:13:53 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5107lb # [ 101.267882] niks3-server[967]: 2026/09/23 05:13:53 INFO Completed upload id=325108lb # [ 101.270662] niks3-server[967]: 2026/09/23 05:13:53 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo5109client # copying 1 paths...5110worker2 # [ 101.373572] nix-grpc-daemon[769]: ts=2026-09-23T05:13:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=15111worker1 # [ 101.313535] nix-grpc-daemon[766]: ts=2026-09-23T05:13:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:490905112worker2 # [ 101.388420] nix-daemon[855]: reaped child process 1272, status = succeeded5113worker1 # [ 101.326748] nix-grpc-daemon[766]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...5114client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5115worker1 # [ 101.371119] nix-grpc-daemon[766]: ts=2026-09-23T05:13:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:490905116worker1 # [ 101.378429] nix-grpc-daemon[766]: ts=2026-09-23T05:13:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=0 paths=1 nar_bytes_out=1205117client # [ 101.532364] nix-daemon[1165]: reaped child process 1273, status = succeeded5118client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.52 seconds)5119client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&25120client # [ 101.715399] nix-daemon[1165]: accepted connection from pid 1292, user root (trusted)5121client # this derivation will be built:5122client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5123worker2 # [ 101.921861] nix-grpc-daemon[769]: ts=2026-09-23T05:13:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:587265124client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5125worker1 # [ 101.856910] nix-grpc-daemon[766]: ts=2026-09-23T05:13:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490905126worker1 # [ 101.867945] nix-grpc-daemon[860]: time=2026-09-23T05:13:54.373Z level=INFO msg="All 1 paths already cached"5127worker2 # [ 101.952158] nix-grpc-daemon[769]: ts=2026-09-23T05:13:54Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05128worker2 # [ 101.958993] nix-grpc-daemon[769]: ts=2026-09-23T05:13:54Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15129worker1 # [ 101.892808] nix-grpc-daemon[766]: ts=2026-09-23T05:13:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490905130worker1 # [ 101.909017] nix-grpc-daemon[766]: ts=2026-09-23T05:13:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:490905131worker1 # [ 101.924092] nix-grpc-daemon[860]: time=2026-09-23T05:13:54.430Z level=INFO msg="All 1 paths already cached"5132client # copying 1 paths to 'lb:50051'...5133client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5134worker1 # [ 101.935520] nix-grpc-daemon[766]: ts=2026-09-23T05:13:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:490905135worker1 # [ 101.951667] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5136worker1 # [ 102.115609] nix-daemon[832]: accepted connection from pid 1106, user nix-grpc-daemon (trusted)5137worker1 # [ 102.134800] nix-daemon[832]: reaped child process 1109, status = succeeded5138lb # [ 102.110288] niks3-server[967]: 2026/09/23 05:13:54 INFO Received uploads request method=POST path=/api/pending_closures5139worker1 # [ 102.151877] nix-grpc-daemon[860]: time=2026-09-23T05:13:54.658Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5140worker1 # [ 102.153681] nix-grpc-daemon[860]: time=2026-09-23T05:13:54.658Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5141worker1 # [ 102.173526] nix-grpc-daemon[860]: time=2026-09-23T05:13:54.679Z level=INFO msg="Uploading 1 narinfos"5142lb # [ 102.141205] niks3-server[967]: 2026/09/23 05:13:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5143lb # [ 102.156134] niks3-server[967]: 2026/09/23 05:13:54 INFO Signed narinfos id=33 count=15144lb # [ 102.164200] niks3-server[967]: 2026/09/23 05:13:54 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5145lb # [ 102.172898] niks3-server[967]: 2026/09/23 05:13:54 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5146lb # [ 102.183323] niks3-server[967]: 2026/09/23 05:13:54 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5147worker1 # [ 102.220823] nix-grpc-daemon[860]: time=2026-09-23T05:13:54.727Z level=INFO msg="Upload complete. (250ms)"5148lb # [ 102.190568] niks3-server[967]: 2026/09/23 05:13:54 INFO Completed upload id=335149lb # [ 102.198460] niks3-server[967]: 2026/09/23 05:13:54 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5150worker1 # [ 102.231357] nix-grpc-daemon[766]: ts=2026-09-23T05:13:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=0 paths=1 nar_bytes_in=5685151worker1 # [ 102.240308] nix-grpc-daemon[766]: ts=2026-09-23T05:13:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:490905152worker1 # [ 102.253004] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5153worker1 # [ 102.258235] nix-daemon[832]: reaped child process 1104, status = succeeded5154client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5155worker1 # [ 102.273266] nix-daemon[832]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5156client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5157client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5158worker1 # [ 103.141428] nix-daemon[832]: reaped child process 1113, status = succeeded5159worker1 # [ 103.262375] nix-daemon[832]: accepted connection from pid 1129, user nix-grpc-daemon (trusted)5160worker1 # [ 103.281974] nix-daemon[832]: reaped child process 1132, status = succeeded5161lb # [ 103.261774] niks3-server[967]: 2026/09/23 05:13:55 INFO Received uploads request method=POST path=/api/pending_closures5162worker1 # [ 103.308085] nix-grpc-daemon[860]: time=2026-09-23T05:13:55.812Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5163worker1 # [ 103.332600] nix-grpc-daemon[860]: time=2026-09-23T05:13:55.838Z level=INFO msg="Uploading 1 narinfos"5164lb # [ 103.300102] niks3-server[967]: 2026/09/23 05:13:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5165lb # [ 103.312447] niks3-server[967]: 2026/09/23 05:13:55 INFO Signed narinfos id=34 count=15166lb # [ 103.315259] niks3-server[967]: 2026/09/23 05:13:55 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5167lb # [ 103.329942] niks3-server[967]: 2026/09/23 05:13:55 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5168worker1 # [ 103.367926] nix-grpc-daemon[860]: time=2026-09-23T05:13:55.874Z level=INFO msg="Upload complete. (283ms)"5169lb # [ 103.338363] niks3-server[967]: 2026/09/23 05:13:55 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5170lb # [ 103.347053] niks3-server[967]: 2026/09/23 05:13:55 INFO Completed upload id=345171client # copying 1 paths...5172lb # [ 103.349321] niks3-server[967]: 2026/09/23 05:13:55 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5173worker1 # [ 103.380969] nix-grpc-daemon[766]: ts=2026-09-23T05:13:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49090 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15174worker2 # [ 103.457518] nix-grpc-daemon[769]: ts=2026-09-23T05:13:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:587265175worker1 # [ 103.400956] nix-daemon[832]: reaped child process 1111, status = succeeded5176worker2 # [ 103.470650] nix-grpc-daemon[769]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...5177client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5178worker2 # [ 103.511478] nix-grpc-daemon[769]: ts=2026-09-23T05:13:55Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:587265179worker2 # [ 103.518888] nix-grpc-daemon[769]: ts=2026-09-23T05:13:55Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:58726 duration_s=0 paths=1 nar_bytes_out=1205180client # [ 103.601852] nix-daemon[1165]: reaped child process 1295, status = succeeded5181client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.07 seconds)5182(finished: subtest: an input only one worker has still reaches the builder, in 8.19 seconds)5183subtest: upload whose reference the worker lacks is completed from the cache5184client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5185client # this derivation will be built:5186client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5187client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5188client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.36 seconds)5189client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5190client # this derivation will be built:5191client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5192client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5193client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.31 seconds)5194client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5195client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.13 seconds)5196worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5197worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5198worker2 # [ 104.727056] systemd[1]: Stopping Nix worker-protocol over gRPC...5199worker2 # [ 105.671313] nix-grpc-daemon[769]: ts=2026-09-23T05:13:58Z level=info event=signal_exit5200lb # [ 105.950449] niks3-server[967]: 2026/09/23 05:13:58 INFO lead: released remote=192.168.1.4:383005201worker1 # [ 106.979460] nix-grpc-daemon[766]: ts=2026-09-23T05:13:59Z level=info event=scheduler_restarting addr=lb:500515202worker2 # [ 107.056653] nix-grpc-daemon[769]: ts=2026-09-23T05:13:59Z level=info event=scheduler_restarting5203worker2 # [ 107.057102] nix-grpc-daemon[769]: ts=2026-09-23T05:13:59Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:587245204worker2 # [ 107.065233] nix-grpc-daemon[769]: ts=2026-09-23T05:13:59Z level=info event=worker_gone addr=192.168.1.3:500515205worker2 # [ 107.065460] nix-grpc-daemon[769]: ts=2026-09-23T05:13:59Z level=info event=scheduler_restarting addr=lb:500515206worker2 # [ 107.065592] nix-grpc-daemon[769]: ts=2026-09-23T05:13:59Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:587245207worker2 # [ 107.065703] nix-grpc-daemon[769]: ts=2026-09-23T05:13:59Z level=info event=worker_gone addr=192.168.1.4:500515208worker2 # [ 107.093055] nix-daemon[855]: reaped child process 857, status = succeeded5209worker1 # [ 107.185201] nix-grpc-daemon[766]: ts=2026-09-23T05:13:59Z level=info event=scheduler_connected addr=lb:500515210worker1 # [ 107.190711] nix-grpc-daemon[766]: ts=2026-09-23T05:13:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5211worker2 # [ 107.278380] nix-grpc-daemon[769]: ts=2026-09-23T05:13:59Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5212worker2 # [ 107.426591] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5213worker2 # [ 107.433244] systemd[1]: Stopped Nix worker-protocol over gRPC.5214worker2 # [ 107.435849] systemd[1]: nix-grpc-daemon.service: Consumed 3.889s CPU time over 1min 27.994s wall clock time, 70.1M memory peak, 112.5K incoming IP traffic, 273.1K outgoing IP traffic.5215worker2 # [ 107.454652] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5216lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5217lb: must succeed: curl -sf localhost:9901/clusters5218worker2 # [ 107.464297] systemd[1]: Closed Nix worker-protocol over gRPC.5219worker1 # [ 107.391586] nix-grpc-daemon[766]: ts=2026-09-23T05:13:59Z level=info event=scheduler_connected addr=lb:500515220worker2 # [ 107.465778] systemd[1]: nix-grpc-daemon.socket: Consumed 623us CPU time over 1min 36.996s wall clock time, 256K memory peak, 120.6K incoming IP traffic, 99.8K outgoing IP traffic.5221worker1 # [ 107.398083] nix-grpc-daemon[766]: ts=2026-09-23T05:13:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5222lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5223lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5224client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25225worker1 # [ 107.598465] nix-grpc-daemon[766]: ts=2026-09-23T05:14:00Z level=info event=scheduler_connected addr=lb:500515226worker1 # [ 107.604433] nix-grpc-daemon[766]: ts=2026-09-23T05:14:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5227worker1 # [ 107.686801] nix-grpc-daemon[766]: ts=2026-09-23T05:14:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:490905228client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5229worker1 # [ 107.694456] nix-grpc-daemon[766]: ts=2026-09-23T05:14:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:490905230client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.25 seconds)5231worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5232worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5233worker1 # [ 107.805035] nix-grpc-daemon[766]: ts=2026-09-23T05:14:00Z level=info event=scheduler_connected addr=lb:500515234worker1 # [ 107.811508] nix-grpc-daemon[766]: ts=2026-09-23T05:14:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5235worker2 # [ 107.886168] systemd[1]: Listening on Nix worker-protocol over gRPC.5236worker1 # [ 107.888886] systemd[1]: Stopping Nix worker-protocol over gRPC...5237worker1 # [ 108.014397] nix-grpc-daemon[766]: ts=2026-09-23T05:14:00Z level=info event=scheduler_connected addr=lb:500515238worker1 # [ 108.019328] nix-grpc-daemon[766]: ts=2026-09-23T05:14:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5239worker1 # [ 108.220312] nix-grpc-daemon[766]: ts=2026-09-23T05:14:00Z level=info event=scheduler_connected addr=lb:500515240worker1 # [ 108.225757] nix-grpc-daemon[766]: ts=2026-09-23T05:14:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5241worker1 # [ 108.426618] nix-grpc-daemon[766]: ts=2026-09-23T05:14:00Z level=info event=scheduler_connected addr=lb:500515242worker1 # [ 108.432223] nix-grpc-daemon[766]: ts=2026-09-23T05:14:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5243worker1 # [ 108.455469] nix-grpc-daemon[766]: ts=2026-09-23T05:14:00Z level=info event=signal_exit5244worker1 # [ 108.632893] nix-grpc-daemon[766]: ts=2026-09-23T05:14:01Z level=info event=scheduler_connected addr=lb:500515245worker1 # [ 108.638353] nix-grpc-daemon[766]: ts=2026-09-23T05:14:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5246worker2 # [ 108.847249] systemd[1]: Starting scheduler-token.service...5247worker1 # [ 108.839102] nix-grpc-daemon[766]: ts=2026-09-23T05:14:01Z level=info event=scheduler_connected addr=lb:500515248worker1 # [ 108.844601] nix-grpc-daemon[766]: ts=2026-09-23T05:14:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5249worker2 # [ 108.932936] systemd[1]: scheduler-token.service: Deactivated successfully.5250worker2 # [ 108.940228] systemd[1]: Finished scheduler-token.service.5251worker2 # [ 108.942285] systemd[1]: scheduler-token.service: Consumed 58ms CPU time over 90ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5252worker2 # [ 108.965900] systemd[1]: Starting Nix worker-protocol over gRPC...5253worker1 # [ 108.979573] nix-grpc-daemon[766]: ts=2026-09-23T05:14:01Z level=info event=scheduler_take_over5254lb # [ 108.956758] niks3-server[967]: 2026/09/23 05:14:01 INFO lead: acquired remote=192.168.1.3:350605255lb # [ 108.957226] niks3-server[967]: 2026/09/23 05:14:01 INFO lead: released remote=192.168.1.3:350605256worker2 # [ 109.084472] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:01Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5257worker2 # [ 109.086764] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:01Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5258worker2 # [ 109.090649] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:01Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15259worker2 # [ 109.096058] nix-grpc-daemon[1321]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5260worker2 # [ 109.097531] nix-grpc-daemon[1321]: E0000 00:00:1790140441.548669 1321 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5261worker2 # [ 109.111980] nix-grpc-daemon[1321]: E0000 00:00:1790140441.554894 1321 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5262lb # [ 109.011422] niks3-server[967]: 2026/09/23 05:14:01 INFO lead: acquired remote=192.168.1.4:359205263worker1 # [ 109.046615] nix-grpc-daemon[766]: ts=2026-09-23T05:14:01Z level=info event=scheduler_connected addr=lb:500515264worker1 # [ 109.052197] nix-grpc-daemon[766]: ts=2026-09-23T05:14:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5265worker2 # [ 109.144193] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:01Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5266worker2 # [ 109.145995] systemd[1]: Started Nix worker-protocol over gRPC.5267worker2 # [ 109.146806] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:01Z level=info event=scheduler_take_over5268worker2 # [ 109.147922] nix-grpc-daemon[1321]: E0000 00:00:1790140441.568461 1361 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5269worker2 # [ 109.156822] nix-grpc-daemon[1321]: E0000 00:00:1790140441.587413 1361 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5270worker2 # [ 109.159778] nix-grpc-daemon[1321]: E0000 00:00:1790140441.587588 1361 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5271worker2 # [ 109.162548] nix-grpc-daemon[1321]: E0000 00:00:1790140441.608823 1361 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5272worker2 # [ 109.170758] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:01Z level=info event=scheduler_connected addr=lb:500515273worker2 # [ 109.178720] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:01Z level=info event=oidc_jwks provider=mock keys=15274worker2 # [ 109.180264] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:01Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:566665275worker2 # [ 109.181948] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:01Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05276lb # [ 109.185227] envoy[895]: conn peer=[2001:db8:1::1]:47274 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3727 tx=2127 ms=21005277lb # [ 109.190407] envoy[895]: conn peer=[2001:db8:1::1]:39868 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=715278lb # [ 109.195275] envoy[895]: conn peer=[2001:db8:1::1]:39878 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3780 tx=2120 ms=16565279lb # [ 109.200485] envoy[895]: conn peer=[2001:db8:1::1]:39888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=705280lb # [ 109.205136] envoy[895]: conn peer=[2001:db8:1::4]:55260 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=57813 tx=3562 ms=875625281lb # [ 109.210107] envoy[895]: conn peer=[2001:db8:1::1]:39898 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=225282worker1 # [ 109.253153] nix-grpc-daemon[766]: ts=2026-09-23T05:14:01Z level=info event=scheduler_connected addr=lb:500515283worker2 # [ 109.325351] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:01Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:566665284worker2 # [ 109.330747] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:01Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05285worker1 # [ 109.981220] nix-grpc-daemon[766]: ts=2026-09-23T05:14:02Z level=info event=scheduler_restarting5286worker2 # [ 110.063557] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:566665287worker2 # [ 110.068567] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:02Z level=info event=worker_gone addr=192.168.1.3:500515288worker1 # [ 110.021822] nix-grpc-daemon[766]: ts=2026-09-23T05:14:02Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5289worker1 # [ 110.042681] nix-daemon[832]: reaped child process 834, status = succeeded5290worker1 # [ 110.623176] nix-grpc-daemon[766]: E0000 00:00:1790140443.128338 812 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5291worker1 # [ 110.630914] nix-grpc-daemon[766]: E0000 00:00:1790140443.137442 812 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5292worker1 # [ 111.297500] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5293worker1 # [ 111.304754] systemd[1]: Stopped Nix worker-protocol over gRPC.5294worker1 # [ 111.307428] systemd[1]: nix-grpc-daemon.service: Consumed 3.011s CPU time over 1min 32.071s wall clock time, 61.3M memory peak, 90.1K incoming IP traffic, 158.1K outgoing IP traffic.5295worker1 # [ 111.324983] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5296worker1 # [ 111.331522] systemd[1]: Closed Nix worker-protocol over gRPC.5297worker1 # [ 111.333868] systemd[1]: nix-grpc-daemon.socket: Consumed 669us CPU time over 1min 40.914s wall clock time, 256K memory peak, 98.6K incoming IP traffic, 88.6K outgoing IP traffic.5298lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5299lb: must succeed: curl -sf localhost:9901/clusters5300lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5301lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)5302client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25303worker2 # [ 111.717359] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:476825304worker2 # [ 111.733268] nix-daemon[855]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5305worker2 # [ 111.763844] nix-grpc-daemon[1321]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 99 ms (attempt 1/5)5306worker2 # [ 111.871319] nix-grpc-daemon[1321]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 153 ms (attempt 2/5)5307worker2 # [ 112.026604] nix-grpc-daemon[1321]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 329 ms (attempt 3/5)5308worker2 # [ 112.357914] nix-grpc-daemon[1321]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 782 ms (attempt 4/5)5309worker2 # [ 113.143342] nix-grpc-daemon[1321]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5310worker2 # [ 113.148811] nix-grpc-daemon[1321]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5311client # copying 2 paths...5312client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5313client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5314worker2 # [ 113.173517] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:476825315worker2 # [ 113.190247] nix-daemon[855]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5316worker2 # [ 113.231664] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:05Z level=info event=niks3_push_started pid=13825317worker2 # [ 113.373121] nix-grpc-daemon[1382]: time=2026-09-23T05:14:05.825Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5318worker2 # [ 113.484183] nix-daemon[855]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5319worker2 # [ 113.505342] nix-daemon[855]: reaped child process 1398, status = succeeded5320lb # [ 113.408730] niks3-server[967]: 2026/09/23 05:14:05 INFO Received uploads request method=POST path=/api/pending_closures5321lb # [ 113.420725] niks3-server[967]: 2026/09/23 05:14:05 INFO Received uploads request method=POST path=/api/pending_closures5322worker2 # [ 113.534993] nix-grpc-daemon[1382]: time=2026-09-23T05:14:05.986Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5323worker2 # [ 113.539295] nix-grpc-daemon[1382]: time=2026-09-23T05:14:05.986Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5324worker2 # [ 113.542208] nix-grpc-daemon[1382]: time=2026-09-23T05:14:05.986Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5325lb # [ 113.482333] niks3-server[967]: 2026/09/23 05:14:05 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5326lb # [ 113.502297] niks3-server[967]: 2026/09/23 05:14:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5327lb # [ 113.516611] niks3-server[967]: 2026/09/23 05:14:06 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5328lb # [ 113.525663] niks3-server[967]: 2026/09/23 05:14:06 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5329worker2 # [ 113.630828] nix-grpc-daemon[1382]: time=2026-09-23T05:14:06.080Z level=INFO msg="Uploading 3 narinfos"5330lb # [ 113.536104] niks3-server[967]: 2026/09/23 05:14:06 INFO Signed narinfos id=36 count=25331lb # [ 113.537513] niks3-server[967]: 2026/09/23 05:14:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5332lb # [ 113.539368] niks3-server[967]: 2026/09/23 05:14:06 INFO Signed narinfos id=35 count=15333lb # [ 113.546314] niks3-server[967]: 2026/09/23 05:14:06 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5334lb # [ 113.558508] niks3-server[967]: 2026/09/23 05:14:06 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5335lb # [ 113.563993] niks3-server[967]: 2026/09/23 05:14:06 INFO Completed upload id=355336lb # [ 113.567347] niks3-server[967]: 2026/09/23 05:14:06 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5337lb # [ 113.572697] niks3-server[967]: 2026/09/23 05:14:06 INFO Completed upload id=365338worker2 # [ 113.674932] nix-grpc-daemon[1382]: time=2026-09-23T05:14:06.127Z level=INFO msg="Upload complete. (298ms)"5339lb # [ 113.576692] niks3-server[967]: 2026/09/23 05:14:06 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5340lb # [ 113.580262] niks3-server[967]: 2026/09/23 05:14:06 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5341lb # [ 113.583624] niks3-server[967]: 2026/09/23 05:14:06 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5342worker2 # [ 113.688410] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47682 duration_s=0 paths=2 nar_bytes_in=3125343worker2 # [ 113.705951] nix-daemon[855]: reaped child process 1380, status = succeeded5344client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 2.24 seconds)5345worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5346worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5347worker1 # [ 113.767377] systemd[1]: Listening on Nix worker-protocol over gRPC.5348worker1 # [ 113.778508] systemd[1]: Starting Nix worker-protocol over gRPC...5349worker1 # [ 113.931152] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:06Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5350worker1 # [ 113.933859] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:06Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5351worker1 # [ 113.938222] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:06Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15352worker1 # [ 113.944071] nix-grpc-daemon[1155]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5353worker1 # [ 113.945623] nix-grpc-daemon[1155]: E0000 00:00:1790140446.449519 1155 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5354lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5355lb: must succeed: curl -sf localhost:9901/clusters5356worker1 # [ 113.991505] nix-grpc-daemon[1155]: E0000 00:00:1790140446.456155 1155 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5357worker1 # [ 113.995126] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:06Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5358worker1 # [ 113.997530] nix-grpc-daemon[1155]: E0000 00:00:1790140446.463128 1195 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5359worker1 # [ 114.000287] nix-grpc-daemon[1155]: E0000 00:00:1790140446.481295 1195 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5360worker1 # [ 114.002522] nix-grpc-daemon[1155]: E0000 00:00:1790140446.481495 1195 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5361worker1 # [ 114.004906] nix-grpc-daemon[1155]: E0000 00:00:1790140446.498054 1195 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5362worker1 # [ 114.010567] systemd[1]: Started Nix worker-protocol over gRPC.5363worker1 # [ 114.013551] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:06Z level=info event=scheduler_connected addr=lb:500515364worker2 # [ 114.085937] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:06Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:566665365worker2 # [ 114.091606] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:06Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05366lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5367lb: must succeed: curl -sf localhost:9901/clusters5368lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5369lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds)5370(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.54 seconds)5371subtest: a niks3 restart does not cost the leader its role5372lb: must succeed: curl -sf localhost:9901/clusters5373lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5374lb: must succeed: curl -sf localhost:9901/clusters5375lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5376lb # [ 115.224853] systemd[1]: Stopping niks3 server...5377lb # [ 115.226739] niks3-server[967]: 2026/09/23 05:14:07 INFO Shutdown signal received, draining in-flight requests timeout=10s5378worker1 # [ 115.259965] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:07Z level=info event=scheduler_lock_lost status=2005379lb # [ 115.234766] niks3-server[967]: 2026/09/23 05:14:07 INFO lead: released remote=192.168.1.4:359205380worker2 # [ 115.338148] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:07Z level=info event=scheduler_lock_lost status=2005381lb # [ 115.252953] systemd[1]: niks3.service: Deactivated successfully.5382lb # [ 115.255718] systemd[1]: Stopped niks3 server.5383lb # [ 115.256764] systemd[1]: niks3.service: Consumed 714ms CPU time over 1min 31.557s wall clock time, 27.7M memory peak, 97.7K incoming IP traffic, 140.9K outgoing IP traffic.5384lb # [ 115.268150] systemd[1]: Starting niks3 server...5385lb # [ 115.331555] niks3-server[1111]: 2026/09/23 05:14:07 goose: no migrations to run. current version: 202609200000005386lb # [ 115.336767] niks3-server[1111]: 2026/09/23 05:14:07 OK 1_commit_pending_closure.sql (6.79ms)5387lb # [ 115.340505] niks3-server[1111]: 2026/09/23 05:14:07 OK 2_object_stats_trigger.sql (3.39ms)5388lb # [ 115.342782] niks3-server[1111]: 2026/09/23 05:14:07 goose: up to current file version: 25389lb # [ 115.343863] niks3-server[1111]: 2026/09/23 05:14:07 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5390lb # [ 115.353786] niks3-server[1111]: 2026/09/23 05:14:07 INFO Read proxy enabled — serving cache objects from S35391lb # [ 115.355132] niks3-server[1111]: 2026/09/23 05:14:07 INFO Using socket-activated listener address=0.0.0.0:57515392lb # [ 115.361298] niks3-server[1111]: 2026/09/23 05:14:07 INFO systemd watchdog enabled interval=15s5393lb # [ 115.364152] niks3-server[1111]: 2026/09/23 05:14:07 INFO Starting HTTP server address=0.0.0.0:57515394lb # [ 115.365668] systemd[1]: Started niks3 server.5395lb: waiting for unit niks3.service5396lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5397lb: must succeed: curl -sf localhost:9901/clusters5398lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5399worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5400worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5401lb: must succeed: curl -sf localhost:9901/clusters5402lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5403lb: must succeed: curl -sf localhost:9901/clusters5404lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5405worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5406worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5407client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25408worker2 # [ 116.007212] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:476825409client # don't know how to build these paths:5410client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5411worker1 # [ 115.956073] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453905412worker1 # [ 115.969782] nix-daemon[832]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5413worker1 # [ 115.996632] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 1/5)5414worker1 # [ 116.079108] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 198 ms (attempt 2/5)5415lb # [ 116.240554] niks3-server[1111]: 2026/09/23 05:14:08 INFO lead: acquired remote=192.168.1.4:406745416worker1 # [ 116.280342] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 3/5)5417worker1 # [ 116.421284] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 4/5)5418worker1 # [ 116.439431] nix-grpc-daemon[1155]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5419worker1 # [ 116.446568] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5420worker2 # [ 116.568893] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:09Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05421worker2 # [ 116.574466] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:09Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15422worker2 # [ 116.611069] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476825423worker2 # [ 116.629395] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:476825424client # copying 1 paths to 'lb:50051'...5425client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5426worker2 # [ 116.645713] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:476825427worker2 # [ 116.661094] nix-daemon[855]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5428worker2 # [ 116.827483] nix-daemon[855]: accepted connection from pid 1417, user nix-grpc-daemon (trusted)5429worker2 # [ 116.848085] nix-daemon[855]: reaped child process 1420, status = succeeded5430lb # [ 116.751125] niks3-server[1111]: 2026/09/23 05:14:09 INFO Received uploads request method=POST path=/api/pending_closures5431worker2 # [ 116.867599] nix-grpc-daemon[1382]: time=2026-09-23T05:14:09.319Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5432worker2 # [ 116.869491] nix-grpc-daemon[1382]: time=2026-09-23T05:14:09.319Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5433lb # [ 116.801921] niks3-server[1111]: 2026/09/23 05:14:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5434lb # [ 116.817255] niks3-server[1111]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5435lb # [ 116.841888] niks3-server[1111]: 2026/09/23 05:14:09 INFO Signed narinfos id=37 count=15436worker2 # [ 116.947701] nix-grpc-daemon[1382]: time=2026-09-23T05:14:09.399Z level=INFO msg="Uploading 1 narinfos"5437lb # [ 116.852850] niks3-server[1111]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5438lb # [ 116.863435] niks3-server[1111]: 2026/09/23 05:14:09 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5439lb # [ 116.868917] niks3-server[1111]: 2026/09/23 05:14:09 INFO Completed upload id=375440worker2 # [ 116.970270] nix-grpc-daemon[1382]: time=2026-09-23T05:14:09.423Z level=INFO msg="Upload complete. (292ms)"5441lb # [ 116.874153] niks3-server[1111]: 2026/09/23 05:14:09 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5442worker2 # [ 116.980451] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47682 duration_s=0 paths=1 nar_bytes_in=5445443worker2 # [ 117.001036] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476825444worker2 # [ 117.014449] nix-daemon[855]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5445worker2 # [ 117.018153] nix-daemon[855]: reaped child process 1415, status = succeeded5446client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5447worker2 # [ 117.031045] nix-daemon[855]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5448client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5449client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5450client # farm-a> LOG-farm-a-after-niks3-restart5451worker2 # [ 118.079801] nix-daemon[855]: reaped child process 1424, status = succeeded5452worker2 # [ 118.192095] nix-daemon[855]: accepted connection from pid 1440, user nix-grpc-daemon (trusted)5453worker2 # [ 118.213351] nix-daemon[855]: reaped child process 1443, status = succeeded5454lb # [ 118.117504] niks3-server[1111]: 2026/09/23 05:14:10 INFO Received uploads request method=POST path=/api/pending_closures5455worker2 # [ 118.235860] nix-grpc-daemon[1382]: time=2026-09-23T05:14:10.685Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5456worker2 # [ 118.237828] nix-grpc-daemon[1382]: time=2026-09-23T05:14:10.685Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5457lb # [ 118.163420] niks3-server[1111]: 2026/09/23 05:14:10 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5458worker2 # [ 118.281160] nix-grpc-daemon[1382]: time=2026-09-23T05:14:10.733Z level=INFO msg="Uploading 1 narinfos"5459lb # [ 118.179055] niks3-server[1111]: 2026/09/23 05:14:10 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5460lb # [ 118.188297] niks3-server[1111]: 2026/09/23 05:14:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5461lb # [ 118.191741] niks3-server[1111]: 2026/09/23 05:14:10 INFO Signed narinfos id=38 count=15462worker2 # [ 118.305952] nix-grpc-daemon[1382]: time=2026-09-23T05:14:10.759Z level=INFO msg="Upload complete. (261ms)"5463lb # [ 118.205073] niks3-server[1111]: 2026/09/23 05:14:10 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5464lb # [ 118.208924] niks3-server[1111]: 2026/09/23 05:14:10 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5465lb # [ 118.215989] niks3-server[1111]: 2026/09/23 05:14:10 INFO Completed upload id=385466lb # [ 118.218743] niks3-server[1111]: 2026/09/23 05:14:10 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5467worker2 # [ 118.325772] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47682 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15468worker2 # [ 118.343197] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:10Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05469worker2 # [ 118.347716] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:10Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15470worker2 # [ 118.352384] nix-daemon[855]: reaped child process 1422, status = succeeded5471worker2 # [ 118.382420] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476825472worker2 # [ 118.398288] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:476825473worker2 # [ 118.414013] nix-grpc-daemon[1382]: time=2026-09-23T05:14:10.867Z level=INFO msg="All 1 paths already cached"5474client # copying 1 paths to 'lb:50051'...5475client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5476worker2 # [ 118.425094] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:476825477worker2 # [ 118.439222] nix-daemon[855]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5478worker2 # [ 118.594405] nix-daemon[855]: accepted connection from pid 1449, user nix-grpc-daemon (trusted)5479worker2 # [ 118.618124] nix-daemon[855]: reaped child process 1452, status = succeeded5480lb # [ 118.514439] niks3-server[1111]: 2026/09/23 05:14:11 INFO Received uploads request method=POST path=/api/pending_closures5481worker2 # [ 118.628108] nix-grpc-daemon[1382]: time=2026-09-23T05:14:11.080Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5482worker2 # [ 118.630029] nix-grpc-daemon[1382]: time=2026-09-23T05:14:11.080Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5483lb # [ 118.547170] niks3-server[1111]: 2026/09/23 05:14:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5484worker2 # [ 118.657485] nix-grpc-daemon[1382]: time=2026-09-23T05:14:11.110Z level=INFO msg="Uploading 1 narinfos"5485lb # [ 118.565183] niks3-server[1111]: 2026/09/23 05:14:11 INFO Signed narinfos id=39 count=15486lb # [ 118.573612] niks3-server[1111]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5487lb # [ 118.583409] niks3-server[1111]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5488lb # [ 118.595996] niks3-server[1111]: 2026/09/23 05:14:11 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5489lb # [ 118.600572] niks3-server[1111]: 2026/09/23 05:14:11 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5490worker2 # [ 118.702453] nix-grpc-daemon[1382]: time=2026-09-23T05:14:11.155Z level=INFO msg="Upload complete. (245ms)"5491lb # [ 118.604080] niks3-server[1111]: 2026/09/23 05:14:11 INFO Completed upload id=395492worker2 # [ 118.713352] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47682 duration_s=0 paths=1 nar_bytes_in=6565493worker2 # [ 118.732298] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476825494worker2 # [ 118.741283] nix-daemon[855]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5495worker2 # [ 118.746456] nix-daemon[855]: reaped child process 1447, status = succeeded5496client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5497worker2 # [ 118.757274] nix-daemon[855]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5498client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5499lb # [ 119.187682] envoy[895]: conn peer=[2001:db8:1::3]:50050 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=13482 tx=3746 ms=905215500lb # [ 119.193713] envoy[895]: conn peer=[2001:db8:1::1]:60900 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1006 tx=282 ms=19885501client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5502client # farm-b> LOG-farm-b-after-niks3-restart5503worker2 # [ 120.212221] nix-daemon[855]: reaped child process 1456, status = succeeded5504worker2 # [ 120.318477] nix-daemon[855]: accepted connection from pid 1472, user nix-grpc-daemon (trusted)5505worker2 # [ 120.338683] nix-daemon[855]: reaped child process 1475, status = succeeded5506lb # [ 120.244728] niks3-server[1111]: 2026/09/23 05:14:12 INFO Received uploads request method=POST path=/api/pending_closures5507worker2 # [ 120.356600] nix-grpc-daemon[1382]: time=2026-09-23T05:14:12.809Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5508worker2 # [ 120.358218] nix-grpc-daemon[1382]: time=2026-09-23T05:14:12.809Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5509lb # [ 120.291434] niks3-server[1111]: 2026/09/23 05:14:12 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5510worker2 # [ 120.404701] nix-grpc-daemon[1382]: time=2026-09-23T05:14:12.857Z level=INFO msg="Uploading 1 narinfos"5511lb # [ 120.304319] niks3-server[1111]: 2026/09/23 05:14:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5512lb # [ 120.316319] niks3-server[1111]: 2026/09/23 05:14:12 INFO Signed narinfos id=40 count=15513lb # [ 120.318561] niks3-server[1111]: 2026/09/23 05:14:12 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5514lb # [ 120.328659] niks3-server[1111]: 2026/09/23 05:14:12 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5515lb # [ 120.336485] niks3-server[1111]: 2026/09/23 05:14:12 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5516lb # [ 120.341168] niks3-server[1111]: 2026/09/23 05:14:12 INFO Completed upload id=405517worker2 # [ 120.443246] nix-grpc-daemon[1382]: time=2026-09-23T05:14:12.896Z level=INFO msg="Upload complete. (272ms)"5518lb # [ 120.344185] niks3-server[1111]: 2026/09/23 05:14:12 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5519worker2 # [ 120.459306] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47682 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15520worker2 # [ 120.483506] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:12Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05521worker2 # [ 120.488681] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:12Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15522worker2 # [ 120.494033] nix-daemon[855]: reaped child process 1454, status = succeeded5523worker2 # [ 120.518654] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476825524worker2 # [ 120.534616] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:476825525worker2 # [ 120.550263] nix-grpc-daemon[1382]: time=2026-09-23T05:14:13.003Z level=INFO msg="All 2 paths already cached"5526client # copying 1 paths to 'lb:50051'...5527client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5528worker2 # [ 120.561830] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:476825529worker2 # [ 120.576288] nix-daemon[855]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5530worker2 # [ 120.730321] nix-daemon[855]: accepted connection from pid 1481, user nix-grpc-daemon (trusted)5531worker2 # [ 120.754278] nix-daemon[855]: reaped child process 1484, status = succeeded5532lb # [ 120.650696] niks3-server[1111]: 2026/09/23 05:14:13 INFO Received uploads request method=POST path=/api/pending_closures5533worker2 # [ 120.763828] nix-grpc-daemon[1382]: time=2026-09-23T05:14:13.216Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5534worker2 # [ 120.765475] nix-grpc-daemon[1382]: time=2026-09-23T05:14:13.216Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5535lb # [ 120.682407] niks3-server[1111]: 2026/09/23 05:14:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5536worker2 # [ 120.791100] nix-grpc-daemon[1382]: time=2026-09-23T05:14:13.244Z level=INFO msg="Uploading 1 narinfos"5537lb # [ 120.698469] niks3-server[1111]: 2026/09/23 05:14:13 INFO Signed narinfos id=41 count=15538lb # [ 120.705558] niks3-server[1111]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5539lb # [ 120.719433] niks3-server[1111]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5540lb # [ 120.726463] niks3-server[1111]: 2026/09/23 05:14:13 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5541lb # [ 120.730747] niks3-server[1111]: 2026/09/23 05:14:13 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5542worker2 # [ 120.837030] nix-grpc-daemon[1382]: time=2026-09-23T05:14:13.290Z level=INFO msg="Upload complete. (244ms)"5543lb # [ 120.738728] niks3-server[1111]: 2026/09/23 05:14:13 INFO Completed upload id=415544worker2 # [ 120.847449] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47682 duration_s=0 paths=1 nar_bytes_in=7845545worker2 # [ 120.864121] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:476825546worker2 # [ 120.870589] nix-daemon[855]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5547worker2 # [ 120.875872] nix-daemon[855]: reaped child process 1479, status = succeeded5548worker2 # [ 120.887481] nix-daemon[855]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5549client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5550client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5551client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5552client # farm-top> LOG-farm-top-after-niks3-restart5553worker2 # [ 122.389281] nix-daemon[855]: reaped child process 1488, status = succeeded5554worker2 # [ 122.460621] nix-daemon[855]: accepted connection from pid 1504, user nix-grpc-daemon (trusted)5555worker2 # [ 122.482408] nix-daemon[855]: reaped child process 1507, status = succeeded5556lb # [ 122.379752] niks3-server[1111]: 2026/09/23 05:14:14 INFO Received uploads request method=POST path=/api/pending_closures5557worker2 # [ 122.492371] nix-grpc-daemon[1382]: time=2026-09-23T05:14:14.944Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5558worker2 # [ 122.493865] nix-grpc-daemon[1382]: time=2026-09-23T05:14:14.944Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5559lb # [ 122.423795] niks3-server[1111]: 2026/09/23 05:14:14 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5560worker2 # [ 122.535228] nix-grpc-daemon[1382]: time=2026-09-23T05:14:14.988Z level=INFO msg="Uploading 1 narinfos"5561lb # [ 122.441380] niks3-server[1111]: 2026/09/23 05:14:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5562lb # [ 122.451881] niks3-server[1111]: 2026/09/23 05:14:14 INFO Signed narinfos id=42 count=15563lb # [ 122.461732] niks3-server[1111]: 2026/09/23 05:14:14 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5564worker2 # [ 122.572587] nix-grpc-daemon[1382]: time=2026-09-23T05:14:15.025Z level=INFO msg="Upload complete. (272ms)"5565lb # [ 122.471676] niks3-server[1111]: 2026/09/23 05:14:14 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5566lb # [ 122.474931] niks3-server[1111]: 2026/09/23 05:14:14 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5567lb # [ 122.481176] niks3-server[1111]: 2026/09/23 05:14:14 INFO Completed upload id=425568lb # [ 122.484354] niks3-server[1111]: 2026/09/23 05:14:14 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5569worker2 # [ 122.583848] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47682 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15570worker2 # [ 122.602595] nix-daemon[855]: reaped child process 1486, status = succeeded5571client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.86 seconds)5572client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5573client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.19 seconds)5574(finished: subtest: a niks3 restart does not cost the leader its role, in 7.62 seconds)5575subtest: scheduler down: the other node takes the lock and keeps it5576lb: must succeed: curl -sf localhost:9901/clusters5577lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5578lb: must succeed: curl -sf localhost:9901/clusters5579lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5580lb: must succeed: curl -sf localhost:9901/clusters5581lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5582lb: must succeed: curl -sf localhost:9901/clusters5583lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5584worker2 # [ 123.070391] systemd[1]: Stopping Nix worker-protocol over gRPC...5585worker2 # [ 123.112036] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:15Z level=info event=signal_exit5586lb # [ 123.251399] niks3-server[1111]: 2026/09/23 05:14:15 INFO lead: released remote=192.168.1.4:406745587worker1 # [ 124.281410] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:16Z level=info event=scheduler_restarting addr=lb:500515588worker2 # [ 124.355503] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:16Z level=info event=scheduler_restarting5589worker2 # [ 124.360850] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:16Z level=info event=scheduler_restarting addr=lb:500515590worker2 # [ 124.365525] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:566665591worker2 # [ 124.371579] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:16Z level=info event=worker_gone addr=192.168.1.4:500515592worker2 # [ 124.380427] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:566665593worker2 # [ 124.388278] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:16Z level=info event=worker_gone addr=192.168.1.3:500515594worker2 # [ 124.404371] nix-daemon[855]: reaped child process 1374, status = succeeded5595worker1 # [ 124.481245] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:16Z level=info event=scheduler_connected addr=lb:500515596worker1 # [ 124.487203] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5597worker2 # [ 124.573497] nix-grpc-daemon[1321]: ts=2026-09-23T05:14:17Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5598worker1 # [ 124.688338] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:17Z level=info event=scheduler_connected addr=lb:500515599worker1 # [ 124.694075] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5600worker1 # [ 124.895134] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:17Z level=info event=scheduler_connected addr=lb:500515601worker1 # [ 124.900968] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5602worker2 # [ 125.145152] nix-grpc-daemon[1321]: E0000 00:00:1790140457.597917 1367 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5603worker2 # [ 125.152295] nix-grpc-daemon[1321]: E0000 00:00:1790140457.598248 1367 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5604worker1 # [ 125.101975] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:17Z level=info event=scheduler_connected addr=lb:500515605worker1 # [ 125.107711] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5606worker1 # [ 125.310821] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:17Z level=info event=scheduler_connected addr=lb:500515607worker1 # [ 125.315203] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5608worker1 # [ 125.511880] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:18Z level=info event=scheduler_connected addr=lb:500515609worker1 # [ 125.517690] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5610worker2 # [ 125.630012] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5611worker2 # [ 125.636581] systemd[1]: Stopped Nix worker-protocol over gRPC.5612worker2 # [ 125.639235] systemd[1]: nix-grpc-daemon.service: Consumed 1.386s CPU time over 16.681s wall clock time, 39.3M memory peak, 45.9K incoming IP traffic, 78K outgoing IP traffic.5613worker2 # [ 125.659341] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5614worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5615worker2 # [ 125.669746] systemd[1]: Closed Nix worker-protocol over gRPC.5616worker2 # [ 125.671360] systemd[1]: nix-grpc-daemon.socket: Consumed 894us CPU time over 17.776s wall clock time, 256K memory peak, 39.1K incoming IP traffic, 29.4K outgoing IP traffic.5617worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)5618worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5619worker1 # [ 125.719814] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:18Z level=info event=scheduler_connected addr=lb:500515620worker1 # [ 125.722515] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5621worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5622worker1 # [ 125.921325] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:18Z level=info event=scheduler_connected addr=lb:500515623worker1 # [ 125.926882] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5624worker1 # [ 126.127712] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:18Z level=info event=scheduler_connected addr=lb:500515625worker1 # [ 126.133243] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5626worker1 # [ 126.266429] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:18Z level=info event=scheduler_take_over5627lb # [ 126.235042] niks3-server[1111]: 2026/09/23 05:14:18 INFO lead: acquired remote=192.168.1.3:605665628worker1 # [ 126.335879] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:18Z level=info event=scheduler_connected addr=lb:500515629worker1 # [ 126.342123] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:18Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5630worker1 # [ 126.536941] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:19Z level=info event=scheduler_connected addr=lb:500515631worker1 # [ 126.546323] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:19Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:492785632worker1 # [ 126.552305] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:19Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05633worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5634worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5635client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25636worker1 # [ 127.015005] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:453905637client # don't know how to build these paths:5638client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5639worker1 # [ 127.025860] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453905640worker1 # [ 127.054400] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:19Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05641worker1 # [ 127.062202] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:19Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15642worker1 # [ 127.100875] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453905643worker1 # [ 127.117316] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453905644client # copying 1 paths to 'lb:50051'...5645client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5646worker1 # [ 127.134298] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453905647worker1 # [ 127.150013] nix-daemon[832]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5648worker1 # [ 127.177966] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:19Z level=info event=niks3_push_started pid=12325649worker1 # [ 127.330459] nix-grpc-daemon[1232]: time=2026-09-23T05:14:19.836Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5650worker1 # [ 127.446091] nix-daemon[832]: accepted connection from pid 1245, user nix-grpc-daemon (trusted)5651worker1 # [ 127.467549] nix-daemon[832]: reaped child process 1248, status = succeeded5652lb # [ 127.441640] niks3-server[1111]: 2026/09/23 05:14:19 INFO Received uploads request method=POST path=/api/pending_closures5653worker1 # [ 127.484078] nix-grpc-daemon[1232]: time=2026-09-23T05:14:19.989Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5654worker1 # [ 127.488616] nix-grpc-daemon[1232]: time=2026-09-23T05:14:19.989Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5655lb # [ 127.493106] niks3-server[1111]: 2026/09/23 05:14:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5656worker1 # [ 127.532065] nix-grpc-daemon[1232]: time=2026-09-23T05:14:20.037Z level=INFO msg="Uploading 1 narinfos"5657lb # [ 127.505049] niks3-server[1111]: 2026/09/23 05:14:20 INFO Signed narinfos id=43 count=15658lb # [ 127.510705] niks3-server[1111]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5659lb # [ 127.522039] niks3-server[1111]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5660lb # [ 127.533819] niks3-server[1111]: 2026/09/23 05:14:20 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5661worker1 # [ 127.570612] nix-grpc-daemon[1232]: time=2026-09-23T05:14:20.077Z level=INFO msg="Upload complete. (236ms)"5662lb # [ 127.541384] niks3-server[1111]: 2026/09/23 05:14:20 INFO Completed upload id=435663lb # [ 127.543325] niks3-server[1111]: 2026/09/23 05:14:20 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5664worker1 # [ 127.588224] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45390 duration_s=0 paths=1 nar_bytes_in=5205665worker1 # [ 127.599537] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453905666worker1 # [ 127.606097] nix-daemon[832]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5667worker1 # [ 127.611045] nix-daemon[832]: reaped child process 1230, status = succeeded5668client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5669worker1 # [ 127.623408] nix-daemon[832]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5670client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5671client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5672client # farm-a> LOG-farm-a-on-standby5673worker1 # [ 128.958359] nix-daemon[832]: reaped child process 1252, status = succeeded5674worker1 # [ 129.050203] nix-daemon[832]: accepted connection from pid 1268, user nix-grpc-daemon (trusted)5675worker1 # [ 129.077665] nix-daemon[832]: reaped child process 1271, status = succeeded5676lb # [ 129.045188] niks3-server[1111]: 2026/09/23 05:14:21 INFO Received uploads request method=POST path=/api/pending_closures5677worker1 # [ 129.092563] nix-grpc-daemon[1232]: time=2026-09-23T05:14:21.596Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5678worker1 # [ 129.094294] nix-grpc-daemon[1232]: time=2026-09-23T05:14:21.596Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5679lb # [ 129.103350] niks3-server[1111]: 2026/09/23 05:14:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5680lb # [ 129.116686] niks3-server[1111]: 2026/09/23 05:14:21 INFO Signed narinfos id=44 count=15681worker1 # [ 129.150140] nix-grpc-daemon[1232]: time=2026-09-23T05:14:21.655Z level=INFO msg="Uploading 1 narinfos"5682lb # [ 129.131414] niks3-server[1111]: 2026/09/23 05:14:21 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5683lb # [ 129.143046] niks3-server[1111]: 2026/09/23 05:14:21 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5684lb # [ 129.153731] niks3-server[1111]: 2026/09/23 05:14:21 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5685lb # [ 129.167153] niks3-server[1111]: 2026/09/23 05:14:21 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5686worker1 # [ 129.204436] nix-grpc-daemon[1232]: time=2026-09-23T05:14:21.710Z level=INFO msg="Upload complete. (364ms)"5687lb # [ 129.172807] niks3-server[1111]: 2026/09/23 05:14:21 INFO Completed upload id=445688lb # [ 129.183606] niks3-server[1111]: 2026/09/23 05:14:21 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5689lb # [ 129.189869] envoy[895]: conn peer=[2001:db8:1::1]:60912 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9792 tx=4446 ms=65975690lb # [ 129.194451] envoy[895]: conn peer=[2001:db8:1::4]:35706 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1708 tx=442 ms=153925691worker1 # [ 129.234517] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45390 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15692worker1 # [ 129.248660] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:21Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05693worker1 # [ 129.257504] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:21Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15694worker1 # [ 129.261871] nix-daemon[832]: reaped child process 1250, status = succeeded5695worker1 # [ 129.286386] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453905696worker1 # [ 129.307945] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453905697worker1 # [ 129.325785] nix-grpc-daemon[1232]: time=2026-09-23T05:14:21.832Z level=INFO msg="All 1 paths already cached"5698client # copying 1 paths to 'lb:50051'...5699worker1 # [ 129.340195] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453905700client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5701worker1 # [ 129.362801] nix-daemon[832]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5702worker1 # [ 129.558370] nix-daemon[832]: accepted connection from pid 1277, user nix-grpc-daemon (trusted)5703worker1 # [ 129.583840] nix-daemon[832]: reaped child process 1280, status = succeeded5704lb # [ 129.550105] niks3-server[1111]: 2026/09/23 05:14:22 INFO Received uploads request method=POST path=/api/pending_closures5705worker1 # [ 129.592920] nix-grpc-daemon[1232]: time=2026-09-23T05:14:22.099Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5706worker1 # [ 129.594472] nix-grpc-daemon[1232]: time=2026-09-23T05:14:22.099Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5707lb # [ 129.585133] niks3-server[1111]: 2026/09/23 05:14:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5708worker1 # [ 129.629381] nix-grpc-daemon[1232]: time=2026-09-23T05:14:22.135Z level=INFO msg="Uploading 1 narinfos"5709lb # [ 129.603210] niks3-server[1111]: 2026/09/23 05:14:22 INFO Signed narinfos id=45 count=15710lb # [ 129.616278] niks3-server[1111]: 2026/09/23 05:14:22 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5711lb # [ 129.627464] niks3-server[1111]: 2026/09/23 05:14:22 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5712lb # [ 129.643577] niks3-server[1111]: 2026/09/23 05:14:22 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5713worker1 # [ 129.681409] nix-grpc-daemon[1232]: time=2026-09-23T05:14:22.187Z level=INFO msg="Upload complete. (298ms)"5714lb # [ 129.653086] niks3-server[1111]: 2026/09/23 05:14:22 INFO Completed upload id=455715lb # [ 129.655707] niks3-server[1111]: 2026/09/23 05:14:22 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5716worker1 # [ 129.692446] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45390 duration_s=0 paths=1 nar_bytes_in=6405717worker1 # [ 129.708928] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453905718worker1 # [ 129.727462] nix-daemon[832]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5719worker1 # [ 129.734160] nix-daemon[832]: reaped child process 1275, status = succeeded5720client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5721worker1 # [ 129.749829] nix-daemon[832]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5722client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5723client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5724client # farm-b> LOG-farm-b-on-standby5725worker1 # [ 131.086080] nix-daemon[832]: reaped child process 1284, status = succeeded5726worker1 # [ 131.173287] nix-daemon[832]: accepted connection from pid 1300, user nix-grpc-daemon (trusted)5727worker1 # [ 131.198167] nix-daemon[832]: reaped child process 1303, status = succeeded5728lb # [ 131.166523] niks3-server[1111]: 2026/09/23 05:14:23 INFO Received uploads request method=POST path=/api/pending_closures5729worker1 # [ 131.216313] nix-grpc-daemon[1232]: time=2026-09-23T05:14:23.721Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5730worker1 # [ 131.217971] nix-grpc-daemon[1232]: time=2026-09-23T05:14:23.721Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5731lb # [ 131.230073] niks3-server[1111]: 2026/09/23 05:14:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5732worker1 # [ 131.271319] nix-grpc-daemon[1232]: time=2026-09-23T05:14:23.777Z level=INFO msg="Uploading 1 narinfos"5733lb # [ 131.247048] niks3-server[1111]: 2026/09/23 05:14:23 INFO Signed narinfos id=46 count=15734lb # [ 131.259055] niks3-server[1111]: 2026/09/23 05:14:23 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5735lb # [ 131.269743] niks3-server[1111]: 2026/09/23 05:14:23 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5736lb # [ 131.275398] niks3-server[1111]: 2026/09/23 05:14:23 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5737lb # [ 131.283960] niks3-server[1111]: 2026/09/23 05:14:23 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5738lb # [ 131.291725] niks3-server[1111]: 2026/09/23 05:14:23 INFO Completed upload id=465739worker1 # [ 131.324319] nix-grpc-daemon[1232]: time=2026-09-23T05:14:23.830Z level=INFO msg="Upload complete. (354ms)"5740lb # [ 131.294777] niks3-server[1111]: 2026/09/23 05:14:23 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5741worker1 # [ 131.348164] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45390 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15742worker1 # [ 131.372555] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:23Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05743worker1 # [ 131.377697] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:23Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15744worker1 # [ 131.385027] nix-daemon[832]: reaped child process 1282, status = succeeded5745worker1 # [ 131.398623] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453905746worker1 # [ 131.419607] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453905747worker1 # [ 131.439025] nix-grpc-daemon[1232]: time=2026-09-23T05:14:23.945Z level=INFO msg="All 2 paths already cached"5748client # copying 1 paths to 'lb:50051'...5749client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5750worker1 # [ 131.455091] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453905751worker1 # [ 131.474806] nix-daemon[832]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5752worker1 # [ 131.669966] nix-daemon[832]: accepted connection from pid 1309, user nix-grpc-daemon (trusted)5753worker1 # [ 131.694541] nix-daemon[832]: reaped child process 1312, status = succeeded5754lb # [ 131.661131] niks3-server[1111]: 2026/09/23 05:14:24 INFO Received uploads request method=POST path=/api/pending_closures5755worker1 # [ 131.705690] nix-grpc-daemon[1232]: time=2026-09-23T05:14:24.211Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5756worker1 # [ 131.707206] nix-grpc-daemon[1232]: time=2026-09-23T05:14:24.211Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5757lb # [ 131.709774] niks3-server[1111]: 2026/09/23 05:14:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5758worker1 # [ 131.749058] nix-grpc-daemon[1232]: time=2026-09-23T05:14:24.255Z level=INFO msg="Uploading 1 narinfos"5759lb # [ 131.724576] niks3-server[1111]: 2026/09/23 05:14:24 INFO Signed narinfos id=47 count=15760lb # [ 131.730333] niks3-server[1111]: 2026/09/23 05:14:24 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5761lb # [ 131.743641] niks3-server[1111]: 2026/09/23 05:14:24 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5762lb # [ 131.758325] niks3-server[1111]: 2026/09/23 05:14:24 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5763lb # [ 131.767995] niks3-server[1111]: 2026/09/23 05:14:24 INFO Completed upload id=475764worker1 # [ 131.799493] nix-grpc-daemon[1232]: time=2026-09-23T05:14:24.305Z level=INFO msg="Upload complete. (301ms)"5765lb # [ 131.772452] niks3-server[1111]: 2026/09/23 05:14:24 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5766worker1 # [ 131.810754] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45390 duration_s=0 paths=1 nar_bytes_in=7685767worker1 # [ 131.833933] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453905768worker1 # [ 131.840467] nix-daemon[832]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5769worker1 # [ 131.846125] nix-daemon[832]: reaped child process 1307, status = succeeded5770client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5771client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5772worker1 # [ 131.858526] nix-daemon[832]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5773client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5774client # farm-top> LOG-farm-top-on-standby5775worker1 # [ 132.533599] nix-daemon[832]: reaped child process 1316, status = succeeded5776worker1 # [ 132.600100] nix-daemon[832]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5777worker1 # [ 132.618633] nix-daemon[832]: reaped child process 1335, status = succeeded5778lb # [ 132.593983] niks3-server[1111]: 2026/09/23 05:14:25 INFO Received uploads request method=POST path=/api/pending_closures5779worker1 # [ 132.636405] nix-grpc-daemon[1232]: time=2026-09-23T05:14:25.141Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5780worker1 # [ 132.637899] nix-grpc-daemon[1232]: time=2026-09-23T05:14:25.141Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5781lb # [ 132.638519] niks3-server[1111]: 2026/09/23 05:14:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5782worker1 # [ 132.678118] nix-grpc-daemon[1232]: time=2026-09-23T05:14:25.184Z level=INFO msg="Uploading 1 narinfos"5783lb # [ 132.653456] niks3-server[1111]: 2026/09/23 05:14:25 INFO Signed narinfos id=48 count=15784lb # [ 132.659857] niks3-server[1111]: 2026/09/23 05:14:25 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5785lb # [ 132.667746] niks3-server[1111]: 2026/09/23 05:14:25 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5786lb # [ 132.676185] niks3-server[1111]: 2026/09/23 05:14:25 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5787lb # [ 132.679964] niks3-server[1111]: 2026/09/23 05:14:25 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5788worker1 # [ 132.711743] nix-grpc-daemon[1232]: time=2026-09-23T05:14:25.218Z level=INFO msg="Upload complete. (241ms)"5789lb # [ 132.683950] niks3-server[1111]: 2026/09/23 05:14:25 INFO Completed upload id=485790lb # [ 132.687351] niks3-server[1111]: 2026/09/23 05:14:25 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5791worker1 # [ 132.728496] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45390 duration_s=0 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15792worker1 # [ 132.740413] nix-daemon[832]: reaped child process 1314, status = succeeded5793client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.98 seconds)5794client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5795client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5796worker2 # [ 133.117065] systemd[1]: Listening on Nix worker-protocol over gRPC.5797worker2 # [ 133.123948] systemd[1]: Starting scheduler-token.service...5798worker2 # [ 133.199022] systemd[1]: scheduler-token.service: Deactivated successfully.5799worker2 # [ 133.203428] systemd[1]: Finished scheduler-token.service.5800worker2 # [ 133.208615] systemd[1]: scheduler-token.service: Consumed 43ms CPU time over 78ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5801worker2 # [ 133.220818] systemd[1]: Starting Nix worker-protocol over gRPC...5802worker2 # [ 133.336197] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5803worker2 # [ 133.338462] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5804worker2 # [ 133.342526] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15805worker2 # [ 133.348052] nix-grpc-daemon[1528]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5806worker2 # [ 133.349483] nix-grpc-daemon[1528]: E0000 00:00:1790140465.800295 1528 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5807worker2 # [ 133.392250] nix-grpc-daemon[1528]: E0000 00:00:1790140465.806621 1528 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5808worker2 # [ 133.394521] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:25Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5809lb: must succeed: curl -sf localhost:9901/clusters5810worker2 # [ 133.398988] nix-grpc-daemon[1528]: E0000 00:00:1790140465.814760 1568 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5811worker2 # [ 133.401935] nix-grpc-daemon[1528]: E0000 00:00:1790140465.829933 1568 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5812worker2 # [ 133.404424] nix-grpc-daemon[1528]: E0000 00:00:1790140465.830124 1568 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5813worker2 # [ 133.406796] systemd[1]: Started Nix worker-protocol over gRPC.5814worker2 # [ 133.407604] nix-grpc-daemon[1528]: E0000 00:00:1790140465.851027 1568 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5815worker2 # [ 133.417253] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:25Z level=info event=scheduler_connected addr=lb:500515816worker1 # [ 133.358288] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:25Z level=info event=oidc_jwks provider=mock keys=15817worker1 # [ 133.362950] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:25Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:492785818worker1 # [ 133.368472] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05819lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5820worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5821worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5822lb: must succeed: curl -sf localhost:9901/clusters5823lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5824lb: must succeed: curl -sf localhost:9901/clusters5825lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5826worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5827worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5828(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 10.89 seconds)5829subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5830client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5831client # [ 133.736803] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5832client # Running as unit: rs.service; invocation ID: ae2c02fa55c9441a8a1a5b9ecf05ec585833client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5834worker2 # [ 133.991328] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:560745835worker2 # [ 134.006375] nix-daemon[855]: accepted connection from pid 1528, user nix-grpc-daemon (trusted)5836client # [ 133.976137] nix[1418]: don't know how to build these paths:5837client # [ 133.977752] nix[1418]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5838worker1 # [ 133.971886] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:26Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05839worker1 # [ 133.978919] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:26Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15840worker1 # [ 134.017262] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453905841worker1 # [ 134.033132] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:453905842client # [ 134.068564] nix[1418]: copying 1 paths to 'lb:50051'...5843worker1 # [ 134.048124] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:453905844client # [ 134.070967] nix[1418]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5845worker1 # [ 134.066364] nix-daemon[832]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5846worker1 # [ 134.225758] nix-daemon[832]: accepted connection from pid 1348, user nix-grpc-daemon (trusted)5847worker1 # [ 134.248559] nix-daemon[832]: reaped child process 1351, status = succeeded5848lb # [ 134.218848] niks3-server[1111]: 2026/09/23 05:14:26 INFO Received uploads request method=POST path=/api/pending_closures5849worker1 # [ 134.264880] nix-grpc-daemon[1232]: time=2026-09-23T05:14:26.770Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5850worker1 # [ 134.266532] nix-grpc-daemon[1232]: time=2026-09-23T05:14:26.770Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5851lb # [ 134.252127] niks3-server[1111]: 2026/09/23 05:14:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5852worker1 # [ 134.290325] nix-grpc-daemon[1232]: time=2026-09-23T05:14:26.796Z level=INFO msg="Uploading 1 narinfos"5853lb # [ 134.260439] niks3-server[1111]: 2026/09/23 05:14:26 INFO Signed narinfos id=49 count=15854lb # [ 134.268753] niks3-server[1111]: 2026/09/23 05:14:26 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5855lb # [ 134.275533] niks3-server[1111]: 2026/09/23 05:14:26 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5856lb # [ 134.285361] niks3-server[1111]: 2026/09/23 05:14:26 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5857lb # [ 134.292749] niks3-server[1111]: 2026/09/23 05:14:26 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5858worker1 # [ 134.324504] nix-grpc-daemon[1232]: time=2026-09-23T05:14:26.831Z level=INFO msg="Upload complete. (241ms)"5859lb # [ 134.295527] niks3-server[1111]: 2026/09/23 05:14:26 INFO Completed upload id=495860worker1 # [ 134.332379] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45390 duration_s=0 paths=1 nar_bytes_in=4165861worker1 # [ 134.351185] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:453905862worker1 # [ 134.358930] nix-daemon[832]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5863worker1 # [ 134.364867] nix-daemon[832]: reaped child process 1346, status = succeeded5864worker1 # [ 134.378109] nix-daemon[832]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5865client # [ 134.397090] nix[1418]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5866lb: must succeed: curl -sf localhost:9901/clusters5867lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5868lb: must succeed: curl -sf localhost:9901/clusters5869lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5870lb: must succeed: curl -sf localhost:9901/clusters5871lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5872lb: must succeed: curl -sf localhost:9901/clusters5873lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5874worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5875worker2 # [ 136.598449] systemd-journald[269]: Received client request to rotate journal, rotating.5876worker2 # [ 136.621992] systemd-journald[269]: Vacuuming done, freed 0B of archived journals from /var/log/journal/d2d4cc527b124966b5a15337f83eea32.5877worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5878worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5879worker1 # [ 136.592882] systemd[1]: Stopping Nix worker-protocol over gRPC...5880worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5881client: must succeed: sleep 5.05882worker1 # [ 136.966808] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:29Z level=info event=draining5883lb # [ 139.188737] envoy[895]: conn peer=[2001:db8:1::1]:54444 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9580 tx=4368 ms=57235884client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5885worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5886worker1 # [ 142.088493] nix-daemon[832]: reaped child process 1355, status = succeeded5887worker1 # [ 142.163517] nix-daemon[832]: accepted connection from pid 1396, user nix-grpc-daemon (trusted)5888worker1 # [ 142.183954] nix-daemon[832]: reaped child process 1399, status = succeeded5889lb # [ 142.158508] niks3-server[1111]: 2026/09/23 05:14:34 INFO Received uploads request method=POST path=/api/pending_closures5890worker1 # [ 142.202949] nix-grpc-daemon[1232]: time=2026-09-23T05:14:34.706Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5891worker1 # [ 142.204967] nix-grpc-daemon[1232]: time=2026-09-23T05:14:34.706Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5892lb # [ 142.200743] niks3-server[1111]: 2026/09/23 05:14:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5893worker1 # [ 142.242978] nix-grpc-daemon[1232]: time=2026-09-23T05:14:34.749Z level=INFO msg="Uploading 1 narinfos"5894lb # [ 142.215494] niks3-server[1111]: 2026/09/23 05:14:34 INFO Signed narinfos id=50 count=15895lb # [ 142.222856] niks3-server[1111]: 2026/09/23 05:14:34 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5896lb # [ 142.230466] niks3-server[1111]: 2026/09/23 05:14:34 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5897lb # [ 142.238756] niks3-server[1111]: 2026/09/23 05:14:34 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5898worker1 # [ 142.275831] nix-grpc-daemon[1232]: time=2026-09-23T05:14:34.782Z level=INFO msg="Upload complete. (279ms)"5899lb # [ 142.245112] niks3-server[1111]: 2026/09/23 05:14:34 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5900lb # [ 142.249506] niks3-server[1111]: 2026/09/23 05:14:34 INFO Completed upload id=505901lb # [ 142.253276] niks3-server[1111]: 2026/09/23 05:14:34 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5902worker1 # [ 142.292351] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45390 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15903worker1 # [ 142.309836] nix-daemon[832]: reaped child process 1353, status = succeeded5904client # [ 142.345970] systemd[1]: rs.service: Deactivated successfully.5905client # [ 142.351275] systemd[1]: rs.service: Consumed 196ms CPU time over 8.610s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.5906worker1 # [ 142.967871] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:35Z level=info event=signal_exit5907lb # [ 143.245012] niks3-server[1111]: 2026/09/23 05:14:35 INFO lead: released remote=192.168.1.3:605665908lb # [ 143.258676] niks3-server[1111]: 2026/09/23 05:14:35 INFO lead: acquired remote=192.168.1.4:487345909worker2 # [ 143.365984] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:35Z level=info event=scheduler_take_over5910worker2 # [ 144.340271] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:36Z level=info event=scheduler_restarting addr=lb:500515911worker1 # [ 144.276896] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:36Z level=info event=scheduler_restarting5912worker1 # [ 144.280823] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:36Z level=info event=scheduler_restarting addr=lb:500515913worker1 # [ 144.285391] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:492785914worker1 # [ 144.290306] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:36Z level=info event=worker_gone addr=192.168.1.3:500515915worker1 # [ 144.294710] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:492785916worker1 # [ 144.300192] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:36Z level=info event=worker_gone addr=192.168.1.4:500515917worker1 # [ 144.323964] nix-daemon[832]: reaped child process 1208, status = succeeded5918worker1 # [ 144.388206] nix-grpc-daemon[1155]: E0000 00:00:1790140476.894377 1201 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5919worker1 # [ 144.394830] nix-grpc-daemon[1155]: E0000 00:00:1790140476.894723 1201 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5920worker2 # [ 144.543034] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:36Z level=info event=scheduler_connected addr=lb:500515921worker2 # [ 144.548377] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5922worker1 # [ 144.493657] nix-grpc-daemon[1155]: ts=2026-09-23T05:14:36Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5923worker2 # [ 144.749622] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:37Z level=info event=scheduler_connected addr=lb:500515924worker2 # [ 144.767279] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:37Z level=info event=oidc_jwks provider=mock keys=15925worker2 # [ 144.771789] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:37Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:475665926worker2 # [ 144.777243] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:37Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05927worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.24 seconds)5928client: waiting for success: ! systemctl is-active rs5929client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5930client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5931client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5932client: must fail: journalctl -u rs | grep -q 'reconnecting'5933client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5934lb: must succeed: curl -sf localhost:9901/clusters5935lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5936worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5937worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5938worker1 # [ 145.522119] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5939worker1 # [ 145.526921] systemd[1]: Stopped Nix worker-protocol over gRPC.5940worker1 # [ 145.531491] systemd[1]: nix-grpc-daemon.service: Consumed 1.786s CPU time over 31.748s wall clock time, 38.8M memory peak, 47.9K incoming IP traffic, 96K outgoing IP traffic.5941worker1 # [ 145.552338] systemd[1]: Starting Nix worker-protocol over gRPC...5942worker1 # [ 145.698041] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:38Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5943worker1 # [ 145.700409] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:38Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5944worker1 # [ 145.704458] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:38Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15945worker1 # [ 145.711222] nix-grpc-daemon[1406]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5946worker1 # [ 145.728169] nix-grpc-daemon[1406]: E0000 00:00:1790140478.215560 1406 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5947worker1 # [ 145.730576] nix-grpc-daemon[1406]: E0000 00:00:1790140478.217706 1406 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5948worker1 # [ 145.754998] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:38Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5949worker1 # [ 145.757568] nix-grpc-daemon[1406]: E0000 00:00:1790140478.225844 1446 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5950worker1 # [ 145.761098] nix-grpc-daemon[1406]: E0000 00:00:1790140478.246180 1446 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5951worker1 # [ 145.766500] nix-grpc-daemon[1406]: E0000 00:00:1790140478.246337 1446 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5952worker1 # [ 145.771035] nix-grpc-daemon[1406]: E0000 00:00:1790140478.260762 1446 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5953worker1 # [ 145.773718] systemd[1]: Started Nix worker-protocol over gRPC.5954worker1 # [ 145.774563] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:38Z level=info event=scheduler_connected addr=lb:500515955worker2 # [ 145.842663] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:38Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:475665956worker2 # [ 145.848477] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:38Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05957lb: must succeed: curl -sf localhost:9901/clusters5958lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5959worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5960worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5961(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.91 seconds)5962subtest: scheduler restart mid-build: build finishes, no double build5963client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5964client # [ 146.644817] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5965client # Running as unit: mid.service; invocation ID: d68625aa1e1b4250a53f1101f4f0e1ba5966client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)5967worker2 # [ 146.897051] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:560745968client # [ 146.855205] nix[1454]: don't know how to build these paths:5969client # [ 146.856342] nix[1454]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5970worker2 # [ 146.917525] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:39Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05971worker2 # [ 146.924631] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:39Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15972worker2 # [ 146.961787] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560745973worker2 # [ 146.978130] nix-grpc-daemon[1528]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 1/5)5974worker2 # [ 147.007328] nix-grpc-daemon[1528]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 2/5)5975worker2 # [ 147.104108] nix-grpc-daemon[1528]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 310 ms (attempt 3/5)5976worker2 # [ 147.416095] nix-grpc-daemon[1528]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 644 ms (attempt 4/5)5977worker2 # [ 148.064784] nix-grpc-daemon[1528]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5978worker2 # [ 148.068647] nix-grpc-daemon[1528]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5979worker2 # [ 148.082668] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:560745980client # [ 148.053566] nix[1454]: copying 1 paths to 'lb:50051'...5981worker2 # [ 148.103685] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:560745982client # [ 148.056696] nix[1454]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5983worker2 # [ 148.122123] nix-daemon[855]: accepted connection from pid 1528, user nix-grpc-daemon (trusted)5984worker2 # [ 148.148104] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:40Z level=info event=niks3_push_started pid=16615985worker2 # [ 148.298881] nix-grpc-daemon[1661]: time=2026-09-23T05:14:40.751Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5986worker2 # [ 148.404456] nix-daemon[855]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)5987worker2 # [ 148.422836] nix-daemon[855]: reaped child process 1676, status = succeeded5988lb # [ 148.326880] niks3-server[1111]: 2026/09/23 05:14:40 INFO Received uploads request method=POST path=/api/pending_closures5989worker2 # [ 148.438945] nix-grpc-daemon[1661]: time=2026-09-23T05:14:40.891Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5990worker2 # [ 148.442839] nix-grpc-daemon[1661]: time=2026-09-23T05:14:40.891Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5991lb # [ 148.374195] niks3-server[1111]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5992worker2 # [ 148.485690] nix-grpc-daemon[1661]: time=2026-09-23T05:14:40.938Z level=INFO msg="Uploading 1 narinfos"5993lb # [ 148.385837] niks3-server[1111]: 2026/09/23 05:14:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5994lb # [ 148.395241] niks3-server[1111]: 2026/09/23 05:14:40 INFO Signed narinfos id=51 count=15995lb # [ 148.400988] niks3-server[1111]: 2026/09/23 05:14:40 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5996lb # [ 148.412731] niks3-server[1111]: 2026/09/23 05:14:40 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5997worker2 # [ 148.520676] nix-grpc-daemon[1661]: time=2026-09-23T05:14:40.973Z level=INFO msg="Upload complete. (218ms)"5998lb # [ 148.421436] niks3-server[1111]: 2026/09/23 05:14:40 INFO Completed upload id=515999worker2 # [ 148.528137] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56074 duration_s=0 paths=1 nar_bytes_in=4246000worker2 # [ 148.540594] nix-grpc-daemon[1528]: ts=2026-09-23T05:14:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560746001worker2 # [ 148.551520] nix-daemon[855]: accepted connection from pid 1528, user nix-grpc-daemon (trusted)6002worker2 # [ 148.556897] nix-daemon[855]: reaped child process 1659, status = succeeded6003worker2 # [ 148.568151] nix-daemon[855]: accepted connection from pid 1528, user nix-grpc-daemon (trusted)6004client # [ 148.516548] nix[1454]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6005lb # [ 149.190048] envoy[895]: conn peer=[2001:db8:1::1]:52254 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=1517 ms=83926006lb # [ 149.195284] envoy[895]: conn peer=[2001:db8:1::3]:56932 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2852 tx=977 ms=304756007lb: must succeed: curl -sf localhost:9901/clusters6008lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6009lb: must succeed: curl -sf localhost:9901/clusters6010lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6011worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6012worker2 # [ 150.572439] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1528 (nix-grpc-daemon) on client request.6013worker2 # [ 150.574049] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6014lb # [ 150.491675] niks3-server[1111]: 2026/09/23 05:14:42 INFO lead: released remote=192.168.1.4:487346015worker2 # [ 150.597148] systemd[1]: Starting scheduler-token.service...6016client # [ 150.544926] nix[1454]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6017worker1 # [ 150.524732] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6018client # [ 150.551282] nix[1454]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6019worker2 # [ 150.608452] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6020worker2 # [ 150.609943] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6021worker2 # [ 150.611285] systemd[1]: nix-grpc-daemon.service: Consumed 496ms CPU time over 17.391s wall clock time, 34.7M memory peak, 15.3K incoming IP traffic, 25.6K outgoing IP traffic.6022worker2 # [ 150.655478] nix-daemon[855]: reaped child process 1678, status = succeeded6023worker2 # [ 150.657596] nix-daemon[855]: reaped child process 1588, status = succeeded6024worker2 # [ 150.664500] nix-daemon[1680]: unexpected Nix daemon error: error: interrupted by the user6025worker2 # [ 150.676600] nix-daemon[855]: reaped child process 1680, status = failed with exit code 16026worker2 # [ 150.693441] systemd[1]: scheduler-token.service: Deactivated successfully.6027worker2 # [ 150.694679] systemd[1]: Finished scheduler-token.service.6028worker2 # [ 150.695797] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 93ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6029worker2 # [ 150.699729] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6030worker2 # [ 150.704260] systemd[1]: Starting scheduler-token.service...6031worker2 # [ 150.745292] systemd[1]: scheduler-token.service: Deactivated successfully.6032worker2 # [ 150.748139] systemd[1]: Finished scheduler-token.service.6033worker2 # [ 150.748961] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 47ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6034worker2 # [ 150.753294] systemd[1]: Starting Nix worker-protocol over gRPC...6035lb # [ 150.689538] niks3-server[1111]: 2026/09/23 05:14:43 INFO lead: acquired remote=192.168.1.3:405666036worker1 # [ 150.724728] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:43Z level=info event=scheduler_connected addr=lb:500516037worker1 # [ 150.730096] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:43Z level=info event=scheduler_take_over6038worker2 # [ 150.853751] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:43Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6039worker2 # [ 150.855983] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6040worker2 # [ 150.859803] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16041worker2 # [ 150.865275] nix-grpc-daemon[1723]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6042worker2 # [ 150.866716] nix-grpc-daemon[1723]: E0000 00:00:1790140483.317737 1723 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6043worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.36 seconds)6044client: must succeed: sleep 5.06045worker1 # [ 150.820566] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6046worker2 # [ 150.904153] nix-grpc-daemon[1723]: E0000 00:00:1790140483.323613 1723 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6047worker2 # [ 150.906389] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6048worker2 # [ 150.907977] nix-grpc-daemon[1723]: E0000 00:00:1790140483.338806 1765 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6049worker2 # [ 150.926371] systemd[1]: Started Nix worker-protocol over gRPC.6050worker2 # [ 150.927642] nix-grpc-daemon[1723]: E0000 00:00:1790140483.364352 1765 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6051worker2 # [ 150.930247] nix-grpc-daemon[1723]: E0000 00:00:1790140483.364484 1765 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6052worker2 # [ 150.935167] nix-grpc-daemon[1723]: E0000 00:00:1790140483.378939 1765 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6053worker2 # [ 150.939848] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:43Z level=info event=scheduler_connected addr=lb:500516054worker2 # [ 150.941262] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6055worker1 # [ 151.021487] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:43Z level=info event=scheduler_connected addr=lb:500516056client # [ 151.047903] nix[1454]: scheduler at lb:50051: no healthy upstream, reconnecting6057worker1 # [ 151.026826] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6058worker2 # [ 151.141392] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:43Z level=info event=scheduler_connected addr=lb:500516059worker2 # [ 151.146266] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6060worker1 # [ 151.227674] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:43Z level=info event=scheduler_connected addr=lb:500516061worker1 # [ 151.232991] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6062worker2 # [ 151.347274] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:43Z level=info event=scheduler_connected addr=lb:500516063worker2 # [ 151.352421] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6064worker1 # [ 151.433712] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:43Z level=info event=scheduler_connected addr=lb:500516065worker1 # [ 151.438863] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6066worker2 # [ 151.553317] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500516067worker2 # [ 151.558332] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6068worker1 # [ 151.641370] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500516069worker1 # [ 151.645549] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6070worker2 # [ 151.759315] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500516071worker2 # [ 151.764448] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6072worker1 # [ 151.841879] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500516073worker1 # [ 151.847087] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6074worker2 # [ 151.967249] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500516075worker2 # [ 151.971408] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6076client # [ 152.051994] nix[1454]: scheduler at lb:50051: no healthy upstream, reconnecting6077worker1 # [ 152.047878] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500516078worker1 # [ 152.053176] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6079worker2 # [ 152.167923] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500516080worker2 # [ 152.173009] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6081worker1 # [ 152.253743] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500516082worker1 # [ 152.258852] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6083worker2 # [ 152.374023] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500516084worker2 # [ 152.379087] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6085worker1 # [ 152.459728] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:44Z level=info event=scheduler_connected addr=lb:500516086worker1 # [ 152.465058] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6087worker2 # [ 152.579873] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:45Z level=info event=scheduler_connected addr=lb:500516088worker2 # [ 152.585044] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6089worker1 # [ 152.666995] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:45Z level=info event=scheduler_connected addr=lb:500516090worker1 # [ 152.686527] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:383766091worker1 # [ 152.692827] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06092worker2 # [ 152.785849] nix-grpc-daemon[1723]: ts=2026-09-23T05:14:45Z level=info event=scheduler_connected addr=lb:500516093worker1 # [ 152.725787] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:45Z level=info event=oidc_jwks provider=mock keys=16094worker1 # [ 152.730250] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:45Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:383766095worker1 # [ 152.735792] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:45Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06096worker1 # [ 154.044515] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:46Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06097worker1 # [ 154.049911] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:46Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=16098worker1 # [ 154.101948] nix-grpc-daemon[1406]: ts=2026-09-23T05:14:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:383806099worker1 # [ 154.118522] nix-daemon[832]: accepted connection from pid 1406, user nix-grpc-daemon (trusted)6100worker1 # [ 154.141702] nix-grpc-daemon[1406]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 1/5)6101worker1 # [ 154.166830] nix-grpc-daemon[1406]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 2/5)6102worker1 # [ 154.208575] nix-grpc-daemon[1406]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 372 ms (attempt 3/5)6103worker1 # [ 154.582583] nix-grpc-daemon[1406]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 185 ms (attempt 4/5)6104worker1 # [ 154.770644] nix-grpc-daemon[1406]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6105worker1 # [ 154.776434] nix-grpc-daemon[1406]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6106worker1 # [ 154.793530] nix-grpc-daemon[1406]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...6107worker1 # [ 154.821302] nix-daemon[832]: accepted connection from pid 1406, user nix-grpc-daemon (trusted)6108client # [ 154.859599] nix[1454]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6109worker1 # [ 154.843292] nix-daemon[832]: accepted connection from pid 1406, user nix-grpc-daemon (trusted)6110client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6111client: waiting for success: ! systemctl is-active mid6112lb # [ 159.190362] envoy[895]: conn peer=[2001:db8:1::4]:47436 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2696 tx=393 ms=171896113worker1 # [ 246.342152] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:18Z level=info event=niks3_push_started pid=15156114worker1 # [ 246.449189] nix-daemon[832]: reaped child process 1490, status = succeeded6115worker1 # [ 246.522976] nix-grpc-daemon[1515]: time=2026-09-23T05:16:19.029Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6116worker1 # [ 246.635918] nix-daemon[832]: accepted connection from pid 1527, user nix-grpc-daemon (trusted)6117worker1 # [ 246.653066] nix-daemon[832]: reaped child process 1530, status = succeeded6118lb # [ 246.629887] niks3-server[1111]: 2026/09/23 05:16:19 INFO Received uploads request method=POST path=/api/pending_closures6119worker1 # [ 246.681645] nix-grpc-daemon[1515]: time=2026-09-23T05:16:19.186Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6120worker1 # [ 246.683133] nix-grpc-daemon[1515]: time=2026-09-23T05:16:19.186Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6121lb # [ 246.707273] niks3-server[1111]: 2026/09/23 05:16:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6122worker1 # [ 246.748140] nix-grpc-daemon[1515]: time=2026-09-23T05:16:19.253Z level=INFO msg="Uploading 1 narinfos"6123lb # [ 246.720194] niks3-server[1111]: 2026/09/23 05:16:19 INFO Signed narinfos id=52 count=16124lb # [ 246.731946] niks3-server[1111]: 2026/09/23 05:16:19 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6125lb # [ 246.739771] niks3-server[1111]: 2026/09/23 05:16:19 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6126lb # [ 246.745967] niks3-server[1111]: 2026/09/23 05:16:19 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6127lb # [ 246.753825] niks3-server[1111]: 2026/09/23 05:16:19 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6128lb # [ 246.759481] niks3-server[1111]: 2026/09/23 05:16:19 INFO Completed upload id=526129worker1 # [ 246.791333] nix-grpc-daemon[1515]: time=2026-09-23T05:16:19.297Z level=INFO msg="Upload complete. (264ms)"6130lb # [ 246.761854] niks3-server[1111]: 2026/09/23 05:16:19 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo6131worker1 # [ 246.808416] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38380 duration_s=92 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16132worker1 # [ 246.829274] nix-daemon[832]: reaped child process 1488, status = succeeded6133client # [ 246.874874] systemd[1]: mid.service: Deactivated successfully.6134client # [ 246.880801] systemd[1]: mid.service: Consumed 285ms CPU time over 1min 40.235s wall clock time, 41.2M memory peak, 4K written to disk, 10.8K incoming IP traffic, 12.2K outgoing IP traffic.6135client: (finished: waiting for success: ! systemctl is-active mid, in 91.46 seconds)6136client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6137client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)6138(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 101.13 seconds)6139subtest: interrupting the client stops the build on the worker6140client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr6141client # [ 247.744973] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.6142client # Running as unit: intr.service; invocation ID: bfe621e9eafc4e99afa39235b41c06e16143client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)6144client # [ 247.932755] nix[1915]: don't know how to build these paths:6145client # [ 247.933753] nix[1915]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6146worker1 # [ 247.912370] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:383806147worker1 # [ 247.927388] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:20Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06148worker1 # [ 247.933489] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:20Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=16149worker2 # [ 248.061797] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473326150worker2 # [ 248.081104] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6151worker2 # [ 248.111572] nix-grpc-daemon[1723]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 67 ms (attempt 1/5)6152worker2 # [ 248.186564] nix-grpc-daemon[1723]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 2/5)6153worker2 # [ 248.211578] nix-grpc-daemon[1723]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 333 ms (attempt 3/5)6154worker2 # [ 248.603801] nix-grpc-daemon[1723]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 561 ms (attempt 4/5)6155worker2 # [ 249.169706] nix-grpc-daemon[1723]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6156worker2 # [ 249.176879] nix-grpc-daemon[1723]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6157worker2 # [ 249.195774] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:473326158client # [ 249.162245] nix[1915]: copying 1 paths to 'lb:50051'...6159client # [ 249.166961] nix[1915]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6160worker2 # [ 249.217234] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:473326161worker2 # [ 249.234230] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6162worker2 # [ 249.260822] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:21Z level=info event=niks3_push_started pid=18056163lb # [ 249.198078] envoy[895]: conn peer=[2001:db8:1::1]:45116 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4480 tx=2190 ms=999946164worker2 # [ 249.434091] nix-grpc-daemon[1805]: time=2026-09-23T05:16:21.884Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6165worker2 # [ 249.539391] nix-daemon[855]: accepted connection from pid 1818, user nix-grpc-daemon (trusted)6166worker2 # [ 249.557375] nix-daemon[855]: reaped child process 1821, status = succeeded6167lb # [ 249.462465] niks3-server[1111]: 2026/09/23 05:16:21 INFO Received uploads request method=POST path=/api/pending_closures6168worker2 # [ 249.576067] nix-grpc-daemon[1805]: time=2026-09-23T05:16:22.028Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6169worker2 # [ 249.577542] nix-grpc-daemon[1805]: time=2026-09-23T05:16:22.028Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6170lb # [ 249.504409] niks3-server[1111]: 2026/09/23 05:16:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6171worker2 # [ 249.622365] nix-grpc-daemon[1805]: time=2026-09-23T05:16:22.074Z level=INFO msg="Uploading 1 narinfos"6172lb # [ 249.520383] niks3-server[1111]: 2026/09/23 05:16:22 INFO Signed narinfos id=53 count=16173lb # [ 249.529936] niks3-server[1111]: 2026/09/23 05:16:22 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6174lb # [ 249.548781] niks3-server[1111]: 2026/09/23 05:16:22 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6175worker2 # [ 249.663706] nix-grpc-daemon[1805]: time=2026-09-23T05:16:22.116Z level=INFO msg="Upload complete. (230ms)"6176lb # [ 249.562608] niks3-server[1111]: 2026/09/23 05:16:22 INFO Completed upload id=536177lb # [ 249.570838] niks3-server[1111]: 2026/09/23 05:16:22 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6178worker2 # [ 249.673499] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=0 paths=1 nar_bytes_in=4246179lb # [ 249.578467] niks3-server[1111]: 2026/09/23 05:16:22 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6180worker2 # [ 249.689723] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473326181worker2 # [ 249.696265] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6182worker2 # [ 249.701397] nix-daemon[855]: reaped child process 1803, status = succeeded6183worker2 # [ 249.713499] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6184client # [ 249.662024] nix[1915]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6185client: must succeed: systemctl kill -s INT intr6186client # [ 251.405432] systemd[1]: intr.service: Sent signal SIGINT to main process 1915 (nix) on client request.6187worker2 # [ 251.465512] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:23Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6188client # [ 251.416741] nix[1915]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6189client # [ 251.428374] nix[1915]: error: interrupted by the user6190client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)6191client # [ 251.450840] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6192client # [ 251.455334] systemd[1]: intr.service: Failed with result 'exit-code'.6193client # [ 251.457361] systemd[1]: intr.service: Consumed 182ms CPU time over 3.708s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6194worker2 # [ 251.540599] nix-daemon[1825]: unexpected Nix daemon error: error: interrupted by the user6195worker2 # [ 251.550127] nix-daemon[855]: reaped child process 1823, status = succeeded6196worker2 # [ 251.551479] nix-daemon[855]: reaped child process 1825, status = failed with exit code 16197(finished: subtest: interrupting the client stops the build on the worker, in 3.82 seconds)6198subtest: a deploy mid-build drains: the build finishes, the new generation takes over6199client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6200client # [ 251.593969] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6201client # Running as unit: sw.service; invocation ID: ae3a99780f77415c8aaaca5bfe15a8986202client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds)6203client # [ 251.793795] nix[1936]: don't know how to build these paths:6204client # [ 251.794929] nix[1936]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6205worker1 # [ 251.773119] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:383806206worker1 # [ 251.790793] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:24Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06207worker1 # [ 251.797931] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:24Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=16208worker1 # [ 251.836122] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:383806209worker1 # [ 251.856640] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:383806210client # [ 251.891183] nix[1936]: copying 1 paths to 'lb:50051'...6211worker1 # [ 251.871211] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:383806212client # [ 251.894526] nix[1936]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6213worker1 # [ 251.886267] nix-daemon[832]: accepted connection from pid 1406, user nix-grpc-daemon (trusted)6214worker1 # [ 252.040187] nix-daemon[832]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)6215worker1 # [ 252.058916] nix-daemon[832]: reaped child process 1567, status = succeeded6216lb # [ 252.031782] niks3-server[1111]: 2026/09/23 05:16:24 INFO Received uploads request method=POST path=/api/pending_closures6217worker1 # [ 252.078863] nix-grpc-daemon[1515]: time=2026-09-23T05:16:24.584Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6218worker1 # [ 252.080466] nix-grpc-daemon[1515]: time=2026-09-23T05:16:24.584Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6219lb # [ 252.066627] niks3-server[1111]: 2026/09/23 05:16:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6220worker1 # [ 252.109078] nix-grpc-daemon[1515]: time=2026-09-23T05:16:24.615Z level=INFO msg="Uploading 1 narinfos"6221lb # [ 252.084169] niks3-server[1111]: 2026/09/23 05:16:24 INFO Signed narinfos id=54 count=16222lb # [ 252.086841] niks3-server[1111]: 2026/09/23 05:16:24 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6223lb # [ 252.104610] niks3-server[1111]: 2026/09/23 05:16:24 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6224lb # [ 252.108604] niks3-server[1111]: 2026/09/23 05:16:24 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6225worker1 # [ 252.144510] nix-grpc-daemon[1515]: time=2026-09-23T05:16:24.651Z level=INFO msg="Upload complete. (239ms)"6226lb # [ 252.114743] niks3-server[1111]: 2026/09/23 05:16:24 INFO Completed upload id=546227lb # [ 252.116834] niks3-server[1111]: 2026/09/23 05:16:24 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6228worker1 # [ 252.156146] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38380 duration_s=0 paths=1 nar_bytes_in=4166229worker1 # [ 252.171487] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:383806230worker1 # [ 252.178008] nix-daemon[832]: accepted connection from pid 1406, user nix-grpc-daemon (trusted)6231worker1 # [ 252.182926] nix-daemon[832]: reaped child process 1562, status = succeeded6232worker1 # [ 252.194868] nix-daemon[832]: accepted connection from pid 1406, user nix-grpc-daemon (trusted)6233client # [ 252.213740] nix[1936]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6234worker1: must succeed: systemctl reload nix-grpc-daemon.service6235worker1 # [ 254.172336] systemd[1]: Reloading Nix worker-protocol over gRPC...6236worker1 # [ 254.208999] systemd[1]: Reloaded Nix worker-protocol over gRPC.6237worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)6238worker1: must succeed: pgrep -f 'read -t [9]0 x'6239worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6240client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26241worker2 # [ 254.561753] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:473326242client # don't know how to build these paths:6243client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6244worker2 # [ 254.573505] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:473326245worker1 # [ 254.527706] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:27Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06246worker1 # [ 254.529774] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:27Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=16247worker2 # [ 254.644588] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473326248worker2 # [ 254.659309] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:473326249client # copying 1 paths to 'lb:50051'...6250worker2 # [ 254.675138] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:473326251client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6252worker2 # [ 254.693299] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6253worker1 # [ 254.755609] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:27Z level=info event=draining6254worker2 # [ 254.854824] nix-daemon[855]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)6255worker2 # [ 254.874122] nix-daemon[855]: reaped child process 1875, status = succeeded6256lb # [ 254.778951] niks3-server[1111]: 2026/09/23 05:16:27 INFO Received uploads request method=POST path=/api/pending_closures6257worker2 # [ 254.890804] nix-grpc-daemon[1805]: time=2026-09-23T05:16:27.343Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6258worker2 # [ 254.894185] nix-grpc-daemon[1805]: time=2026-09-23T05:16:27.346Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6259lb # [ 254.813496] niks3-server[1111]: 2026/09/23 05:16:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6260worker2 # [ 254.923645] nix-grpc-daemon[1805]: time=2026-09-23T05:16:27.376Z level=INFO msg="Uploading 1 narinfos"6261lb # [ 254.824886] niks3-server[1111]: 2026/09/23 05:16:27 INFO Signed narinfos id=55 count=16262lb # [ 254.836799] niks3-server[1111]: 2026/09/23 05:16:27 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6263lb # [ 254.848660] niks3-server[1111]: 2026/09/23 05:16:27 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6264lb # [ 254.863147] niks3-server[1111]: 2026/09/23 05:16:27 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6265lb # [ 254.873225] niks3-server[1111]: 2026/09/23 05:16:27 INFO Completed upload id=556266lb # [ 254.875392] niks3-server[1111]: 2026/09/23 05:16:27 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6267worker2 # [ 254.972310] nix-grpc-daemon[1805]: time=2026-09-23T05:16:27.425Z level=INFO msg="Upload complete. (261ms)"6268worker2 # [ 254.984575] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=0 paths=1 nar_bytes_in=5286269worker2 # [ 255.000523] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473326270worker2 # [ 255.013987] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6271worker2 # [ 255.018239] nix-daemon[855]: reaped child process 1870, status = succeeded6272client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6273worker2 # [ 255.032701] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6274client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6275client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6276client # farm-a> LOG-farm-a-during-drain6277worker2 # [ 256.106361] nix-daemon[855]: reaped child process 1879, status = succeeded6278worker2 # [ 256.218997] nix-daemon[855]: accepted connection from pid 1895, user nix-grpc-daemon (trusted)6279worker2 # [ 256.239316] nix-daemon[855]: reaped child process 1898, status = succeeded6280lb # [ 256.142436] niks3-server[1111]: 2026/09/23 05:16:28 INFO Received uploads request method=POST path=/api/pending_closures6281worker2 # [ 256.255776] nix-grpc-daemon[1805]: time=2026-09-23T05:16:28.706Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6282worker2 # [ 256.258954] nix-grpc-daemon[1805]: time=2026-09-23T05:16:28.706Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6283lb # [ 256.185491] niks3-server[1111]: 2026/09/23 05:16:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6284worker2 # [ 256.295140] nix-grpc-daemon[1805]: time=2026-09-23T05:16:28.748Z level=INFO msg="Uploading 1 narinfos"6285lb # [ 256.199315] niks3-server[1111]: 2026/09/23 05:16:28 INFO Signed narinfos id=56 count=16286lb # [ 256.206986] niks3-server[1111]: 2026/09/23 05:16:28 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6287lb # [ 256.214970] niks3-server[1111]: 2026/09/23 05:16:28 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6288worker2 # [ 256.327623] nix-grpc-daemon[1805]: time=2026-09-23T05:16:28.780Z level=INFO msg="Upload complete. (278ms)"6289lb # [ 256.226867] niks3-server[1111]: 2026/09/23 05:16:28 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6290lb # [ 256.234579] niks3-server[1111]: 2026/09/23 05:16:28 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6291lb # [ 256.239299] niks3-server[1111]: 2026/09/23 05:16:28 INFO Completed upload id=566292worker1 # [ 256.276822] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:28Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06293worker2 # [ 256.341226] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16294worker1 # [ 256.278666] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:28Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16295lb # [ 256.243953] niks3-server[1111]: 2026/09/23 05:16:28 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6296worker2 # [ 256.359055] nix-daemon[855]: reaped child process 1877, status = succeeded6297worker2 # [ 256.401020] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473326298worker2 # [ 256.415382] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:473326299worker2 # [ 256.429641] nix-grpc-daemon[1805]: time=2026-09-23T05:16:28.882Z level=INFO msg="All 1 paths already cached"6300client # copying 1 paths to 'lb:50051'...6301client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6302worker2 # [ 256.440454] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:473326303worker2 # [ 256.457172] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6304worker2 # [ 256.614283] nix-daemon[855]: accepted connection from pid 1902, user nix-grpc-daemon (trusted)6305worker2 # [ 256.632983] nix-daemon[855]: reaped child process 1905, status = succeeded6306lb # [ 256.536559] niks3-server[1111]: 2026/09/23 05:16:29 INFO Received uploads request method=POST path=/api/pending_closures6307worker2 # [ 256.647098] nix-grpc-daemon[1805]: time=2026-09-23T05:16:29.099Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6308worker2 # [ 256.648776] nix-grpc-daemon[1805]: time=2026-09-23T05:16:29.100Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6309worker2 # [ 256.673613] nix-grpc-daemon[1805]: time=2026-09-23T05:16:29.126Z level=INFO msg="Uploading 1 narinfos"6310lb # [ 256.565456] niks3-server[1111]: 2026/09/23 05:16:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6311lb # [ 256.579169] niks3-server[1111]: 2026/09/23 05:16:29 INFO Signed narinfos id=57 count=16312lb # [ 256.588699] niks3-server[1111]: 2026/09/23 05:16:29 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6313lb # [ 256.602553] niks3-server[1111]: 2026/09/23 05:16:29 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6314lb # [ 256.607310] niks3-server[1111]: 2026/09/23 05:16:29 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6315worker2 # [ 256.717549] nix-grpc-daemon[1805]: time=2026-09-23T05:16:29.170Z level=INFO msg="Upload complete. (243ms)"6316lb # [ 256.616163] niks3-server[1111]: 2026/09/23 05:16:29 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6317lb # [ 256.624076] niks3-server[1111]: 2026/09/23 05:16:29 INFO Completed upload id=576318worker2 # [ 256.728299] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=0 paths=1 nar_bytes_in=6406319worker2 # [ 256.742522] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473326320worker2 # [ 256.752529] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6321worker2 # [ 256.755968] nix-daemon[855]: reaped child process 1900, status = succeeded6322client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6323worker2 # [ 256.770452] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6324client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6325client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6326client # farm-b> LOG-farm-b-during-drain6327worker2 # [ 258.064405] nix-daemon[855]: reaped child process 1909, status = succeeded6328worker2 # [ 258.153959] nix-daemon[855]: accepted connection from pid 1925, user nix-grpc-daemon (trusted)6329worker2 # [ 258.171682] nix-daemon[855]: reaped child process 1928, status = succeeded6330lb # [ 258.077863] niks3-server[1111]: 2026/09/23 05:16:30 INFO Received uploads request method=POST path=/api/pending_closures6331worker2 # [ 258.190294] nix-grpc-daemon[1805]: time=2026-09-23T05:16:30.642Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6332worker2 # [ 258.191847] nix-grpc-daemon[1805]: time=2026-09-23T05:16:30.642Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6333lb # [ 258.116453] niks3-server[1111]: 2026/09/23 05:16:30 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6334lb # [ 258.128186] niks3-server[1111]: 2026/09/23 05:16:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6335worker2 # [ 258.234881] nix-grpc-daemon[1805]: time=2026-09-23T05:16:30.687Z level=INFO msg="Uploading 1 narinfos"6336lb # [ 258.140713] niks3-server[1111]: 2026/09/23 05:16:30 INFO Signed narinfos id=58 count=16337lb # [ 258.143077] niks3-server[1111]: 2026/09/23 05:16:30 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6338lb # [ 258.153759] niks3-server[1111]: 2026/09/23 05:16:30 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6339lb # [ 258.161986] niks3-server[1111]: 2026/09/23 05:16:30 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6340worker2 # [ 258.263986] nix-grpc-daemon[1805]: time=2026-09-23T05:16:30.717Z level=INFO msg="Upload complete. (278ms)"6341lb # [ 258.165864] niks3-server[1111]: 2026/09/23 05:16:30 INFO Completed upload id=586342lb # [ 258.167697] niks3-server[1111]: 2026/09/23 05:16:30 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6343worker1 # [ 258.208994] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:30Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06344worker1 # [ 258.210896] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:30Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16345worker2 # [ 258.277066] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16346worker2 # [ 258.296248] nix-daemon[855]: reaped child process 1907, status = succeeded6347worker2 # [ 258.326069] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473326348worker2 # [ 258.341141] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:473326349worker2 # [ 258.355500] nix-grpc-daemon[1805]: time=2026-09-23T05:16:30.808Z level=INFO msg="All 2 paths already cached"6350client # copying 1 paths to 'lb:50051'...6351client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6352worker2 # [ 258.366672] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:473326353worker2 # [ 258.380962] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6354worker2 # [ 258.532709] nix-daemon[855]: accepted connection from pid 1932, user nix-grpc-daemon (trusted)6355worker2 # [ 258.548757] nix-daemon[855]: reaped child process 1935, status = succeeded6356lb # [ 258.454765] niks3-server[1111]: 2026/09/23 05:16:30 INFO Received uploads request method=POST path=/api/pending_closures6357worker2 # [ 258.565437] nix-grpc-daemon[1805]: time=2026-09-23T05:16:31.018Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6358worker2 # [ 258.566991] nix-grpc-daemon[1805]: time=2026-09-23T05:16:31.018Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6359lb # [ 258.484251] niks3-server[1111]: 2026/09/23 05:16:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6360worker2 # [ 258.594729] nix-grpc-daemon[1805]: time=2026-09-23T05:16:31.047Z level=INFO msg="Uploading 1 narinfos"6361lb # [ 258.500614] niks3-server[1111]: 2026/09/23 05:16:31 INFO Signed narinfos id=59 count=16362lb # [ 258.503564] niks3-server[1111]: 2026/09/23 05:16:31 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6363lb # [ 258.519761] niks3-server[1111]: 2026/09/23 05:16:31 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6364lb # [ 258.525878] niks3-server[1111]: 2026/09/23 05:16:31 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6365worker2 # [ 258.621604] nix-grpc-daemon[1805]: time=2026-09-23T05:16:31.074Z level=INFO msg="Upload complete. (223ms)"6366lb # [ 258.531677] niks3-server[1111]: 2026/09/23 05:16:31 INFO Completed upload id=596367worker2 # [ 258.632855] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=0 paths=1 nar_bytes_in=7686368lb # [ 258.534252] niks3-server[1111]: 2026/09/23 05:16:31 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6369worker2 # [ 258.642611] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473326370worker2 # [ 258.649106] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6371worker2 # [ 258.653976] nix-daemon[855]: reaped child process 1930, status = succeeded6372client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6373client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6374worker2 # [ 258.667294] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6375client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6376lb # [ 259.199581] envoy[895]: conn peer=[2001:db8:1::1]:45824 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=851 ms=35136377client # farm-top> LOG-farm-top-during-drain6378worker2 # [ 259.752565] nix-daemon[855]: reaped child process 1939, status = succeeded6379worker2 # [ 259.849631] nix-daemon[855]: accepted connection from pid 1955, user nix-grpc-daemon (trusted)6380worker2 # [ 259.867884] nix-daemon[855]: reaped child process 1958, status = succeeded6381lb # [ 259.773996] niks3-server[1111]: 2026/09/23 05:16:32 INFO Received uploads request method=POST path=/api/pending_closures6382worker2 # [ 259.887101] nix-grpc-daemon[1805]: time=2026-09-23T05:16:32.338Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6383worker2 # [ 259.889041] nix-grpc-daemon[1805]: time=2026-09-23T05:16:32.338Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6384lb # [ 259.816947] niks3-server[1111]: 2026/09/23 05:16:32 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6385worker2 # [ 259.928742] nix-grpc-daemon[1805]: time=2026-09-23T05:16:32.381Z level=INFO msg="Uploading 1 narinfos"6386lb # [ 259.830541] niks3-server[1111]: 2026/09/23 05:16:32 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6387lb # [ 259.839746] niks3-server[1111]: 2026/09/23 05:16:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6388lb # [ 259.850134] niks3-server[1111]: 2026/09/23 05:16:32 INFO Signed narinfos id=60 count=16389worker2 # [ 259.952081] nix-grpc-daemon[1805]: time=2026-09-23T05:16:32.405Z level=INFO msg="Upload complete. (273ms)"6390lb # [ 259.853381] niks3-server[1111]: 2026/09/23 05:16:32 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6391lb # [ 259.858070] niks3-server[1111]: 2026/09/23 05:16:32 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6392lb # [ 259.862394] niks3-server[1111]: 2026/09/23 05:16:32 INFO Completed upload id=606393lb # [ 259.864503] niks3-server[1111]: 2026/09/23 05:16:32 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6394worker2 # [ 259.965159] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16395worker2 # [ 259.982396] nix-daemon[855]: reaped child process 1937, status = succeeded6396client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.68 seconds)6397client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6398client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.17 seconds)6399worker1: must succeed: pkill -f 'read -t [9]0 x'6400worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6401client: waiting for success: ! systemctl is-active sw6402worker1 # [ 260.227506] nix-daemon[832]: reaped child process 1571, status = succeeded6403worker1 # [ 260.322059] nix-daemon[832]: accepted connection from pid 1618, user nix-grpc-daemon (trusted)6404worker1 # [ 260.338593] nix-daemon[832]: reaped child process 1621, status = succeeded6405lb # [ 260.314176] niks3-server[1111]: 2026/09/23 05:16:32 INFO Received uploads request method=POST path=/api/pending_closures6406worker1 # [ 260.356053] nix-grpc-daemon[1515]: time=2026-09-23T05:16:32.861Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6407worker1 # [ 260.357517] nix-grpc-daemon[1515]: time=2026-09-23T05:16:32.861Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6408worker1 # [ 260.389697] nix-grpc-daemon[1515]: time=2026-09-23T05:16:32.896Z level=INFO msg="Uploading 1 narinfos"6409lb # [ 260.351905] niks3-server[1111]: 2026/09/23 05:16:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6410lb # [ 260.367251] niks3-server[1111]: 2026/09/23 05:16:32 INFO Signed narinfos id=61 count=16411lb # [ 260.373541] niks3-server[1111]: 2026/09/23 05:16:32 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6412lb # [ 260.387005] niks3-server[1111]: 2026/09/23 05:16:32 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6413worker1 # [ 260.427255] nix-grpc-daemon[1515]: time=2026-09-23T05:16:32.933Z level=INFO msg="Upload complete. (213ms)"6414lb # [ 260.397132] niks3-server[1111]: 2026/09/23 05:16:32 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6415lb # [ 260.404137] niks3-server[1111]: 2026/09/23 05:16:32 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6416worker1 # [ 260.436750] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38380 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16417lb # [ 260.409465] niks3-server[1111]: 2026/09/23 05:16:32 INFO Completed upload id=616418lb # [ 260.412133] niks3-server[1111]: 2026/09/23 05:16:32 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6419worker1 # [ 260.451616] nix-daemon[832]: reaped child process 1569, status = succeeded6420client # [ 260.502809] systemd[1]: sw.service: Deactivated successfully.6421client # [ 260.507856] systemd[1]: sw.service: Consumed 191ms CPU time over 8.909s wall clock time, 41.2M memory peak, 7.2K incoming IP traffic, 8.5K outgoing IP traffic.6422worker1 # [ 260.759063] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:33Z level=info event=signal_exit6423client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6424client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6425client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6426worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6427lb # [ 261.693284] niks3-server[1111]: 2026/09/23 05:16:34 INFO lead: released remote=192.168.1.3:405666428worker1 # [ 262.724934] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:35Z level=info event=scheduler_restarting6429worker2 # [ 262.796634] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:35Z level=info event=scheduler_restarting addr=lb:500516430worker1 # [ 262.736799] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:383766431worker1 # [ 262.743044] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:35Z level=info event=worker_gone addr=192.168.1.4:500516432worker1 # [ 262.747773] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:35Z level=info event=scheduler_restarting addr=lb:500516433worker1 # [ 262.752454] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:383766434worker1 # [ 262.757643] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:35Z level=info event=worker_gone addr=192.168.1.3:500516435worker1 # [ 262.781492] nix-daemon[832]: reaped child process 1481, status = succeeded6436worker2 # [ 262.996831] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:35Z level=info event=scheduler_connected addr=lb:500516437worker2 # [ 263.002217] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6438worker1 # [ 262.955068] nix-grpc-daemon[1406]: ts=2026-09-23T05:16:35Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6439worker2 # [ 263.203313] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:35Z level=info event=scheduler_connected addr=lb:500516440worker2 # [ 263.208625] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6441worker2 # [ 263.409770] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:35Z level=info event=scheduler_connected addr=lb:500516442worker2 # [ 263.414850] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6443worker2 # [ 263.615959] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:36Z level=info event=scheduler_connected addr=lb:500516444worker2 # [ 263.621058] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6445worker1 # [ 263.681948] nix-grpc-daemon[1406]: E0000 00:00:1790140596.188220 1452 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6446worker1 # [ 263.682281] nix-grpc-daemon[1406]: E0000 00:00:1790140596.188473 1452 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6447worker2 # [ 263.823619] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:36Z level=info event=scheduler_connected addr=lb:500516448worker2 # [ 263.827795] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6449worker1 # [ 263.848813] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6450worker1 # [ 263.855333] systemd[1]: nix-grpc-daemon.service: Consumed 1.018s CPU time over 1min 58.300s wall clock time, 37.4M memory peak, 30.1K incoming IP traffic, 38.5K outgoing IP traffic.6451worker1 # [ 263.876801] systemd[1]: Starting Nix worker-protocol over gRPC...6452worker2 # [ 264.024602] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:36Z level=info event=scheduler_connected addr=lb:500516453worker2 # [ 264.029616] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6454worker1 # [ 264.020880] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:36Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6455worker1 # [ 264.023631] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:36Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6456worker1 # [ 264.027658] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:36Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16457worker1 # [ 264.034788] nix-grpc-daemon[1635]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6458lb # [ 264.013653] niks3-server[1111]: 2026/09/23 05:16:36 INFO lead: acquired remote=192.168.1.3:485326459worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.78 seconds)6460worker1 # [ 264.050377] nix-grpc-daemon[1635]: E0000 00:00:1790140596.539194 1635 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6461lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6462lb: must succeed: curl -sf localhost:9901/clusters6463worker1 # [ 264.080258] nix-grpc-daemon[1635]: E0000 00:00:1790140596.541294 1635 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6464worker1 # [ 264.082352] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:36Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6465worker1 # [ 264.083953] nix-grpc-daemon[1635]: E0000 00:00:1790140596.550928 1675 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6466worker1 # [ 264.087681] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:36Z level=info event=scheduler_take_over6467worker1 # [ 264.089428] nix-grpc-daemon[1635]: E0000 00:00:1790140596.570092 1675 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6468worker1 # [ 264.091840] nix-grpc-daemon[1635]: E0000 00:00:1790140596.570261 1675 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6469worker1 # [ 264.094340] systemd[1]: Started Nix worker-protocol over gRPC.6470worker1 # [ 264.095136] nix-grpc-daemon[1635]: E0000 00:00:1790140596.593317 1675 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6471worker1 # [ 264.103525] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:36Z level=info event=scheduler_connected addr=lb:500516472worker1 # [ 264.105405] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6473lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6474worker2 # [ 264.230570] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:36Z level=info event=scheduler_connected addr=lb:500516475worker2 # [ 264.235928] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6476worker1 # [ 264.306420] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:36Z level=info event=scheduler_connected addr=lb:500516477worker1 # [ 264.311522] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6478worker2 # [ 264.437402] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:36Z level=info event=scheduler_connected addr=lb:500516479worker2 # [ 264.442474] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6480worker1 # [ 264.512256] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:37Z level=info event=scheduler_connected addr=lb:500516481worker1 # [ 264.517337] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6482worker2 # [ 264.643743] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:37Z level=info event=scheduler_connected addr=lb:500516483worker2 # [ 264.649212] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6484worker1 # [ 264.718143] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:37Z level=info event=scheduler_connected addr=lb:500516485worker1 # [ 264.723259] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6486worker2 # [ 264.851901] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:37Z level=info event=scheduler_connected addr=lb:500516487worker2 # [ 264.856184] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6488worker1 # [ 264.923887] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:37Z level=info event=scheduler_connected addr=lb:500516489worker1 # [ 264.932142] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:37Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:556746490worker1 # [ 264.937680] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:37Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06491worker2 # [ 265.052546] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:37Z level=info event=scheduler_connected addr=lb:500516492worker1 # [ 264.993713] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:37Z level=info event=oidc_jwks provider=mock keys=16493worker1 # [ 264.998191] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:37Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:556746494worker1 # [ 265.003565] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:37Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06495lb: must succeed: curl -sf localhost:9901/clusters6496lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6497lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.14 seconds)6498lb: must succeed: curl -sf localhost:9901/clusters6499lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6500worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6501worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6502(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 13.83 seconds)6503subtest: queue gauges show a burst while every build blocks6504lb: must succeed: curl -sf localhost:9901/clusters6505lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6506lb: must succeed: curl -sf localhost:9901/clusters6507lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6508worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6509worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6510client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6511client # [ 265.593545] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6512client # Running as unit: burst.service; invocation ID: 41f8e2843a464f08b12c6b713d1b987b6513client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds)6514lb: must succeed: curl -sf localhost:9901/clusters6515lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6516lb: must succeed: curl -sf localhost:9901/clusters6517lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6518worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6519worker1 # [ 265.776959] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:556906520worker1 # [ 265.784826] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6521worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6522client # [ 265.819954] nix[1992]: don't know how to build these paths:6523client # [ 265.821226] nix[1992]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6524client # [ 265.822575] nix[1992]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6525client # [ 265.824176] nix[1992]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6526client # [ 265.825459] nix[1992]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6527worker1 # [ 265.816235] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:38Z level=debug event=want client=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06528worker1 # [ 265.821360] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:38Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.3:50051 clients=16529worker1 # [ 265.827528] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:38Z level=debug event=want client=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06530worker1 # [ 265.832569] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:38Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.4:50051 clients=16531worker1 # [ 265.837531] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:38Z level=debug event=want client=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06532worker1 # [ 265.842062] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:38Z level=debug event=want client=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06533worker1 # [ 265.869866] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906534worker2 # [ 265.941323] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473326535worker1 # [ 265.887791] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 91 ms (attempt 1/5)6536worker2 # [ 265.959445] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:473326537client # [ 265.924232] nix[1992]: copying 1 paths to 'lb:50051'...6538worker2 # [ 265.974558] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:473326539client # [ 265.926599] nix[1992]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6540worker2 # [ 265.992150] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6541worker1 # [ 265.987758] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 2/5)6542worker2 # [ 266.150877] nix-daemon[855]: accepted connection from pid 1962, user nix-grpc-daemon (trusted)6543worker2 # [ 266.173111] nix-daemon[855]: reaped child process 1965, status = succeeded6544lb # [ 266.071395] niks3-server[1111]: 2026/09/23 05:16:38 INFO Received uploads request method=POST path=/api/pending_closures6545worker2 # [ 266.183365] nix-grpc-daemon[1805]: time=2026-09-23T05:16:38.635Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6546worker2 # [ 266.185174] nix-grpc-daemon[1805]: time=2026-09-23T05:16:38.635Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6547lb # [ 266.108830] niks3-server[1111]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6548worker1 # [ 266.145825] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 305 ms (attempt 3/5)6549worker2 # [ 266.222052] nix-grpc-daemon[1805]: time=2026-09-23T05:16:38.674Z level=INFO msg="Uploading 1 narinfos"6550lb # [ 266.120893] niks3-server[1111]: 2026/09/23 05:16:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6551lb # [ 266.131835] niks3-server[1111]: 2026/09/23 05:16:38 INFO Signed narinfos id=62 count=16552lb # [ 266.140294] niks3-server[1111]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6553lb # [ 266.151634] niks3-server[1111]: 2026/09/23 05:16:38 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6554lb # [ 266.157639] niks3-server[1111]: 2026/09/23 05:16:38 INFO Completed upload id=626555worker2 # [ 266.259538] nix-grpc-daemon[1805]: time=2026-09-23T05:16:38.712Z level=INFO msg="Upload complete. (250ms)"6556lb # [ 266.162351] niks3-server[1111]: 2026/09/23 05:16:38 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6557worker2 # [ 266.271660] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=0 paths=1 nar_bytes_in=4166558worker2 # [ 266.287208] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473326559worker2 # [ 266.294779] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6560worker2 # [ 266.300269] nix-daemon[855]: reaped child process 1960, status = succeeded6561client # [ 266.261829] nix[1992]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6562worker2 # [ 266.315793] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6563worker1 # [ 266.452902] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 683 ms (attempt 4/5)6564lb: must succeed: curl -sf localhost:9901/clusters6565lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6566lb: must succeed: curl -sf localhost:9901/clusters6567lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6568worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6569worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6570worker1 # [ 267.141433] nix-grpc-daemon[1635]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6571worker1 # [ 267.145872] nix-grpc-daemon[1635]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6572worker1 # [ 267.166268] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556906573client # [ 267.201127] nix[1992]: copying 1 paths to 'lb:50051'...6574worker1 # [ 267.181900] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556906575client # [ 267.205864] nix[1992]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6576worker1 # [ 267.197440] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6577worker1 # [ 267.225439] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:39Z level=info event=niks3_push_started pid=17196578worker1 # [ 267.374728] nix-grpc-daemon[1719]: time=2026-09-23T05:16:39.880Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6579worker1 # [ 267.477356] nix-daemon[832]: accepted connection from pid 1731, user nix-grpc-daemon (trusted)6580worker1 # [ 267.496147] nix-daemon[832]: reaped child process 1734, status = succeeded6581lb # [ 267.475037] niks3-server[1111]: 2026/09/23 05:16:39 INFO Received uploads request method=POST path=/api/pending_closures6582worker1 # [ 267.517736] nix-grpc-daemon[1719]: time=2026-09-23T05:16:40.023Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6583worker1 # [ 267.519234] nix-grpc-daemon[1719]: time=2026-09-23T05:16:40.023Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6584lb # [ 267.524823] niks3-server[1111]: 2026/09/23 05:16:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6585worker1 # [ 267.562831] nix-grpc-daemon[1719]: time=2026-09-23T05:16:40.068Z level=INFO msg="Uploading 1 narinfos"6586lb # [ 267.531789] niks3-server[1111]: 2026/09/23 05:16:40 INFO Signed narinfos id=63 count=16587lb # [ 267.539846] niks3-server[1111]: 2026/09/23 05:16:40 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6588lb # [ 267.550166] niks3-server[1111]: 2026/09/23 05:16:40 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6589lb # [ 267.559044] niks3-server[1111]: 2026/09/23 05:16:40 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6590lb # [ 267.566702] niks3-server[1111]: 2026/09/23 05:16:40 INFO Completed upload id=636591worker1 # [ 267.597767] nix-grpc-daemon[1719]: time=2026-09-23T05:16:40.104Z level=INFO msg="Upload complete. (220ms)"6592lb # [ 267.572252] niks3-server[1111]: 2026/09/23 05:16:40 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6593worker1 # [ 267.608251] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=0 paths=1 nar_bytes_in=4166594worker1 # [ 267.623411] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906595worker1 # [ 267.630289] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6596worker1 # [ 267.638146] nix-daemon[832]: reaped child process 1717, status = succeeded6597worker1 # [ 267.646662] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6598client # [ 267.666178] nix[1992]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6599worker2 # [ 268.586006] nix-daemon[855]: reaped child process 1969, status = succeeded6600worker2 # [ 268.701761] nix-daemon[855]: accepted connection from pid 1994, user nix-grpc-daemon (trusted)6601worker2 # [ 268.722635] nix-daemon[855]: reaped child process 1997, status = succeeded6602lb # [ 268.626748] niks3-server[1111]: 2026/09/23 05:16:41 INFO Received uploads request method=POST path=/api/pending_closures6603worker2 # [ 268.740438] nix-grpc-daemon[1805]: time=2026-09-23T05:16:41.191Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6604worker2 # [ 268.742139] nix-grpc-daemon[1805]: time=2026-09-23T05:16:41.191Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6605lb # [ 268.671158] niks3-server[1111]: 2026/09/23 05:16:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6606worker2 # [ 268.782201] nix-grpc-daemon[1805]: time=2026-09-23T05:16:41.235Z level=INFO msg="Uploading 1 narinfos"6607lb # [ 268.685008] niks3-server[1111]: 2026/09/23 05:16:41 INFO Signed narinfos id=64 count=16608lb # [ 268.697222] niks3-server[1111]: 2026/09/23 05:16:41 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6609lb # [ 268.708674] niks3-server[1111]: 2026/09/23 05:16:41 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6610lb # [ 268.723908] niks3-server[1111]: 2026/09/23 05:16:41 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6611lb # [ 268.737232] niks3-server[1111]: 2026/09/23 05:16:41 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6612lb # [ 268.743571] niks3-server[1111]: 2026/09/23 05:16:41 INFO Completed upload id=646613worker2 # [ 268.845169] nix-grpc-daemon[1805]: time=2026-09-23T05:16:41.298Z level=INFO msg="Upload complete. (316ms)"6614lb # [ 268.747021] niks3-server[1111]: 2026/09/23 05:16:41 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6615worker2 # [ 268.857202] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=2 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=2 outputs=16616worker1 # [ 268.790117] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:41Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.4:50051 clients=16617worker2 # [ 268.879462] nix-daemon[855]: reaped child process 1967, status = succeeded6618worker2 # [ 268.906279] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473326619worker2 # [ 268.923318] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:473326620client # [ 268.889232] nix[1992]: copying 1 paths to 'lb:50051'...6621worker2 # [ 268.938082] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:473326622client # [ 268.889554] nix[1992]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6623worker2 # [ 268.955351] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6624worker2 # [ 269.117177] nix-daemon[855]: accepted connection from pid 2001, user nix-grpc-daemon (trusted)6625worker2 # [ 269.138460] nix-daemon[855]: reaped child process 2004, status = succeeded6626lb # [ 269.041240] niks3-server[1111]: 2026/09/23 05:16:41 INFO Received uploads request method=POST path=/api/pending_closures6627worker2 # [ 269.151819] nix-grpc-daemon[1805]: time=2026-09-23T05:16:41.604Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6628worker2 # [ 269.153573] nix-grpc-daemon[1805]: time=2026-09-23T05:16:41.604Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6629lb # [ 269.072555] niks3-server[1111]: 2026/09/23 05:16:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6630worker2 # [ 269.180577] nix-grpc-daemon[1805]: time=2026-09-23T05:16:41.633Z level=INFO msg="Uploading 1 narinfos"6631lb # [ 269.086317] niks3-server[1111]: 2026/09/23 05:16:41 INFO Signed narinfos id=65 count=16632lb # [ 269.094359] niks3-server[1111]: 2026/09/23 05:16:41 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6633worker2 # [ 269.208793] nix-grpc-daemon[1805]: time=2026-09-23T05:16:41.661Z level=INFO msg="Upload complete. (236ms)"6634lb # [ 269.107625] niks3-server[1111]: 2026/09/23 05:16:41 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6635lb # [ 269.114858] niks3-server[1111]: 2026/09/23 05:16:41 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6636worker2 # [ 269.216415] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=0 paths=1 nar_bytes_in=4166637lb # [ 269.119135] niks3-server[1111]: 2026/09/23 05:16:41 INFO Completed upload id=656638lb # [ 269.121323] niks3-server[1111]: 2026/09/23 05:16:41 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6639worker2 # [ 269.228488] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473326640worker2 # [ 269.239507] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6641worker2 # [ 269.244825] nix-daemon[855]: reaped child process 1999, status = succeeded6642client # [ 269.204696] nix[1992]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6643worker2 # [ 269.257278] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6644lb # [ 269.197993] envoy[895]: conn peer=[2001:db8:1::1]:47282 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9582 tx=4429 ms=54216645lb # [ 269.202998] envoy[895]: conn peer=[2001:db8:1::1]:47276 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1516 ms=86906646lb # [ 269.208004] envoy[895]: conn peer=[2001:db8:1::3]:39458 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2388 tx=995 ms=1171776647worker1 # [ 269.489373] nix-daemon[832]: reaped child process 1739, status = succeeded6648worker1 # [ 269.599478] nix-daemon[832]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)6649worker1 # [ 269.618007] nix-daemon[832]: reaped child process 1767, status = succeeded6650lb # [ 269.593916] niks3-server[1111]: 2026/09/23 05:16:42 INFO Received uploads request method=POST path=/api/pending_closures6651worker1 # [ 269.636634] nix-grpc-daemon[1719]: time=2026-09-23T05:16:42.141Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6652worker1 # [ 269.641446] nix-grpc-daemon[1719]: time=2026-09-23T05:16:42.141Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6653lb # [ 269.641995] niks3-server[1111]: 2026/09/23 05:16:42 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6654worker1 # [ 269.689359] nix-grpc-daemon[1719]: time=2026-09-23T05:16:42.195Z level=INFO msg="Uploading 1 narinfos"6655lb # [ 269.657207] niks3-server[1111]: 2026/09/23 05:16:42 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6656lb # [ 269.667956] niks3-server[1111]: 2026/09/23 05:16:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6657lb # [ 269.675788] niks3-server[1111]: 2026/09/23 05:16:42 INFO Signed narinfos id=66 count=16658worker1 # [ 269.716917] nix-grpc-daemon[1719]: time=2026-09-23T05:16:42.223Z level=INFO msg="Upload complete. (258ms)"6659lb # [ 269.686640] niks3-server[1111]: 2026/09/23 05:16:42 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6660lb # [ 269.693213] niks3-server[1111]: 2026/09/23 05:16:42 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6661lb # [ 269.699694] niks3-server[1111]: 2026/09/23 05:16:42 INFO Completed upload id=666662worker1 # [ 269.728356] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=2 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=1 outputs=16663lb # [ 269.702579] niks3-server[1111]: 2026/09/23 05:16:42 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6664worker1 # [ 269.747230] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:42Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.3:50051 clients=16665worker1 # [ 269.752490] nix-daemon[832]: reaped child process 1737, status = succeeded6666worker1 # [ 269.773560] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906667worker1 # [ 269.793453] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556906668client # [ 269.828256] nix[1992]: copying 1 paths to 'lb:50051'...6669worker1 # [ 269.810259] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556906670client # [ 269.832372] nix[1992]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6671worker1 # [ 269.825735] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6672worker1 # [ 269.974773] nix-daemon[832]: accepted connection from pid 1771, user nix-grpc-daemon (trusted)6673worker1 # [ 269.997401] nix-daemon[832]: reaped child process 1774, status = succeeded6674lb # [ 269.964281] niks3-server[1111]: 2026/09/23 05:16:42 INFO Received uploads request method=POST path=/api/pending_closures6675worker1 # [ 270.005639] nix-grpc-daemon[1719]: time=2026-09-23T05:16:42.512Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6676worker1 # [ 270.007161] nix-grpc-daemon[1719]: time=2026-09-23T05:16:42.512Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6677lb # [ 269.997751] niks3-server[1111]: 2026/09/23 05:16:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6678worker1 # [ 270.038075] nix-grpc-daemon[1719]: time=2026-09-23T05:16:42.544Z level=INFO msg="Uploading 1 narinfos"6679lb # [ 270.012974] niks3-server[1111]: 2026/09/23 05:16:42 INFO Signed narinfos id=67 count=16680lb # [ 270.020760] niks3-server[1111]: 2026/09/23 05:16:42 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6681lb # [ 270.032226] niks3-server[1111]: 2026/09/23 05:16:42 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6682worker1 # [ 270.068900] nix-grpc-daemon[1719]: time=2026-09-23T05:16:42.575Z level=INFO msg="Upload complete. (227ms)"6683lb # [ 270.037683] niks3-server[1111]: 2026/09/23 05:16:42 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6684lb # [ 270.041426] niks3-server[1111]: 2026/09/23 05:16:42 INFO Completed upload id=676685lb # [ 270.048243] niks3-server[1111]: 2026/09/23 05:16:42 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6686worker1 # [ 270.079210] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=0 paths=1 nar_bytes_in=4166687worker1 # [ 270.092373] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906688worker1 # [ 270.100338] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6689worker1 # [ 270.102877] nix-daemon[832]: reaped child process 1769, status = succeeded6690worker1 # [ 270.114063] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6691client # [ 270.132922] nix[1992]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6692worker2 # [ 270.936283] nix-daemon[855]: reaped child process 2008, status = succeeded6693worker2 # [ 271.034284] nix-daemon[855]: accepted connection from pid 2033, user nix-grpc-daemon (trusted)6694worker2 # [ 271.055098] nix-daemon[855]: reaped child process 2036, status = succeeded6695lb # [ 270.958306] niks3-server[1111]: 2026/09/23 05:16:43 INFO Received uploads request method=POST path=/api/pending_closures6696worker2 # [ 271.072408] nix-grpc-daemon[1805]: time=2026-09-23T05:16:43.522Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6697worker2 # [ 271.074008] nix-grpc-daemon[1805]: time=2026-09-23T05:16:43.522Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6698lb # [ 271.001099] niks3-server[1111]: 2026/09/23 05:16:43 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6699lb # [ 271.015400] niks3-server[1111]: 2026/09/23 05:16:43 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6700lb # [ 271.026164] niks3-server[1111]: 2026/09/23 05:16:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6701worker2 # [ 271.135065] nix-grpc-daemon[1805]: time=2026-09-23T05:16:43.587Z level=INFO msg="Uploading 1 narinfos"6702lb # [ 271.036690] niks3-server[1111]: 2026/09/23 05:16:43 INFO Signed narinfos id=68 count=16703lb # [ 271.042807] niks3-server[1111]: 2026/09/23 05:16:43 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6704lb # [ 271.055553] niks3-server[1111]: 2026/09/23 05:16:43 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6705lb # [ 271.063428] niks3-server[1111]: 2026/09/23 05:16:43 INFO Completed upload id=686706worker2 # [ 271.164151] nix-grpc-daemon[1805]: time=2026-09-23T05:16:43.617Z level=INFO msg="Upload complete. (296ms)"6707lb # [ 271.066239] niks3-server[1111]: 2026/09/23 05:16:43 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6708worker2 # [ 271.177268] nix-grpc-daemon[1723]: ts=2026-09-23T05:16:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=3 outputs=16709worker2 # [ 271.195230] nix-daemon[855]: reaped child process 2006, status = succeeded6710worker1 # [ 271.992114] nix-daemon[832]: reaped child process 1778, status = succeeded6711worker1 # [ 272.093857] nix-daemon[832]: accepted connection from pid 1803, user nix-grpc-daemon (trusted)6712worker1 # [ 272.110366] nix-daemon[832]: reaped child process 1806, status = succeeded6713lb # [ 272.084619] niks3-server[1111]: 2026/09/23 05:16:44 INFO Received uploads request method=POST path=/api/pending_closures6714worker1 # [ 272.126576] nix-grpc-daemon[1719]: time=2026-09-23T05:16:44.632Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6715worker1 # [ 272.128368] nix-grpc-daemon[1719]: time=2026-09-23T05:16:44.632Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6716lb # [ 272.126455] niks3-server[1111]: 2026/09/23 05:16:44 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6717worker1 # [ 272.169496] nix-grpc-daemon[1719]: time=2026-09-23T05:16:44.675Z level=INFO msg="Uploading 1 narinfos"6718lb # [ 272.141880] niks3-server[1111]: 2026/09/23 05:16:44 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6719lb # [ 272.151170] niks3-server[1111]: 2026/09/23 05:16:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6720lb # [ 272.161766] niks3-server[1111]: 2026/09/23 05:16:44 INFO Signed narinfos id=69 count=16721worker1 # [ 272.192727] nix-grpc-daemon[1719]: time=2026-09-23T05:16:44.699Z level=INFO msg="Upload complete. (264ms)"6722lb # [ 272.163960] niks3-server[1111]: 2026/09/23 05:16:44 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6723lb # [ 272.170274] niks3-server[1111]: 2026/09/23 05:16:44 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6724lb # [ 272.174525] niks3-server[1111]: 2026/09/23 05:16:44 INFO Completed upload id=696725lb # [ 272.177119] niks3-server[1111]: 2026/09/23 05:16:44 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6726worker1 # [ 272.205107] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=2 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=4 outputs=16727worker1 # [ 272.224328] nix-daemon[832]: reaped child process 1776, status = succeeded6728client # [ 272.273325] systemd[1]: burst.service: Deactivated successfully.6729client # [ 272.277437] systemd[1]: burst.service: Consumed 226ms CPU time over 6.683s wall clock time, 53.7M memory peak, 13.8K incoming IP traffic, 18.5K outgoing IP traffic.6730client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6731client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds)6732lb: must succeed: curl -sf localhost:9901/clusters6733lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6734lb: must succeed: curl -sf localhost:9901/clusters6735lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6736worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6737worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6738(finished: subtest: queue gauges show a burst while every build blocks, in 8.03 seconds)6739subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6740client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16741worker1 # [ 273.589317] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:556906742worker1 # [ 273.596097] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556906743worker1 # [ 273.641159] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:46Z level=debug event=want client=2 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06744worker1 # [ 273.648251] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:46Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16745worker1 # [ 273.654171] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:46Z level=debug event=want client=2 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06746worker1 # [ 273.659443] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:46Z level=debug event=want client=2 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06747worker1 # [ 273.688496] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906748worker1 # [ 273.704515] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556906749worker1 # [ 273.718791] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556906750worker1 # [ 273.733723] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6751worker1 # [ 273.885578] nix-daemon[832]: accepted connection from pid 1819, user nix-grpc-daemon (trusted)6752worker1 # [ 273.907802] nix-daemon[832]: reaped child process 1822, status = succeeded6753lb # [ 273.877459] niks3-server[1111]: 2026/09/23 05:16:46 INFO Received uploads request method=POST path=/api/pending_closures6754worker1 # [ 273.918595] nix-grpc-daemon[1719]: time=2026-09-23T05:16:46.424Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6755worker1 # [ 273.920312] nix-grpc-daemon[1719]: time=2026-09-23T05:16:46.424Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6756lb # [ 273.908263] niks3-server[1111]: 2026/09/23 05:16:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6757worker1 # [ 273.947387] nix-grpc-daemon[1719]: time=2026-09-23T05:16:46.453Z level=INFO msg="Uploading 1 narinfos"6758lb # [ 273.921682] niks3-server[1111]: 2026/09/23 05:16:46 INFO Signed narinfos id=70 count=16759lb # [ 273.929884] niks3-server[1111]: 2026/09/23 05:16:46 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6760worker1 # [ 273.976488] nix-grpc-daemon[1719]: time=2026-09-23T05:16:46.483Z level=INFO msg="Upload complete. (226ms)"6761lb # [ 273.944784] niks3-server[1111]: 2026/09/23 05:16:46 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6762lb # [ 273.955670] niks3-server[1111]: 2026/09/23 05:16:46 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6763worker1 # [ 273.986347] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=0 paths=1 nar_bytes_in=5126764lb # [ 273.963245] niks3-server[1111]: 2026/09/23 05:16:46 INFO Completed upload id=706765lb # [ 273.965326] niks3-server[1111]: 2026/09/23 05:16:46 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6766worker1 # [ 273.999440] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906767worker1 # [ 274.008304] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6768worker1 # [ 274.011100] nix-daemon[832]: reaped child process 1817, status = succeeded6769worker1 # [ 274.021701] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6770worker1 # [ 274.689971] nix-daemon[832]: reaped child process 1826, status = succeeded6771worker1 # [ 274.821187] nix-daemon[832]: accepted connection from pid 1842, user nix-grpc-daemon (trusted)6772worker1 # [ 274.837802] nix-daemon[832]: reaped child process 1845, status = succeeded6773lb # [ 274.812277] niks3-server[1111]: 2026/09/23 05:16:47 INFO Received uploads request method=POST path=/api/pending_closures6774worker1 # [ 274.854400] nix-grpc-daemon[1719]: time=2026-09-23T05:16:47.359Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6775worker1 # [ 274.856200] nix-grpc-daemon[1719]: time=2026-09-23T05:16:47.359Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6776lb # [ 274.840567] niks3-server[1111]: 2026/09/23 05:16:47 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6777lb # [ 274.862886] niks3-server[1111]: 2026/09/23 05:16:47 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6778worker1 # [ 274.909440] nix-grpc-daemon[1719]: time=2026-09-23T05:16:47.415Z level=INFO msg="Uploading 1 narinfos"6779lb # [ 274.872113] niks3-server[1111]: 2026/09/23 05:16:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6780lb # [ 274.882934] niks3-server[1111]: 2026/09/23 05:16:47 INFO Signed narinfos id=71 count=16781lb # [ 274.889843] niks3-server[1111]: 2026/09/23 05:16:47 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6782lb # [ 274.902488] niks3-server[1111]: 2026/09/23 05:16:47 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6783lb # [ 274.910619] niks3-server[1111]: 2026/09/23 05:16:47 INFO Completed upload id=716784worker1 # [ 274.941912] nix-grpc-daemon[1719]: time=2026-09-23T05:16:47.447Z level=INFO msg="Upload complete. (304ms)"6785lb # [ 274.914187] niks3-server[1111]: 2026/09/23 05:16:47 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6786worker1 # [ 274.960969] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=0 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=5 outputs=16787worker1 # [ 274.980315] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:47Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16788worker1 # [ 274.988105] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906789worker1 # [ 274.998026] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:47Z level=debug event=want client=2 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06790worker1 # [ 275.007585] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556906791worker1 # [ 275.014233] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556906792worker1 # [ 275.022446] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6793worker1 # [ 275.026310] nix-daemon[832]: reaped child process 1824, status = succeeded6794worker1 # [ 275.141507] nix-daemon[832]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)6795worker1 # [ 275.158678] nix-daemon[832]: reaped child process 1854, status = succeeded6796lb # [ 275.133211] niks3-server[1111]: 2026/09/23 05:16:47 INFO Received uploads request method=POST path=/api/pending_closures6797worker1 # [ 275.174133] nix-grpc-daemon[1719]: time=2026-09-23T05:16:47.680Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6798worker1 # [ 275.175582] nix-grpc-daemon[1719]: time=2026-09-23T05:16:47.680Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6799lb # [ 275.164245] niks3-server[1111]: 2026/09/23 05:16:47 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6800worker1 # [ 275.206635] nix-grpc-daemon[1719]: time=2026-09-23T05:16:47.713Z level=INFO msg="Uploading 1 narinfos"6801lb # [ 275.177892] niks3-server[1111]: 2026/09/23 05:16:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6802lb # [ 275.188253] niks3-server[1111]: 2026/09/23 05:16:47 INFO Signed narinfos id=72 count=16803lb # [ 275.190731] niks3-server[1111]: 2026/09/23 05:16:47 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6804worker1 # [ 275.235794] nix-grpc-daemon[1719]: time=2026-09-23T05:16:47.742Z level=INFO msg="Upload complete. (222ms)"6805lb # [ 275.204759] niks3-server[1111]: 2026/09/23 05:16:47 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6806lb # [ 275.210332] niks3-server[1111]: 2026/09/23 05:16:47 INFO Completed upload id=726807worker1 # [ 275.242273] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=0 paths=1 nar_bytes_in=5126808lb # [ 275.213906] niks3-server[1111]: 2026/09/23 05:16:47 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6809worker1 # [ 275.252479] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906810worker1 # [ 275.261832] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6811worker1 # [ 275.266675] nix-daemon[832]: reaped child process 1849, status = succeeded6812worker1 # [ 275.278971] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6813worker1 # [ 276.316782] nix-daemon[832]: reaped child process 1858, status = succeeded6814worker1 # [ 276.430931] nix-daemon[832]: accepted connection from pid 1874, user nix-grpc-daemon (trusted)6815worker1 # [ 276.449337] nix-daemon[832]: reaped child process 1877, status = succeeded6816lb # [ 276.416710] niks3-server[1111]: 2026/09/23 05:16:48 INFO Received uploads request method=POST path=/api/pending_closures6817worker1 # [ 276.460374] nix-grpc-daemon[1719]: time=2026-09-23T05:16:48.965Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6818worker1 # [ 276.461825] nix-grpc-daemon[1719]: time=2026-09-23T05:16:48.965Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6819lb # [ 276.467815] niks3-server[1111]: 2026/09/23 05:16:48 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6820worker1 # [ 276.507886] nix-grpc-daemon[1719]: time=2026-09-23T05:16:49.014Z level=INFO msg="Uploading 1 narinfos"6821lb # [ 276.477439] niks3-server[1111]: 2026/09/23 05:16:48 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6822lb # [ 276.488541] niks3-server[1111]: 2026/09/23 05:16:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6823lb # [ 276.491644] niks3-server[1111]: 2026/09/23 05:16:48 INFO Signed narinfos id=73 count=16824lb # [ 276.496279] niks3-server[1111]: 2026/09/23 05:16:48 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6825lb # [ 276.503502] niks3-server[1111]: 2026/09/23 05:16:49 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6826lb # [ 276.509134] niks3-server[1111]: 2026/09/23 05:16:49 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6827lb # [ 276.513758] niks3-server[1111]: 2026/09/23 05:16:49 INFO Completed upload id=736828worker1 # [ 276.545301] nix-grpc-daemon[1719]: time=2026-09-23T05:16:49.051Z level=INFO msg="Upload complete. (257ms)"6829worker1 # [ 276.565182] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=6 outputs=16830worker1 # [ 276.581203] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:49Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16831worker1 # [ 276.590967] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906832worker1 # [ 276.596869] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:49Z level=debug event=want client=2 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06833worker1 # [ 276.608294] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556906834worker1 # [ 276.611909] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556906835worker1 # [ 276.623877] nix-daemon[832]: reaped child process 1856, status = succeeded6836worker1 # [ 276.628373] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6837worker1 # [ 276.748836] nix-daemon[832]: accepted connection from pid 1883, user nix-grpc-daemon (trusted)6838worker1 # [ 276.766866] nix-daemon[832]: reaped child process 1886, status = succeeded6839lb # [ 276.737705] niks3-server[1111]: 2026/09/23 05:16:49 INFO Received uploads request method=POST path=/api/pending_closures6840worker1 # [ 276.778592] nix-grpc-daemon[1719]: time=2026-09-23T05:16:49.284Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6841worker1 # [ 276.780101] nix-grpc-daemon[1719]: time=2026-09-23T05:16:49.284Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6842worker1 # [ 276.798456] nix-grpc-daemon[1719]: time=2026-09-23T05:16:49.304Z level=INFO msg="Uploading 1 narinfos"6843lb # [ 276.766197] niks3-server[1111]: 2026/09/23 05:16:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6844lb # [ 276.778107] niks3-server[1111]: 2026/09/23 05:16:49 INFO Signed narinfos id=74 count=16845lb # [ 276.788476] niks3-server[1111]: 2026/09/23 05:16:49 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6846lb # [ 276.800727] niks3-server[1111]: 2026/09/23 05:16:49 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6847worker1 # [ 276.839719] nix-grpc-daemon[1719]: time=2026-09-23T05:16:49.346Z level=INFO msg="Upload complete. (216ms)"6848lb # [ 276.808633] niks3-server[1111]: 2026/09/23 05:16:49 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6849lb # [ 276.817331] niks3-server[1111]: 2026/09/23 05:16:49 INFO Completed upload id=746850worker1 # [ 276.848239] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=0 paths=1 nar_bytes_in=5126851lb # [ 276.819688] niks3-server[1111]: 2026/09/23 05:16:49 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6852worker1 # [ 276.862314] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906853worker1 # [ 276.870817] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6854worker1 # [ 276.874265] nix-daemon[832]: reaped child process 1881, status = succeeded6855worker1 # [ 276.884123] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6856worker1 # [ 278.340795] nix-daemon[832]: reaped child process 1890, status = succeeded6857worker1 # [ 278.457188] nix-daemon[832]: accepted connection from pid 1906, user nix-grpc-daemon (trusted)6858worker1 # [ 278.473372] nix-daemon[832]: reaped child process 1909, status = succeeded6859lb # [ 278.448002] niks3-server[1111]: 2026/09/23 05:16:50 INFO Received uploads request method=POST path=/api/pending_closures6860worker1 # [ 278.491595] nix-grpc-daemon[1719]: time=2026-09-23T05:16:50.996Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6861worker1 # [ 278.493739] nix-grpc-daemon[1719]: time=2026-09-23T05:16:50.996Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6862lb # [ 278.487799] niks3-server[1111]: 2026/09/23 05:16:50 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6863worker1 # [ 278.532041] nix-grpc-daemon[1719]: time=2026-09-23T05:16:51.038Z level=INFO msg="Uploading 1 narinfos"6864lb # [ 278.500898] niks3-server[1111]: 2026/09/23 05:16:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6865lb # [ 278.512327] niks3-server[1111]: 2026/09/23 05:16:51 INFO Signed narinfos id=75 count=16866lb # [ 278.518020] niks3-server[1111]: 2026/09/23 05:16:51 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6867lb # [ 278.528281] niks3-server[1111]: 2026/09/23 05:16:51 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6868worker1 # [ 278.567368] nix-grpc-daemon[1719]: time=2026-09-23T05:16:51.073Z level=INFO msg="Upload complete. (268ms)"6869lb # [ 278.536847] niks3-server[1111]: 2026/09/23 05:16:51 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6870lb # [ 278.540699] niks3-server[1111]: 2026/09/23 05:16:51 INFO Completed upload id=756871lb # [ 278.544490] niks3-server[1111]: 2026/09/23 05:16:51 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6872worker1 # [ 278.585462] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=7 outputs=16873worker1 # [ 278.604725] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:51Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16874worker1 # [ 278.613714] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906875worker1 # [ 278.619360] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:51Z level=debug event=want client=2 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06876worker1 # [ 278.625179] nix-daemon[832]: reaped child process 1888, status = succeeded6877worker1 # [ 278.627236] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556906878worker1 # [ 278.632759] nix-grpc-daemon[1719]: time=2026-09-23T05:16:51.138Z level=INFO msg="All 1 paths already cached"6879worker1 # [ 278.636797] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556906880worker1 # [ 278.647506] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6881worker1 # [ 278.779886] nix-daemon[832]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)6882worker1 # [ 278.798713] nix-daemon[832]: reaped child process 1918, status = succeeded6883lb # [ 278.770546] niks3-server[1111]: 2026/09/23 05:16:51 INFO Received uploads request method=POST path=/api/pending_closures6884worker1 # [ 278.811901] nix-grpc-daemon[1719]: time=2026-09-23T05:16:51.318Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6885worker1 # [ 278.813492] nix-grpc-daemon[1719]: time=2026-09-23T05:16:51.318Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6886worker1 # [ 278.831940] nix-grpc-daemon[1719]: time=2026-09-23T05:16:51.338Z level=INFO msg="Uploading 1 narinfos"6887lb # [ 278.799665] niks3-server[1111]: 2026/09/23 05:16:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6888lb # [ 278.814376] niks3-server[1111]: 2026/09/23 05:16:51 INFO Signed narinfos id=76 count=16889lb # [ 278.823219] niks3-server[1111]: 2026/09/23 05:16:51 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6890lb # [ 278.833196] niks3-server[1111]: 2026/09/23 05:16:51 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6891worker1 # [ 278.875024] nix-grpc-daemon[1719]: time=2026-09-23T05:16:51.381Z level=INFO msg="Upload complete. (216ms)"6892lb # [ 278.843515] niks3-server[1111]: 2026/09/23 05:16:51 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6893lb # [ 278.852472] niks3-server[1111]: 2026/09/23 05:16:51 INFO Completed upload id=766894worker1 # [ 278.883737] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=0 paths=1 nar_bytes_in=6246895lb # [ 278.855991] niks3-server[1111]: 2026/09/23 05:16:51 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6896worker1 # [ 278.895977] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906897worker1 # [ 278.905513] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6898worker1 # [ 278.908261] nix-daemon[832]: reaped child process 1913, status = succeeded6899worker1 # [ 278.918038] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6900lb # [ 279.201287] envoy[895]: conn peer=[2001:db8:1::1]:48088 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9675 tx=5207 ms=64666901worker1 # [ 279.953889] nix-daemon[832]: reaped child process 1922, status = succeeded6902worker1 # [ 280.060190] nix-daemon[832]: accepted connection from pid 1938, user nix-grpc-daemon (trusted)6903worker1 # [ 280.079769] nix-daemon[832]: reaped child process 1941, status = succeeded6904lb # [ 280.054333] niks3-server[1111]: 2026/09/23 05:16:52 INFO Received uploads request method=POST path=/api/pending_closures6905worker1 # [ 280.097300] nix-grpc-daemon[1719]: time=2026-09-23T05:16:52.602Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6906worker1 # [ 280.098878] nix-grpc-daemon[1719]: time=2026-09-23T05:16:52.602Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6907lb # [ 280.097831] niks3-server[1111]: 2026/09/23 05:16:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6908worker1 # [ 280.139644] nix-grpc-daemon[1719]: time=2026-09-23T05:16:52.646Z level=INFO msg="Uploading 1 narinfos"6909lb # [ 280.114305] niks3-server[1111]: 2026/09/23 05:16:52 INFO Signed narinfos id=77 count=16910lb # [ 280.121579] niks3-server[1111]: 2026/09/23 05:16:52 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6911lb # [ 280.132709] niks3-server[1111]: 2026/09/23 05:16:52 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6912worker1 # [ 280.177441] nix-grpc-daemon[1719]: time=2026-09-23T05:16:52.684Z level=INFO msg="Upload complete. (248ms)"6913lb # [ 280.146357] niks3-server[1111]: 2026/09/23 05:16:52 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6914lb # [ 280.153006] niks3-server[1111]: 2026/09/23 05:16:52 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6915lb # [ 280.157385] niks3-server[1111]: 2026/09/23 05:16:52 INFO Completed upload id=776916worker1 # [ 280.188924] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=8 outputs=16917lb # [ 280.160548] niks3-server[1111]: 2026/09/23 05:16:52 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6918worker1 # [ 280.203711] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:52Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16919worker1 # [ 280.208320] nix-daemon[832]: reaped child process 1920, status = succeeded6920worker1 # [ 280.230958] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906921worker1 # [ 280.238890] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:52Z level=debug event=want client=2 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06922worker1 # [ 280.249751] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556906923worker1 # [ 280.263187] nix-grpc-daemon[1719]: time=2026-09-23T05:16:52.769Z level=INFO msg="All 1 paths already cached"6924worker1 # [ 280.271966] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556906925worker1 # [ 280.283488] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6926worker1 # [ 280.427471] nix-daemon[832]: accepted connection from pid 1947, user nix-grpc-daemon (trusted)6927worker1 # [ 280.445206] nix-daemon[832]: reaped child process 1950, status = succeeded6928lb # [ 280.418726] niks3-server[1111]: 2026/09/23 05:16:52 INFO Received uploads request method=POST path=/api/pending_closures6929worker1 # [ 280.459464] nix-grpc-daemon[1719]: time=2026-09-23T05:16:52.965Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6930worker1 # [ 280.461126] nix-grpc-daemon[1719]: time=2026-09-23T05:16:52.965Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6931lb # [ 280.454349] niks3-server[1111]: 2026/09/23 05:16:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6932worker1 # [ 280.493074] nix-grpc-daemon[1719]: time=2026-09-23T05:16:52.999Z level=INFO msg="Uploading 1 narinfos"6933lb # [ 280.466754] niks3-server[1111]: 2026/09/23 05:16:52 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6934lb # [ 280.475938] niks3-server[1111]: 2026/09/23 05:16:52 INFO Signed narinfos id=78 count=16935worker1 # [ 280.516072] nix-grpc-daemon[1719]: time=2026-09-23T05:16:53.022Z level=INFO msg="Upload complete. (219ms)"6936lb # [ 280.485214] niks3-server[1111]: 2026/09/23 05:16:52 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6937worker1 # [ 280.523727] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=0 paths=1 nar_bytes_in=6246938lb # [ 280.493507] niks3-server[1111]: 2026/09/23 05:16:52 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6939lb # [ 280.497103] niks3-server[1111]: 2026/09/23 05:16:52 INFO Completed upload id=786940lb # [ 280.499104] niks3-server[1111]: 2026/09/23 05:16:53 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6941worker1 # [ 280.538800] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906942worker1 # [ 280.545032] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6943worker1 # [ 280.549890] nix-daemon[832]: reaped child process 1945, status = succeeded6944worker1 # [ 280.562734] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6945worker1 # [ 282.273386] nix-daemon[832]: reaped child process 1954, status = succeeded6946worker1 # [ 282.459517] nix-daemon[832]: accepted connection from pid 1970, user nix-grpc-daemon (trusted)6947worker1 # [ 282.483750] nix-daemon[832]: reaped child process 1973, status = succeeded6948lb # [ 282.463495] niks3-server[1111]: 2026/09/23 05:16:54 INFO Received uploads request method=POST path=/api/pending_closures6949worker1 # [ 282.512590] nix-grpc-daemon[1719]: time=2026-09-23T05:16:55.017Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6950worker1 # [ 282.514227] nix-grpc-daemon[1719]: time=2026-09-23T05:16:55.017Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6951lb # [ 282.541808] niks3-server[1111]: 2026/09/23 05:16:55 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6952lb # [ 282.557284] niks3-server[1111]: 2026/09/23 05:16:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign6953worker1 # [ 282.598199] nix-grpc-daemon[1719]: time=2026-09-23T05:16:55.103Z level=INFO msg="Uploading 1 narinfos"6954lb # [ 282.572333] niks3-server[1111]: 2026/09/23 05:16:55 INFO Signed narinfos id=79 count=16955lb # [ 282.578569] niks3-server[1111]: 2026/09/23 05:16:55 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6956lb # [ 282.595270] niks3-server[1111]: 2026/09/23 05:16:55 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6957lb # [ 282.605880] niks3-server[1111]: 2026/09/23 05:16:55 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete6958lb # [ 282.615150] niks3-server[1111]: 2026/09/23 05:16:55 INFO Completed upload id=796959lb # [ 282.618448] niks3-server[1111]: 2026/09/23 05:16:55 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6960worker1 # [ 282.644191] nix-grpc-daemon[1719]: time=2026-09-23T05:16:55.150Z level=INFO msg="Upload complete. (467ms)"6961worker1 # [ 282.673475] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=2 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=9 outputs=16962worker1 # [ 282.696910] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:55Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16963worker1 # [ 282.712436] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906964worker1 # [ 282.721181] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:55Z level=debug event=want client=2 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06965worker1 # [ 282.732729] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556906966worker1 # [ 282.741003] nix-daemon[832]: reaped child process 1952, status = succeeded6967worker1 # [ 282.745851] nix-grpc-daemon[1719]: time=2026-09-23T05:16:55.252Z level=INFO msg="All 1 paths already cached"6968worker1 # [ 282.759757] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556906969worker1 # [ 282.778804] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6970worker1 # [ 283.024906] nix-daemon[832]: accepted connection from pid 1979, user nix-grpc-daemon (trusted)6971worker1 # [ 283.047442] nix-daemon[832]: reaped child process 1982, status = succeeded6972lb # [ 283.023895] niks3-server[1111]: 2026/09/23 05:16:55 INFO Received uploads request method=POST path=/api/pending_closures6973worker1 # [ 283.069967] nix-grpc-daemon[1719]: time=2026-09-23T05:16:55.576Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6974worker1 # [ 283.071527] nix-grpc-daemon[1719]: time=2026-09-23T05:16:55.576Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6975lb # [ 283.075672] niks3-server[1111]: 2026/09/23 05:16:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign6976worker1 # [ 283.120173] nix-grpc-daemon[1719]: time=2026-09-23T05:16:55.626Z level=INFO msg="Uploading 1 narinfos"6977lb # [ 283.094060] niks3-server[1111]: 2026/09/23 05:16:55 INFO Signed narinfos id=80 count=16978lb # [ 283.101097] niks3-server[1111]: 2026/09/23 05:16:55 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6979lb # [ 283.115846] niks3-server[1111]: 2026/09/23 05:16:55 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6980lb # [ 283.130397] niks3-server[1111]: 2026/09/23 05:16:55 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete6981worker1 # [ 283.171035] nix-grpc-daemon[1719]: time=2026-09-23T05:16:55.677Z level=INFO msg="Upload complete. (371ms)"6982lb # [ 283.142506] niks3-server[1111]: 2026/09/23 05:16:55 INFO Completed upload id=806983lb # [ 283.145618] niks3-server[1111]: 2026/09/23 05:16:55 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6984worker1 # [ 283.184642] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=0 paths=1 nar_bytes_in=6246985worker1 # [ 283.204434] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556906986worker1 # [ 283.219611] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6987worker1 # [ 283.247311] nix-daemon[832]: reaped child process 1977, status = succeeded6988worker1 # [ 283.259297] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6989worker1 # [ 284.960461] nix-daemon[832]: reaped child process 1986, status = succeeded6990worker1 # [ 285.061979] nix-daemon[832]: accepted connection from pid 2002, user nix-grpc-daemon (trusted)6991worker1 # [ 285.078908] nix-daemon[832]: reaped child process 2005, status = succeeded6992lb # [ 285.055402] niks3-server[1111]: 2026/09/23 05:16:57 INFO Received uploads request method=POST path=/api/pending_closures6993worker1 # [ 285.099612] nix-grpc-daemon[1719]: time=2026-09-23T05:16:57.604Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6994worker1 # [ 285.101649] nix-grpc-daemon[1719]: time=2026-09-23T05:16:57.604Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6995lb # [ 285.098503] niks3-server[1111]: 2026/09/23 05:16:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign6996worker1 # [ 285.143397] nix-grpc-daemon[1719]: time=2026-09-23T05:16:57.649Z level=INFO msg="Uploading 1 narinfos"6997lb # [ 285.114168] niks3-server[1111]: 2026/09/23 05:16:57 INFO Signed narinfos id=81 count=16998lb # [ 285.121222] niks3-server[1111]: 2026/09/23 05:16:57 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6999lb # [ 285.128615] niks3-server[1111]: 2026/09/23 05:16:57 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst7000lb # [ 285.134629] niks3-server[1111]: 2026/09/23 05:16:57 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls7001lb # [ 285.145120] niks3-server[1111]: 2026/09/23 05:16:57 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7002lb # [ 285.151967] niks3-server[1111]: 2026/09/23 05:16:57 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo7003lb # [ 285.156728] niks3-server[1111]: 2026/09/23 05:16:57 INFO Completed upload id=817004worker1 # [ 285.188957] nix-grpc-daemon[1719]: time=2026-09-23T05:16:57.695Z level=INFO msg="Upload complete. (308ms)"7005worker1 # [ 285.207145] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=2 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=10 outputs=17006worker1 # [ 285.228344] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:57Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=17007worker1 # [ 285.240424] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556907008worker1 # [ 285.249228] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:57Z level=debug event=want client=2 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=07009worker1 # [ 285.263718] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556907010worker1 # [ 285.274583] nix-daemon[832]: reaped child process 1984, status = succeeded7011worker1 # [ 285.279742] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7012worker1 # [ 285.282934] nix-grpc-daemon[1719]: time=2026-09-23T05:16:57.752Z level=INFO msg="All 2 paths already cached"7013worker1 # [ 285.288422] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556907014worker1 # [ 285.411682] nix-daemon[832]: accepted connection from pid 2011, user nix-grpc-daemon (trusted)7015worker1 # [ 285.428131] nix-daemon[832]: reaped child process 2014, status = succeeded7016lb # [ 285.403854] niks3-server[1111]: 2026/09/23 05:16:57 INFO Received uploads request method=POST path=/api/pending_closures7017worker1 # [ 285.449510] nix-grpc-daemon[1719]: time=2026-09-23T05:16:57.955Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7018worker1 # [ 285.451079] nix-grpc-daemon[1719]: time=2026-09-23T05:16:57.955Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"7019worker1 # [ 285.481588] nix-grpc-daemon[1719]: time=2026-09-23T05:16:57.987Z level=INFO msg="Uploading 1 narinfos"7020lb # [ 285.439395] niks3-server[1111]: 2026/09/23 05:16:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7021lb # [ 285.457021] niks3-server[1111]: 2026/09/23 05:16:57 INFO Signed narinfos id=82 count=17022lb # [ 285.464785] niks3-server[1111]: 2026/09/23 05:16:57 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls7023lb # [ 285.476248] niks3-server[1111]: 2026/09/23 05:16:57 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst7024worker1 # [ 285.515126] nix-grpc-daemon[1719]: time=2026-09-23T05:16:58.021Z level=INFO msg="Upload complete. (232ms)"7025lb # [ 285.485394] niks3-server[1111]: 2026/09/23 05:16:57 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7026lb # [ 285.492809] niks3-server[1111]: 2026/09/23 05:16:57 INFO Completed upload id=827027worker1 # [ 285.523739] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=0 paths=1 nar_bytes_in=7527028lb # [ 285.495808] niks3-server[1111]: 2026/09/23 05:16:58 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo7029worker1 # [ 285.540525] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556907030worker1 # [ 285.548729] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7031worker1 # [ 285.553903] nix-daemon[832]: reaped child process 2009, status = succeeded7032worker1 # [ 285.567519] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7033worker1 # [ 287.240655] nix-daemon[832]: reaped child process 2018, status = succeeded7034worker1 # [ 287.348671] nix-daemon[832]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)7035worker1 # [ 287.366730] nix-daemon[832]: reaped child process 2037, status = succeeded7036lb # [ 287.342230] niks3-server[1111]: 2026/09/23 05:16:59 INFO Received uploads request method=POST path=/api/pending_closures7037worker1 # [ 287.384418] nix-grpc-daemon[1719]: time=2026-09-23T05:16:59.890Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7038worker1 # [ 287.386111] nix-grpc-daemon[1719]: time=2026-09-23T05:16:59.890Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"7039lb # [ 287.378367] niks3-server[1111]: 2026/09/23 05:16:59 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv7040worker1 # [ 287.437612] nix-grpc-daemon[1719]: time=2026-09-23T05:16:59.943Z level=INFO msg="Uploading 1 narinfos"7041lb # [ 287.400545] niks3-server[1111]: 2026/09/23 05:16:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7042lb # [ 287.411340] niks3-server[1111]: 2026/09/23 05:16:59 INFO Signed narinfos id=83 count=17043lb # [ 287.416659] niks3-server[1111]: 2026/09/23 05:16:59 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst7044lb # [ 287.429545] niks3-server[1111]: 2026/09/23 05:16:59 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7045worker1 # [ 287.460632] nix-grpc-daemon[1719]: time=2026-09-23T05:16:59.967Z level=INFO msg="Upload complete. (283ms)"7046lb # [ 287.433898] niks3-server[1111]: 2026/09/23 05:16:59 INFO Completed upload id=837047lb # [ 287.436530] niks3-server[1111]: 2026/09/23 05:16:59 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls7048lb # [ 287.440704] niks3-server[1111]: 2026/09/23 05:16:59 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo7049worker1 # [ 287.477102] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=11 outputs=17050worker1 # [ 287.493148] nix-grpc-daemon[1635]: ts=2026-09-23T05:16:59Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=17051worker1 # [ 287.498103] nix-daemon[832]: reaped child process 2016, status = succeeded7052worker1 # [ 287.517012] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556907053worker1 # [ 287.532442] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556907054worker1 # [ 287.547883] nix-grpc-daemon[1719]: time=2026-09-23T05:17:00.054Z level=INFO msg="All 2 paths already cached"7055worker1 # [ 287.558219] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556907056worker1 # [ 287.571848] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7057worker1 # [ 287.720886] nix-daemon[832]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)7058worker1 # [ 287.740713] nix-daemon[832]: reaped child process 2044, status = succeeded7059lb # [ 287.712907] niks3-server[1111]: 2026/09/23 05:17:00 INFO Received uploads request method=POST path=/api/pending_closures7060worker1 # [ 287.754634] nix-grpc-daemon[1719]: time=2026-09-23T05:17:00.260Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7061worker1 # [ 287.756433] nix-grpc-daemon[1719]: time=2026-09-23T05:17:00.260Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"7062lb # [ 287.743437] niks3-server[1111]: 2026/09/23 05:17:00 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls7063lb # [ 287.755498] niks3-server[1111]: 2026/09/23 05:17:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7064worker1 # [ 287.793913] nix-grpc-daemon[1719]: time=2026-09-23T05:17:00.300Z level=INFO msg="Uploading 1 narinfos"7065lb # [ 287.768373] niks3-server[1111]: 2026/09/23 05:17:00 INFO Signed narinfos id=84 count=17066lb # [ 287.773797] niks3-server[1111]: 2026/09/23 05:17:00 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst7067lb # [ 287.783331] niks3-server[1111]: 2026/09/23 05:17:00 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7068lb # [ 287.789671] niks3-server[1111]: 2026/09/23 05:17:00 INFO Completed upload id=847069worker1 # [ 287.821464] nix-grpc-daemon[1719]: time=2026-09-23T05:17:00.328Z level=INFO msg="Upload complete. (234ms)"7070lb # [ 287.792587] niks3-server[1111]: 2026/09/23 05:17:00 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo7071worker1 # [ 287.832246] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=0 paths=1 nar_bytes_in=7527072worker1 # [ 287.847859] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556907073worker1 # [ 287.854090] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7074worker1 # [ 287.859077] nix-daemon[832]: reaped child process 2039, status = succeeded7075worker1 # [ 287.870087] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7076worker1 # [ 289.144177] nix-daemon[832]: reaped child process 2048, status = succeeded7077worker1 # [ 289.245039] nix-daemon[832]: accepted connection from pid 2064, user nix-grpc-daemon (trusted)7078worker1 # [ 289.265765] nix-daemon[832]: reaped child process 2067, status = succeeded7079lb # [ 289.232906] niks3-server[1111]: 2026/09/23 05:17:01 INFO Received uploads request method=POST path=/api/pending_closures7080worker1 # [ 289.277729] nix-grpc-daemon[1719]: time=2026-09-23T05:17:01.783Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7081worker1 # [ 289.279362] nix-grpc-daemon[1719]: time=2026-09-23T05:17:01.783Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"7082lb # [ 289.271709] niks3-server[1111]: 2026/09/23 05:17:01 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls7083worker1 # [ 289.318427] nix-grpc-daemon[1719]: time=2026-09-23T05:17:01.824Z level=INFO msg="Uploading 1 narinfos"7084lb # [ 289.285919] niks3-server[1111]: 2026/09/23 05:17:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7085lb # [ 289.297961] niks3-server[1111]: 2026/09/23 05:17:01 INFO Signed narinfos id=85 count=17086lb # [ 289.304321] niks3-server[1111]: 2026/09/23 05:17:01 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv7087lb # [ 289.315374] niks3-server[1111]: 2026/09/23 05:17:01 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst7088worker1 # [ 289.354962] nix-grpc-daemon[1719]: time=2026-09-23T05:17:01.861Z level=INFO msg="Upload complete. (273ms)"7089lb # [ 289.323566] niks3-server[1111]: 2026/09/23 05:17:01 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7090lb # [ 289.327440] niks3-server[1111]: 2026/09/23 05:17:01 INFO Completed upload id=857091lb # [ 289.331893] niks3-server[1111]: 2026/09/23 05:17:01 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo7092worker1 # [ 289.369388] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=12 outputs=17093worker1 # [ 289.384400] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:01Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=17094worker1 # [ 289.390213] nix-daemon[832]: reaped child process 2046, status = succeeded7095worker1 # [ 289.408901] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556907096worker1 # [ 289.423623] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556907097worker1 # [ 289.438496] nix-grpc-daemon[1719]: time=2026-09-23T05:17:01.945Z level=INFO msg="All 2 paths already cached"7098worker1 # [ 289.448593] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:556907099worker1 # [ 289.461572] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7100worker1 # [ 289.606304] nix-daemon[832]: accepted connection from pid 2071, user nix-grpc-daemon (trusted)7101worker1 # [ 289.622305] nix-daemon[832]: reaped child process 2074, status = succeeded7102lb # [ 289.598157] niks3-server[1111]: 2026/09/23 05:17:02 INFO Received uploads request method=POST path=/api/pending_closures7103worker1 # [ 289.638881] nix-grpc-daemon[1719]: time=2026-09-23T05:17:02.145Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7104worker1 # [ 289.640535] nix-grpc-daemon[1719]: time=2026-09-23T05:17:02.145Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"7105worker1 # [ 289.659925] nix-grpc-daemon[1719]: time=2026-09-23T05:17:02.166Z level=INFO msg="Uploading 1 narinfos"7106lb # [ 289.627608] niks3-server[1111]: 2026/09/23 05:17:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign7107lb # [ 289.641745] niks3-server[1111]: 2026/09/23 05:17:02 INFO Signed narinfos id=86 count=17108lb # [ 289.649401] niks3-server[1111]: 2026/09/23 05:17:02 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst7109worker1 # [ 289.696188] nix-grpc-daemon[1719]: time=2026-09-23T05:17:02.202Z level=INFO msg="Upload complete. (219ms)"7110lb # [ 289.664559] niks3-server[1111]: 2026/09/23 05:17:02 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls7111lb # [ 289.670445] niks3-server[1111]: 2026/09/23 05:17:02 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete7112lb # [ 289.680288] niks3-server[1111]: 2026/09/23 05:17:02 INFO Completed upload id=867113worker1 # [ 289.704121] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=0 paths=1 nar_bytes_in=7527114lb # [ 289.682935] niks3-server[1111]: 2026/09/23 05:17:02 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo7115worker1 # [ 289.719004] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:556907116worker1 # [ 289.725420] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7117worker1 # [ 289.730475] nix-daemon[832]: reaped child process 2069, status = succeeded7118worker1 # [ 289.741406] nix-daemon[832]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)7119worker1 # [ 290.567259] nix-daemon[832]: reaped child process 2078, status = succeeded7120worker1 # [ 290.663669] nix-daemon[832]: accepted connection from pid 2094, user nix-grpc-daemon (trusted)7121worker1 # [ 290.679631] nix-daemon[832]: reaped child process 2097, status = succeeded7122lb # [ 290.658007] niks3-server[1111]: 2026/09/23 05:17:03 INFO Received uploads request method=POST path=/api/pending_closures7123worker1 # [ 290.700080] nix-grpc-daemon[1719]: time=2026-09-23T05:17:03.204Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7124worker1 # [ 290.701666] nix-grpc-daemon[1719]: time=2026-09-23T05:17:03.204Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"7125lb # [ 290.699282] niks3-server[1111]: 2026/09/23 05:17:03 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv7126worker1 # [ 290.745240] nix-grpc-daemon[1719]: time=2026-09-23T05:17:03.251Z level=INFO msg="Uploading 1 narinfos"7127lb # [ 290.715053] niks3-server[1111]: 2026/09/23 05:17:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign7128lb # [ 290.725803] niks3-server[1111]: 2026/09/23 05:17:03 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls7129lb # [ 290.730633] niks3-server[1111]: 2026/09/23 05:17:03 INFO Signed narinfos id=87 count=17130lb # [ 290.739147] niks3-server[1111]: 2026/09/23 05:17:03 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst7131worker1 # [ 290.776788] nix-grpc-daemon[1719]: time=2026-09-23T05:17:03.283Z level=INFO msg="Upload complete. (224ms)"7132lb # [ 290.746653] niks3-server[1111]: 2026/09/23 05:17:03 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete7133lb # [ 290.753629] niks3-server[1111]: 2026/09/23 05:17:03 INFO Completed upload id=877134lb # [ 290.757454] niks3-server[1111]: 2026/09/23 05:17:03 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo7135worker1 # [ 290.789018] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55690 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=13 outputs=17136worker1 # [ 290.805246] nix-daemon[832]: reaped child process 2076, status = succeeded7137client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 17.49 seconds)7138worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7139worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7140client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'7141client # [ 291.020997] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7142client # Running as unit: gpu.service; invocation ID: 556ad90cda7d4022bd3d9bc537d5b0007143client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.10 seconds)7144lb: must succeed: curl -sf localhost:9901/clusters7145lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7146lb: must succeed: curl -sf localhost:9901/clusters7147lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7148worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7149worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7150worker1 # [ 291.207763] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:556907151client # [ 291.234240] nix[2066]: don't know how to build these paths:7152client # [ 291.235202] nix[2066]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv7153worker2 # [ 291.287755] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:473327154client # [ 291.265897] nix[2066]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker7155worker1 # [ 291.245103] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:03Z level=debug event=want client=3 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=07156lb: must succeed: curl -sf localhost:9901/clusters7157lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7158lb: must succeed: curl -sf localhost:9901/clusters7159lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7160worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7161worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7162client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7163lb # [ 299.203484] envoy[895]: conn peer=[2001:db8:1::1]:45676 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27226 tx=12434 ms=172237164lb # [ 315.538186] postgres[798]: [798] LOG: checkpoint starting: time7165client # [ 321.267438] nix[2066]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}7166client # [ 321.285392] nix[2066]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed7167client # [ 321.317761] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7168client # [ 321.326079] systemd[1]: gpu.service: Failed with result 'exit-code'.7169client # [ 321.330000] systemd[1]: gpu.service: Consumed 202ms CPU time over 30.303s wall clock time, 37.4M memory peak, 5.6K incoming IP traffic, 5.3K outgoing IP traffic.7170client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.07 seconds)7171client: must succeed: journalctl -u gpu -o cat7172client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)7173lb: must succeed: curl -sf localhost:9901/clusters7174lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7175lb: must succeed: curl -sf localhost:9901/clusters7176lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7177worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7178worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7179(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 48.29 seconds)7180subtest: low disk drains a worker and builds go to the other7181worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7182worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.14 seconds)7183worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7184worker1 # [ 323.067643] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:35Z level=info event=unhealthy reason=min_free7185worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)7186client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&27187client # don't know how to build these paths:7188client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7189worker1 # [ 324.136994] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:556907190worker1 # [ 324.144620] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:556907191worker1 # [ 324.168124] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:36Z level=debug event=want client=4 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=07192worker1 # [ 324.174609] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:36Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=17193worker2 # [ 324.291503] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473327194worker2 # [ 324.308797] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:473327195client # copying 1 paths to 'lb:50051'...7196client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...7197worker2 # [ 324.324711] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:473327198worker2 # [ 324.342717] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)7199worker2 # [ 324.518912] nix-daemon[855]: accepted connection from pid 2049, user nix-grpc-daemon (trusted)7200worker2 # [ 324.542534] nix-daemon[855]: reaped child process 2052, status = succeeded7201lb # [ 324.442181] niks3-server[1111]: 2026/09/23 05:17:36 INFO Received uploads request method=POST path=/api/pending_closures7202worker2 # [ 324.552442] nix-grpc-daemon[1805]: time=2026-09-23T05:17:37.005Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7203worker2 # [ 324.553946] nix-grpc-daemon[1805]: time=2026-09-23T05:17:37.005Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"7204lb # [ 324.483111] niks3-server[1111]: 2026/09/23 05:17:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign7205worker2 # [ 324.592591] nix-grpc-daemon[1805]: time=2026-09-23T05:17:37.045Z level=INFO msg="Uploading 1 narinfos"7206lb # [ 324.494896] niks3-server[1111]: 2026/09/23 05:17:37 INFO Signed narinfos id=88 count=17207lb # [ 324.502865] niks3-server[1111]: 2026/09/23 05:17:37 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls7208lb # [ 324.513661] niks3-server[1111]: 2026/09/23 05:17:37 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst7209lb # [ 324.526895] niks3-server[1111]: 2026/09/23 05:17:37 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete7210lb # [ 324.532296] niks3-server[1111]: 2026/09/23 05:17:37 INFO Completed upload id=887211worker2 # [ 324.632066] nix-grpc-daemon[1805]: time=2026-09-23T05:17:37.085Z level=INFO msg="Upload complete. (267ms)"7212lb # [ 324.539949] niks3-server[1111]: 2026/09/23 05:17:37 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo7213worker2 # [ 324.642335] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=0 paths=1 nar_bytes_in=5127214worker2 # [ 324.659991] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473327215worker2 # [ 324.669624] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)7216worker2 # [ 324.672483] nix-daemon[855]: reaped child process 2047, status = succeeded7217client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...7218worker2 # [ 324.685633] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)7219client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7220client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7221client # farm-a> LOG-farm-a-drain7222worker2 # [ 325.914823] nix-daemon[855]: reaped child process 2056, status = succeeded7223worker2 # [ 326.024393] nix-daemon[855]: accepted connection from pid 2072, user nix-grpc-daemon (trusted)7224worker2 # [ 326.045248] nix-daemon[855]: reaped child process 2075, status = succeeded7225lb # [ 325.950016] niks3-server[1111]: 2026/09/23 05:17:38 INFO Received uploads request method=POST path=/api/pending_closures7226worker2 # [ 326.062931] nix-grpc-daemon[1805]: time=2026-09-23T05:17:38.514Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7227worker2 # [ 326.064527] nix-grpc-daemon[1805]: time=2026-09-23T05:17:38.514Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7228lb # [ 325.992145] niks3-server[1111]: 2026/09/23 05:17:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign7229worker2 # [ 326.100083] nix-grpc-daemon[1805]: time=2026-09-23T05:17:38.553Z level=INFO msg="Uploading 1 narinfos"7230lb # [ 326.005707] niks3-server[1111]: 2026/09/23 05:17:38 INFO Signed narinfos id=89 count=17231lb # [ 326.012420] niks3-server[1111]: 2026/09/23 05:17:38 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7232lb # [ 326.024225] niks3-server[1111]: 2026/09/23 05:17:38 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7233worker2 # [ 326.132979] nix-grpc-daemon[1805]: time=2026-09-23T05:17:38.586Z level=INFO msg="Upload complete. (238ms)"7234lb # [ 326.033065] niks3-server[1111]: 2026/09/23 05:17:38 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7235lb # [ 326.038042] niks3-server[1111]: 2026/09/23 05:17:38 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete7236lb # [ 326.043127] niks3-server[1111]: 2026/09/23 05:17:38 INFO Completed upload id=897237worker2 # [ 326.144955] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=14 outputs=17238lb # [ 326.047688] niks3-server[1111]: 2026/09/23 05:17:38 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7239worker1 # [ 326.081939] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:38Z level=debug event=want client=4 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07240worker2 # [ 326.161015] nix-daemon[855]: reaped child process 2054, status = succeeded7241worker1 # [ 326.088005] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:38Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17242worker2 # [ 326.198286] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473327243worker2 # [ 326.213360] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:473327244worker2 # [ 326.228100] nix-grpc-daemon[1805]: time=2026-09-23T05:17:38.681Z level=INFO msg="All 1 paths already cached"7245client # copying 1 paths to 'lb:50051'...7246client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7247worker2 # [ 326.239441] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:473327248worker2 # [ 326.254800] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)7249worker2 # [ 326.405641] nix-daemon[855]: accepted connection from pid 2079, user nix-grpc-daemon (trusted)7250worker2 # [ 326.422469] nix-daemon[855]: reaped child process 2082, status = succeeded7251lb # [ 326.328039] niks3-server[1111]: 2026/09/23 05:17:38 INFO Received uploads request method=POST path=/api/pending_closures7252worker2 # [ 326.438631] nix-grpc-daemon[1805]: time=2026-09-23T05:17:38.891Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7253worker2 # [ 326.440349] nix-grpc-daemon[1805]: time=2026-09-23T05:17:38.891Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7254lb # [ 326.355686] niks3-server[1111]: 2026/09/23 05:17:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign7255worker2 # [ 326.463165] nix-grpc-daemon[1805]: time=2026-09-23T05:17:38.916Z level=INFO msg="Uploading 1 narinfos"7256lb # [ 326.367931] niks3-server[1111]: 2026/09/23 05:17:38 INFO Signed narinfos id=90 count=17257lb # [ 326.374448] niks3-server[1111]: 2026/09/23 05:17:38 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7258lb # [ 326.386220] niks3-server[1111]: 2026/09/23 05:17:38 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete7259worker2 # [ 326.492432] nix-grpc-daemon[1805]: time=2026-09-23T05:17:38.945Z level=INFO msg="Upload complete. (220ms)"7260lb # [ 326.391515] niks3-server[1111]: 2026/09/23 05:17:38 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7261lb # [ 326.396594] niks3-server[1111]: 2026/09/23 05:17:38 INFO Completed upload id=907262lb # [ 326.399110] niks3-server[1111]: 2026/09/23 05:17:38 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7263worker2 # [ 326.500110] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=0 paths=1 nar_bytes_in=6327264worker2 # [ 326.513589] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473327265worker2 # [ 326.519362] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)7266worker2 # [ 326.523972] nix-daemon[855]: reaped child process 2077, status = succeeded7267client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7268worker2 # [ 326.534857] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)7269client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7270client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7271client # farm-b> LOG-farm-b-drain7272worker2 # [ 327.401843] nix-daemon[855]: reaped child process 2086, status = succeeded7273worker2 # [ 327.514914] nix-daemon[855]: accepted connection from pid 2102, user nix-grpc-daemon (trusted)7274worker2 # [ 327.532198] nix-daemon[855]: reaped child process 2105, status = succeeded7275lb # [ 327.438263] niks3-server[1111]: 2026/09/23 05:17:39 INFO Received uploads request method=POST path=/api/pending_closures7276worker2 # [ 327.552414] nix-grpc-daemon[1805]: time=2026-09-23T05:17:40.002Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7277worker2 # [ 327.555590] nix-grpc-daemon[1805]: time=2026-09-23T05:17:40.002Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7278lb # [ 327.476292] niks3-server[1111]: 2026/09/23 05:17:39 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7279lb # [ 327.491820] niks3-server[1111]: 2026/09/23 05:17:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7280worker2 # [ 327.601217] nix-grpc-daemon[1805]: time=2026-09-23T05:17:40.054Z level=INFO msg="Uploading 1 narinfos"7281lb # [ 327.505602] niks3-server[1111]: 2026/09/23 05:17:40 INFO Signed narinfos id=91 count=17282lb # [ 327.512371] niks3-server[1111]: 2026/09/23 05:17:40 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7283lb # [ 327.523036] niks3-server[1111]: 2026/09/23 05:17:40 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7284worker2 # [ 327.631351] nix-grpc-daemon[1805]: time=2026-09-23T05:17:40.084Z level=INFO msg="Upload complete. (282ms)"7285lb # [ 327.532548] niks3-server[1111]: 2026/09/23 05:17:40 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7286lb # [ 327.539011] niks3-server[1111]: 2026/09/23 05:17:40 INFO Completed upload id=917287lb # [ 327.541703] niks3-server[1111]: 2026/09/23 05:17:40 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7288worker2 # [ 327.645160] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=15 outputs=17289worker1 # [ 327.582082] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:40Z level=debug event=want client=4 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07290worker1 # [ 327.587864] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:40Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17291worker2 # [ 327.664158] nix-daemon[855]: reaped child process 2084, status = succeeded7292worker2 # [ 327.698045] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473327293worker2 # [ 327.712915] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:473327294worker2 # [ 327.726590] nix-grpc-daemon[1805]: time=2026-09-23T05:17:40.179Z level=INFO msg="All 2 paths already cached"7295client # copying 1 paths to 'lb:50051'...7296worker2 # [ 327.737474] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:473327297client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7298worker2 # [ 327.752358] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)7299worker2 # [ 327.907372] nix-daemon[855]: accepted connection from pid 2109, user nix-grpc-daemon (trusted)7300worker2 # [ 327.924257] nix-daemon[855]: reaped child process 2112, status = succeeded7301lb # [ 327.828488] niks3-server[1111]: 2026/09/23 05:17:40 INFO Received uploads request method=POST path=/api/pending_closures7302worker2 # [ 327.939545] nix-grpc-daemon[1805]: time=2026-09-23T05:17:40.392Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7303worker2 # [ 327.941323] nix-grpc-daemon[1805]: time=2026-09-23T05:17:40.392Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7304lb # [ 327.857966] niks3-server[1111]: 2026/09/23 05:17:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7305worker2 # [ 327.966258] nix-grpc-daemon[1805]: time=2026-09-23T05:17:40.419Z level=INFO msg="Uploading 1 narinfos"7306lb # [ 327.870505] niks3-server[1111]: 2026/09/23 05:17:40 INFO Signed narinfos id=92 count=17307lb # [ 327.879986] niks3-server[1111]: 2026/09/23 05:17:40 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7308lb # [ 327.895755] niks3-server[1111]: 2026/09/23 05:17:40 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7309worker2 # [ 328.008815] nix-grpc-daemon[1805]: time=2026-09-23T05:17:40.461Z level=INFO msg="Upload complete. (238ms)"7310lb # [ 327.906423] niks3-server[1111]: 2026/09/23 05:17:40 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7311lb # [ 327.911295] niks3-server[1111]: 2026/09/23 05:17:40 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7312lb # [ 327.917894] niks3-server[1111]: 2026/09/23 05:17:40 INFO Completed upload id=927313worker2 # [ 328.017638] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=0 paths=1 nar_bytes_in=7607314worker2 # [ 328.033584] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:473327315worker2 # [ 328.040484] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)7316worker2 # [ 328.046033] nix-daemon[855]: reaped child process 2107, status = succeeded7317client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7318worker2 # [ 328.058193] nix-daemon[855]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)7319client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7320client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7321client # farm-top> LOG-farm-top-drain7322worker2 # [ 329.164636] nix-daemon[855]: reaped child process 2116, status = succeeded7323worker2 # [ 329.236901] nix-daemon[855]: accepted connection from pid 2132, user nix-grpc-daemon (trusted)7324worker2 # [ 329.257214] nix-daemon[855]: reaped child process 2135, status = succeeded7325lb # [ 329.161242] niks3-server[1111]: 2026/09/23 05:17:41 INFO Received uploads request method=POST path=/api/pending_closures7326worker2 # [ 329.274034] nix-grpc-daemon[1805]: time=2026-09-23T05:17:41.726Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7327worker2 # [ 329.275739] nix-grpc-daemon[1805]: time=2026-09-23T05:17:41.726Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7328lb # [ 329.204360] niks3-server[1111]: 2026/09/23 05:17:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7329worker2 # [ 329.312660] nix-grpc-daemon[1805]: time=2026-09-23T05:17:41.765Z level=INFO msg="Uploading 1 narinfos"7330lb # [ 329.218248] niks3-server[1111]: 2026/09/23 05:17:41 INFO Signed narinfos id=93 count=17331lb # [ 329.228809] envoy[895]: conn peer=[2001:db8:1::1]:46908 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=300677332lb # [ 329.236295] niks3-server[1111]: 2026/09/23 05:17:41 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7333lb # [ 329.245663] niks3-server[1111]: 2026/09/23 05:17:41 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7334worker2 # [ 329.352133] nix-grpc-daemon[1805]: time=2026-09-23T05:17:41.805Z level=INFO msg="Upload complete. (280ms)"7335lb # [ 329.251412] niks3-server[1111]: 2026/09/23 05:17:41 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7336lb # [ 329.255526] niks3-server[1111]: 2026/09/23 05:17:41 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7337lb # [ 329.262632] niks3-server[1111]: 2026/09/23 05:17:41 INFO Completed upload id=937338worker2 # [ 329.363327] nix-grpc-daemon[1723]: ts=2026-09-23T05:17:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47332 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=16 outputs=17339lb # [ 329.265872] niks3-server[1111]: 2026/09/23 05:17:41 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7340worker2 # [ 329.383029] nix-daemon[855]: reaped child process 2114, status = succeeded7341client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.43 seconds)7342client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7343client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds)7344worker1: must succeed: rm /nix/.rw-store/fill7345worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)7346worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7347worker1 # [ 330.069917] nix-grpc-daemon[1635]: ts=2026-09-23T05:17:42Z level=info event=healthy reason=min_free7348worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7349(finished: subtest: low disk drains a worker and builds go to the other, in 9.21 seconds)7350(finished: run the VM test script, in 331.87 seconds)7351test script finished in 332.12s7352cleanup7353kill QemuMachine (pid 49)7354client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7355client # [2026-09-23T05:17:43Z INFO virtiofsd] Client disconnected, shutting down7356client # [2026-09-23T05:17:43Z INFO virtiofsd] Client disconnected, shutting down7357client # [2026-09-23T05:17:43Z INFO virtiofsd] Client disconnected, shutting down7358kill QemuMachine (pid 50)7359lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7360lb # [2026-09-23T05:17:44Z INFO virtiofsd] Client disconnected, shutting down7361lb # [2026-09-23T05:17:44Z INFO virtiofsd] Client disconnected, shutting down7362lb # [2026-09-23T05:17:44Z INFO virtiofsd] Client disconnected, shutting down7363kill QemuMachine (pid 51)7364worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7365worker1 # [2026-09-23T05:17:44Z INFO virtiofsd] Client disconnected, shutting down7366worker1 # [2026-09-23T05:17:44Z INFO virtiofsd] Client disconnected, shutting down7367worker1 # [2026-09-23T05:17:44Z INFO virtiofsd] Client disconnected, shutting down7368kill QemuMachine (pid 52)7369worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7370worker2 # [2026-09-23T05:17:45Z INFO virtiofsd] Client disconnected, shutting down7371worker2 # [2026-09-23T05:17:45Z INFO virtiofsd] Client disconnected, shutting down7372worker2 # [2026-09-23T05:17:45Z INFO virtiofsd] Client disconnected, shutting down7373(finished: cleanup, in 1.61 seconds)7374additionally exposed symbols:7375 client, lb, worker1, worker2,7376 vlan1,7377 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh