vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #554
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2: QEMU running (pid 52)14lb: QEMU running (pid 50)15lb # Disk image does not exist, creating the virtualisation disk image...16worker2 # Formatting '/build/vm-state-worker2/tmp.qDedR6x5CK', fmt=raw size=107374182417lb # Formatting '/build/vm-state-lb/tmp.hDAsLQ6xCu', fmt=raw size=107374182418worker2 # mke2fs 1.47.4 (6-Mar-2025)19lb # mke2fs 1.47.4 (6-Mar-2025)20worker2 # Discarding device blocks: 0/262144 done21lb # Discarding device blocks: 0/262144 done22worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes23lb # Creating filesystem with 262144 4k blocks and 65536 inodes24worker2 # Filesystem UUID: 7be8e5b9-c3c1-4c67-82e2-cc31ea16c28a25lb # Filesystem UUID: dd36236e-2b36-4221-be14-0e277d17f91326worker2 # Superblock backups stored on blocks:27lb # Superblock backups stored on blocks:28worker2 # 32768, 98304, 163840, 22937629lb # 32768, 98304, 163840, 22937630worker2 # 31lb # 32worker1 # Disk image does not exist, creating the virtualisation disk image...33worker1: QEMU running (pid 51)34worker2 # Allocating group tables: 0/8 done35lb # Allocating group tables: 0/8 done36worker2 # Writing inode tables: 0/8 done37lb # Writing inode tables: 0/8 done38worker2 # Creating journal (8192 blocks): done39lb # Creating journal (8192 blocks): done40worker2 # Writing superblocks and filesystem accounting information: 0/8 done41lb # Writing superblocks and filesystem accounting information: 0/8 done42worker2 # 43lb # 44worker2 # Virtualisation disk image created.45lb # Virtualisation disk image created.46worker2 # Starting virtiofs daemons...47lb # Starting virtiofs daemons...48worker2 # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)49lb # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)50worker2 # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether51lb # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether52worker2 # [2026-09-25T14:58:33Z INFO virtiofsd] Waiting for vhost-user socket connection...53lb # [2026-09-25T14:58:33Z INFO virtiofsd] Waiting for vhost-user socket connection...54worker2 # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)55lb # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)56worker2 # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether57lb # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether58worker2 # [2026-09-25T14:58:33Z INFO virtiofsd] Waiting for vhost-user socket connection...59lb # [2026-09-25T14:58:33Z INFO virtiofsd] Waiting for vhost-user socket connection...60worker2 # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)61lb # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)62worker2 # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether63lb # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether64worker2 # [2026-09-25T14:58:33Z INFO virtiofsd] Waiting for vhost-user socket connection...65lb # [2026-09-25T14:58:33Z INFO virtiofsd] Waiting for vhost-user socket connection...66worker2 # [2026-09-25T14:58:33Z INFO virtiofsd] Client connected, servicing requests67lb # [2026-09-25T14:58:33Z INFO virtiofsd] Client connected, servicing requests68worker2 # [2026-09-25T14:58:33Z INFO virtiofsd] Client connected, servicing requests69lb # [2026-09-25T14:58:33Z INFO virtiofsd] Client connected, servicing requests70worker2 # [2026-09-25T14:58:33Z INFO virtiofsd] Client connected, servicing requests71lb # [2026-09-25T14:58:33Z INFO virtiofsd] Client connected, servicing requests72worker1 # Formatting '/build/vm-state-worker1/tmp.zSXc6YKcvU', fmt=raw size=107374182473worker1 # mke2fs 1.47.4 (6-Mar-2025)74worker1 # Discarding device blocks: 0/262144 done75worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes76worker1 # Filesystem UUID: 94a731a1-a9c1-407e-b62b-7191944152cb77worker1 # Superblock backups stored on blocks:78worker1 # 32768, 98304, 163840, 22937679worker1 # 80worker1 # Allocating group tables: 0/8 done81worker1 # Writing inode tables: 0/8 done82worker1 # Creating journal (8192 blocks): done83worker1 # Writing superblocks and filesystem accounting information: 0/8 done84worker1 # 85worker1 # Virtualisation disk image created.86worker1 # Starting virtiofs daemons...87worker1 # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)88worker1 # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker1 # [2026-09-25T14:58:33Z INFO virtiofsd] Waiting for vhost-user socket connection...90worker1 # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)91worker1 # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92worker1 # [2026-09-25T14:58:33Z INFO virtiofsd] Waiting for vhost-user socket connection...93worker1 # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)94worker1 # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether95worker1 # [2026-09-25T14:58:33Z INFO virtiofsd] Waiting for vhost-user socket connection...96worker1 # [2026-09-25T14:58:33Z INFO virtiofsd] Client connected, servicing requests97worker1 # [2026-09-25T14:58:33Z INFO virtiofsd] Client connected, servicing requests98worker1 # [2026-09-25T14:58:33Z INFO virtiofsd] Client connected, servicing requests99client: QEMU running (pid 49)100client # Disk image does not exist, creating the virtualisation disk image...101client # Formatting '/build/vm-state-client/tmp.DWS7rOVk4d', fmt=raw size=1073741824102(finished: start all VMs, in 0.34 seconds)103client # mke2fs 1.47.4 (6-Mar-2025)104client: waiting for unit network-addresses-eth1.service105client # Discarding device blocks: 0/262144 done106client: waiting for the VM to finish booting107client # Creating filesystem with 262144 4k blocks and 65536 inodes108client # Filesystem UUID: d874307a-9036-4fce-9348-0ddd02912399109client # Superblock backups stored on blocks:110client # 32768, 98304, 163840, 229376111client # 112client # Allocating group tables: 0/8 done113client # Writing inode tables: 0/8 done114client # Creating journal (8192 blocks): done115client # Writing superblocks and filesystem accounting information: 0/8 done116client # 117client # Virtualisation disk image created.118client # Starting virtiofs daemons...119client # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120client # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121client # [2026-09-25T14:58:33Z INFO virtiofsd] Waiting for vhost-user socket connection...122client # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123client # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124client # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)125client # [2026-09-25T14:58:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether126client # [2026-09-25T14:58:33Z INFO virtiofsd] Waiting for vhost-user socket connection...127client # [2026-09-25T14:58:33Z INFO virtiofsd] Waiting for vhost-user socket connection...128client # [2026-09-25T14:58:33Z INFO virtiofsd] Client connected, servicing requests129client # [2026-09-25T14:58:33Z INFO virtiofsd] Client connected, servicing requests130client # [2026-09-25T14:58:33Z INFO virtiofsd] Client connected, servicing requests131lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135lb # 136lb # 137lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...139lb # 140lb # 141lb # 142lb # 143worker2 # 144worker2 # 145lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00146worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00147worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...148worker2 # 149worker2 # 150worker2 # 151worker2 # 152lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...153lb # 154lb # 155worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00156client # 157client # 158client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00159worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...160worker2 # 161worker2 # 162client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...163client # 164client # 165client # 166client # 167client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00168client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...169client # 170client # 171worker1 # 172worker1 # 173worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00174worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...175worker1 # 176worker1 # 177worker1 # 178worker1 # 179worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00180worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181worker1 # 182worker1 # 183lb # Booting from ROM...184worker2 # Booting from ROM...185client # Booting from ROM...186worker1 # Booting from ROM...187lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0189lb # [ 0.000000] BIOS-provided physical RAM map:190lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved196lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved197lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved198lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved199lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved200lb # [ 0.000000] NX (Execute Disable) protection: active201lb # [ 0.000000] APIC: Static calls initialized202lb # [ 0.000000] SMBIOS 2.8 present.203lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014204lb # [ 0.000000] DMI: Memory slots populated: 1/1205lb # [ 0.000000] Hypervisor detected: KVM206lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000207lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00208lb # [ 0.000000] kvm-clock: using sched offset of 531835813 cycles209lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns210lb # [ 0.000005] tsc: Detected 2400.012 MHz processor211lb # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000212lb # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs213worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026214lb # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT215client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026216lb # [ 0.002731] found SMP MP-table at [mem 0x000f5450-0x000f545f]217lb # [ 0.002741] Using GB pages for direct mapping218lb # [ 0.002903] RAMDISK: [mem 0x5e36e000-0x5ffcffff]219lb # [ 0.002911] ACPI: Early table checksum verification disabled220lb # [ 0.002914] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )221worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zgairylzc1j9i3djbnwh9i9md3fl2hg7-nixos-system-worker2-test/init regInfo=/nix/store/lcx43qpj54674ygiwf9d9f03hycqp5g8-closure-info/registration console=ttyS0,115200n8 console=tty0222worker2 # [ 0.000000] BIOS-provided physical RAM map:223lb # [ 0.002917] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)224client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9wqg33lnyl1gb812wn81xc3mq1m01kvz-nixos-system-client-test/init regInfo=/nix/store/sz1p3lkavwn9ixnd6zfkvzvkb1ljriph-closure-info/registration console=ttyS0,115200n8 console=tty0225client # [ 0.000000] BIOS-provided physical RAM map:226worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable227lb # [ 0.002922] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)228client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable229worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved230lb # [ 0.002929] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)231client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved232worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved233lb # [ 0.002931] ACPI: FACS 0x000000005FFE0000 000040234client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved235worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable236lb # [ 0.002933] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)237client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable238worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved239client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved240lb # [ 0.002934] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)241worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved242client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved243worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved244lb # [ 0.002936] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)245client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved246worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved247lb # [ 0.002937] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)248client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved249worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved250lb # [ 0.002939] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]251client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved252worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved253lb # [ 0.002940] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]254worker2 # [ 0.000000] NX (Execute Disable) protection: active255client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved256worker2 # [ 0.000000] APIC: Static calls initialized257lb # [ 0.002940] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]258client # [ 0.000000] NX (Execute Disable) protection: active259worker2 # [ 0.000000] SMBIOS 2.8 present.260client # [ 0.000000] APIC: Static calls initialized261lb # [ 0.002941] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]262client # [ 0.000000] SMBIOS 2.8 present.263worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014264lb # [ 0.002941] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]265worker2 # [ 0.000000] DMI: Memory slots populated: 1/1266client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014267worker2 # [ 0.000000] Hypervisor detected: KVM268lb # [ 0.002942] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]269client # [ 0.000000] DMI: Memory slots populated: 1/1270worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000271client # [ 0.000000] Hypervisor detected: KVM272lb # [ 0.002942] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]273worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00274client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000275lb # [ 0.003161] No NUMA configuration found276worker2 # [ 0.000001] kvm-clock: using sched offset of 530309369 cycles277client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00278lb # [ 0.003162] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]279client # [ 0.000001] kvm-clock: using sched offset of 518642157 cycles280lb # [ 0.003164] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]281lb # [ 0.005443] Zone ranges:282worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns283worker2 # [ 0.000004] tsc: Detected 2400.012 MHz processor284lb # [ 0.005443] DMA [mem 0x0000000000001000-0x0000000000ffffff]285client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns286worker2 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000287client # [ 0.000005] tsc: Detected 2400.012 MHz processor288lb # [ 0.005445] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]289lb # [ 0.005446] Normal empty290client # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000291lb # [ 0.005447] Device empty292worker2 # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs293lb # [ 0.005448] Movable zone start for each node294client # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs295worker2 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT296lb # [ 0.005448] Early memory node ranges297client # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT298worker2 # [ 0.002730] found SMP MP-table at [mem 0x000f5450-0x000f545f]299lb # [ 0.005449] node 0: [mem 0x0000000000001000-0x000000000009efff]300worker2 # [ 0.002741] Using GB pages for direct mapping301client # [ 0.002739] found SMP MP-table at [mem 0x000f5450-0x000f545f]302lb # [ 0.005450] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]303worker2 # [ 0.002819] RAMDISK: [mem 0x5e36f000-0x5ffcffff]304client # [ 0.002750] Using GB pages for direct mapping305lb # [ 0.005451] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]306worker2 # [ 0.002828] ACPI: Early table checksum verification disabled307client # [ 0.002858] RAMDISK: [mem 0x5e370000-0x5ffcffff]308lb # [ 0.005471] On node 0, zone DMA: 1 pages in unavailable ranges309worker2 # [ 0.002831] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )310client # [ 0.002866] ACPI: Early table checksum verification disabled311lb # [ 0.005735] On node 0, zone DMA: 97 pages in unavailable ranges312client # [ 0.002869] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )313worker2 # [ 0.002834] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)314lb # [ 0.032825] On node 0, zone DMA32: 40 pages in unavailable ranges315client # [ 0.002873] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)316lb # [ 0.033274] ACPI: PM-Timer IO Port: 0x608317worker2 # [ 0.002838] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)318lb # [ 0.033285] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])319client # [ 0.002877] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)320worker2 # [ 0.002847] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)321lb # [ 0.033312] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23322worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026323client # [ 0.002884] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)324worker2 # [ 0.002849] ACPI: FACS 0x000000005FFE0000 000040325lb # [ 0.033315] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)326client # [ 0.002886] ACPI: FACS 0x000000005FFE0000 000040327worker2 # [ 0.002850] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)328lb # [ 0.033316] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)329client # [ 0.002887] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)330worker2 # [ 0.002852] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)331lb # [ 0.033317] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)332client # [ 0.002889] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)333lb # [ 0.033319] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)334worker2 # [ 0.002853] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)335lb # [ 0.033319] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)336client # [ 0.002890] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)337worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/v1j6w5h3z5q2s56m2lfbxi84hm0ia44j-nixos-system-worker1-test/init regInfo=/nix/store/gn0awja2kmjggbyr30cqfdv4hv0iz25m-closure-info/registration console=ttyS0,115200n8 console=tty0338worker2 # [ 0.002855] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)339worker1 # [ 0.000000] BIOS-provided physical RAM map:340lb # [ 0.033322] ACPI: Using ACPI (MADT) for SMP configuration information341client # [ 0.002892] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)342worker2 # [ 0.002856] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]343worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable344lb # [ 0.033323] ACPI: HPET id: 0x8086a201 base: 0xfed00000345lb # [ 0.033326] TSC deadline timer available346client # [ 0.002893] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]347worker2 # [ 0.002857] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]348worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved349lb # [ 0.033330] CPU topo: Max. logical packages: 1350worker2 # [ 0.002858] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]351client # [ 0.002894] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]352worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved353lb # [ 0.033330] CPU topo: Max. logical dies: 1354lb # [ 0.033331] CPU topo: Max. dies per package: 1355client # [ 0.002895] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]356worker2 # [ 0.002858] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]357worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable358lb # [ 0.033334] CPU topo: Max. threads per core: 1359client # [ 0.002895] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]360worker2 # [ 0.002859] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]361lb # [ 0.033335] CPU topo: Num. cores per package: 1362worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved363client # [ 0.002895] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]364lb # [ 0.033335] CPU topo: Num. threads per package: 1365worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved366worker2 # [ 0.002859] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]367lb # [ 0.033335] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs368client # [ 0.002896] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]369worker2 # [ 0.002860] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]370worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved371worker2 # [ 0.003100] No NUMA configuration found372lb # [ 0.033352] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()373client # [ 0.002896] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]374worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved375client # [ 0.003123] No NUMA configuration found376worker2 # [ 0.003101] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]377lb # [ 0.033380] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]378worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved379client # [ 0.003124] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]380worker2 # [ 0.003103] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]381worker2 # [ 0.005502] Zone ranges:382lb # [ 0.033382] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]383worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved384client # [ 0.003126] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]385worker2 # [ 0.005503] DMA [mem 0x0000000000001000-0x0000000000ffffff]386client # [ 0.005454] Zone ranges:387lb # [ 0.033383] [mem 0x60000000-0xafffffff] available for PCI devices388worker1 # [ 0.000000] NX (Execute Disable) protection: active389lb # [ 0.033385] Booting paravirtualized kernel on KVM390worker1 # [ 0.000000] APIC: Static calls initialized391worker2 # [ 0.005504] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]392client # [ 0.005455] DMA [mem 0x0000000000001000-0x0000000000ffffff]393worker2 # [ 0.005505] Normal empty394worker1 # [ 0.000000] SMBIOS 2.8 present.395worker2 # [ 0.005506] Device empty396client # [ 0.005456] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]397client # [ 0.005457] Normal empty398worker2 # [ 0.005507] Movable zone start for each node399client # [ 0.005458] Device empty400worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014401worker2 # [ 0.005507] Early memory node ranges402lb # [ 0.033388] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns403client # [ 0.005459] Movable zone start for each node404lb # [ 0.037867] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1405worker1 # [ 0.000000] DMI: Memory slots populated: 1/1406worker2 # [ 0.005508] node 0: [mem 0x0000000000001000-0x000000000009efff]407client # [ 0.005460] Early memory node ranges408worker1 # [ 0.000000] Hypervisor detected: KVM409lb # [ 0.040345] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152410client # [ 0.005460] node 0: [mem 0x0000000000001000-0x000000000009efff]411worker2 # [ 0.005509] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]412worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000413lb # [ 0.040388] kvm-guest: PV spinlocks disabled, single CPU414worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00415client # [ 0.005461] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]416worker2 # [ 0.005510] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]417worker1 # [ 0.000001] kvm-clock: using sched offset of 572826336 cycles418worker2 # [ 0.005531] On node 0, zone DMA: 1 pages in unavailable ranges419client # [ 0.005462] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]420worker2 # [ 0.005806] On node 0, zone DMA: 97 pages in unavailable ranges421client # [ 0.005483] On node 0, zone DMA: 1 pages in unavailable ranges422worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns423client # [ 0.005757] On node 0, zone DMA: 97 pages in unavailable ranges424worker2 # [ 0.034006] On node 0, zone DMA32: 40 pages in unavailable ranges425worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor426worker2 # [ 0.034477] ACPI: PM-Timer IO Port: 0x608427client # [ 0.032948] On node 0, zone DMA32: 40 pages in unavailable ranges428worker1 # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000429lb # [ 0.040390] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0430worker2 # [ 0.034489] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])431client # [ 0.033403] ACPI: PM-Timer IO Port: 0x608432worker1 # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs433client # [ 0.033414] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])434worker2 # [ 0.034518] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23435worker1 # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT436worker2 # [ 0.034520] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)437client # [ 0.033440] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23438lb # [ 0.040488] Unknown kernel command line parameters "regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration", will be passed to user space.439lb # [ 0.040500] random: crng init done440worker1 # [ 0.002724] found SMP MP-table at [mem 0x000f5450-0x000f545f]441client # [ 0.033443] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)442worker2 # [ 0.034522] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)443worker1 # [ 0.002736] Using GB pages for direct mapping444lb # [ 0.040501] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes445client # [ 0.033444] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)446worker1 # [ 0.002860] RAMDISK: [mem 0x5e370000-0x5ffcffff]447worker2 # [ 0.034524] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)448client # [ 0.033445] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)449lb # [ 0.042673] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)450worker1 # [ 0.002868] ACPI: Early table checksum verification disabled451worker2 # [ 0.034525] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)452worker1 # [ 0.002871] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )453client # [ 0.033446] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)454worker2 # [ 0.034525] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)455lb # [ 0.043751] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)456lb # [ 0.043781] Fallback order for Node 0: 0457client # [ 0.033447] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)458worker2 # [ 0.034528] ACPI: Using ACPI (MADT) for SMP configuration information459worker1 # [ 0.002875] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)460worker2 # [ 0.034529] ACPI: HPET id: 0x8086a201 base: 0xfed00000461lb # [ 0.043784] Built 1 zonelists, mobility grouping on. Total pages: 393078462client # [ 0.033449] ACPI: Using ACPI (MADT) for SMP configuration information463lb # [ 0.043785] Policy zone: DMA32464worker2 # [ 0.034532] TSC deadline timer available465worker1 # [ 0.002879] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)466client # [ 0.033450] ACPI: HPET id: 0x8086a201 base: 0xfed00000467worker2 # [ 0.034537] CPU topo: Max. logical packages: 1468lb # [ 0.046289] mem auto-init: stack:all(zero), heap alloc:on, heap free:off469client # [ 0.033453] TSC deadline timer available470worker2 # [ 0.034537] CPU topo: Max. logical dies: 1471worker1 # [ 0.002886] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)472client # [ 0.033457] CPU topo: Max. logical packages: 1473lb # [ 0.049365] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1474worker2 # [ 0.034538] CPU topo: Max. dies per package: 1475worker1 # [ 0.002888] ACPI: FACS 0x000000005FFE0000 000040476client # [ 0.033458] CPU topo: Max. logical dies: 1477lb # [ 0.053016] allocated 3145728 bytes of page_ext478worker2 # [ 0.034541] CPU topo: Max. threads per core: 1479client # [ 0.033458] CPU topo: Max. dies per package: 1480lb # [ 0.062607] ftrace: allocating 48787 entries in 192 pages481worker1 # [ 0.002890] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)482worker2 # [ 0.034542] CPU topo: Num. cores per package: 1483client # [ 0.033461] CPU topo: Max. threads per core: 1484lb # [ 0.062609] ftrace: allocated 192 pages with 2 groups485worker2 # [ 0.034542] CPU topo: Num. threads per package: 1486client # [ 0.033462] CPU topo: Num. cores per package: 1487lb # [ 0.063447] Dynamic Preempt: lazy488worker1 # [ 0.002891] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)489worker2 # [ 0.034542] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs490client # [ 0.033462] CPU topo: Num. threads per package: 1491lb # [ 0.063572] rcu: Preemptible hierarchical RCU implementation.492worker1 # [ 0.002893] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)493client # [ 0.033463] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs494worker2 # [ 0.034562] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()495lb # [ 0.063573] rcu: RCU event tracing is enabled.496lb # [ 0.063573] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.497client # [ 0.033481] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()498worker1 # [ 0.002894] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)499worker2 # [ 0.034594] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]500lb # [ 0.063574] Trampoline variant of Tasks RCU enabled.501client # [ 0.033521] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]502worker1 # [ 0.002896] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]503lb # [ 0.063575] Rude variant of Tasks RCU enabled.504worker2 # [ 0.034596] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]505lb # [ 0.063575] Tracing variant of Tasks RCU enabled.506worker1 # [ 0.002897] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]507worker2 # [ 0.034598] [mem 0x60000000-0xafffffff] available for PCI devices508client # [ 0.033523] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]509worker2 # [ 0.034599] Booting paravirtualized kernel on KVM510client # [ 0.033525] [mem 0x60000000-0xafffffff] available for PCI devices511worker1 # [ 0.002897] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]512lb # [ 0.063576] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.513client # [ 0.033526] Booting paravirtualized kernel on KVM514lb # [ 0.063576] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1515worker1 # [ 0.002898] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]516worker2 # [ 0.034602] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns517worker1 # [ 0.002898] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]518client # [ 0.033529] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns519worker2 # [ 0.039099] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1520lb # [ 0.063630] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.521worker1 # [ 0.002899] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]522client # [ 0.038011] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1523worker2 # [ 0.041893] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152524lb # [ 0.063632] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.525worker2 # [ 0.041943] kvm-guest: PV spinlocks disabled, single CPU526worker1 # [ 0.002899] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]527client # [ 0.040492] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152528worker1 # [ 0.003131] No NUMA configuration found529lb # [ 0.063632] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.530client # [ 0.040545] kvm-guest: PV spinlocks disabled, single CPU531worker1 # [ 0.003131] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]532lb # [ 0.068002] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16533worker1 # [ 0.003134] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]534lb # [ 0.068280] rcu: srcu_init: Setting srcu_struct sizes based on contention.535worker1 # [ 0.005511] Zone ranges:536worker1 # [ 0.005512] DMA [mem 0x0000000000001000-0x0000000000ffffff]537lb # [ 0.068287] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns538worker2 # [ 0.041944] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/zgairylzc1j9i3djbnwh9i9md3fl2hg7-nixos-system-worker2-test/init regInfo=/nix/store/lcx43qpj54674ygiwf9d9f03hycqp5g8-closure-info/registration console=ttyS0,115200n8 console=tty0539worker1 # [ 0.005513] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]540worker1 # [ 0.005515] Normal empty541client # [ 0.040546] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9wqg33lnyl1gb812wn81xc3mq1m01kvz-nixos-system-client-test/init regInfo=/nix/store/sz1p3lkavwn9ixnd6zfkvzvkb1ljriph-closure-info/registration console=ttyS0,115200n8 console=tty0542worker1 # [ 0.005516] Device empty543lb # [ 0.068387] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)544lb # [ 0.071899] Console: colour VGA+ 80x25545worker1 # [ 0.005516] Movable zone start for each node546worker2 # [ 0.042042] Unknown kernel command line parameters "regInfo=/nix/store/lcx43qpj54674ygiwf9d9f03hycqp5g8-closure-info/registration", will be passed to user space.547worker1 # [ 0.005517] Early memory node ranges548lb # [ 0.071902] printk: legacy console [tty0] enabled549worker2 # [ 0.042054] random: crng init done550lb # [ 0.101262] printk: legacy console [ttyS0] enabled551client # [ 0.040637] Unknown kernel command line parameters "regInfo=/nix/store/sz1p3lkavwn9ixnd6zfkvzvkb1ljriph-closure-info/registration", will be passed to user space.552worker1 # [ 0.005517] node 0: [mem 0x0000000000001000-0x000000000009efff]553client # [ 0.040650] random: crng init done554worker2 # [ 0.042055] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes555lb # [ 0.199158] ACPI: Core revision 20250807556worker1 # [ 0.005519] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]557client # [ 0.040650] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes558worker2 # [ 0.044325] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)559worker1 # [ 0.005520] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]560client # [ 0.042802] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)561lb # [ 0.200035] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns562worker2 # [ 0.045464] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)563worker1 # [ 0.005540] On node 0, zone DMA: 1 pages in unavailable ranges564worker2 # [ 0.045496] Fallback order for Node 0: 0565worker1 # [ 0.005819] On node 0, zone DMA: 97 pages in unavailable ranges566client # [ 0.043881] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)567lb # [ 0.201503] APIC: Switch to symmetric I/O mode setup568client # [ 0.043913] Fallback order for Node 0: 0569worker2 # [ 0.045499] Built 1 zonelists, mobility grouping on. Total pages: 393078570worker1 # [ 0.033848] On node 0, zone DMA32: 40 pages in unavailable ranges571lb # [ 0.202421] x2apic enabled572worker2 # [ 0.045500] Policy zone: DMA32573worker1 # [ 0.034314] ACPI: PM-Timer IO Port: 0x608574client # [ 0.043916] Built 1 zonelists, mobility grouping on. Total pages: 393078575client # [ 0.043917] Policy zone: DMA32576worker2 # [ 0.048147] mem auto-init: stack:all(zero), heap alloc:on, heap free:off577worker1 # [ 0.034325] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])578lb # [ 0.203146] APIC: Switched APIC routing to: physical x2apic579client # [ 0.046406] mem auto-init: stack:all(zero), heap alloc:on, heap free:off580worker2 # [ 0.051503] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1581worker1 # [ 0.034353] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23582worker2 # [ 0.055323] allocated 3145728 bytes of page_ext583client # [ 0.049497] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1584worker1 # [ 0.034355] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)585worker2 # [ 0.065572] ftrace: allocating 48787 entries in 192 pages586lb # [ 0.204913] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1587client # [ 0.053136] allocated 3145728 bytes of page_ext588worker1 # [ 0.034357] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)589worker2 # [ 0.065574] ftrace: allocated 192 pages with 2 groups590client # [ 0.062788] ftrace: allocating 48787 entries in 192 pages591worker2 # [ 0.066450] Dynamic Preempt: lazy592worker1 # [ 0.034358] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)593client # [ 0.062790] ftrace: allocated 192 pages with 2 groups594worker2 # [ 0.066622] rcu: Preemptible hierarchical RCU implementation.595lb # [ 0.205842] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns596client # [ 0.063638] Dynamic Preempt: lazy597worker1 # [ 0.034359] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)598worker2 # [ 0.066622] rcu: RCU event tracing is enabled.599client # [ 0.063765] rcu: Preemptible hierarchical RCU implementation.600worker1 # [ 0.034360] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)601lb # [ 0.207394] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)602worker2 # [ 0.066623] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.603client # [ 0.063765] rcu: RCU event tracing is enabled.604worker2 # [ 0.066625] Trampoline variant of Tasks RCU enabled.605worker1 # [ 0.034362] ACPI: Using ACPI (MADT) for SMP configuration information606client # [ 0.063766] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.607lb # [ 0.209477] x86/cpu: User Mode Instruction Prevention (UMIP) activated608worker2 # [ 0.066625] Rude variant of Tasks RCU enabled.609client # [ 0.063767] Trampoline variant of Tasks RCU enabled.610worker1 # [ 0.034363] ACPI: HPET id: 0x8086a201 base: 0xfed00000611worker1 # [ 0.034367] TSC deadline timer available612worker2 # [ 0.066626] Tracing variant of Tasks RCU enabled.613client # [ 0.063768] Rude variant of Tasks RCU enabled.614lb # [ 0.210507] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127615worker1 # [ 0.034371] CPU topo: Max. logical packages: 1616client # [ 0.063768] Tracing variant of Tasks RCU enabled.617worker2 # [ 0.066626] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.618worker1 # [ 0.034372] CPU topo: Max. logical dies: 1619lb # [ 0.211391] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0620client # [ 0.063769] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.621worker1 # [ 0.034372] CPU topo: Max. dies per package: 1622worker2 # [ 0.066627] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1623worker1 # [ 0.034375] CPU topo: Max. threads per core: 1624client # [ 0.063769] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1625worker2 # [ 0.066637] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.626worker1 # [ 0.034376] CPU topo: Num. cores per package: 1627lb # [ 0.212395] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto628client # [ 0.063832] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.629worker1 # [ 0.034376] CPU topo: Num. threads per package: 1630worker2 # [ 0.066638] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.631worker1 # [ 0.034377] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs632lb # [ 0.213391] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl633client # [ 0.063834] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.634worker2 # [ 0.066639] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.635worker1 # [ 0.034397] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()636lb # [ 0.215392] Transient Scheduler Attacks: Vulnerable: No microcode637client # [ 0.063835] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.638worker2 # [ 0.071088] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16639worker1 # [ 0.034428] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]640client # [ 0.068206] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16641lb # [ 0.216391] Spectre V2 : Mitigation: Enhanced / Automatic IBRS642worker2 # [ 0.071365] rcu: srcu_init: Setting srcu_struct sizes based on contention.643worker1 # [ 0.034429] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]644client # [ 0.068490] rcu: srcu_init: Setting srcu_struct sizes based on contention.645lb # [ 0.217391] Speculative Return Stack Overflow: Mitigation: Safe RET646worker1 # [ 0.034430] [mem 0x60000000-0xafffffff] available for PCI devices647worker2 # [ 0.071371] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns648worker1 # [ 0.034432] Booting paravirtualized kernel on KVM649client # [ 0.068496] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns650worker2 # [ 0.071479] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)651worker2 # [ 0.075015] Console: colour VGA+ 80x25652worker1 # [ 0.034435] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns653client # [ 0.068605] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)654worker2 # [ 0.075018] printk: legacy console [tty0] enabled655client # [ 0.072134] Console: colour VGA+ 80x25656worker2 # [ 0.104493] printk: legacy console [ttyS0] enabled657worker1 # [ 0.038913] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1658client # [ 0.072137] printk: legacy console [tty0] enabled659client # [ 0.101606] printk: legacy console [ttyS0] enabled660worker2 # [ 0.200226] ACPI: Core revision 20250807661lb # [ 0.218322] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization662worker1 # [ 0.041166] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152663worker1 # [ 0.041215] kvm-guest: PV spinlocks disabled, single CPU664client # [ 0.197353] ACPI: Core revision 20250807665lb # [ 0.219398] Spectre V2 : Enabling IBPB for BPF666worker2 # [ 0.201133] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns667lb # [ 0.220896] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier668client # [ 0.198209] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns669worker2 # [ 0.202674] APIC: Switch to symmetric I/O mode setup670lb # [ 0.221392] active return thunk: srso_alias_return_thunk671client # [ 0.199704] APIC: Switch to symmetric I/O mode setup672worker2 # [ 0.203666] x2apic enabled673client # [ 0.200685] x2apic enabled674lb # [ 0.223009] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'675worker1 # [ 0.041216] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/v1j6w5h3z5q2s56m2lfbxi84hm0ia44j-nixos-system-worker1-test/init regInfo=/nix/store/gn0awja2kmjggbyr30cqfdv4hv0iz25m-closure-info/registration console=ttyS0,115200n8 console=tty0676worker2 # [ 0.204464] APIC: Switched APIC routing to: physical x2apic677client # [ 0.201462] APIC: Switched APIC routing to: physical x2apic678lb # [ 0.224266] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'679worker1 # [ 0.041312] Unknown kernel command line parameters "regInfo=/nix/store/gn0awja2kmjggbyr30cqfdv4hv0iz25m-closure-info/registration", will be passed to user space.680lb # [ 0.225122] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'681worker2 # [ 0.206351] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1682worker1 # [ 0.041324] random: crng init done683client # [ 0.203283] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1684lb # [ 0.226141] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'685worker1 # [ 0.041325] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes686lb # [ 0.227116] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'687worker2 # [ 0.207334] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns688worker1 # [ 0.043546] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)689client # [ 0.204258] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns690lb # [ 0.228098] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'691worker1 # [ 0.044935] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)692worker2 # [ 0.208985] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)693worker1 # [ 0.044966] Fallback order for Node 0: 0694client # [ 0.205909] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)695lb # [ 0.228391] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'696worker1 # [ 0.044969] Built 1 zonelists, mobility grouping on. Total pages: 393078697worker2 # [ 0.210071] x86/cpu: User Mode Instruction Prevention (UMIP) activated698worker1 # [ 0.044971] Policy zone: DMA32699client # [ 0.206992] x86/cpu: User Mode Instruction Prevention (UMIP) activated700lb # [ 0.230274] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256701worker2 # [ 0.211103] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127702worker1 # [ 0.047549] mem auto-init: stack:all(zero), heap alloc:on, heap free:off703lb # [ 0.231086] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64704client # [ 0.208022] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127705worker1 # [ 0.050855] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1706worker2 # [ 0.212705] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0707worker1 # [ 0.054305] allocated 3145728 bytes of page_ext708lb # [ 0.232078] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512709client # [ 0.208907] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0710worker1 # [ 0.064453] ftrace: allocating 48787 entries in 192 pages711lb # [ 0.232391] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024712worker1 # [ 0.064455] ftrace: allocated 192 pages with 2 groups713worker2 # [ 0.213733] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto714worker1 # [ 0.065326] Dynamic Preempt: lazy715client # [ 0.209910] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto716lb # [ 0.233391] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8717worker1 # [ 0.065458] rcu: Preemptible hierarchical RCU implementation.718worker2 # [ 0.214982] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl719worker1 # [ 0.065459] rcu: RCU event tracing is enabled.720client # [ 0.211906] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl721lb # [ 0.234391] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.722worker1 # [ 0.065460] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.723worker2 # [ 0.215982] Transient Scheduler Attacks: Vulnerable: No microcode724client # [ 0.212906] Transient Scheduler Attacks: Vulnerable: No microcode725worker1 # [ 0.065461] Trampoline variant of Tasks RCU enabled.726worker2 # [ 0.217688] Spectre V2 : Mitigation: Enhanced / Automatic IBRS727worker1 # [ 0.065462] Rude variant of Tasks RCU enabled.728client # [ 0.213906] Spectre V2 : Mitigation: Enhanced / Automatic IBRS729worker1 # [ 0.065462] Tracing variant of Tasks RCU enabled.730worker2 # [ 0.218689] Speculative Return Stack Overflow: Mitigation: Safe RET731client # [ 0.214906] Speculative Return Stack Overflow: Mitigation: Safe RET732worker1 # [ 0.065462] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.733worker1 # [ 0.065463] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1734worker1 # [ 0.065523] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.735worker1 # [ 0.065525] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.736worker2 # [ 0.218982] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization737client # [ 0.215906] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization738worker2 # [ 0.220990] Spectre V2 : Enabling IBPB for BPF739worker1 # [ 0.065526] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.740client # [ 0.217913] Spectre V2 : Enabling IBPB for BPF741worker1 # [ 0.070332] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16742worker2 # [ 0.221983] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier743worker1 # [ 0.070614] rcu: srcu_init: Setting srcu_struct sizes based on contention.744client # [ 0.218907] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier745worker2 # [ 0.222982] active return thunk: srso_alias_return_thunk746client # [ 0.219907] active return thunk: srso_alias_return_thunk747worker1 # [ 0.070622] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns748worker2 # [ 0.224633] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'749client # [ 0.220927] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'750worker1 # [ 0.070733] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)751worker2 # [ 0.225881] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'752worker1 # [ 0.074255] Console: colour VGA+ 80x25753client # [ 0.221907] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'754worker1 # [ 0.074258] printk: legacy console [tty0] enabled755worker2 # [ 0.226735] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'756worker1 # [ 0.103707] printk: legacy console [ttyS0] enabled757client # [ 0.223639] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'758worker2 # [ 0.227704] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'759worker1 # [ 0.200632] ACPI: Core revision 20250807760client # [ 0.224644] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'761worker2 # [ 0.228745] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'762client # [ 0.224906] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'763worker1 # [ 0.201534] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns764worker2 # [ 0.229734] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'765client # [ 0.226647] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'766worker1 # [ 0.203078] APIC: Switch to symmetric I/O mode setup767worker2 # [ 0.230767] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'768worker1 # [ 0.204090] x2apic enabled769client # [ 0.227703] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'770worker2 # [ 0.231982] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256771worker1 # [ 0.204867] APIC: Switched APIC routing to: physical x2apic772client # [ 0.228906] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256773worker2 # [ 0.232982] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64774client # [ 0.229906] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64775worker2 # [ 0.233982] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512776worker1 # [ 0.206731] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1777client # [ 0.230906] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512778worker2 # [ 0.234982] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024779client # [ 0.231906] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024780worker2 # [ 0.235982] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8781worker1 # [ 0.207741] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns782client # [ 0.232906] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8783worker1 # [ 0.209481] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)784worker2 # [ 0.236982] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.785client # [ 0.233906] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.786worker1 # [ 0.210567] x86/cpu: User Mode Instruction Prevention (UMIP) activated787worker1 # [ 0.212586] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127788worker1 # [ 0.213477] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0789worker1 # [ 0.214483] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto790worker1 # [ 0.215479] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl791worker1 # [ 0.216479] Transient Scheduler Attacks: Vulnerable: No microcode792worker1 # [ 0.217479] Spectre V2 : Mitigation: Enhanced / Automatic IBRS793worker1 # [ 0.218478] Speculative Return Stack Overflow: Mitigation: Safe RET794lb # [ 0.268198] Freeing SMP alternatives memory: 44K795lb # [ 0.268965] pid_max: default: 32768 minimum: 301796lb # [ 0.269487] LSM: initializing lsm=capability,landlock,yama,bpf,ima797lb # [ 0.270506] landlock: Up and running.798lb # [ 0.271392] Yama: becoming mindful.799worker1 # [ 0.219478] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization800worker1 # [ 0.221487] Spectre V2 : Enabling IBPB for BPF801lb # [ 0.272573] LSM support for eBPF active802lb # [ 0.273321] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)803worker1 # [ 0.222479] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier804worker1 # [ 0.223479] active return thunk: srso_alias_return_thunk805lb # [ 0.274433] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)806worker1 # [ 0.224379] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'807worker1 # [ 0.225478] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'808lb # [ 0.276923] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)809worker1 # [ 0.226478] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'810lb # [ 0.277914] Performance Events: Fam17h+ core perfctr, AMD PMU driver.811worker1 # [ 0.227479] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'812lb # [ 0.278396] ... version: 2813worker1 # [ 0.228479] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'814lb # [ 0.279138] ... bit width: 48815lb # [ 0.279394] ... generic counters: 6816worker1 # [ 0.229478] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'817lb # [ 0.280125] ... generic bitmap: 000000000000003f818worker1 # [ 0.230479] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'819lb # [ 0.280439] ... fixed-purpose counters: 0820worker1 # [ 0.232479] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256821lb # [ 0.281157] ... fixed-purpose bitmap: 0000000000000000822worker1 # [ 0.233479] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64823lb # [ 0.281394] ... value mask: 0000ffffffffffff824worker1 # [ 0.234479] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512825lb # [ 0.282322] ... max period: 00007fffffffffff826lb # [ 0.283117] ... global_ctrl mask: 000000000000003f827worker1 # [ 0.235479] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024828lb # [ 0.283502] signal: max sigframe size: 3376829worker1 # [ 0.236478] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8830lb # [ 0.284310] rcu: Hierarchical SRCU implementation.831lb # [ 0.285034] rcu: Max phase no-delay instances is 400.832worker1 # [ 0.237478] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.833client # [ 0.267815] Freeing SMP alternatives memory: 44K834worker2 # [ 0.271789] Freeing SMP alternatives memory: 44K835client # [ 0.268488] pid_max: default: 32768 minimum: 301836worker2 # [ 0.272564] pid_max: default: 32768 minimum: 301837client # [ 0.268993] LSM: initializing lsm=capability,landlock,yama,bpf,ima838worker2 # [ 0.273082] LSM: initializing lsm=capability,landlock,yama,bpf,ima839client # [ 0.270008] landlock: Up and running.840worker2 # [ 0.274080] landlock: Up and running.841client # [ 0.270907] Yama: becoming mindful.842lb # [ 0.289935] smp: Bringing up secondary CPUs ...843worker2 # [ 0.274983] Yama: becoming mindful.844client # [ 0.272076] LSM support for eBPF active845lb # [ 0.290407] smp: Brought up 1 node, 1 CPU846worker2 # [ 0.276134] LSM support for eBPF active847lb # [ 0.291078] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)848client # [ 0.272845] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)849worker2 # [ 0.276905] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)850client # [ 0.273945] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)851worker2 # [ 0.278022] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)852lb # [ 0.291564] Memory: 1454592K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)853lb # [ 0.292623] devtmpfs: initialized854lb # [ 0.293487] x86/mm: Memory block size: 128MB855client # [ 0.276418] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)856worker2 # [ 0.280448] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)857client # [ 0.277450] Performance Events: Fam17h+ core perfctr, AMD PMU driver.858worker2 # [ 0.281534] Performance Events: Fam17h+ core perfctr, AMD PMU driver.859lb # [ 0.295206] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)860client # [ 0.277911] ... version: 2861worker2 # [ 0.281987] ... version: 2862client # [ 0.278644] ... bit width: 48863worker2 # [ 0.282709] ... bit width: 48864lb # [ 0.296317] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).865client # [ 0.278908] ... generic counters: 6866worker2 # [ 0.282984] ... generic counters: 6867lb # [ 0.297467] pinctrl core: initialized pinctrl subsystem868client # [ 0.279607] ... generic bitmap: 000000000000003f869worker2 # [ 0.283729] ... generic bitmap: 000000000000003f870client # [ 0.279919] ... fixed-purpose counters: 0871lb # [ 0.298610] PM: RTC time: 14:58:33, date: 2026-09-25872worker2 # [ 0.284047] ... fixed-purpose counters: 0873client # [ 0.280634] ... fixed-purpose bitmap: 0000000000000000874worker2 # [ 0.284776] ... fixed-purpose bitmap: 0000000000000000875client # [ 0.280908] ... value mask: 0000ffffffffffff876worker2 # [ 0.284985] ... value mask: 0000ffffffffffff877client # [ 0.281829] ... max period: 00007fffffffffff878worker2 # [ 0.285900] ... max period: 00007fffffffffff879client # [ 0.282620] ... global_ctrl mask: 000000000000003f880worker2 # [ 0.286703] ... global_ctrl mask: 000000000000003f881lb # [ 0.302423] NET: Registered PF_NETLINK/PF_ROUTE protocol family882client # [ 0.283032] signal: max sigframe size: 3376883worker2 # [ 0.287099] signal: max sigframe size: 3376884client # [ 0.283814] rcu: Hierarchical SRCU implementation.885worker2 # [ 0.287899] rcu: Hierarchical SRCU implementation.886lb # [ 0.303908] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations887client # [ 0.284479] rcu: Max phase no-delay instances is 400.888worker2 # [ 0.288540] rcu: Max phase no-delay instances is 400.889lb # [ 0.304416] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations890lb # [ 0.305683] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations891lb # [ 0.306403] audit: initializing netlink subsys (disabled)892lb # [ 0.307621] thermal_sys: Registered thermal governor 'fair_share'893client # [ 0.289460] smp: Bringing up secondary CPUs ...894worker2 # [ 0.293621] smp: Bringing up secondary CPUs ...895lb # [ 0.307623] thermal_sys: Registered thermal governor 'bang_bang'896client # [ 0.289922] smp: Brought up 1 node, 1 CPU897worker2 # [ 0.293999] smp: Brought up 1 node, 1 CPU898client # [ 0.290562] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)899lb # [ 0.308398] audit: type=2000 audit(1790348314.641:1): state=initialized audit_enabled=0 res=1900worker2 # [ 0.294653] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)901lb # [ 0.310395] thermal_sys: Registered thermal governor 'step_wise'902lb # [ 0.310396] thermal_sys: Registered thermal governor 'user_space'903client # [ 0.291127] Memory: 1454592K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)904worker2 # [ 0.295226] Memory: 1454600K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)905lb # [ 0.311346] thermal_sys: Registered thermal governor 'power_allocator'906client # [ 0.292135] devtmpfs: initialized907lb # [ 0.312121] cpuidle: using governor menu908worker2 # [ 0.296268] devtmpfs: initialized909client # [ 0.292963] x86/mm: Memory block size: 128MB910worker1 # [ 0.272631] Freeing SMP alternatives memory: 44K911worker2 # [ 0.297109] x86/mm: Memory block size: 128MB912worker1 # [ 0.273446] pid_max: default: 32768 minimum: 301913lb # [ 0.313596] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5914worker1 # [ 0.274146] LSM: initializing lsm=capability,landlock,yama,bpf,ima915client # [ 0.294739] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)916lb # [ 0.314675] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]917worker2 # [ 0.298908] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)918worker1 # [ 0.275334] landlock: Up and running.919worker1 # [ 0.275907] Yama: becoming mindful.920client # [ 0.295832] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).921lb # [ 0.315397] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry922worker2 # [ 0.299881] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).923worker1 # [ 0.276692] LSM support for eBPF active924client # [ 0.296984] pinctrl core: initialized pinctrl subsystem925lb # [ 0.316405] PCI: Using configuration type 1 for base access926worker2 # [ 0.301070] pinctrl core: initialized pinctrl subsystem927client # [ 0.298050] PM: RTC time: 14:58:33, date: 2026-09-25928worker1 # [ 0.277621] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)929lb # [ 0.317474] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.930worker2 # [ 0.302211] PM: RTC time: 14:58:33, date: 2026-09-25931worker1 # [ 0.278518] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)932client # [ 0.301675] NET: Registered PF_NETLINK/PF_ROUTE protocol family933worker1 # [ 0.281373] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)934worker2 # [ 0.306186] NET: Registered PF_NETLINK/PF_ROUTE protocol family935client # [ 0.302386] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations936worker1 # [ 0.282035] Performance Events: Fam17h+ core perfctr, AMD PMU driver.937worker1 # [ 0.282484] ... version: 2938worker2 # [ 0.307512] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations939client # [ 0.302930] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations940worker1 # [ 0.283192] ... bit width: 48941lb # [ 0.324408] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages942worker2 # [ 0.308010] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations943worker1 # [ 0.283481] ... generic counters: 6944client # [ 0.304515] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations945lb # [ 0.325394] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page946worker1 # [ 0.284219] ... generic bitmap: 000000000000003f947client # [ 0.304918] audit: initializing netlink subsys (disabled)948worker2 # [ 0.309272] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations949worker1 # [ 0.284528] ... fixed-purpose counters: 0950worker2 # [ 0.309994] audit: initializing netlink subsys (disabled)951worker1 # [ 0.285215] ... fixed-purpose bitmap: 0000000000000000952client # [ 0.306053] thermal_sys: Registered thermal governor 'fair_share'953worker1 # [ 0.285482] ... value mask: 0000ffffffffffff954client # [ 0.306055] thermal_sys: Registered thermal governor 'bang_bang'955worker2 # [ 0.310986] audit: type=2000 audit(1790348314.666:1): state=initialized audit_enabled=0 res=1956worker1 # [ 0.286360] ... max period: 00007fffffffffff957client # [ 0.306910] thermal_sys: Registered thermal governor 'step_wise'958worker2 # [ 0.312191] thermal_sys: Registered thermal governor 'fair_share'959worker1 # [ 0.287304] ... global_ctrl mask: 000000000000003f960client # [ 0.307833] thermal_sys: Registered thermal governor 'user_space'961lb # [ 0.330395] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages962worker2 # [ 0.312193] thermal_sys: Registered thermal governor 'bang_bang'963worker1 # [ 0.288305] signal: max sigframe size: 3376964lb # [ 0.331394] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page965worker2 # [ 0.312986] thermal_sys: Registered thermal governor 'step_wise'966client # [ 0.308629] audit: type=2000 audit(1790348314.664:1): state=initialized audit_enabled=0 res=1967worker1 # [ 0.289138] rcu: Hierarchical SRCU implementation.968worker2 # [ 0.313913] thermal_sys: Registered thermal governor 'user_space'969client # [ 0.309911] thermal_sys: Registered thermal governor 'power_allocator'970worker1 # [ 0.289484] rcu: Max phase no-delay instances is 400.971client # [ 0.309925] cpuidle: using governor menu972worker2 # [ 0.314672] thermal_sys: Registered thermal governor 'power_allocator'973worker2 # [ 0.315007] cpuidle: using governor menu974client # [ 0.312132] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5975worker2 # [ 0.317235] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5976client # [ 0.313136] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]977worker2 # [ 0.318225] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]978client # [ 0.313911] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry979worker1 # [ 0.294934] smp: Bringing up secondary CPUs ...980client # [ 0.314921] PCI: Using configuration type 1 for base access981worker2 # [ 0.318988] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry982worker1 # [ 0.295493] smp: Brought up 1 node, 1 CPU983worker2 # [ 0.319996] PCI: Using configuration type 1 for base access984worker1 # [ 0.296192] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)985client # [ 0.316032] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.986worker2 # [ 0.321070] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.987worker1 # [ 0.296639] Memory: 1454612K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)988lb # [ 0.342633] ACPI: Added _OSI(Module Device)989worker1 # [ 0.297769] devtmpfs: initialized990lb # [ 0.343395] ACPI: Added _OSI(Processor Device)991worker1 # [ 0.298611] x86/mm: Memory block size: 128MB992lb # [ 0.344131] ACPI: Added _OSI(Processor Aggregator Device)993client # [ 0.321150] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages994worker1 # [ 0.300451] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)995client # [ 0.321910] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page996worker1 # [ 0.301448] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).997worker2 # [ 0.328009] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages998worker1 # [ 0.302574] pinctrl core: initialized pinctrl subsystem999worker2 # [ 0.328985] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1000worker1 # [ 0.303788] PM: RTC time: 14:58:33, date: 2026-09-251001lb # [ 0.349693] ACPI: 1 ACPI AML tables successfully acquired and loaded1002client # [ 0.326909] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1003client # [ 0.327908] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1004worker1 # [ 0.307348] NET: Registered PF_NETLINK/PF_ROUTE protocol family1005worker2 # [ 0.333986] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1006lb # [ 0.353803] ACPI: Interpreter enabled1007lb # [ 0.354409] ACPI: PM: (supports S0 S3 S4 S5)1008worker2 # [ 0.334985] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1009worker1 # [ 0.308775] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations1010lb # [ 0.355115] ACPI: Using IOAPIC for interrupt routing1011worker1 # [ 0.309506] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations1012worker1 # [ 0.310778] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations1013lb # [ 0.357430] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1014worker1 # [ 0.311492] audit: initializing netlink subsys (disabled)1015lb # [ 0.358394] PCI: Using E820 reservations for host bridge windows1016worker1 # [ 0.312721] thermal_sys: Registered thermal governor 'fair_share'1017worker1 # [ 0.312724] thermal_sys: Registered thermal governor 'bang_bang'1018client # [ 0.338360] ACPI: Added _OSI(Module Device)1019worker1 # [ 0.313484] audit: type=2000 audit(1790348314.697:1): state=initialized audit_enabled=0 res=11020lb # [ 0.361513] ACPI: Enabled 2 GPEs in block 00 to 3F1021client # [ 0.338910] ACPI: Added _OSI(Processor Device)1022worker1 # [ 0.315482] thermal_sys: Registered thermal governor 'step_wise'1023client # [ 0.339666] ACPI: Added _OSI(Processor Aggregator Device)1024worker1 # [ 0.315483] thermal_sys: Registered thermal governor 'user_space'1025worker1 # [ 0.316454] thermal_sys: Registered thermal governor 'power_allocator'1026worker2 # [ 0.346259] ACPI: Added _OSI(Module Device)1027worker1 # [ 0.317269] cpuidle: using governor menu1028worker2 # [ 0.346986] ACPI: Added _OSI(Processor Device)1029worker2 # [ 0.347748] ACPI: Added _OSI(Processor Aggregator Device)1030worker1 # [ 0.319527] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51031worker1 # [ 0.320708] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1032lb # [ 0.368897] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1033worker1 # [ 0.321484] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1034worker1 # [ 0.322493] PCI: Using configuration type 1 for base access1035lb # [ 0.369401] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1036client # [ 0.347266] ACPI: 1 ACPI AML tables successfully acquired and loaded1037lb # [ 0.370484] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1038worker1 # [ 0.323623] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1039worker2 # [ 0.353279] ACPI: 1 ACPI AML tables successfully acquired and loaded1040lb # [ 0.371525] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1041lb # [ 0.372726] PCI host bridge to bus 0000:001042client # [ 0.351314] ACPI: Interpreter enabled1043lb # [ 0.373398] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1044client # [ 0.351923] ACPI: PM: (supports S0 S3 S4 S5)1045lb # [ 0.374394] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1046worker2 # [ 0.357392] ACPI: Interpreter enabled1047client # [ 0.352635] ACPI: Using IOAPIC for interrupt routing1048worker2 # [ 0.357999] ACPI: PM: (supports S0 S3 S4 S5)1049lb # [ 0.375394] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1050worker2 # [ 0.358705] ACPI: Using IOAPIC for interrupt routing1051lb # [ 0.376394] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1052worker1 # [ 0.330496] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1053lb # [ 0.377393] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1054worker1 # [ 0.331482] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1055client # [ 0.354983] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1056client # [ 0.355910] PCI: Using E820 reservations for host bridge windows1057lb # [ 0.378393] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1058worker2 # [ 0.361037] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1059lb # [ 0.379394] pci_bus 0000:00: root bus resource [bus 00-ff]1060worker2 # [ 0.361986] PCI: Using E820 reservations for host bridge windows1061lb # [ 0.380405] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1062client # [ 0.358995] ACPI: Enabled 2 GPEs in block 00 to 3F1063worker1 # [ 0.336482] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1064worker2 # [ 0.365075] ACPI: Enabled 2 GPEs in block 00 to 3F1065lb # [ 0.381854] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1066worker1 # [ 0.337481] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1067client # [ 0.366990] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1068worker2 # [ 0.372609] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1069client # [ 0.367914] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1070lb # [ 0.384465] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1071worker2 # [ 0.372991] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1072lb # [ 0.385407] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1073client # [ 0.368988] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1074worker2 # [ 0.374062] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1075lb # [ 0.386348] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1076client # [ 0.370032] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1077worker1 # [ 0.348759] ACPI: Added _OSI(Module Device)1078worker2 # [ 0.375108] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1079client # [ 0.371265] PCI host bridge to bus 0000:001080lb # [ 0.387475] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1081worker1 # [ 0.349459] ACPI: Added _OSI(Processor Device)1082worker2 # [ 0.376412] PCI host bridge to bus 0000:001083client # [ 0.371913] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1084worker1 # [ 0.350036] ACPI: Added _OSI(Processor Aggregator Device)1085worker2 # [ 0.376989] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1086client # [ 0.372909] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1087lb # [ 0.389339] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1088worker2 # [ 0.377985] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1089client # [ 0.373908] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1090worker2 # [ 0.378986] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1091client # [ 0.374909] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1092worker2 # [ 0.379984] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1093lb # [ 0.392411] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1094client # [ 0.375908] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1095lb # [ 0.393227] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1096worker2 # [ 0.380984] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1097client # [ 0.376909] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1098lb # [ 0.394110] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1099client # [ 0.377909] pci_bus 0000:00: root bus resource [bus 00-ff]1100worker1 # [ 0.355968] ACPI: 1 ACPI AML tables successfully acquired and loaded1101worker2 # [ 0.381984] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1102lb # [ 0.394400] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1103worker2 # [ 0.382985] pci_bus 0000:00: root bus resource [bus 00-ff]1104client # [ 0.378911] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1105worker2 # [ 0.383951] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1106lb # [ 0.396318] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1107client # [ 0.380349] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1108worker1 # [ 0.359897] ACPI: Interpreter enabled1109worker2 # [ 0.385571] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1110worker1 # [ 0.360497] ACPI: PM: (supports S0 S3 S4 S5)1111worker1 # [ 0.361190] ACPI: Using IOAPIC for interrupt routing1112lb # [ 0.398412] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1113lb # [ 0.399285] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1114lb # [ 0.400192] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1115worker1 # [ 0.363544] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1116lb # [ 0.401839] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1117worker1 # [ 0.364482] PCI: Using E820 reservations for host bridge windows1118client # [ 0.382993] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1119worker1 # [ 0.367619] ACPI: Enabled 2 GPEs in block 00 to 3F1120client # [ 0.383921] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1121worker2 # [ 0.388179] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1122lb # [ 0.404401] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1123worker2 # [ 0.389038] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1124client # [ 0.384887] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1125lb # [ 0.406038] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1126worker2 # [ 0.390006] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1127lb # [ 0.406415] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1128client # [ 0.386185] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1129worker2 # [ 0.391310] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1130lb # [ 0.407969] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1131client # [ 0.387623] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1132worker2 # [ 0.392716] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1133worker1 # [ 0.375085] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1134lb # [ 0.409412] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1135client # [ 0.390942] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1136lb # [ 0.410296] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1137worker1 # [ 0.375488] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1138client # [ 0.391787] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1139worker2 # [ 0.396025] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1140lb # [ 0.411133] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1141worker1 # [ 0.376559] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1142client # [ 0.392663] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1143worker2 # [ 0.396836] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1144worker1 # [ 0.377611] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1145lb # [ 0.411399] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1146worker2 # [ 0.397691] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1147client # [ 0.392915] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1148worker1 # [ 0.378842] PCI host bridge to bus 0000:001149lb # [ 0.412945] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1150worker2 # [ 0.397991] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1151worker1 # [ 0.379486] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1152client # [ 0.394638] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1153worker1 # [ 0.380481] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1154worker2 # [ 0.399573] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1155lb # [ 0.414416] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1156worker1 # [ 0.381481] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1157lb # [ 0.415374] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1158worker1 # [ 0.382481] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1159client # [ 0.397824] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1160client # [ 0.398581] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1161worker2 # [ 0.401889] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1162worker1 # [ 0.383482] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1163lb # [ 0.416807] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1164worker2 # [ 0.402620] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1165client # [ 0.398931] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1166worker1 # [ 0.384487] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1167worker2 # [ 0.403006] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1168worker1 # [ 0.385482] pci_bus 0000:00: root bus resource [bus 00-ff]1169lb # [ 0.418416] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1170client # [ 0.400497] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1171worker1 # [ 0.386418] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1172worker2 # [ 0.404544] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1173lb # [ 0.419367] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1174worker1 # [ 0.387917] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1175lb # [ 0.420983] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1176client # [ 0.403943] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1177lb # [ 0.422416] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1178client # [ 0.404861] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1179worker2 # [ 0.408023] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1180lb # [ 0.423409] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1181worker2 # [ 0.408873] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1182client # [ 0.405674] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1183worker2 # [ 0.409690] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1184lb # [ 0.424935] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1185client # [ 0.406448] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1186worker2 # [ 0.410583] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1187lb # [ 0.426415] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1188client # [ 0.408941] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1189worker1 # [ 0.390682] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1190lb # [ 0.427416] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1191client # [ 0.409819] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1192worker2 # [ 0.412965] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1193worker1 # [ 0.391494] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1194worker2 # [ 0.413635] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1195lb # [ 0.428937] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1196worker1 # [ 0.392458] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1197client # [ 0.410620] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1198client # [ 0.411736] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1199worker2 # [ 0.414006] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1200worker1 # [ 0.393822] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1201lb # [ 0.430419] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1202worker2 # [ 0.415866] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1203lb # [ 0.431270] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1204client # [ 0.413285] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1205worker1 # [ 0.395219] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1206lb # [ 0.432139] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1207worker2 # [ 0.417335] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1208client # [ 0.414953] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1209lb # [ 0.432928] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1210client # [ 0.415930] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1211worker2 # [ 0.419030] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1212worker1 # [ 0.398501] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1213worker2 # [ 0.420006] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1214worker1 # [ 0.399327] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1215client # [ 0.417562] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1216worker1 # [ 0.400203] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1217lb # [ 0.434677] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1218worker2 # [ 0.421563] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1219lb # [ 0.436032] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1220worker1 # [ 0.400487] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1221lb # [ 0.436415] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1222client # [ 0.418922] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1223worker1 # [ 0.402044] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1224client # [ 0.420686] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1225worker2 # [ 0.423469] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1226lb # [ 0.437976] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1227worker2 # [ 0.424723] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1228client # [ 0.422342] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1229lb # [ 0.439132] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1230worker1 # [ 0.404499] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1231worker2 # [ 0.425519] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1232worker1 # [ 0.405326] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1233lb # [ 0.439596] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1234client # [ 0.423921] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1235worker1 # [ 0.406197] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1236client # [ 0.424908] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1237worker2 # [ 0.426998] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1238lb # [ 0.441046] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1239worker2 # [ 0.427955] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1240worker1 # [ 0.407061] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1241client # [ 0.426437] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1242lb # [ 0.441595] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1243worker2 # [ 0.429481] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1244lb # [ 0.443090] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1245client # [ 0.428266] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1246worker1 # [ 0.410432] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1247client # [ 0.429708] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1248worker2 # [ 0.432030] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1249worker1 # [ 0.411112] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1250worker2 # [ 0.433006] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1251worker1 # [ 0.411503] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1252client # [ 0.431290] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1253worker2 # [ 0.434527] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1254worker1 # [ 0.413045] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1255client # [ 0.433668] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1256client # [ 0.433916] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1257worker2 # [ 0.436694] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1258worker1 # [ 0.415469] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1259client # [ 0.434930] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1260worker2 # [ 0.436992] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1261worker1 # [ 0.416110] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1262worker2 # [ 0.437954] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1263worker1 # [ 0.416503] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1264client # [ 0.436469] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1265worker1 # [ 0.417487] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1266lb # [ 0.443607] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1267worker2 # [ 0.439417] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1268lb # [ 0.445995] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1269worker1 # [ 0.418963] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1270client # [ 0.438602] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1271client # [ 0.438916] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1272lb # [ 0.446661] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1273worker2 # [ 0.442747] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1274client # [ 0.439897] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1275worker1 # [ 0.420506] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1276worker2 # [ 0.443648] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1277lb # [ 0.447676] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1278worker1 # [ 0.421425] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1279worker2 # [ 0.444006] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1280client # [ 0.441305] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1281lb # [ 0.448657] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1282worker1 # [ 0.422837] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1283worker2 # [ 0.445551] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1284client # [ 0.442607] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1285lb # [ 0.450432] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1286worker2 # [ 0.446741] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1287lb # [ 0.451242] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1288client # [ 0.443114] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1289worker1 # [ 0.424529] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1290lb # [ 0.452502] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1291worker2 # [ 0.447193] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1292client # [ 0.444747] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1293worker1 # [ 0.426226] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1294lb # [ 0.454074] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1295worker2 # [ 0.448812] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1296client # [ 0.445712] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1297worker1 # [ 0.427104] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1298client # [ 0.447613] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1299worker2 # [ 0.449825] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1300worker1 # [ 0.428495] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1301worker2 # [ 0.451715] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1302worker1 # [ 0.429445] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1303lb # [ 0.457425] ACPI: PCI: Interrupt link LNKA configured for IRQ 101304lb # [ 0.458507] ACPI: PCI: Interrupt link LNKB configured for IRQ 101305worker1 # [ 0.430861] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1306lb # [ 0.459493] ACPI: PCI: Interrupt link LNKC configured for IRQ 111307worker1 # [ 0.432494] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1308lb # [ 0.460497] ACPI: PCI: Interrupt link LNKD configured for IRQ 111309worker1 # [ 0.434243] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1310lb # [ 0.461534] ACPI: PCI: Interrupt link LNKE configured for IRQ 101311lb # [ 0.462496] ACPI: PCI: Interrupt link LNKF configured for IRQ 101312worker1 # [ 0.435027] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1313lb # [ 0.463495] ACPI: PCI: Interrupt link LNKG configured for IRQ 111314lb # [ 0.464497] ACPI: PCI: Interrupt link LNKH configured for IRQ 111315worker1 # [ 0.437379] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1316client # [ 0.448315] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1317worker1 # [ 0.438100] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1318lb # [ 0.466439] ACPI: PCI: Interrupt link GSIA configured for IRQ 161319client # [ 0.450688] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1320worker2 # [ 0.452221] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1321worker1 # [ 0.438503] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1322lb # [ 0.467407] ACPI: PCI: Interrupt link GSIB configured for IRQ 171323lb # [ 0.468378] ACPI: PCI: Interrupt link GSIC configured for IRQ 181324client # [ 0.451880] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1325lb # [ 0.469194] ACPI: PCI: Interrupt link GSID configured for IRQ 191326worker1 # [ 0.440051] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1327lb # [ 0.470159] ACPI: PCI: Interrupt link GSIE configured for IRQ 201328client # [ 0.453201] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1329lb # [ 0.470409] ACPI: PCI: Interrupt link GSIF configured for IRQ 211330client # [ 0.454155] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1331lb # [ 0.471409] ACPI: PCI: Interrupt link GSIG configured for IRQ 221332lb # [ 0.472381] ACPI: PCI: Interrupt link GSIH configured for IRQ 231333worker2 # [ 0.454764] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1334client # [ 0.455938] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1335lb # [ 0.474166] iommu: Default domain type: Translated1336client # [ 0.456809] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1337worker2 # [ 0.456300] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1338worker1 # [ 0.442935] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1339lb # [ 0.474401] iommu: DMA domain TLB invalidation policy: lazy mode1340client # [ 0.457921] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1341worker1 # [ 0.443488] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1342lb # [ 0.475589] ACPI: bus type USB registered1343worker2 # [ 0.457329] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1344lb # [ 0.476307] usbcore: registered new interface driver usbfs1345worker1 # [ 0.444461] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1346lb # [ 0.477067] usbcore: registered new interface driver hub1347worker2 # [ 0.458302] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1348client # [ 0.459594] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1349lb # [ 0.477403] usbcore: registered new device driver usb1350lb # [ 0.479136] NetLabel: Initializing1351lb # [ 0.479400] NetLabel: domain hash size = 1281352worker2 # [ 0.461002] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1353lb # [ 0.480098] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1354worker2 # [ 0.461882] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1355lb # [ 0.480427] NetLabel: unlabeled traffic allowed by default1356worker1 # [ 0.445958] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1357lb # [ 0.481366] PCI: Using ACPI for IRQ routing1358client # [ 0.464068] ACPI: PCI: Interrupt link LNKA configured for IRQ 101359client # [ 0.465019] ACPI: PCI: Interrupt link LNKB configured for IRQ 101360worker1 # [ 0.447510] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1361worker2 # [ 0.464824] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1362client # [ 0.466023] ACPI: PCI: Interrupt link LNKC configured for IRQ 111363worker1 # [ 0.448593] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1364client # [ 0.467008] ACPI: PCI: Interrupt link LNKD configured for IRQ 111365worker2 # [ 0.466951] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1366worker1 # [ 0.450378] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1367worker1 # [ 0.451393] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1368worker2 # [ 0.470824] ACPI: PCI: Interrupt link LNKA configured for IRQ 101369client # [ 0.468004] ACPI: PCI: Interrupt link LNKE configured for IRQ 101370worker1 # [ 0.453389] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1371worker2 # [ 0.471885] ACPI: PCI: Interrupt link LNKB configured for IRQ 101372client # [ 0.469016] ACPI: PCI: Interrupt link LNKF configured for IRQ 101373worker2 # [ 0.472796] ACPI: PCI: Interrupt link LNKC configured for IRQ 111374client # [ 0.470017] ACPI: PCI: Interrupt link LNKG configured for IRQ 111375worker2 # [ 0.473815] ACPI: PCI: Interrupt link LNKD configured for IRQ 111376client # [ 0.471011] ACPI: PCI: Interrupt link LNKH configured for IRQ 111377worker2 # [ 0.475086] ACPI: PCI: Interrupt link LNKE configured for IRQ 101378client # [ 0.471954] ACPI: PCI: Interrupt link GSIA configured for IRQ 161379worker2 # [ 0.476092] ACPI: PCI: Interrupt link LNKF configured for IRQ 101380client # [ 0.472924] ACPI: PCI: Interrupt link GSIB configured for IRQ 171381worker2 # [ 0.477085] ACPI: PCI: Interrupt link LNKG configured for IRQ 111382client # [ 0.473923] ACPI: PCI: Interrupt link GSIC configured for IRQ 181383worker2 # [ 0.478082] ACPI: PCI: Interrupt link LNKH configured for IRQ 111384client # [ 0.474920] ACPI: PCI: Interrupt link GSID configured for IRQ 191385client # [ 0.475924] ACPI: PCI: Interrupt link GSIE configured for IRQ 201386worker2 # [ 0.479023] ACPI: PCI: Interrupt link GSIA configured for IRQ 161387client # [ 0.476921] ACPI: PCI: Interrupt link GSIF configured for IRQ 211388worker2 # [ 0.479997] ACPI: PCI: Interrupt link GSIB configured for IRQ 171389client # [ 0.477903] ACPI: PCI: Interrupt link GSIG configured for IRQ 221390worker2 # [ 0.480938] ACPI: PCI: Interrupt link GSIC configured for IRQ 181391client # [ 0.478896] ACPI: PCI: Interrupt link GSIH configured for IRQ 231392worker2 # [ 0.481719] ACPI: PCI: Interrupt link GSID configured for IRQ 191393worker2 # [ 0.481999] ACPI: PCI: Interrupt link GSIE configured for IRQ 201394client # [ 0.480768] iommu: Default domain type: Translated1395worker2 # [ 0.483001] ACPI: PCI: Interrupt link GSIF configured for IRQ 211396worker1 # [ 0.454335] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1397client # [ 0.481528] iommu: DMA domain TLB invalidation policy: lazy mode1398worker2 # [ 0.483931] ACPI: PCI: Interrupt link GSIG configured for IRQ 221399client # [ 0.482173] ACPI: bus type USB registered1400worker2 # [ 0.484711] ACPI: PCI: Interrupt link GSIH configured for IRQ 231401client # [ 0.482934] usbcore: registered new interface driver usbfs1402client # [ 0.483885] usbcore: registered new interface driver hub1403worker2 # [ 0.486069] iommu: Default domain type: Translated1404client # [ 0.484583] usbcore: registered new device driver usb1405worker2 # [ 0.486880] iommu: DMA domain TLB invalidation policy: lazy mode1406worker1 # [ 0.457180] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1407worker2 # [ 0.488053] ACPI: bus type USB registered1408client # [ 0.485770] NetLabel: Initializing1409client # [ 0.486370] NetLabel: domain hash size = 1281410worker2 # [ 0.488803] usbcore: registered new interface driver usbfs1411worker1 # [ 0.457787] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1412client # [ 0.486908] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1413worker2 # [ 0.489661] usbcore: registered new interface driver hub1414client # [ 0.487833] NetLabel: unlabeled traffic allowed by default1415worker2 # [ 0.489995] usbcore: registered new device driver usb1416worker1 # [ 0.458780] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1417client # [ 0.488569] PCI: Using ACPI for IRQ routing1418worker2 # [ 0.491752] NetLabel: Initializing1419worker1 # [ 0.459747] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1420worker2 # [ 0.492002] NetLabel: domain hash size = 1281421worker2 # [ 0.492814] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1422worker2 # [ 0.493787] NetLabel: unlabeled traffic allowed by default1423worker1 # [ 0.462521] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1424worker2 # [ 0.494714] PCI: Using ACPI for IRQ routing1425worker1 # [ 0.463380] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1426worker1 # [ 0.464808] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1427worker1 # [ 0.466388] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1428lb # [ 0.524369] pci 0000:00:01.0: vgaarb: setting as boot VGA device1429lb # [ 0.524390] pci 0000:00:01.0: vgaarb: bridge control possible1430worker1 # [ 0.470308] ACPI: PCI: Interrupt link LNKA configured for IRQ 101431worker1 # [ 0.471317] ACPI: PCI: Interrupt link LNKB configured for IRQ 101432lb # [ 0.524390] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1433lb # [ 0.524398] vgaarb: loaded1434worker1 # [ 0.472317] ACPI: PCI: Interrupt link LNKC configured for IRQ 111435lb # [ 0.525146] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01436worker1 # [ 0.473541] ACPI: PCI: Interrupt link LNKD configured for IRQ 111437lb # [ 0.525394] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1438worker1 # [ 0.474578] ACPI: PCI: Interrupt link LNKE configured for IRQ 101439worker1 # [ 0.475577] ACPI: PCI: Interrupt link LNKF configured for IRQ 101440worker1 # [ 0.476584] ACPI: PCI: Interrupt link LNKG configured for IRQ 111441worker1 # [ 0.477581] ACPI: PCI: Interrupt link LNKH configured for IRQ 111442worker1 # [ 0.478524] ACPI: PCI: Interrupt link GSIA configured for IRQ 161443lb # [ 0.530476] clocksource: Switched to clocksource kvm-clock1444worker1 # [ 0.479497] ACPI: PCI: Interrupt link GSIB configured for IRQ 171445worker1 # [ 0.480460] ACPI: PCI: Interrupt link GSIC configured for IRQ 181446worker1 # [ 0.481288] ACPI: PCI: Interrupt link GSID configured for IRQ 191447worker1 # [ 0.482285] ACPI: PCI: Interrupt link GSIE configured for IRQ 201448lb # [ 0.533966] VFS: Disk quotas dquot_6.6.01449worker1 # [ 0.483406] ACPI: PCI: Interrupt link GSIF configured for IRQ 211450lb # [ 0.534717] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1451worker1 # [ 0.484251] ACPI: PCI: Interrupt link GSIG configured for IRQ 221452lb # [ 0.536097] pnp: PnP ACPI init1453worker1 # [ 0.484498] ACPI: PCI: Interrupt link GSIH configured for IRQ 231454lb # [ 0.536908] ACPI: IRQ 4 override to edge(!), high(!)1455lb # [ 0.537890] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1456worker1 # [ 0.486517] iommu: Default domain type: Translated1457lb # [ 0.539513] pnp: PnP ACPI: found 5 devices1458worker1 # [ 0.487323] iommu: DMA domain TLB invalidation policy: lazy mode1459worker1 # [ 0.488532] ACPI: bus type USB registered1460worker1 # [ 0.489340] usbcore: registered new interface driver usbfs1461worker1 # [ 0.490423] usbcore: registered new interface driver hub1462worker1 # [ 0.491179] usbcore: registered new device driver usb1463worker1 # [ 0.492659] NetLabel: Initializing1464worker1 # [ 0.493316] NetLabel: domain hash size = 1281465worker1 # [ 0.494053] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1466lb # [ 0.547187] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1467worker1 # [ 0.494517] NetLabel: unlabeled traffic allowed by default1468lb # [ 0.548664] clocksource: Switched to clocksource acpi_pm1469worker1 # [ 0.495433] PCI: Using ACPI for IRQ routing1470lb # [ 0.549635] NET: Registered PF_INET protocol family1471lb # [ 0.550809] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1472client # [ 0.532078] pci 0000:00:01.0: vgaarb: setting as boot VGA device1473client # [ 0.532905] pci 0000:00:01.0: vgaarb: bridge control possible1474client # [ 0.532905] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1475client # [ 0.532913] vgaarb: loaded1476client # [ 0.533664] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01477worker2 # [ 0.537966] pci 0000:00:01.0: vgaarb: setting as boot VGA device1478client # [ 0.533911] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1479worker2 # [ 0.537981] pci 0000:00:01.0: vgaarb: bridge control possible1480worker2 # [ 0.537981] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1481worker2 # [ 0.537990] vgaarb: loaded1482worker2 # [ 0.538750] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01483client # [ 0.537037] clocksource: Switched to clocksource kvm-clock1484worker2 # [ 0.538985] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1485client # [ 0.540496] VFS: Disk quotas dquot_6.6.01486worker2 # [ 0.542070] clocksource: Switched to clocksource kvm-clock1487client # [ 0.541251] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1488client # [ 0.542575] pnp: PnP ACPI init1489client # [ 0.543397] ACPI: IRQ 4 override to edge(!), high(!)1490lb # [ 0.567441] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1491client # [ 0.544425] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1492worker2 # [ 0.545669] VFS: Disk quotas dquot_6.6.01493client # [ 0.545988] pnp: PnP ACPI: found 5 devices1494lb # [ 0.569019] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1495worker2 # [ 0.546387] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1496worker2 # [ 0.547749] pnp: PnP ACPI init1497lb # [ 0.570350] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1498worker2 # [ 0.548582] ACPI: IRQ 4 override to edge(!), high(!)1499lb # [ 0.571776] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1500worker2 # [ 0.549565] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1501lb # [ 0.573030] TCP: Hash tables configured (established 16384 bind 16384)1502worker2 # [ 0.551139] pnp: PnP ACPI: found 5 devices1503lb # [ 0.574216] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1504lb # [ 0.575524] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1505lb # [ 0.576672] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1506client # [ 0.553601] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1507lb # [ 0.577926] NET: Registered PF_UNIX/PF_LOCAL protocol family1508client # [ 0.555083] clocksource: Switched to clocksource acpi_pm1509lb # [ 0.578920] NET: Registered PF_XDP protocol family1510client # [ 0.556096] NET: Registered PF_INET protocol family1511lb # [ 0.579794] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1512lb # [ 0.580817] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1513client # [ 0.557268] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1514lb # [ 0.581900] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1515worker2 # [ 0.558747] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1516worker2 # [ 0.560195] clocksource: Switched to clocksource acpi_pm1517lb # [ 0.583007] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1518worker2 # [ 0.561201] NET: Registered PF_INET protocol family1519lb # [ 0.584132] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1520lb # [ 0.585252] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1521worker2 # [ 0.562378] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1522lb # [ 0.587173] ACPI: \_SB_.GSIA: Enabled at IRQ 161523lb # [ 0.589364] ACPI: \_SB_.GSIB: Enabled at IRQ 171524lb # [ 0.591357] ACPI: \_SB_.GSIC: Enabled at IRQ 181525worker1 # [ 0.538293] pci 0000:00:01.0: vgaarb: setting as boot VGA device1526lb # [ 0.593375] ACPI: \_SB_.GSID: Enabled at IRQ 191527worker1 # [ 0.538477] pci 0000:00:01.0: vgaarb: bridge control possible1528worker1 # [ 0.538477] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1529lb # [ 0.595066] PCI: CLS 0 bytes, default 641530worker1 # [ 0.538486] vgaarb: loaded1531worker1 # [ 0.539243] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01532lb # [ 0.596072] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1533worker1 # [ 0.539481] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1534lb # [ 0.597915] Trying to unpack rootfs image as initramfs...1535client # [ 0.574086] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1536client # [ 0.575641] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1537client # [ 0.577051] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1538worker1 # [ 0.542621] clocksource: Switched to clocksource kvm-clock1539client # [ 0.578457] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1540worker2 # [ 0.579217] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1541client # [ 0.579732] TCP: Hash tables configured (established 16384 bind 16384)1542worker2 # [ 0.580810] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1543client # [ 0.580886] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1544worker1 # [ 0.546188] VFS: Disk quotas dquot_6.6.01545worker2 # [ 0.582181] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1546client # [ 0.582140] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1547worker1 # [ 0.546976] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1548worker2 # [ 0.583528] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1549worker1 # [ 0.548290] pnp: PnP ACPI init1550client # [ 0.583216] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1551worker2 # [ 0.584808] TCP: Hash tables configured (established 16384 bind 16384)1552worker1 # [ 0.549140] ACPI: IRQ 4 override to edge(!), high(!)1553client # [ 0.584532] NET: Registered PF_UNIX/PF_LOCAL protocol family1554client # [ 0.585515] NET: Registered PF_XDP protocol family1555worker2 # [ 0.585917] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1556worker1 # [ 0.550117] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1557client # [ 0.586447] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1558worker2 # [ 0.587173] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1559worker1 # [ 0.551747] pnp: PnP ACPI: found 5 devices1560client # [ 0.587457] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1561worker2 # [ 0.588253] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1562client # [ 0.588485] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1563worker2 # [ 0.589510] NET: Registered PF_UNIX/PF_LOCAL protocol family1564worker2 # [ 0.590514] NET: Registered PF_XDP protocol family1565client # [ 0.589621] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1566worker2 # [ 0.591373] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1567client # [ 0.590738] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1568worker2 # [ 0.592400] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1569client # [ 0.591849] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1570worker2 # [ 0.593447] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1571client # [ 0.593611] ACPI: \_SB_.GSIA: Enabled at IRQ 161572worker2 # [ 0.594604] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1573worker2 # [ 0.595753] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1574worker1 # [ 0.559402] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1575client # [ 0.596112] ACPI: \_SB_.GSIB: Enabled at IRQ 171576worker1 # [ 0.560916] clocksource: Switched to clocksource acpi_pm1577worker2 # [ 0.596888] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1578worker1 # [ 0.561885] NET: Registered PF_INET protocol family1579worker2 # [ 0.598655] ACPI: \_SB_.GSIA: Enabled at IRQ 161580client # [ 0.598048] ACPI: \_SB_.GSIC: Enabled at IRQ 181581worker1 # [ 0.563104] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1582client # [ 0.599992] ACPI: \_SB_.GSID: Enabled at IRQ 191583worker2 # [ 0.600821] ACPI: \_SB_.GSIB: Enabled at IRQ 171584client # [ 0.601649] PCI: CLS 0 bytes, default 641585worker2 # [ 0.602917] ACPI: \_SB_.GSIC: Enabled at IRQ 181586client # [ 0.602575] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1587worker2 # [ 0.605023] ACPI: \_SB_.GSID: Enabled at IRQ 191588client # [ 0.604362] Trying to unpack rootfs image as initramfs...1589worker2 # [ 0.607135] PCI: CLS 0 bytes, default 641590worker2 # [ 0.608146] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1591worker2 # [ 0.609939] Trying to unpack rootfs image as initramfs...1592worker1 # [ 0.579987] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1593worker1 # [ 0.581578] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1594lb # [ 0.641570] Initialise system trusted keyrings1595worker1 # [ 0.582976] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1596worker1 # [ 0.584348] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1597worker1 # [ 0.585639] TCP: Hash tables configured (established 16384 bind 16384)1598lb # [ 0.645176] workingset: timestamp_bits=40 max_order=19 bucket_order=01599worker1 # [ 0.586772] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1600worker1 # [ 0.588103] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1601worker1 # [ 0.589230] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1602worker1 # [ 0.590491] NET: Registered PF_UNIX/PF_LOCAL protocol family1603worker1 # [ 0.591491] NET: Registered PF_XDP protocol family1604worker1 # [ 0.592374] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1605worker1 # [ 0.593422] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1606worker1 # [ 0.594480] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1607worker1 # [ 0.595623] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1608worker1 # [ 0.596765] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1609worker1 # [ 0.597942] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1610worker1 # [ 0.599706] ACPI: \_SB_.GSIA: Enabled at IRQ 161611worker1 # [ 0.601914] ACPI: \_SB_.GSIB: Enabled at IRQ 171612worker1 # [ 0.604001] ACPI: \_SB_.GSIC: Enabled at IRQ 181613worker1 # [ 0.606051] ACPI: \_SB_.GSID: Enabled at IRQ 191614lb # [ 0.665362] Key type asymmetric registered1615lb # [ 0.666064] Asymmetric key parser 'x509' registered1616worker1 # [ 0.608126] PCI: CLS 0 bytes, default 641617worker1 # [ 0.609043] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1618lb # [ 0.669169] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1619worker1 # [ 0.610813] Trying to unpack rootfs image as initramfs...1620lb # [ 0.672172] io scheduler mq-deadline registered1621lb # [ 0.672966] io scheduler kyber registered1622client # [ 0.649840] Initialise system trusted keyrings1623client # [ 0.650810] workingset: timestamp_bits=40 max_order=19 bucket_order=01624lb # [ 0.675729] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1625worker2 # [ 0.653667] Initialise system trusted keyrings1626lb # [ 0.677048] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1627worker2 # [ 0.656689] workingset: timestamp_bits=40 max_order=19 bucket_order=01628lb # [ 0.682538] Linux agpgart interface v0.1031629lb # [ 0.683316] ACPI: bus type drm_connector registered1630lb # [ 0.686582] usbcore: registered new interface driver usbserial_generic1631lb # [ 0.687686] usbserial: USB Serial support registered for generic1632lb # [ 0.690144] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1633lb # [ 0.690144] Please enable it if your BIOS has the CPPC option.1634lb # [ 0.692377] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1635lb # [ 0.696258] drop_monitor: Initializing network drop monitor service1636client # [ 0.673808] Key type asymmetric registered1637lb # [ 0.697458] NET: Registered PF_INET6 protocol family1638client # [ 0.676445] Asymmetric key parser 'x509' registered1639lb # [ 0.700641] Segment Routing with IPv61640worker2 # [ 0.677927] Key type asymmetric registered1641client # [ 0.677346] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1642lb # [ 0.703151] In-situ OAM (IOAM) with IPv61643worker2 # [ 0.680648] Asymmetric key parser 'x509' registered1644lb # [ 0.704245] IPI shorthand broadcast: enabled1645client # [ 0.680577] io scheduler mq-deadline registered1646client # [ 0.681357] io scheduler kyber registered1647worker2 # [ 0.681542] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1648worker2 # [ 0.682916] io scheduler mq-deadline registered1649worker2 # [ 0.685643] io scheduler kyber registered1650client # [ 0.685001] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1651client # [ 0.686303] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1652worker2 # [ 0.688862] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1653worker1 # [ 0.653087] Initialise system trusted keyrings1654lb # [ 0.712665] sched_clock: Marking stable (577015565, 135103655)->(788107479, -75988259)1655worker2 # [ 0.690158] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1656client # [ 0.691498] Linux agpgart interface v0.1031657worker1 # [ 0.656042] workingset: timestamp_bits=40 max_order=19 bucket_order=01658client # [ 0.692237] ACPI: bus type drm_connector registered1659lb # [ 0.718311] registered taskstats version 11660worker2 # [ 0.695401] Linux agpgart interface v0.1031661client # [ 0.694559] usbcore: registered new interface driver usbserial_generic1662worker2 # [ 0.696190] ACPI: bus type drm_connector registered1663lb # [ 0.719283] Loading compiled-in X.509 certificates1664client # [ 0.695671] usbserial: USB Serial support registered for generic1665worker2 # [ 0.699100] usbcore: registered new interface driver usbserial_generic1666client # [ 0.698452] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1667worker2 # [ 0.700260] usbserial: USB Serial support registered for generic1668client # [ 0.698452] Please enable it if your BIOS has the CPPC option.1669client # [ 0.700823] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1670worker2 # [ 0.702646] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1671worker2 # [ 0.702646] Please enable it if your BIOS has the CPPC option.1672worker2 # [ 0.704898] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1673client # [ 0.706583] drop_monitor: Initializing network drop monitor service1674client # [ 0.707844] NET: Registered PF_INET6 protocol family1675worker2 # [ 0.709766] drop_monitor: Initializing network drop monitor service1676worker2 # [ 0.710977] NET: Registered PF_INET6 protocol family1677client # [ 0.711917] Segment Routing with IPv61678client # [ 0.712646] In-situ OAM (IOAM) with IPv61679worker1 # [ 0.677946] Key type asymmetric registered1680worker2 # [ 0.714170] Segment Routing with IPv61681worker1 # [ 0.678655] Asymmetric key parser 'x509' registered1682worker2 # [ 0.714850] In-situ OAM (IOAM) with IPv61683lb # [ 0.738126] Demotion targets for Node 0: null1684worker1 # [ 0.679528] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1685lb # [ 0.739004] Key type .fscrypt registered1686client # [ 0.715730] IPI shorthand broadcast: enabled1687lb # [ 0.739752] Key type fscrypt-provisioning registered1688lb # [ 0.740713] ima: No TPM chip found, activating TPM-bypass!1689worker2 # [ 0.717947] IPI shorthand broadcast: enabled1690worker1 # [ 0.684994] io scheduler mq-deadline registered1691worker1 # [ 0.685799] io scheduler kyber registered1692lb # [ 0.745129] ima: Allocated hash algorithm: sha11693lb # [ 0.745938] ima: No architecture policies found1694worker1 # [ 0.688435] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1695lb # [ 0.748142] PM: Magic number: 10:824:9911696client # [ 0.723940] sched_clock: Marking stable (590015660, 133409133)->(799182544, -75757751)1697worker1 # [ 0.689730] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1698worker2 # [ 0.726219] sched_clock: Marking stable (593019722, 132609702)->(804270068, -78640644)1699lb # [ 0.749752] RAS: Correctable Errors collector initialized.1700client # [ 0.726512] registered taskstats version 11701client # [ 0.727491] Loading compiled-in X.509 certificates1702worker2 # [ 0.730851] registered taskstats version 11703worker1 # [ 0.695511] Linux agpgart interface v0.1031704worker2 # [ 0.731885] Loading compiled-in X.509 certificates1705worker1 # [ 0.696313] ACPI: bus type drm_connector registered1706worker1 # [ 0.699336] usbcore: registered new interface driver usbserial_generic1707lb # [ 0.758805] clk: Disabling unused clocks1708worker1 # [ 0.700471] usbserial: USB Serial support registered for generic1709lb # [ 0.761153] PM: genpd: Disabling unused power domains1710worker1 # [ 0.702899] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1711worker1 # [ 0.702899] Please enable it if your BIOS has the CPPC option.1712worker1 # [ 0.705209] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1713worker1 # [ 0.709019] drop_monitor: Initializing network drop monitor service1714worker1 # [ 0.710230] NET: Registered PF_INET6 protocol family1715client # [ 0.746444] Demotion targets for Node 0: null1716client # [ 0.747310] Key type .fscrypt registered1717worker2 # [ 0.750639] Demotion targets for Node 0: null1718worker1 # [ 0.714985] Segment Routing with IPv61719client # [ 0.750429] Key type fscrypt-provisioning registered1720worker1 # [ 0.715610] In-situ OAM (IOAM) with IPv61721worker2 # [ 0.751553] Key type .fscrypt registered1722client # [ 0.751361] ima: No TPM chip found, activating TPM-bypass!1723worker1 # [ 0.716689] IPI shorthand broadcast: enabled1724client # [ 0.752311] ima: Allocated hash algorithm: sha11725worker2 # [ 0.754650] Key type fscrypt-provisioning registered1726client # [ 0.754460] ima: No architecture policies found1727worker2 # [ 0.755645] ima: No TPM chip found, activating TPM-bypass!1728client # [ 0.755447] PM: Magic number: 10:824:9911729worker2 # [ 0.756580] ima: Allocated hash algorithm: sha11730worker2 # [ 0.757435] ima: No architecture policies found1731client # [ 0.757052] RAS: Correctable Errors collector initialized.1732worker2 # [ 0.759636] PM: Magic number: 10:824:9911733worker2 # [ 0.761369] RAS: Correctable Errors collector initialized.1734worker1 # [ 0.726970] sched_clock: Marking stable (593017696, 133855586)->(812199729, -85326447)1735worker1 # [ 0.728678] registered taskstats version 11736worker1 # [ 0.729669] Loading compiled-in X.509 certificates1737client # [ 0.768125] clk: Disabling unused clocks1738client # [ 0.770439] PM: genpd: Disabling unused power domains1739worker2 # [ 0.772327] clk: Disabling unused clocks1740worker2 # [ 0.774646] PM: genpd: Disabling unused power domains1741worker1 # [ 0.748878] Demotion targets for Node 0: null1742worker1 # [ 0.749767] Key type .fscrypt registered1743worker1 # [ 0.751882] Key type fscrypt-provisioning registered1744worker1 # [ 0.752828] ima: No TPM chip found, activating TPM-bypass!1745worker1 # [ 0.753773] ima: Allocated hash algorithm: sha11746worker1 # [ 0.757909] ima: No architecture policies found1747worker1 # [ 0.758895] PM: Magic number: 10:824:9911748worker1 # [ 0.760554] RAS: Correctable Errors collector initialized.1749worker1 # [ 0.769537] clk: Disabling unused clocks1750worker1 # [ 0.772901] PM: genpd: Disabling unused power domains1751lb # [ 0.927697] Freeing initrd memory: 29064K1752lb # [ 0.930871] Freeing unused decrypted memory: 2028K1753lb # [ 0.933540] Freeing unused kernel image (initmem) memory: 3652K1754lb # [ 0.934657] Write protecting the kernel read-only data: 32768k1755lb # [ 0.936563] Freeing unused kernel image (text/rodata gap) memory: 1184K1756lb # [ 0.938065] Freeing unused kernel image (rodata/data gap) memory: 720K1757client # [ 0.934084] Freeing initrd memory: 29056K1758client # [ 0.937318] Freeing unused decrypted memory: 2028K1759client # [ 0.940004] Freeing unused kernel image (initmem) memory: 3652K1760worker2 # [ 0.941309] Freeing initrd memory: 29060K1761client # [ 0.941136] Write protecting the kernel read-only data: 32768k1762client # [ 0.943016] Freeing unused kernel image (text/rodata gap) memory: 1184K1763worker2 # [ 0.944638] Freeing unused decrypted memory: 2028K1764client # [ 0.944596] Freeing unused kernel image (rodata/data gap) memory: 720K1765worker2 # [ 0.947371] Freeing unused kernel image (initmem) memory: 3652K1766worker2 # [ 0.948528] Write protecting the kernel read-only data: 32768k1767worker2 # [ 0.950479] Freeing unused kernel image (text/rodata gap) memory: 1184K1768worker2 # [ 0.952066] Freeing unused kernel image (rodata/data gap) memory: 720K1769lb # [ 0.989034] x86/mm: Checked W+X mappings: passed, no W+X pages found.1770lb # [ 0.990168] Run /init as init process1771lb # [ 1.001268] systemd[1]: Inserted module 'autofs4'1772worker1 # [ 0.942401] Freeing initrd memory: 29056K1773worker1 # [ 0.945714] Freeing unused decrypted memory: 2028K1774worker1 # [ 0.948449] Freeing unused kernel image (initmem) memory: 3652K1775worker1 # [ 0.949598] Write protecting the kernel read-only data: 32768k1776worker1 # [ 0.951576] Freeing unused kernel image (text/rodata gap) memory: 1184K1777worker1 # [ 0.953130] Freeing unused kernel image (rodata/data gap) memory: 720K1778client # [ 0.995391] x86/mm: Checked W+X mappings: passed, no W+X pages found.1779client # [ 0.996517] Run /init as init process1780lb # [ 1.021016] fuse: init (API version 7.45)1781worker2 # [ 1.002724] x86/mm: Checked W+X mappings: passed, no W+X pages found.1782worker2 # [ 1.003865] Run /init as init process1783lb # [ 1.028385] ACPI: \_SB_.GSIG: Enabled at IRQ 221784lb # [ 1.030807] ACPI: \_SB_.GSIH: Enabled at IRQ 231785client # [ 1.007320] systemd[1]: Inserted module 'autofs4'1786lb # [ 1.033613] ACPI: \_SB_.GSIE: Enabled at IRQ 201787lb # [ 1.035516] ACPI: \_SB_.GSIF: Enabled at IRQ 211788worker2 # [ 1.015145] systemd[1]: Inserted module 'autofs4'1789lb # [ 1.040628] virtiofs virtio5: discovered new tag: nix-store1790lb # [ 1.042220] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1791lb # [ 1.049403] virtiofs virtio6: discovered new tag: shared1792client # [ 1.027179] fuse: init (API version 7.45)1793lb # [ 1.051040] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1794lb # [ 1.054296] virtiofs virtio7: discovered new tag: xchg1795lb # [ 1.055807] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1796worker2 # [ 1.035090] fuse: init (API version 7.45)1797client # [ 1.034519] ACPI: \_SB_.GSIG: Enabled at IRQ 221798client # [ 1.037960] ACPI: \_SB_.GSIH: Enabled at IRQ 231799worker1 # [ 1.004138] x86/mm: Checked W+X mappings: passed, no W+X pages found.1800worker1 # [ 1.005245] Run /init as init process1801client # [ 1.041722] ACPI: \_SB_.GSIE: Enabled at IRQ 201802worker2 # [ 1.042579] ACPI: \_SB_.GSIG: Enabled at IRQ 221803client # [ 1.044000] ACPI: \_SB_.GSIF: Enabled at IRQ 211804worker2 # [ 1.045220] ACPI: \_SB_.GSIH: Enabled at IRQ 231805worker2 # [ 1.048857] ACPI: \_SB_.GSIE: Enabled at IRQ 201806client # [ 1.049602] virtiofs virtio5: discovered new tag: nix-store1807worker2 # [ 1.051030] ACPI: \_SB_.GSIF: Enabled at IRQ 211808lb # [ 1.074892] systemd[1]: Successfully made /usr/ read-only.1809worker1 # [ 1.016158] systemd[1]: Inserted module 'autofs4'1810client # [ 1.051238] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1811worker2 # [ 1.056926] virtiofs virtio5: discovered new tag: nix-store1812worker2 # [ 1.058575] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1813client # [ 1.058289] virtiofs virtio6: discovered new tag: shared1814client # [ 1.059802] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1815client # [ 1.063213] virtiofs virtio7: discovered new tag: xchg1816client # [ 1.064721] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1817worker2 # [ 1.066461] virtiofs virtio6: discovered new tag: shared1818worker2 # [ 1.068053] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1819worker2 # [ 1.071768] virtiofs virtio7: discovered new tag: xchg1820worker1 # [ 1.036075] fuse: init (API version 7.45)1821worker2 # [ 1.073297] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1822worker1 # [ 1.043515] ACPI: \_SB_.GSIG: Enabled at IRQ 221823worker1 # [ 1.046294] ACPI: \_SB_.GSIH: Enabled at IRQ 231824worker1 # [ 1.049455] ACPI: \_SB_.GSIE: Enabled at IRQ 201825client # [ 1.085230] systemd[1]: Successfully made /usr/ read-only.1826worker1 # [ 1.051614] ACPI: \_SB_.GSIF: Enabled at IRQ 211827worker1 # [ 1.057388] virtiofs virtio5: discovered new tag: nix-store1828worker2 # [ 1.094791] systemd[1]: Successfully made /usr/ read-only.1829worker1 # [ 1.058953] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1830worker1 # [ 1.067104] virtiofs virtio6: discovered new tag: shared1831worker1 # [ 1.068714] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1832worker1 # [ 1.071989] virtiofs virtio7: discovered new tag: xchg1833worker1 # [ 1.073505] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1834worker1 # [ 1.095032] systemd[1]: Successfully made /usr/ read-only.1835lb # [ 1.410493] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836lb # [ 1.415324] systemd[1]: Detected virtualization kvm.1837lb # [ 1.416162] systemd[1]: Detected architecture x86-64.1838lb # [ 1.416991] systemd[1]: Running in initrd.1839lb # [ 1.418012] systemd[1]: Initializing machine ID from random generator.1840lb # [ 1.419187] systemd[1]: Hostname set to <lb>.1841client # [ 1.420666] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1842client # [ 1.425936] systemd[1]: Detected virtualization kvm.1843client # [ 1.426834] systemd[1]: Detected architecture x86-64.1844client # [ 1.427726] systemd[1]: Running in initrd.1845client # [ 1.428818] systemd[1]: Initializing machine ID from random generator.1846client # [ 1.430007] systemd[1]: Hostname set to <client>.1847worker2 # [ 1.431352] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1848worker2 # [ 1.443349] systemd[1]: Detected virtualization kvm.1849worker2 # [ 1.445504] systemd[1]: Detected architecture x86-64.1850worker2 # [ 1.447675] systemd[1]: Running in initrd.1851worker2 # [ 1.450156] systemd[1]: Initializing machine ID from random generator.1852worker2 # [ 1.453043] systemd[1]: Hostname set to <worker2>.1853worker1 # [ 1.431648] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1854worker1 # [ 1.443552] systemd[1]: Detected virtualization kvm.1855worker1 # [ 1.445681] systemd[1]: Detected architecture x86-64.1856worker1 # [ 1.447921] systemd[1]: Running in initrd.1857worker1 # [ 1.450415] systemd[1]: Initializing machine ID from random generator.1858worker1 # [ 1.453315] systemd[1]: Hostname set to <worker1>.1859lb # [ 1.620437] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860client # [ 1.620763] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861lb # [ 1.671762] systemd[1]: Queued start job for default target Initrd Default Target.1862lb # [ 1.675356] systemd[1]: Created slice Slice /system/modprobe.1863lb # [ 1.676485] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1864lb # [ 1.677884] systemd[1]: Expecting device /dev/disk/by-label/nixos...1865worker2 # [ 1.655292] systemd[1]: bpf-restrict-fs: LSM BPF program attached1866lb # [ 1.678947] systemd[1]: Reached target Path Units.1867lb # [ 1.679812] systemd[1]: Reached target Slice Units.1868lb # [ 1.680693] systemd[1]: Reached target Swaps.1869lb # [ 1.681463] systemd[1]: Reached target Timer Units.1870client # [ 1.658173] systemd[1]: Queued start job for default target Initrd Default Target.1871lb # [ 1.682410] systemd[1]: Listening on D-Bus System Message Bus Socket.1872lb # [ 1.683552] systemd[1]: Listening on Journal Socket (/dev/log).1873lb # [ 1.684691] systemd[1]: Listening on Journal Sockets.1874lb # [ 1.685677] systemd[1]: Listening on udev Control Socket.1875client # [ 1.662709] systemd[1]: Created slice Slice /system/modprobe.1876lb # [ 1.686681] systemd[1]: Listening on udev Kernel Socket.1877lb # [ 1.687595] systemd[1]: Reached target Socket Units.1878client # [ 1.663895] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1879client # [ 1.665333] systemd[1]: Expecting device /dev/disk/by-label/nixos...1880lb # [ 1.689306] systemd[1]: Starting Create List of Static Device Nodes...1881client # [ 1.666464] systemd[1]: Reached target Path Units.1882client # [ 1.667355] systemd[1]: Reached target Slice Units.1883client # [ 1.668279] systemd[1]: Reached target Swaps.1884client # [ 1.669123] systemd[1]: Reached target Timer Units.1885lb # [ 1.692926] systemd[1]: Starting Load Kernel Module configfs...1886client # [ 1.670082] systemd[1]: Listening on D-Bus System Message Bus Socket.1887client # [ 1.671338] systemd[1]: Listening on Journal Socket (/dev/log).1888client # [ 1.672575] systemd[1]: Listening on Journal Sockets.1889client # [ 1.673627] systemd[1]: Listening on udev Control Socket.1890client # [ 1.674679] systemd[1]: Listening on udev Kernel Socket.1891client # [ 1.675678] systemd[1]: Reached target Socket Units.1892client # [ 1.677478] systemd[1]: Starting Create List of Static Device Nodes...1893client # [ 1.681220] systemd[1]: Starting Load Kernel Module configfs...1894lb # [ 1.703577] systemd[1]: Starting Journal Service...1895worker2 # [ 1.694501] systemd[1]: Queued start job for default target Initrd Default Target.1896worker2 # [ 1.698977] systemd[1]: Created slice Slice /system/modprobe.1897worker2 # [ 1.700183] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1898worker2 # [ 1.701695] systemd[1]: Expecting device /dev/disk/by-label/nixos...1899client # [ 1.690495] systemd[1]: Starting Journal Service...1900worker2 # [ 1.702817] systemd[1]: Reached target Path Units.1901worker2 # [ 1.703721] systemd[1]: Reached target Slice Units.1902worker2 # [ 1.704632] systemd[1]: Reached target Swaps.1903lb # [ 1.728206] systemd[1]: Starting Load Kernel Modules...1904worker2 # [ 1.705443] systemd[1]: Reached target Timer Units.1905worker2 # [ 1.706407] systemd[1]: Listening on D-Bus System Message Bus Socket.1906worker2 # [ 1.707686] systemd[1]: Listening on Journal Socket (/dev/log).1907worker2 # [ 1.708874] systemd[1]: Listening on Journal Sockets.1908lb # [ 1.732199] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1909worker2 # [ 1.709873] systemd[1]: Listening on udev Control Socket.1910worker2 # [ 1.710862] systemd[1]: Listening on udev Kernel Socket.1911worker2 # [ 1.711827] systemd[1]: Reached target Socket Units.1912worker2 # [ 1.713693] systemd[1]: Starting Create List of Static Device Nodes...1913lb # [ 1.737296] systemd[1]: Starting Coldplug All udev Devices...1914client # [ 1.714043] systemd[1]: Starting Load Kernel Modules...1915worker1 # [ 1.677417] systemd[1]: bpf-restrict-fs: LSM BPF program attached1916worker2 # [ 1.717463] systemd[1]: Starting Load Kernel Module configfs...1917client # [ 1.718586] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1918lb # [ 1.746703] systemd-journald[66]: Collecting audit messages is disabled.1919client # [ 1.725322] systemd[1]: Starting Coldplug All udev Devices...1920lb # [ 1.751635] systemd[1]: Finished Create List of Static Device Nodes.1921lb # [ 1.759731] systemd[1]: modprobe@configfs.service: Deactivated successfully.1922client # [ 1.736945] systemd-journald[66]: Collecting audit messages is disabled.1923worker2 # [ 1.725819] systemd[1]: Starting Journal Service...1924client # [ 1.739983] systemd[1]: Finished Create List of Static Device Nodes.1925lb # [ 1.764508] systemd[1]: Finished Load Kernel Module configfs.1926client # [ 1.746020] systemd[1]: modprobe@configfs.service: Deactivated successfully.1927lb # [ 1.770536] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1928client # [ 1.752474] systemd[1]: Finished Load Kernel Module configfs.1929worker2 # [ 1.753840] systemd[1]: Starting Load Kernel Modules...1930worker2 # [ 1.757733] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1931client # [ 1.757831] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1932lb # [ 1.782187] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1933lb # [ 1.785725] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1934worker2 # [ 1.766817] systemd[1]: Starting Coldplug All udev Devices...1935worker1 # [ 1.732104] systemd[1]: Queued start job for default target Initrd Default Target.1936client # [ 1.768189] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1937worker1 # [ 1.736264] systemd[1]: Created slice Slice /system/modprobe.1938worker1 # [ 1.737421] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1939worker1 # [ 1.738782] systemd[1]: Expecting device /dev/disk/by-label/nixos...1940worker1 # [ 1.739874] systemd[1]: Reached target Path Units.1941lb # [ 1.798264] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1942worker1 # [ 1.740686] systemd[1]: Reached target Slice Units.1943worker1 # [ 1.741566] systemd[1]: Reached target Swaps.1944worker1 # [ 1.742370] systemd[1]: Reached target Timer Units.1945worker2 # [ 1.778173] systemd-journald[66]: Collecting audit messages is disabled.1946worker1 # [ 1.743331] systemd[1]: Listening on D-Bus System Message Bus Socket.1947worker1 # [ 1.744530] systemd[1]: Listening on Journal Socket (/dev/log).1948worker1 # [ 1.745664] systemd[1]: Listening on Journal Sockets.1949worker2 # [ 1.781742] systemd[1]: Finished Create List of Static Device Nodes.1950worker1 # [ 1.746615] systemd[1]: Listening on udev Control Socket.1951client # [ 1.781791] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1952worker1 # [ 1.747669] systemd[1]: Listening on udev Kernel Socket.1953worker1 # [ 1.748579] systemd[1]: Reached target Socket Units.1954worker2 # [ 1.784319] systemd[1]: modprobe@configfs.service: Deactivated successfully.1955worker1 # [ 1.750432] systemd[1]: Starting Create List of Static Device Nodes...1956client # [ 1.789618] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1957worker1 # [ 1.754939] systemd[1]: Starting Load Kernel Module configfs...1958worker2 # [ 1.793522] systemd[1]: Finished Load Kernel Module configfs.1959lb # [ 1.820191] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1960worker2 # [ 1.801078] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1961lb # [ 1.829344] systemd[1]: Starting Create Static Device Nodes in /dev...1962client # [ 1.808991] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1963worker1 # [ 1.764403] systemd[1]: Starting Journal Service...1964worker2 # [ 1.812708] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1965lb # [ 1.839647] systemd[1]: Finished Load Kernel Modules.1966client # [ 1.818645] systemd[1]: Starting Create Static Device Nodes in /dev...1967worker2 # [ 1.820366] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1968lb # [ 1.847768] systemd[1]: Starting Apply Kernel Variables...1969worker1 # [ 1.790104] systemd[1]: Starting Load Kernel Modules...1970worker2 # [ 1.829850] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1971worker1 # [ 1.793975] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1972client # [ 1.834732] systemd[1]: Finished Load Kernel Modules.1973worker1 # [ 1.801055] systemd[1]: Starting Coldplug All udev Devices...1974client # [ 1.841621] systemd[1]: Starting Apply Kernel Variables...1975lb # [ 1.866475] systemd[1]: Finished Create Static Device Nodes in /dev.1976lb # [ 1.730939] systemd-modules-load[67]: Inserted module 'dm_mod'1977lb # [ 1.734948] systemd-modules-load[67]: Inserted module 'virtio_balloon'1978lb # [ 1.735979] systemd-modules-load[67]: Inserted module 'virtio_gpu'1979lb # [ 1.873374] systemd[1]: Started Journal Service.1980worker1 # [ 1.815130] systemd-journald[66]: Collecting audit messages is disabled.1981worker1 # [ 1.819142] systemd[1]: Finished Create List of Static Device Nodes.1982worker2 # [ 1.855234] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1983client # [ 1.857532] systemd[1]: Finished Create Static Device Nodes in /dev.1984worker1 # [ 1.824619] systemd[1]: modprobe@configfs.service: Deactivated successfully.1985lb # [ 1.749117] systemd[1]: Finished Apply Kernel Variables.1986client # [ 1.861660] systemd[1]: Started Journal Service.1987client # [ 1.726714] systemd-modules-load[67]: Inserted module 'dm_mod'1988lb # [ 1.752072] systemd[1]: Reached target Preparation for Local File Systems.1989lb # [ 1.753091] systemd[1]: Reached target Local File Systems.1990client # [ 1.731054] systemd-modules-load[67]: Inserted module 'virtio_balloon'1991worker2 # [ 1.865025] systemd[1]: Starting Create Static Device Nodes in /dev...1992client # [ 1.733146] systemd-modules-load[67]: Inserted module 'virtio_gpu'1993worker1 # [ 1.831281] systemd[1]: Finished Load Kernel Module configfs.1994lb # [ 1.755846] systemd[1]: Starting Create System Files and Directories...1995client # [ 1.736226] systemd[1]: Reached target Preparation for Local File Systems.1996client # [ 1.737419] systemd[1]: Reached target Local File Systems.1997worker1 # [ 1.836310] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1998lb # [ 1.761133] systemd[1]: Starting Rule-based Manager for Device Events and Files...1999client # [ 1.743420] systemd[1]: Starting Create System Files and Directories...2000worker2 # [ 1.877729] systemd[1]: Finished Load Kernel Modules.
loading 2,474 hidden lines…
4475worker2 # [ 43.631261] nix-grpc-daemon[784]: ts=2026-09-25T14:59:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14476worker2 # [ 43.645892] nix-daemon[871]: reaped child process 968, status = succeeded4477client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 5.76 seconds)4478client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4479client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.12 seconds)4480client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4481client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4482client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4483client # copying 3 paths...4484client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4485client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4486client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4487client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.17 seconds)4488(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.13 seconds)4489subtest: repeat is answered Cached by the scheduler without building4490worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4491worker1 # finding garbage collector roots...4492worker1 # deleting specified paths...4493worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4494worker1 # deleting unused links...4495worker1 # note: hard linking is currently saving 0.0 KiB4496worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.18 seconds)4497worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4498worker2 # finding garbage collector roots...4499worker2 # deleting specified paths...4500worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4501worker2 # deleting unused links...4502worker2 # note: hard linking is currently saving 0.0 KiB4503worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.15 seconds)4504client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24505worker2 # [ 44.516194] nix-grpc-daemon[784]: ts=2026-09-25T14:59:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334124506client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4507client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4508worker2 # [ 44.534197] nix-grpc-daemon[784]: ts=2026-09-25T14:59:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334124509worker2 # [ 44.542861] nix-grpc-daemon[885]: time=2026-09-25T14:59:18.571Z level=INFO msg="All 2 paths already cached"4510worker2 # [ 44.557255] nix-grpc-daemon[784]: ts=2026-09-25T14:59:18Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14511client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.21 seconds)4512client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4513client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.10 seconds)4514lb: must succeed: curl -sf localhost:9901/clusters4515lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4516lb: must succeed: curl -sf localhost:9901/clusters4517lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4518worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4519worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4520(finished: subtest: repeat is answered Cached by the scheduler without building, in 0.82 seconds)4521subtest: inputs already in the cache are not sent to the scheduler4522lb: must succeed: curl -sf localhost:9901/clusters4523lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4524lb: must succeed: curl -sf localhost:9901/clusters4525lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4526worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4527worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4528client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&24529client # don't know how to build these paths:4530client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4531worker1 # [ 45.061369] nix-grpc-daemon[510]: ts=2026-09-25T14:59:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:446604532worker1 # [ 45.066706] nix-grpc-daemon[510]: ts=2026-09-25T14:59:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446604533worker2 # [ 45.127392] nix-grpc-daemon[784]: ts=2026-09-25T14:59:19Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04534worker2 # [ 45.132848] nix-grpc-daemon[784]: ts=2026-09-25T14:59:19Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.4:50051 clients=14535worker2 # [ 45.139304] nix-grpc-daemon[784]: ts=2026-09-25T14:59:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334124536worker2 # [ 45.149805] nix-grpc-daemon[784]: ts=2026-09-25T14:59:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334124537worker2 # [ 45.162639] nix-grpc-daemon[885]: time=2026-09-25T14:59:19.190Z level=INFO msg="All 2 paths already cached"4538client # copying 1 paths to 'lb:50051'...4539client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4540worker2 # [ 45.171151] nix-grpc-daemon[784]: ts=2026-09-25T14:59:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334124541worker2 # [ 45.179921] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4542worker2 # [ 45.279327] nix-daemon[871]: accepted connection from pid 1018, user nix-grpc-daemon (trusted)4543worker2 # [ 45.292154] nix-daemon[871]: reaped child process 1021, status = succeeded4544lb # [ 45.320584] niks3-server[987]: 2026/09/25 14:59:18 INFO Received push request method=POST path=/api/pushes4545worker2 # [ 45.308276] nix-grpc-daemon[885]: time=2026-09-25T14:59:19.336Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4546worker2 # [ 45.310329] nix-grpc-daemon[885]: time=2026-09-25T14:59:19.338Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4547lb # [ 45.354086] niks3-server[987]: 2026/09/25 14:59:18 INFO Received sign narinfos request method=POST path=/api/pushes/7/sign4548worker2 # [ 45.341577] nix-grpc-daemon[885]: time=2026-09-25T14:59:19.370Z level=INFO msg="Uploading 1 narinfos"4549lb # [ 45.361813] niks3-server[987]: 2026/09/25 14:59:18 INFO Signed narinfos id=7 count=14550lb # [ 45.372817] niks3-server[987]: 2026/09/25 14:59:18 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4551lb # [ 45.377900] envoy[906]: conn peer=[2001:db8:1::1]:55112 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=715 tx=364 ms=11234552lb # [ 45.380695] envoy[906]: conn peer=[2001:db8:1::1]:55122 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=614553lb # [ 45.384785] envoy[906]: conn peer=[2001:db8:1::1]:55126 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=164554lb # [ 45.387574] envoy[906]: conn peer=[2001:db8:1::1]:55130 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=124555lb # [ 45.390334] envoy[906]: conn peer=[2001:db8:1::1]:55134 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9438 tx=4382 ms=56164556lb # [ 45.393150] envoy[906]: conn peer=[2001:db8:1::1]:47810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=544557lb # [ 45.397128] niks3-server[987]: 2026/09/25 14:59:18 INFO Received complete push request method=POST path=/api/pushes/7/complete4558lb # [ 45.401350] niks3-server[987]: 2026/09/25 14:59:18 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4559lb # [ 45.405554] niks3-server[987]: 2026/09/25 14:59:18 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4560worker2 # [ 45.384390] nix-grpc-daemon[885]: time=2026-09-25T14:59:19.412Z level=INFO msg="Upload complete. (190ms)"4561worker2 # [ 45.389289] nix-grpc-daemon[784]: ts=2026-09-25T14:59:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=7524562worker2 # [ 45.395308] nix-grpc-daemon[784]: ts=2026-09-25T14:59:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334124563worker2 # [ 45.408114] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4564worker2 # [ 45.418293] nix-daemon[871]: reaped child process 1016, status = succeeded4565client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4566worker2 # [ 45.425905] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4567client # farm-top> worker2: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4568worker2 # [ 45.761188] nix-grpc-daemon[784]: E0000 00:00:1790348359.789014 824 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4569worker2 # [ 45.793290] nix-grpc-daemon[784]: E0000 00:00:1790348359.820404 824 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4570worker2 # [ 45.802143] nix-grpc-daemon[784]: E0000 00:00:1790348359.820698 824 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4571worker2 # [ 45.824066] nix-grpc-daemon[784]: E0000 00:00:1790348359.852457 824 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4572client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4573client # farm-top> LOG-farm-top-t1b4574worker2 # [ 46.465330] nix-daemon[871]: reaped child process 1025, status = succeeded4575worker2 # [ 46.541339] nix-daemon[871]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)4576worker2 # [ 46.554219] nix-daemon[871]: reaped child process 1044, status = succeeded4577lb # [ 46.582252] niks3-server[987]: 2026/09/25 14:59:19 INFO Received push request method=POST path=/api/pushes4578worker2 # [ 46.569981] nix-grpc-daemon[885]: time=2026-09-25T14:59:20.598Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4579worker2 # [ 46.571695] nix-grpc-daemon[885]: time=2026-09-25T14:59:20.600Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4580lb # [ 46.625359] niks3-server[987]: 2026/09/25 14:59:19 INFO Received sign narinfos request method=POST path=/api/pushes/8/sign4581worker2 # [ 46.609306] nix-grpc-daemon[885]: time=2026-09-25T14:59:20.637Z level=INFO msg="Uploading 1 narinfos"4582lb # [ 46.630589] niks3-server[987]: 2026/09/25 14:59:19 INFO Signed narinfos id=8 count=14583lb # [ 46.638715] niks3-server[987]: 2026/09/25 14:59:19 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4584lb # [ 46.646771] niks3-server[987]: 2026/09/25 14:59:19 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4585lb # [ 46.650760] niks3-server[987]: 2026/09/25 14:59:19 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4586lb # [ 46.662534] niks3-server[987]: 2026/09/25 14:59:19 INFO Received complete push request method=POST path=/api/pushes/8/complete4587lb # [ 46.670879] niks3-server[987]: 2026/09/25 14:59:19 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4588worker2 # [ 46.649989] nix-grpc-daemon[885]: time=2026-09-25T14:59:20.678Z level=INFO msg="Upload complete. (216ms)"4589worker2 # [ 46.657603] nix-grpc-daemon[784]: ts=2026-09-25T14:59:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14590worker2 # [ 46.672270] nix-daemon[871]: reaped child process 1023, status = succeeded4591client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.74 seconds)4592client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4593client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.12 seconds)4594lb: must succeed: curl -sf localhost:9901/clusters4595lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4596lb: must succeed: curl -sf localhost:9901/clusters4597lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4598worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4599worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4600(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.08 seconds)4601subtest: outputs read back through any worker carry the cache signature4602client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&24603worker2 # [ 47.050686] nix-grpc-daemon[784]: ts=2026-09-25T14:59:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334124604client # don't know how to build these paths:4605client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4606worker1 # [ 47.025380] nix-grpc-daemon[510]: ts=2026-09-25T14:59:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446604607worker2 # [ 47.081431] nix-grpc-daemon[784]: ts=2026-09-25T14:59:21Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=04608worker2 # [ 47.085894] nix-grpc-daemon[784]: ts=2026-09-25T14:59:21Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.3:50051 clients=14609worker1 # [ 47.050811] nix-grpc-daemon[510]: ts=2026-09-25T14:59:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446604610worker1 # [ 47.062365] nix-grpc-daemon[510]: ts=2026-09-25T14:59:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446604611client # copying 1 paths to 'lb:50051'...4612client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'...4613worker1 # [ 47.073549] nix-grpc-daemon[510]: ts=2026-09-25T14:59:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446604614worker1 # [ 47.085068] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4615worker1 # [ 47.106494] nix-grpc-daemon[510]: ts=2026-09-25T14:59:21Z level=info event=niks3_push_started pid=8764616worker1 # [ 47.239274] nix-grpc-daemon[876]: time=2026-09-25T14:59:21.393Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""4617worker1 # [ 47.329493] nix-daemon[849]: accepted connection from pid 889, user nix-grpc-daemon (trusted)4618worker1 # [ 47.344040] nix-daemon[849]: reaped child process 892, status = succeeded4619lb # [ 47.407993] niks3-server[987]: 2026/09/25 14:59:20 INFO Received push request method=POST path=/api/pushes4620worker1 # [ 47.358772] nix-grpc-daemon[876]: time=2026-09-25T14:59:21.514Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4621worker1 # [ 47.360317] nix-grpc-daemon[876]: time=2026-09-25T14:59:21.516Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"4622lb # [ 47.444438] niks3-server[987]: 2026/09/25 14:59:20 INFO Received sign narinfos request method=POST path=/api/pushes/9/sign4623lb # [ 47.449145] niks3-server[987]: 2026/09/25 14:59:20 INFO Signed narinfos id=9 count=14624worker1 # [ 47.394425] nix-grpc-daemon[876]: time=2026-09-25T14:59:21.550Z level=INFO msg="Uploading 1 narinfos"4625lb # [ 47.456204] niks3-server[987]: 2026/09/25 14:59:20 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst4626lb # [ 47.466421] niks3-server[987]: 2026/09/25 14:59:20 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls4627lb # [ 47.474099] niks3-server[987]: 2026/09/25 14:59:20 INFO Received complete push request method=POST path=/api/pushes/9/complete4628lb # [ 47.482220] niks3-server[987]: 2026/09/25 14:59:20 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo4629worker1 # [ 47.429903] nix-grpc-daemon[876]: time=2026-09-25T14:59:21.585Z level=INFO msg="Upload complete. (187ms)"4630worker1 # [ 47.432792] nix-grpc-daemon[510]: ts=2026-09-25T14:59:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=0 paths=1 nar_bytes_in=5044631worker1 # [ 47.437447] nix-grpc-daemon[510]: ts=2026-09-25T14:59:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446604632worker1 # [ 47.446560] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4633worker1 # [ 47.451954] nix-daemon[849]: reaped child process 874, status = succeeded4634worker1 # [ 47.456677] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4635client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...4636client # farm-a> node-a: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4637client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4638client # farm-a> LOG-farm-a-sig4639worker1 # [ 48.501130] nix-daemon[849]: reaped child process 896, status = succeeded4640worker1 # [ 48.576264] nix-daemon[849]: accepted connection from pid 912, user nix-grpc-daemon (trusted)4641worker1 # [ 48.589388] nix-daemon[849]: reaped child process 915, status = succeeded4642lb # [ 48.654534] niks3-server[987]: 2026/09/25 14:59:21 INFO Received push request method=POST path=/api/pushes4643worker1 # [ 48.604660] nix-grpc-daemon[876]: time=2026-09-25T14:59:22.760Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4644worker1 # [ 48.606157] nix-grpc-daemon[876]: time=2026-09-25T14:59:22.761Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4645lb # [ 48.696681] niks3-server[987]: 2026/09/25 14:59:22 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4646lb # [ 48.702521] niks3-server[987]: 2026/09/25 14:59:22 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign4647lb # [ 48.706630] niks3-server[987]: 2026/09/25 14:59:22 INFO Signed narinfos id=10 count=14648worker1 # [ 48.651314] nix-grpc-daemon[876]: time=2026-09-25T14:59:22.807Z level=INFO msg="Uploading 1 narinfos"4649lb # [ 48.709176] niks3-server[987]: 2026/09/25 14:59:22 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4650lb # [ 48.721531] niks3-server[987]: 2026/09/25 14:59:22 INFO Received complete push request method=POST path=/api/pushes/10/complete4651lb # [ 48.729409] niks3-server[987]: 2026/09/25 14:59:22 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4652lb # [ 48.732249] niks3-server[987]: 2026/09/25 14:59:22 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4653worker1 # [ 48.679806] nix-grpc-daemon[876]: time=2026-09-25T14:59:22.835Z level=INFO msg="Upload complete. (215ms)"4654worker1 # [ 48.685973] nix-grpc-daemon[510]: ts=2026-09-25T14:59:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=14655worker2 # [ 48.728920] nix-grpc-daemon[784]: ts=2026-09-25T14:59:22Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04656worker2 # [ 48.733677] nix-grpc-daemon[784]: ts=2026-09-25T14:59:22Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.3:50051 clients=14657worker1 # [ 48.700541] nix-grpc-daemon[510]: ts=2026-09-25T14:59:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446604658worker1 # [ 48.706265] nix-daemon[849]: reaped child process 894, status = succeeded4659worker1 # [ 48.711507] nix-grpc-daemon[510]: ts=2026-09-25T14:59:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446604660worker1 # [ 48.721953] nix-grpc-daemon[876]: time=2026-09-25T14:59:22.877Z level=INFO msg="All 1 paths already cached"4661client # copying 1 paths to 'lb:50051'...4662client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4663worker1 # [ 48.729304] nix-grpc-daemon[510]: ts=2026-09-25T14:59:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446604664worker1 # [ 48.737874] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4665worker1 # [ 48.825748] nix-daemon[849]: accepted connection from pid 919, user nix-grpc-daemon (trusted)4666worker1 # [ 48.839704] nix-daemon[849]: reaped child process 922, status = succeeded4667lb # [ 48.904241] niks3-server[987]: 2026/09/25 14:59:22 INFO Received push request method=POST path=/api/pushes4668worker1 # [ 48.854339] nix-grpc-daemon[876]: time=2026-09-25T14:59:23.010Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4669worker1 # [ 48.855818] nix-grpc-daemon[876]: time=2026-09-25T14:59:23.011Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4670lb # [ 48.936759] niks3-server[987]: 2026/09/25 14:59:22 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign4671lb # [ 48.943404] niks3-server[987]: 2026/09/25 14:59:22 INFO Signed narinfos id=11 count=14672worker1 # [ 48.888726] nix-grpc-daemon[876]: time=2026-09-25T14:59:23.044Z level=INFO msg="Uploading 1 narinfos"4673lb # [ 48.951533] niks3-server[987]: 2026/09/25 14:59:22 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4674lb # [ 48.957400] niks3-server[987]: 2026/09/25 14:59:22 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4675lb # [ 48.968892] niks3-server[987]: 2026/09/25 14:59:22 INFO Received complete push request method=POST path=/api/pushes/11/complete4676lb # [ 48.976537] niks3-server[987]: 2026/09/25 14:59:22 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4677worker1 # [ 48.921945] nix-grpc-daemon[876]: time=2026-09-25T14:59:23.077Z level=INFO msg="Upload complete. (177ms)"4678worker1 # [ 48.925953] nix-grpc-daemon[510]: ts=2026-09-25T14:59:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=0 paths=1 nar_bytes_in=6244679worker1 # [ 48.930351] nix-grpc-daemon[510]: ts=2026-09-25T14:59:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446604680worker1 # [ 48.939798] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4681worker1 # [ 48.947789] nix-daemon[849]: reaped child process 917, status = succeeded4682client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4683worker1 # [ 48.953623] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4684client # farm-b> node-a: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4685client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4686client # farm-b> LOG-farm-b-sig4687worker1 # [ 50.393317] nix-daemon[849]: reaped child process 926, status = succeeded4688worker1 # [ 50.473160] nix-daemon[849]: accepted connection from pid 942, user nix-grpc-daemon (trusted)4689worker1 # [ 50.484480] nix-daemon[849]: reaped child process 945, status = succeeded4690lb # [ 50.550453] niks3-server[987]: 2026/09/25 14:59:23 INFO Received push request method=POST path=/api/pushes4691worker1 # [ 50.502209] nix-grpc-daemon[876]: time=2026-09-25T14:59:24.657Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4692worker1 # [ 50.503728] nix-grpc-daemon[876]: time=2026-09-25T14:59:24.659Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4693lb # [ 50.589230] niks3-server[987]: 2026/09/25 14:59:23 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign4694worker1 # [ 50.539465] nix-grpc-daemon[876]: time=2026-09-25T14:59:24.695Z level=INFO msg="Uploading 1 narinfos"4695lb # [ 50.597217] niks3-server[987]: 2026/09/25 14:59:23 INFO Signed narinfos id=12 count=14696lb # [ 50.603990] niks3-server[987]: 2026/09/25 14:59:23 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4697lb # [ 50.613391] niks3-server[987]: 2026/09/25 14:59:23 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4698lb # [ 50.620120] niks3-server[987]: 2026/09/25 14:59:23 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4699lb # [ 50.625228] niks3-server[987]: 2026/09/25 14:59:23 INFO Received complete push request method=POST path=/api/pushes/12/complete4700lb # [ 50.632735] niks3-server[987]: 2026/09/25 14:59:23 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4701worker1 # [ 50.578243] nix-grpc-daemon[876]: time=2026-09-25T14:59:24.733Z level=INFO msg="Upload complete. (217ms)"4702worker1 # [ 50.584979] nix-grpc-daemon[510]: ts=2026-09-25T14:59:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=14703worker2 # [ 50.628648] nix-grpc-daemon[784]: ts=2026-09-25T14:59:24Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04704worker1 # [ 50.595783] nix-grpc-daemon[510]: ts=2026-09-25T14:59:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446604705worker2 # [ 50.633189] nix-grpc-daemon[784]: ts=2026-09-25T14:59:24Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.3:50051 clients=14706worker1 # [ 50.608529] nix-grpc-daemon[510]: ts=2026-09-25T14:59:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446604707worker1 # [ 50.615841] nix-daemon[849]: reaped child process 924, status = succeeded4708worker1 # [ 50.618794] nix-grpc-daemon[876]: time=2026-09-25T14:59:24.773Z level=INFO msg="All 2 paths already cached"4709client # copying 1 paths to 'lb:50051'...4710client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4711worker1 # [ 50.625814] nix-grpc-daemon[510]: ts=2026-09-25T14:59:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446604712worker1 # [ 50.639135] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4713worker1 # [ 50.726236] nix-daemon[849]: accepted connection from pid 949, user nix-grpc-daemon (trusted)4714worker1 # [ 50.739070] nix-daemon[849]: reaped child process 952, status = succeeded4715lb # [ 50.803507] niks3-server[987]: 2026/09/25 14:59:24 INFO Received push request method=POST path=/api/pushes4716worker1 # [ 50.755790] nix-grpc-daemon[876]: time=2026-09-25T14:59:24.911Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4717worker1 # [ 50.757808] nix-grpc-daemon[876]: time=2026-09-25T14:59:24.913Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4718lb # [ 50.833666] niks3-server[987]: 2026/09/25 14:59:24 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign4719worker1 # [ 50.786106] nix-grpc-daemon[876]: time=2026-09-25T14:59:24.941Z level=INFO msg="Uploading 1 narinfos"4720lb # [ 50.843322] niks3-server[987]: 2026/09/25 14:59:24 INFO Signed narinfos id=13 count=14721lb # [ 50.847105] niks3-server[987]: 2026/09/25 14:59:24 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls4722lb # [ 50.855302] niks3-server[987]: 2026/09/25 14:59:24 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst4723lb # [ 50.861836] niks3-server[987]: 2026/09/25 14:59:24 INFO Received complete push request method=POST path=/api/pushes/13/complete4724worker1 # [ 50.808740] nix-grpc-daemon[876]: time=2026-09-25T14:59:24.964Z level=INFO msg="Upload complete. (160ms)"4725lb # [ 50.867071] niks3-server[987]: 2026/09/25 14:59:24 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo4726worker1 # [ 50.812790] nix-grpc-daemon[510]: ts=2026-09-25T14:59:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=0 paths=1 nar_bytes_in=7524727worker1 # [ 50.817908] nix-grpc-daemon[510]: ts=2026-09-25T14:59:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446604728worker1 # [ 50.826065] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4729worker1 # [ 50.830503] nix-daemon[849]: reaped child process 947, status = succeeded4730worker1 # [ 50.835575] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4731client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...4732client # farm-top> node-a: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4733client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4734client # farm-top> LOG-farm-top-sig4735worker1 # [ 52.050604] nix-daemon[849]: reaped child process 956, status = succeeded4736worker1 # [ 52.126668] nix-daemon[849]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4737worker1 # [ 52.140201] nix-daemon[849]: reaped child process 975, status = succeeded4738lb # [ 52.205748] niks3-server[987]: 2026/09/25 14:59:25 INFO Received push request method=POST path=/api/pushes4739worker1 # [ 52.156602] nix-grpc-daemon[876]: time=2026-09-25T14:59:26.312Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4740worker1 # [ 52.158330] nix-grpc-daemon[876]: time=2026-09-25T14:59:26.313Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"4741lb # [ 52.247127] niks3-server[987]: 2026/09/25 14:59:25 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls4742lb # [ 52.252463] niks3-server[987]: 2026/09/25 14:59:25 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4743lb # [ 52.259991] niks3-server[987]: 2026/09/25 14:59:25 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign4744lb # [ 52.266259] niks3-server[987]: 2026/09/25 14:59:25 INFO Signed narinfos id=14 count=14745worker1 # [ 52.211704] nix-grpc-daemon[876]: time=2026-09-25T14:59:26.367Z level=INFO msg="Uploading 1 narinfos"4746lb # [ 52.273111] niks3-server[987]: 2026/09/25 14:59:25 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst4747lb # [ 52.282222] niks3-server[987]: 2026/09/25 14:59:25 INFO Received complete push request method=POST path=/api/pushes/14/complete4748worker1 # [ 52.230121] nix-grpc-daemon[876]: time=2026-09-25T14:59:26.385Z level=INFO msg="Upload complete. (185ms)"4749lb # [ 52.290735] niks3-server[987]: 2026/09/25 14:59:25 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo4750worker1 # [ 52.235053] nix-grpc-daemon[510]: ts=2026-09-25T14:59:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=14751worker1 # [ 52.247708] nix-daemon[849]: reaped child process 954, status = succeeded4752client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 5.38 seconds)4753client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath4754client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.12 seconds)4755client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4756worker1 # [ 52.498764] nix-grpc-daemon[510]: ts=2026-09-25T14:59:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:420624757worker1 # [ 52.504637] nix-grpc-daemon[510]: ts=2026-09-25T14:59:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:420624758client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.13 seconds)4759client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4760worker2 # [ 52.665761] nix-grpc-daemon[784]: ts=2026-09-25T14:59:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:552284761client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4762client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a4763client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b4764client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4765worker2 # [ 52.686683] nix-grpc-daemon[784]: ts=2026-09-25T14:59:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:552284766worker2 # [ 52.693288] nix-grpc-daemon[784]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...4767worker2 # [ 52.707130] nix-grpc-daemon[784]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...4768worker2 # [ 52.719143] nix-grpc-daemon[784]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...4769client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.20 seconds)4770client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4771client # finding garbage collector roots...4772client # removing stale temporary roots file "/nix/var/nix/temproots/823"4773client # removing stale temporary roots file "/nix/var/nix/temproots/885"4774client # removing stale temporary roots file "/nix/var/nix/temproots/900"4775client # removing stale temporary roots file "/nix/var/nix/temproots/856"4776client # removing stale temporary roots file "/nix/var/nix/temproots/877"4777client # removing stale temporary roots file "/nix/var/nix/temproots/864"4778client # removing stale temporary roots file "/nix/var/nix/temproots/844"4779client # removing stale temporary roots file "/nix/var/nix/temproots/808"4780client # deleting specified paths...4781client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'4782client # deleting unused links...4783client # note: hard linking is currently saving 0.0 KiB4784worker1 # [ 52.969479] nix-grpc-daemon[510]: ts=2026-09-25T14:59:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:446604785worker2 # [ 53.015121] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334124786worker2 # [ 53.025739] nix-grpc-daemon[885]: time=2026-09-25T14:59:27.053Z level=INFO msg="All 1 paths already cached"4787worker2 # [ 53.031949] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:334124788worker2 # [ 53.039321] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:334124789client # copying 3 paths...4790worker2 # [ 53.049434] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:334124791client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...4792worker2 # [ 53.065812] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:334124793worker2 # [ 53.071462] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_out=2324794worker1 # [ 53.036165] nix-grpc-daemon[510]: ts=2026-09-25T14:59:27Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:446604795worker1 # [ 53.042182] nix-grpc-daemon[510]: ts=2026-09-25T14:59:27Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=0 paths=1 nar_bytes_out=1764796worker2 # [ 53.080676] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:334124797worker2 # [ 53.085479] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_out=1284798client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...4799client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...4800client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.36 seconds)4801(finished: subtest: outputs read back through any worker carry the cache signature, in 6.19 seconds)4802subtest: two clients wanting the same drv build it once4803client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4804client # [ 53.140331] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4805client # Running as unit: dedup1.service; invocation ID: db66a76ac20d43b48dfb77982ffd32f34806client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)4807client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4808client # [ 53.274955] nix[953]: don't know how to build these paths:4809client # [ 53.275949] nix[953]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4810worker1 # [ 53.237814] nix-grpc-daemon[510]: ts=2026-09-25T14:59:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:446604811worker2 # [ 53.283896] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04812worker2 # [ 53.290813] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14813worker2 # [ 53.295660] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334124814worker2 # [ 53.303474] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334124815client # [ 53.311853] nix[953]: copying 1 paths to 'lb:50051'...4816client # [ 53.312832] nix[953]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4817worker2 # [ 53.314166] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334124818worker2 # [ 53.325150] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4819worker2 # [ 53.426892] nix-daemon[871]: accepted connection from pid 1064, user nix-grpc-daemon (trusted)4820worker2 # [ 53.442103] nix-daemon[871]: reaped child process 1067, status = succeeded4821lb # [ 53.467622] niks3-server[987]: 2026/09/25 14:59:26 INFO Received push request method=POST path=/api/pushes4822worker2 # [ 53.457215] nix-grpc-daemon[885]: time=2026-09-25T14:59:27.485Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4823worker2 # [ 53.458751] nix-grpc-daemon[885]: time=2026-09-25T14:59:27.487Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4824lb # [ 53.498521] niks3-server[987]: 2026/09/25 14:59:26 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign4825lb # [ 53.508748] niks3-server[987]: 2026/09/25 14:59:26 INFO Signed narinfos id=15 count=14826worker2 # [ 53.491086] nix-grpc-daemon[885]: time=2026-09-25T14:59:27.519Z level=INFO msg="Uploading 1 narinfos"4827lb # [ 53.516389] niks3-server[987]: 2026/09/25 14:59:26 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4828lb # [ 53.524160] niks3-server[987]: 2026/09/25 14:59:26 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4829lb # [ 53.532375] niks3-server[987]: 2026/09/25 14:59:26 INFO Received complete push request method=POST path=/api/pushes/15/complete4830worker2 # [ 53.519961] nix-grpc-daemon[885]: time=2026-09-25T14:59:27.548Z level=INFO msg="Upload complete. (183ms)"4831lb # [ 53.541635] niks3-server[987]: 2026/09/25 14:59:26 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4832worker2 # [ 53.524329] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=4324833worker2 # [ 53.530289] nix-grpc-daemon[784]: ts=2026-09-25T14:59:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334124834worker2 # [ 53.540273] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4835worker2 # [ 53.550157] nix-daemon[871]: reaped child process 1062, status = succeeded4836worker2 # [ 53.553550] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4837client # [ 53.552149] nix[953]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4838client # [ 54.198614] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4839client # Running as unit: dedup2.service; invocation ID: 486bb487363e4123814bd9914751f7cf4840client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)4841client: must succeed: sleep 24842client # [ 54.390356] nix[969]: don't know how to build these paths:4843worker1 # [ 54.351622] nix-grpc-daemon[510]: ts=2026-09-25T14:59:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:446604844client # [ 54.391212] nix[969]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4845worker2 # [ 54.402785] nix-grpc-daemon[784]: ts=2026-09-25T14:59:28Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04846worker2 # [ 54.405157] nix-grpc-daemon[784]: ts=2026-09-25T14:59:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334124847client # [ 54.406149] nix[969]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4848lb # [ 55.378665] envoy[906]: conn peer=[2001:db8:1::1]:47814 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4208 tx=1896 ms=15764849lb # [ 55.382530] envoy[906]: conn peer=[2001:db8:1::1]:47826 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9494 tx=4354 ms=52334850lb # [ 55.386223] envoy[906]: conn peer=[2001:db8:1::1]:57792 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1034 tx=2870 ms=1054851lb # [ 55.389782] envoy[906]: conn peer=[2001:db8:1::1]:57814 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=274852lb # [ 55.393389] envoy[906]: conn peer=[2001:db8:1::1]:57810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=354853lb # [ 55.396783] envoy[906]: conn peer=[2001:db8:1::1]:57796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=424854client: (finished: must succeed: sleep 2, in 2.05 seconds)4855client: waiting for success: ! systemctl is-active dedup1 dedup24856worker2 # [ 56.574967] nix-daemon[871]: reaped child process 1071, status = succeeded4857worker2 # [ 56.658458] nix-daemon[871]: accepted connection from pid 1099, user nix-grpc-daemon (trusted)4858worker2 # [ 56.675269] nix-daemon[871]: reaped child process 1102, status = succeeded4859lb # [ 56.701425] niks3-server[987]: 2026/09/25 14:59:30 INFO Received push request method=POST path=/api/pushes4860worker2 # [ 56.691109] nix-grpc-daemon[885]: time=2026-09-25T14:59:30.719Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4861worker2 # [ 56.692794] nix-grpc-daemon[885]: time=2026-09-25T14:59:30.721Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4862lb # [ 56.745844] niks3-server[987]: 2026/09/25 14:59:30 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4863lb # [ 56.754081] niks3-server[987]: 2026/09/25 14:59:30 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign4864lb # [ 56.763668] niks3-server[987]: 2026/09/25 14:59:30 INFO Signed narinfos id=16 count=14865worker2 # [ 56.745595] nix-grpc-daemon[885]: time=2026-09-25T14:59:30.774Z level=INFO msg="Uploading 1 narinfos"4866lb # [ 56.768408] niks3-server[987]: 2026/09/25 14:59:30 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4867lb # [ 56.772646] niks3-server[987]: 2026/09/25 14:59:30 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4868lb # [ 56.783041] niks3-server[987]: 2026/09/25 14:59:30 INFO Received complete push request method=POST path=/api/pushes/16/complete4869lb # [ 56.789367] niks3-server[987]: 2026/09/25 14:59:30 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4870worker2 # [ 56.774206] nix-grpc-daemon[885]: time=2026-09-25T14:59:30.802Z level=INFO msg="Upload complete. (212ms)"4871worker2 # [ 56.782539] nix-grpc-daemon[784]: ts=2026-09-25T14:59:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=14872worker2 # [ 56.791539] nix-grpc-daemon[784]: ts=2026-09-25T14:59:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=2 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=14873worker2 # [ 56.805131] nix-daemon[871]: reaped child process 1069, status = succeeded4874client # [ 56.832783] systemd[1]: dedup2.service: Deactivated successfully.4875client # [ 56.836097] systemd[1]: dedup2.service: Consumed 147ms CPU time over 2.637s wall clock time, 35.4M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.4876client # [ 56.843162] systemd[1]: dedup1.service: Deactivated successfully.4877client # [ 56.846058] systemd[1]: dedup1.service: Consumed 121ms CPU time over 3.703s wall clock time, 43.6M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.4878client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)4879client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4880client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4881worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4882worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4883worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4884worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4885worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4886worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4887worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4888worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4889(finished: subtest: two clients wanting the same drv build it once, in 4.68 seconds)4890subtest: --timeout and --max-silent-time reach the builder4891worker1 # [ 57.913668] nix-grpc-daemon[510]: ts=2026-09-25T14:59:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:446604892worker2 # [ 57.965839] nix-grpc-daemon[784]: ts=2026-09-25T14:59:31Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=04893worker1 # [ 57.930810] nix-grpc-daemon[510]: ts=2026-09-25T14:59:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446604894worker2 # [ 57.970109] nix-grpc-daemon[784]: ts=2026-09-25T14:59:31Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=14895worker1 # [ 57.942606] nix-grpc-daemon[510]: ts=2026-09-25T14:59:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446604896worker1 # [ 57.953437] nix-grpc-daemon[510]: ts=2026-09-25T14:59:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446604897worker1 # [ 57.965327] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4898worker1 # [ 58.076078] nix-daemon[849]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4899worker1 # [ 58.092485] nix-daemon[849]: reaped child process 999, status = succeeded4900lb # [ 58.157203] niks3-server[987]: 2026/09/25 14:59:31 INFO Received push request method=POST path=/api/pushes4901worker1 # [ 58.112204] nix-grpc-daemon[876]: time=2026-09-25T14:59:32.267Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4902worker1 # [ 58.113623] nix-grpc-daemon[876]: time=2026-09-25T14:59:32.267Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"4903lb # [ 58.189666] niks3-server[987]: 2026/09/25 14:59:31 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign4904worker1 # [ 58.139527] nix-grpc-daemon[876]: time=2026-09-25T14:59:32.295Z level=INFO msg="Uploading 1 narinfos"4905lb # [ 58.196965] niks3-server[987]: 2026/09/25 14:59:31 INFO Signed narinfos id=17 count=14906lb # [ 58.209420] niks3-server[987]: 2026/09/25 14:59:31 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls4907lb # [ 58.214112] niks3-server[987]: 2026/09/25 14:59:31 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst4908lb # [ 58.221676] niks3-server[987]: 2026/09/25 14:59:31 INFO Received complete push request method=POST path=/api/pushes/17/complete4909lb # [ 58.231419] niks3-server[987]: 2026/09/25 14:59:31 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo4910worker1 # [ 58.173189] nix-grpc-daemon[876]: time=2026-09-25T14:59:32.328Z level=INFO msg="Upload complete. (196ms)"4911worker1 # [ 58.177987] nix-grpc-daemon[510]: ts=2026-09-25T14:59:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=0 paths=1 nar_bytes_in=4244912worker1 # [ 58.183766] nix-grpc-daemon[510]: ts=2026-09-25T14:59:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446604913worker1 # [ 58.196109] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4914worker1 # [ 58.204192] nix-daemon[849]: reaped child process 994, status = succeeded4915worker1 # [ 58.209697] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4916worker1 # [ 64.222316] nix-grpc-daemon[510]: ts=2026-09-25T14:59:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=04917worker1 # [ 64.234282] nix-daemon[849]: reaped child process 1001, status = succeeded4918worker1 # [ 64.235918] nix-daemon[849]: reaped child process 1003, status = succeeded4919worker2 # [ 64.429618] nix-grpc-daemon[784]: ts=2026-09-25T14:59:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334124920worker2 # [ 64.444443] nix-grpc-daemon[784]: ts=2026-09-25T14:59:38Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=04921worker2 # [ 64.449687] nix-grpc-daemon[784]: ts=2026-09-25T14:59:38Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=14922worker2 # [ 64.455590] nix-grpc-daemon[784]: ts=2026-09-25T14:59:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334124923worker2 # [ 64.467528] nix-grpc-daemon[784]: ts=2026-09-25T14:59:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334124924worker2 # [ 64.479066] nix-grpc-daemon[784]: ts=2026-09-25T14:59:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334124925worker2 # [ 64.491354] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4926worker2 # [ 64.602649] nix-daemon[871]: accepted connection from pid 1119, user nix-grpc-daemon (trusted)4927worker2 # [ 64.619356] nix-daemon[871]: reaped child process 1122, status = succeeded4928lb # [ 64.647373] niks3-server[987]: 2026/09/25 14:59:37 INFO Received push request method=POST path=/api/pushes4929worker2 # [ 64.635843] nix-grpc-daemon[885]: time=2026-09-25T14:59:38.663Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4930worker2 # [ 64.637226] nix-grpc-daemon[885]: time=2026-09-25T14:59:38.663Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"4931lb # [ 64.678579] niks3-server[987]: 2026/09/25 14:59:37 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign4932lb # [ 64.688598] niks3-server[987]: 2026/09/25 14:59:37 INFO Signed narinfos id=18 count=14933worker2 # [ 64.671383] nix-grpc-daemon[885]: time=2026-09-25T14:59:38.699Z level=INFO msg="Uploading 1 narinfos"4934lb # [ 64.694068] niks3-server[987]: 2026/09/25 14:59:38 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls4935lb # [ 64.703550] niks3-server[987]: 2026/09/25 14:59:38 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst4936lb # [ 64.713521] niks3-server[987]: 2026/09/25 14:59:38 INFO Received complete push request method=POST path=/api/pushes/18/complete4937lb # [ 64.721606] niks3-server[987]: 2026/09/25 14:59:38 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo4938worker2 # [ 64.703936] nix-grpc-daemon[885]: time=2026-09-25T14:59:38.732Z level=INFO msg="Upload complete. (200ms)"4939worker2 # [ 64.708366] nix-grpc-daemon[784]: ts=2026-09-25T14:59:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=4244940worker2 # [ 64.713292] nix-grpc-daemon[784]: ts=2026-09-25T14:59:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334124941worker2 # [ 64.724112] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4942worker2 # [ 64.732868] nix-daemon[871]: reaped child process 1117, status = succeeded4943worker2 # [ 64.737230] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)4944lb # [ 65.379406] envoy[906]: conn peer=[2001:db8:1::1]:57822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2994 tx=1521 ms=35254945lb # [ 65.383199] envoy[906]: conn peer=[2001:db8:1::1]:57828 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1453 tx=1194 ms=24164946lb # [ 65.386683] envoy[906]: conn peer=[2001:db8:1::1]:57832 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=933 ms=63244947worker2 # [ 70.753732] nix-grpc-daemon[784]: ts=2026-09-25T14:59:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=04948worker2 # [ 70.782395] nix-daemon[871]: reaped child process 1124, status = succeeded4949worker2 # [ 70.784863] nix-daemon[871]: reaped child process 1126, status = succeeded4950(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.10 seconds)4951subtest: the first client leaving does not fail the second4952client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share4953client # [ 70.939073] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.4954client # Running as unit: share1.service; invocation ID: fcfce5dcf66445bc843e5acdc57dd43a4955client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)4956client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share4957worker1 # [ 71.045874] nix-grpc-daemon[510]: ts=2026-09-25T14:59:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:446604958client # [ 71.088158] nix[1040]: don't know how to build these paths:4959client # [ 71.089089] nix[1040]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv4960worker2 # [ 71.097397] nix-grpc-daemon[784]: ts=2026-09-25T14:59:45Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=04961worker2 # [ 71.101486] nix-grpc-daemon[784]: ts=2026-09-25T14:59:45Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.3:50051 clients=14962worker1 # [ 71.071737] nix-grpc-daemon[510]: ts=2026-09-25T14:59:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446604963worker1 # [ 71.084744] nix-grpc-daemon[510]: ts=2026-09-25T14:59:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446604964client # [ 71.130826] nix[1040]: copying 1 paths to 'lb:50051'...4965client # [ 71.131720] nix[1040]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...4966worker1 # [ 71.095939] nix-grpc-daemon[510]: ts=2026-09-25T14:59:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446604967worker1 # [ 71.107781] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4968worker1 # [ 71.220583] nix-daemon[849]: accepted connection from pid 1026, user nix-grpc-daemon (trusted)4969worker1 # [ 71.236769] nix-daemon[849]: reaped child process 1029, status = succeeded4970lb # [ 71.302362] niks3-server[987]: 2026/09/25 14:59:44 INFO Received push request method=POST path=/api/pushes4971worker1 # [ 71.253647] nix-grpc-daemon[876]: time=2026-09-25T14:59:45.409Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4972worker1 # [ 71.255841] nix-grpc-daemon[876]: time=2026-09-25T14:59:45.411Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"4973lb # [ 71.336681] niks3-server[987]: 2026/09/25 14:59:44 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign4974worker1 # [ 71.287167] nix-grpc-daemon[876]: time=2026-09-25T14:59:45.442Z level=INFO msg="Uploading 1 narinfos"4975lb # [ 71.344332] niks3-server[987]: 2026/09/25 14:59:44 INFO Signed narinfos id=19 count=14976lb # [ 71.352166] niks3-server[987]: 2026/09/25 14:59:44 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls4977lb # [ 71.365246] niks3-server[987]: 2026/09/25 14:59:44 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst4978lb # [ 71.371502] niks3-server[987]: 2026/09/25 14:59:44 INFO Received complete push request method=POST path=/api/pushes/19/complete4979lb # [ 71.381064] niks3-server[987]: 2026/09/25 14:59:44 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo4980worker1 # [ 71.323180] nix-grpc-daemon[876]: time=2026-09-25T14:59:45.478Z level=INFO msg="Upload complete. (203ms)"4981worker1 # [ 71.327841] nix-grpc-daemon[510]: ts=2026-09-25T14:59:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=0 paths=1 nar_bytes_in=4324982worker1 # [ 71.334344] nix-grpc-daemon[510]: ts=2026-09-25T14:59:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446604983worker1 # [ 71.346112] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4984worker1 # [ 71.354387] nix-daemon[849]: reaped child process 1024, status = succeeded4985worker1 # [ 71.359631] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)4986client # [ 71.395193] nix[1040]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...4987client # [ 72.010724] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.4988client # Running as unit: share2.service; invocation ID: b83bf63292d54de0b3c8ca333e0d49194989client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)4990worker2 # [ 72.156395] nix-grpc-daemon[784]: ts=2026-09-25T14:59:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334124991client # [ 72.160959] nix[1056]: don't know how to build these paths:4992client # [ 72.161929] nix[1056]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv4993worker2 # [ 72.168994] nix-grpc-daemon[784]: ts=2026-09-25T14:59:46Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=04994client # [ 72.173121] nix[1056]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...4995worker1 # [ 72.136745] nix-grpc-daemon[510]: ts=2026-09-25T14:59:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446604996client: must succeed: sleep 2; systemctl kill -s INT share14997client # [ 75.278470] systemd[1]: share1.service: Sent signal SIGINT to main process 1040 (nix) on client request.4998client # [ 75.283778] nix[1040]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4999client # [ 75.289405] nix[1040]: error: interrupted by the user5000client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5001client: waiting for success: ! systemctl is-active share15002client # [ 75.332887] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5003client # [ 75.334456] systemd[1]: share1.service: Failed with result 'exit-code'.5004client # [ 75.336528] systemd[1]: share1.service: Consumed 120ms CPU time over 4.398s wall clock time, 41.4M memory peak, 6.1K incoming IP traffic, 8K outgoing IP traffic.5005lb # [ 75.379446] envoy[906]: conn peer=[2001:db8:1::1]:51400 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2952 tx=935 ms=63415006lb # [ 75.380998] envoy[906]: conn peer=[2001:db8:1::1]:51402 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=770 ms=42195007client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5008client: waiting for success: ! systemctl is-active share25009worker1 # [ 76.583079] nix-daemon[849]: reaped child process 1033, status = succeeded5010worker1 # [ 76.655755] nix-daemon[849]: accepted connection from pid 1067, user nix-grpc-daemon (trusted)5011worker1 # [ 76.669265] nix-daemon[849]: reaped child process 1070, status = succeeded5012lb # [ 76.732903] niks3-server[987]: 2026/09/25 14:59:50 INFO Received push request method=POST path=/api/pushes5013worker1 # [ 76.681776] nix-grpc-daemon[876]: time=2026-09-25T14:59:50.837Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5014worker1 # [ 76.683249] nix-grpc-daemon[876]: time=2026-09-25T14:59:50.839Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5015lb # [ 76.758702] niks3-server[987]: 2026/09/25 14:59:50 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5016lb # [ 76.761363] niks3-server[987]: 2026/09/25 14:59:50 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5017lb # [ 76.764052] niks3-server[987]: 2026/09/25 14:59:50 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign5018lb # [ 76.765830] niks3-server[987]: 2026/09/25 14:59:50 INFO Signed narinfos id=20 count=15019worker1 # [ 76.708540] nix-grpc-daemon[876]: time=2026-09-25T14:59:50.864Z level=INFO msg="Uploading 1 narinfos"5020lb # [ 76.771237] niks3-server[987]: 2026/09/25 14:59:50 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5021lb # [ 76.775061] niks3-server[987]: 2026/09/25 14:59:50 INFO Received complete push request method=POST path=/api/pushes/20/complete5022worker1 # [ 76.722488] nix-grpc-daemon[876]: time=2026-09-25T14:59:50.878Z level=INFO msg="Upload complete. (151ms)"5023lb # [ 76.782881] niks3-server[987]: 2026/09/25 14:59:50 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5024worker1 # [ 76.725387] nix-grpc-daemon[510]: ts=2026-09-25T14:59:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15025worker1 # [ 76.729148] nix-grpc-daemon[510]: ts=2026-09-25T14:59:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15026worker1 # [ 76.734725] nix-daemon[849]: reaped child process 1031, status = succeeded5027client # [ 76.794789] systemd[1]: share2.service: Deactivated successfully.5028client # [ 76.798770] systemd[1]: share2.service: Consumed 122ms CPU time over 4.787s wall clock time, 35.4M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.5029client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds)5030client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5031client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5032(finished: subtest: the first client leaving does not fail the second, in 6.77 seconds)5033subtest: build hook: nix-daemon with builders = grpc://lb5034client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5035client # [ 77.757975] systemd[1]: Started Nix Daemon.5036client # [ 77.847497] nix-daemon[1108]: accepted connection from pid 1105, user root (trusted)5037worker2 # [ 77.964912] nix-grpc-daemon[784]: ts=2026-09-25T14:59:51Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:334125038worker1 # [ 77.935677] nix-grpc-daemon[510]: ts=2026-09-25T14:59:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446605039worker2 # [ 77.986571] nix-grpc-daemon[784]: ts=2026-09-25T14:59:52Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05040worker2 # [ 77.991703] nix-grpc-daemon[784]: ts=2026-09-25T14:59:52Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.4:50051 clients=15041worker2 # [ 77.997644] nix-grpc-daemon[784]: ts=2026-09-25T14:59:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334125042worker2 # [ 78.006954] nix-grpc-daemon[784]: ts=2026-09-25T14:59:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334125043worker2 # [ 78.016359] nix-grpc-daemon[784]: ts=2026-09-25T14:59:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334125044worker2 # [ 78.028334] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5045worker2 # [ 78.131638] nix-daemon[871]: accepted connection from pid 1171, user nix-grpc-daemon (trusted)5046worker2 # [ 78.146255] nix-daemon[871]: reaped child process 1174, status = succeeded5047lb # [ 78.174878] niks3-server[987]: 2026/09/25 14:59:51 INFO Received push request method=POST path=/api/pushes5048worker2 # [ 78.159231] nix-grpc-daemon[885]: time=2026-09-25T14:59:52.187Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5049worker2 # [ 78.160715] nix-grpc-daemon[885]: time=2026-09-25T14:59:52.187Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5050lb # [ 78.186531] niks3-server[987]: 2026/09/25 14:59:51 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign5051lb # [ 78.190446] niks3-server[987]: 2026/09/25 14:59:51 INFO Signed narinfos id=21 count=15052worker2 # [ 78.169881] nix-grpc-daemon[885]: time=2026-09-25T14:59:52.198Z level=INFO msg="Uploading 1 narinfos"5053lb # [ 78.193156] niks3-server[987]: 2026/09/25 14:59:51 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5054lb # [ 78.195854] niks3-server[987]: 2026/09/25 14:59:51 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5055lb # [ 78.200148] niks3-server[987]: 2026/09/25 14:59:51 INFO Received complete push request method=POST path=/api/pushes/21/complete5056worker2 # [ 78.183966] nix-grpc-daemon[885]: time=2026-09-25T14:59:52.212Z level=INFO msg="Upload complete. (144ms)"5057worker2 # [ 78.186243] nix-grpc-daemon[784]: ts=2026-09-25T14:59:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=5125058lb # [ 78.208512] niks3-server[987]: 2026/09/25 14:59:51 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5059worker2 # [ 78.190536] nix-grpc-daemon[784]: ts=2026-09-25T14:59:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334125060worker2 # [ 78.197327] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5061worker2 # [ 78.199751] nix-daemon[871]: reaped child process 1169, status = succeeded5062worker2 # [ 78.205037] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5063worker2 # [ 79.652140] nix-daemon[871]: reaped child process 1178, status = succeeded5064worker2 # [ 79.736887] nix-daemon[871]: accepted connection from pid 1194, user nix-grpc-daemon (trusted)5065worker2 # [ 79.750390] nix-daemon[871]: reaped child process 1197, status = succeeded5066lb # [ 79.777935] niks3-server[987]: 2026/09/25 14:59:53 INFO Received push request method=POST path=/api/pushes5067worker2 # [ 79.761842] nix-grpc-daemon[885]: time=2026-09-25T14:59:53.790Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5068worker2 # [ 79.763324] nix-grpc-daemon[885]: time=2026-09-25T14:59:53.792Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5069lb # [ 79.797149] niks3-server[987]: 2026/09/25 14:59:53 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign5070lb # [ 79.800169] niks3-server[987]: 2026/09/25 14:59:53 INFO Signed narinfos id=22 count=15071worker2 # [ 79.780318] nix-grpc-daemon[885]: time=2026-09-25T14:59:53.808Z level=INFO msg="Uploading 1 narinfos"5072lb # [ 79.804061] niks3-server[987]: 2026/09/25 14:59:53 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5073lb # [ 79.806871] niks3-server[987]: 2026/09/25 14:59:53 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5074lb # [ 79.810870] niks3-server[987]: 2026/09/25 14:59:53 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5075lb # [ 79.812920] niks3-server[987]: 2026/09/25 14:59:53 INFO Received complete push request method=POST path=/api/pushes/22/complete5076worker2 # [ 79.797510] nix-grpc-daemon[885]: time=2026-09-25T14:59:53.826Z level=INFO msg="Upload complete. (183ms)"5077lb # [ 79.819263] niks3-server[987]: 2026/09/25 14:59:53 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5078worker2 # [ 79.800318] nix-grpc-daemon[784]: ts=2026-09-25T14:59:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15079worker2 # [ 79.809185] nix-daemon[871]: reaped child process 1176, status = succeeded5080worker1 # [ 79.772707] nix-grpc-daemon[510]: ts=2026-09-25T14:59:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:446605081worker1 # [ 79.779286] nix-grpc-daemon[510]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5082worker2 # [ 79.840320] nix-grpc-daemon[784]: ts=2026-09-25T14:59:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:334125083worker2 # [ 79.842592] nix-grpc-daemon[784]: ts=2026-09-25T14:59:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_out=1285084worker1 # [ 79.983335] nix-grpc-daemon[510]: ts=2026-09-25T14:59:54Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:446605085worker2 # [ 80.028164] nix-grpc-daemon[784]: ts=2026-09-25T14:59:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334125086worker2 # [ 80.036139] nix-grpc-daemon[885]: time=2026-09-25T14:59:54.064Z level=INFO msg="All 1 paths already cached"5087worker2 # [ 80.049462] nix-grpc-daemon[784]: ts=2026-09-25T14:59:54Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05088worker2 # [ 80.054270] nix-grpc-daemon[784]: ts=2026-09-25T14:59:54Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.4:50051 clients=15089worker2 # [ 80.060276] nix-grpc-daemon[784]: ts=2026-09-25T14:59:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334125090worker2 # [ 80.069621] nix-grpc-daemon[784]: ts=2026-09-25T14:59:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334125091worker2 # [ 80.079993] nix-grpc-daemon[885]: time=2026-09-25T14:59:54.108Z level=INFO msg="All 1 paths already cached"5092worker2 # [ 80.086967] nix-grpc-daemon[784]: ts=2026-09-25T14:59:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334125093worker2 # [ 80.095635] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5094worker2 # [ 80.197848] nix-daemon[871]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)5095worker2 # [ 80.213074] nix-daemon[871]: reaped child process 1206, status = succeeded5096lb # [ 80.239940] niks3-server[987]: 2026/09/25 14:59:53 INFO Received push request method=POST path=/api/pushes5097worker2 # [ 80.223757] nix-grpc-daemon[885]: time=2026-09-25T14:59:54.252Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5098worker2 # [ 80.225289] nix-grpc-daemon[885]: time=2026-09-25T14:59:54.253Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5099lb # [ 80.256107] niks3-server[987]: 2026/09/25 14:59:53 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign5100lb # [ 80.258430] niks3-server[987]: 2026/09/25 14:59:53 INFO Signed narinfos id=23 count=15101worker2 # [ 80.238115] nix-grpc-daemon[885]: time=2026-09-25T14:59:54.266Z level=INFO msg="Uploading 1 narinfos"5102lb # [ 80.260415] niks3-server[987]: 2026/09/25 14:59:53 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5103lb # [ 80.263619] niks3-server[987]: 2026/09/25 14:59:53 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5104lb # [ 80.268150] niks3-server[987]: 2026/09/25 14:59:53 INFO Received complete push request method=POST path=/api/pushes/23/complete5105worker2 # [ 80.252109] nix-grpc-daemon[885]: time=2026-09-25T14:59:54.280Z level=INFO msg="Upload complete. (144ms)"5106lb # [ 80.273911] niks3-server[987]: 2026/09/25 14:59:53 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5107worker2 # [ 80.254368] nix-grpc-daemon[784]: ts=2026-09-25T14:59:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=6245108worker2 # [ 80.259390] nix-grpc-daemon[784]: ts=2026-09-25T14:59:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334125109worker2 # [ 80.265518] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5110worker2 # [ 80.267864] nix-daemon[871]: reaped child process 1201, status = succeeded5111worker2 # [ 80.273289] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5112worker2 # [ 81.719576] nix-daemon[871]: reaped child process 1210, status = succeeded5113worker2 # [ 81.800547] nix-daemon[871]: accepted connection from pid 1226, user nix-grpc-daemon (trusted)5114worker2 # [ 81.815202] nix-daemon[871]: reaped child process 1229, status = succeeded5115lb # [ 81.844034] niks3-server[987]: 2026/09/25 14:59:55 INFO Received push request method=POST path=/api/pushes5116worker2 # [ 81.830024] nix-grpc-daemon[885]: time=2026-09-25T14:59:55.858Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5117worker2 # [ 81.831605] nix-grpc-daemon[885]: time=2026-09-25T14:59:55.860Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5118lb # [ 81.869958] niks3-server[987]: 2026/09/25 14:59:55 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5119lb # [ 81.873389] niks3-server[987]: 2026/09/25 14:59:55 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign5120lb # [ 81.875687] niks3-server[987]: 2026/09/25 14:59:55 INFO Signed narinfos id=24 count=15121worker2 # [ 81.856774] nix-grpc-daemon[885]: time=2026-09-25T14:59:55.885Z level=INFO msg="Uploading 1 narinfos"5122lb # [ 81.881095] niks3-server[987]: 2026/09/25 14:59:55 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5123lb # [ 81.883718] niks3-server[987]: 2026/09/25 14:59:55 INFO Received complete push request method=POST path=/api/pushes/24/complete5124lb # [ 81.887248] niks3-server[987]: 2026/09/25 14:59:55 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5125lb # [ 81.891135] niks3-server[987]: 2026/09/25 14:59:55 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5126worker2 # [ 81.873736] nix-grpc-daemon[885]: time=2026-09-25T14:59:55.902Z level=INFO msg="Upload complete. (191ms)"5127worker2 # [ 81.877235] nix-grpc-daemon[784]: ts=2026-09-25T14:59:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15128worker2 # [ 81.885305] nix-daemon[871]: reaped child process 1208, status = succeeded5129worker1 # [ 81.846714] nix-grpc-daemon[510]: ts=2026-09-25T14:59:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:446605130worker1 # [ 81.856348] nix-grpc-daemon[510]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5131worker1 # [ 81.878240] nix-grpc-daemon[510]: ts=2026-09-25T14:59:56Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:446605132worker1 # [ 81.883764] nix-grpc-daemon[510]: ts=2026-09-25T14:59:56Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=0 paths=1 nar_bytes_out=1765133worker1 # [ 82.050175] nix-grpc-daemon[510]: ts=2026-09-25T14:59:56Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:446605134worker1 # [ 82.056881] nix-grpc-daemon[510]: ts=2026-09-25T14:59:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446605135worker1 # [ 82.066289] nix-grpc-daemon[876]: time=2026-09-25T14:59:56.221Z level=INFO msg="All 2 paths already cached"5136worker2 # [ 82.117295] nix-grpc-daemon[784]: ts=2026-09-25T14:59:56Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05137worker2 # [ 82.123966] nix-grpc-daemon[784]: ts=2026-09-25T14:59:56Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.4:50051 clients=15138worker2 # [ 82.129231] nix-grpc-daemon[784]: ts=2026-09-25T14:59:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334125139worker2 # [ 82.138773] nix-grpc-daemon[784]: ts=2026-09-25T14:59:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334125140worker2 # [ 82.148951] nix-grpc-daemon[885]: time=2026-09-25T14:59:56.177Z level=INFO msg="All 2 paths already cached"5141worker2 # [ 82.156563] nix-grpc-daemon[784]: ts=2026-09-25T14:59:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334125142worker2 # [ 82.165263] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5143worker2 # [ 82.276346] nix-daemon[871]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5144worker2 # [ 82.293476] nix-daemon[871]: reaped child process 1238, status = succeeded5145lb # [ 82.319467] niks3-server[987]: 2026/09/25 14:59:55 INFO Received push request method=POST path=/api/pushes5146worker2 # [ 82.303484] nix-grpc-daemon[885]: time=2026-09-25T14:59:56.332Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5147worker2 # [ 82.305514] nix-grpc-daemon[885]: time=2026-09-25T14:59:56.333Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5148lb # [ 82.333867] niks3-server[987]: 2026/09/25 14:59:55 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign5149lb # [ 82.336295] niks3-server[987]: 2026/09/25 14:59:55 INFO Signed narinfos id=25 count=15150worker2 # [ 82.317408] nix-grpc-daemon[885]: time=2026-09-25T14:59:56.345Z level=INFO msg="Uploading 1 narinfos"5151lb # [ 82.339856] niks3-server[987]: 2026/09/25 14:59:55 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5152lb # [ 82.342527] niks3-server[987]: 2026/09/25 14:59:55 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5153lb # [ 82.346914] niks3-server[987]: 2026/09/25 14:59:55 INFO Received complete push request method=POST path=/api/pushes/25/complete5154worker2 # [ 82.331456] nix-grpc-daemon[885]: time=2026-09-25T14:59:56.360Z level=INFO msg="Upload complete. (152ms)"5155lb # [ 82.353686] niks3-server[987]: 2026/09/25 14:59:55 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5156worker2 # [ 82.333922] nix-grpc-daemon[784]: ts=2026-09-25T14:59:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=7525157worker2 # [ 82.338400] nix-grpc-daemon[784]: ts=2026-09-25T14:59:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334125158worker2 # [ 82.345192] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5159worker2 # [ 82.347581] nix-daemon[871]: reaped child process 1233, status = succeeded5160worker2 # [ 82.353167] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5161worker2 # [ 83.596786] nix-daemon[871]: reaped child process 1242, status = succeeded5162worker2 # [ 83.681127] nix-daemon[871]: accepted connection from pid 1258, user nix-grpc-daemon (trusted)5163worker2 # [ 83.695985] nix-daemon[871]: reaped child process 1261, status = succeeded5164lb # [ 83.723088] niks3-server[987]: 2026/09/25 14:59:57 INFO Received push request method=POST path=/api/pushes5165worker2 # [ 83.707044] nix-grpc-daemon[885]: time=2026-09-25T14:59:57.735Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5166worker2 # [ 83.709997] nix-grpc-daemon[885]: time=2026-09-25T14:59:57.737Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5167lb # [ 83.743349] niks3-server[987]: 2026/09/25 14:59:57 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign5168lb # [ 83.745484] niks3-server[987]: 2026/09/25 14:59:57 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5169lb # [ 83.747905] niks3-server[987]: 2026/09/25 14:59:57 INFO Signed narinfos id=26 count=15170worker2 # [ 83.728853] nix-grpc-daemon[885]: time=2026-09-25T14:59:57.757Z level=INFO msg="Uploading 1 narinfos"5171lb # [ 83.749297] niks3-server[987]: 2026/09/25 14:59:57 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5172lb # [ 83.751688] niks3-server[987]: 2026/09/25 14:59:57 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5173lb # [ 83.756053] niks3-server[987]: 2026/09/25 14:59:57 INFO Received complete push request method=POST path=/api/pushes/26/complete5174worker2 # [ 83.739856] nix-grpc-daemon[885]: time=2026-09-25T14:59:57.768Z level=INFO msg="Upload complete. (178ms)"5175lb # [ 83.761812] niks3-server[987]: 2026/09/25 14:59:57 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5176worker2 # [ 83.742648] nix-grpc-daemon[784]: ts=2026-09-25T14:59:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15177worker1 # [ 83.711880] nix-grpc-daemon[510]: ts=2026-09-25T14:59:57Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:446605178worker2 # [ 83.751054] nix-daemon[871]: reaped child process 1240, status = succeeded5179worker1 # [ 83.716660] nix-grpc-daemon[510]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5180worker2 # [ 83.771683] nix-grpc-daemon[784]: ts=2026-09-25T14:59:57Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:334125181worker2 # [ 83.773915] nix-grpc-daemon[784]: ts=2026-09-25T14:59:57Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_out=2325182client # [ 83.845722] nix-daemon[1108]: reaped child process 1110, status = succeeded5183client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.19 seconds)5184client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5185client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5186client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5187client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5188client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5189client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5190(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.25 seconds)5191subtest: an input only one worker has still reaches the builder5192client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5193client # this derivation will be built:5194client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5195client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5196client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.24 seconds)5197client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5198client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.10 seconds)5199worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5200worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.10 seconds)5201worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5202worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5203client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25204client # [ 84.473917] nix-daemon[1108]: accepted connection from pid 1189, user root (trusted)5205client # this derivation will be built:5206client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5207worker1 # [ 84.555407] nix-grpc-daemon[510]: ts=2026-09-25T14:59:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:446605208client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5209worker2 # [ 84.595935] nix-grpc-daemon[784]: ts=2026-09-25T14:59:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334125210client # copying 1 paths...5211client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5212worker2 # [ 84.606278] nix-grpc-daemon[784]: ts=2026-09-25T14:59:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334125213worker2 # [ 84.618225] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5214worker2 # [ 84.728298] nix-daemon[871]: accepted connection from pid 1268, user nix-grpc-daemon (trusted)5215worker2 # [ 84.743520] nix-daemon[871]: reaped child process 1271, status = succeeded5216lb # [ 84.770962] niks3-server[987]: 2026/09/25 14:59:58 INFO Received push request method=POST path=/api/pushes5217worker2 # [ 84.755661] nix-grpc-daemon[885]: time=2026-09-25T14:59:58.783Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5218worker2 # [ 84.757094] nix-grpc-daemon[885]: time=2026-09-25T14:59:58.783Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5219lb # [ 84.785057] niks3-server[987]: 2026/09/25 14:59:58 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign5220lb # [ 84.788184] niks3-server[987]: 2026/09/25 14:59:58 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5221lb # [ 84.790208] niks3-server[987]: 2026/09/25 14:59:58 INFO Signed narinfos id=27 count=15222worker2 # [ 84.770080] nix-grpc-daemon[885]: time=2026-09-25T14:59:58.798Z level=INFO msg="Uploading 1 narinfos"5223lb # [ 84.795142] niks3-server[987]: 2026/09/25 14:59:58 INFO Received complete push request method=POST path=/api/pushes/27/complete5224lb # [ 84.798753] niks3-server[987]: 2026/09/25 14:59:58 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5225worker2 # [ 84.781676] nix-grpc-daemon[885]: time=2026-09-25T14:59:58.810Z level=INFO msg="Upload complete. (152ms)"5226lb # [ 84.804217] niks3-server[987]: 2026/09/25 14:59:58 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5227worker2 # [ 84.784600] nix-grpc-daemon[784]: ts=2026-09-25T14:59:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=1205228worker2 # [ 84.791361] nix-daemon[871]: reaped child process 1266, status = succeeded5229worker2 # [ 84.796436] nix-grpc-daemon[784]: ts=2026-09-25T14:59:58Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05230worker2 # [ 84.798385] nix-grpc-daemon[784]: ts=2026-09-25T14:59:58Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15231worker1 # [ 84.761692] nix-grpc-daemon[510]: ts=2026-09-25T14:59:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446605232worker1 # [ 84.767837] nix-grpc-daemon[510]: ts=2026-09-25T14:59:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446605233client # copying 2 paths to 'lb:50051'...5234worker1 # [ 84.775832] nix-grpc-daemon[510]: ts=2026-09-25T14:59:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446605235client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5236client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5237worker1 # [ 84.791654] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)5238worker1 # [ 84.878067] nix-daemon[849]: accepted connection from pid 1082, user nix-grpc-daemon (trusted)5239worker1 # [ 84.892758] nix-daemon[849]: reaped child process 1085, status = succeeded5240lb # [ 84.954939] niks3-server[987]: 2026/09/25 14:59:58 INFO Received push request method=POST path=/api/pushes5241worker1 # [ 84.902055] nix-grpc-daemon[876]: time=2026-09-25T14:59:59.057Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5242worker1 # [ 84.906146] nix-grpc-daemon[876]: time=2026-09-25T14:59:59.059Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5243worker1 # [ 84.908117] nix-grpc-daemon[876]: time=2026-09-25T14:59:59.059Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5244lb # [ 84.978737] niks3-server[987]: 2026/09/25 14:59:58 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5245lb # [ 84.981931] niks3-server[987]: 2026/09/25 14:59:58 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign5246lb # [ 84.984774] niks3-server[987]: 2026/09/25 14:59:58 INFO Signed narinfos id=28 count=25247worker1 # [ 84.927069] nix-grpc-daemon[876]: time=2026-09-25T14:59:59.082Z level=INFO msg="Uploading 2 narinfos"5248lb # [ 84.985826] niks3-server[987]: 2026/09/25 14:59:58 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5249lb # [ 84.987990] niks3-server[987]: 2026/09/25 14:59:58 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5250lb # [ 84.990630] niks3-server[987]: 2026/09/25 14:59:58 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5251lb # [ 84.999041] niks3-server[987]: 2026/09/25 14:59:58 INFO Received complete push request method=POST path=/api/pushes/28/complete5252lb # [ 85.002748] niks3-server[987]: 2026/09/25 14:59:58 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5253worker1 # [ 84.948087] nix-grpc-daemon[876]: time=2026-09-25T14:59:59.103Z level=INFO msg="Upload complete. (149ms)"5254worker1 # [ 84.950368] nix-grpc-daemon[510]: ts=2026-09-25T14:59:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=0 paths=2 nar_bytes_in=9765255lb # [ 85.008805] niks3-server[987]: 2026/09/25 14:59:58 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5256worker1 # [ 84.953560] nix-grpc-daemon[510]: ts=2026-09-25T14:59:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446605257worker1 # [ 84.961829] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)5258worker1 # [ 84.963252] nix-daemon[849]: reaped child process 1080, status = succeeded5259worker1 # [ 84.968328] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)5260client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5261client # uses-input-w1onlya> node-a: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5262client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5263lb # [ 85.380284] envoy[906]: conn peer=[2001:db8:1::1]:51414 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1197 ms=46195264lb # [ 85.384061] envoy[906]: conn peer=[2001:db8:1::1]:46596 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3405 tx=1920 ms=19025265lb # [ 85.387661] envoy[906]: conn peer=[2001:db8:1::1]:46600 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=275266lb # [ 85.391169] envoy[906]: conn peer=[2001:db8:1::1]:46616 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3876 tx=2143 ms=19095267lb # [ 85.394909] envoy[906]: conn peer=[2001:db8:1::1]:46632 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=357 ms=155268lb # [ 85.398300] envoy[906]: conn peer=[2001:db8:1::1]:49930 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=411 tx=410 ms=125269lb # [ 85.401681] envoy[906]: conn peer=[2001:db8:1::1]:46634 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4322 tx=2368 ms=16955270worker1 # [ 85.607841] nix-daemon[849]: reaped child process 1089, status = succeeded5271worker1 # [ 85.690602] nix-daemon[849]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5272worker1 # [ 85.704655] nix-daemon[849]: reaped child process 1108, status = succeeded5273lb # [ 85.769303] niks3-server[987]: 2026/09/25 14:59:59 INFO Received push request method=POST path=/api/pushes5274worker1 # [ 85.721112] nix-grpc-daemon[876]: time=2026-09-25T14:59:59.876Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5275lb # [ 85.802416] niks3-server[987]: 2026/09/25 14:59:59 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5276lb # [ 85.807438] niks3-server[987]: 2026/09/25 14:59:59 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign5277lb # [ 85.812425] niks3-server[987]: 2026/09/25 14:59:59 INFO Signed narinfos id=29 count=15278worker1 # [ 85.757776] nix-grpc-daemon[876]: time=2026-09-25T14:59:59.913Z level=INFO msg="Uploading 1 narinfos"5279lb # [ 85.830154] niks3-server[987]: 2026/09/25 14:59:59 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5280lb # [ 85.836120] niks3-server[987]: 2026/09/25 14:59:59 INFO Received complete push request method=POST path=/api/pushes/29/complete5281lb # [ 85.843867] niks3-server[987]: 2026/09/25 14:59:59 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5282worker1 # [ 85.789424] nix-grpc-daemon[876]: time=2026-09-25T14:59:59.945Z level=INFO msg="Upload complete. (214ms)"5283client # copying 1 paths...5284worker1 # [ 85.795784] nix-grpc-daemon[510]: ts=2026-09-25T14:59:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=0 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15285worker1 # [ 85.803964] nix-grpc-daemon[510]: ts=2026-09-25T14:59:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:446605286client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5287worker1 # [ 85.816378] nix-daemon[849]: reaped child process 1087, status = succeeded5288worker2 # [ 85.861872] nix-grpc-daemon[784]: ts=2026-09-25T14:59:59Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:334125289worker2 # [ 85.873326] nix-grpc-daemon[784]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5290worker2 # [ 85.887449] nix-grpc-daemon[784]: ts=2026-09-25T14:59:59Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_out=1205291client # [ 85.958344] nix-daemon[1108]: reaped child process 1192, status = succeeded5292client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 1.58 seconds)5293client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25294client # [ 86.059861] nix-daemon[1108]: accepted connection from pid 1211, user root (trusted)5295client # this derivation will be built:5296client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5297worker2 # [ 86.179304] nix-grpc-daemon[784]: ts=2026-09-25T15:00:00Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:334125298client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5299worker1 # [ 86.149709] nix-grpc-daemon[510]: ts=2026-09-25T15:00:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446605300worker1 # [ 86.159830] nix-grpc-daemon[876]: time=2026-09-25T15:00:00.314Z level=INFO msg="All 1 paths already cached"5301worker2 # [ 86.211828] nix-grpc-daemon[784]: ts=2026-09-25T15:00:00Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05302worker2 # [ 86.218573] nix-grpc-daemon[784]: ts=2026-09-25T15:00:00Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15303worker2 # [ 86.223555] nix-grpc-daemon[784]: ts=2026-09-25T15:00:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334125304worker2 # [ 86.234600] nix-grpc-daemon[784]: ts=2026-09-25T15:00:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334125305client # copying 1 paths to 'lb:50051'...5306client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5307worker2 # [ 86.253425] nix-grpc-daemon[784]: ts=2026-09-25T15:00:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334125308worker2 # [ 86.265454] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5309worker2 # [ 86.281116] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 1/5)5310worker2 # [ 86.346962] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 2/5)5311worker2 # [ 86.357472] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 3/5)5312worker2 # [ 86.397700] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 597 ms (attempt 4/5)5313worker2 # [ 86.998926] nix-grpc-daemon[784]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5314worker2 # [ 87.002530] nix-grpc-daemon[784]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5315worker2 # [ 87.007869] nix-grpc-daemon[784]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5316worker2 # [ 87.130462] nix-daemon[871]: accepted connection from pid 1285, user nix-grpc-daemon (trusted)5317worker2 # [ 87.148626] nix-daemon[871]: reaped child process 1288, status = succeeded5318lb # [ 87.177516] niks3-server[987]: 2026/09/25 15:00:00 INFO Received push request method=POST path=/api/pushes5319worker2 # [ 87.166223] nix-grpc-daemon[885]: time=2026-09-25T15:00:01.194Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5320worker2 # [ 87.167741] nix-grpc-daemon[885]: time=2026-09-25T15:00:01.196Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5321lb # [ 87.212450] niks3-server[987]: 2026/09/25 15:00:00 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign5322lb # [ 87.217359] niks3-server[987]: 2026/09/25 15:00:00 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5323worker2 # [ 87.203567] nix-grpc-daemon[885]: time=2026-09-25T15:00:01.232Z level=INFO msg="Uploading 1 narinfos"5324lb # [ 87.223554] niks3-server[987]: 2026/09/25 15:00:00 INFO Signed narinfos id=30 count=15325lb # [ 87.231485] niks3-server[987]: 2026/09/25 15:00:00 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5326lb # [ 87.240073] niks3-server[987]: 2026/09/25 15:00:00 INFO Received complete push request method=POST path=/api/pushes/30/complete5327worker2 # [ 87.225135] nix-grpc-daemon[885]: time=2026-09-25T15:00:01.253Z level=INFO msg="Upload complete. (197ms)"5328lb # [ 87.246550] niks3-server[987]: 2026/09/25 15:00:00 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5329worker2 # [ 87.229460] nix-grpc-daemon[784]: ts=2026-09-25T15:00:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=5605330worker2 # [ 87.234569] nix-grpc-daemon[784]: ts=2026-09-25T15:00:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334125331worker2 # [ 87.244271] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5332worker2 # [ 87.253433] nix-daemon[871]: reaped child process 1278, status = succeeded5333worker2 # [ 87.256886] nix-daemon[871]: accepted connection from pid 784, user nix-grpc-daemon (trusted)5334client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5335client # uses-input-w1onlyb> worker2: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5336client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5337worker2 # [ 88.093893] nix-daemon[871]: reaped child process 1292, status = succeeded5338worker2 # [ 88.182099] nix-daemon[871]: accepted connection from pid 1308, user nix-grpc-daemon (trusted)5339worker2 # [ 88.196406] nix-daemon[871]: reaped child process 1311, status = succeeded5340lb # [ 88.223436] niks3-server[987]: 2026/09/25 15:00:01 INFO Received push request method=POST path=/api/pushes5341worker2 # [ 88.210585] nix-grpc-daemon[885]: time=2026-09-25T15:00:02.239Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5342lb # [ 88.251154] niks3-server[987]: 2026/09/25 15:00:01 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5343lb # [ 88.260235] niks3-server[987]: 2026/09/25 15:00:01 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign5344worker2 # [ 88.244244] nix-grpc-daemon[885]: time=2026-09-25T15:00:02.272Z level=INFO msg="Uploading 1 narinfos"5345lb # [ 88.264748] niks3-server[987]: 2026/09/25 15:00:01 INFO Signed narinfos id=31 count=15346lb # [ 88.275703] niks3-server[987]: 2026/09/25 15:00:01 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5347lb # [ 88.283339] niks3-server[987]: 2026/09/25 15:00:01 INFO Received complete push request method=POST path=/api/pushes/31/complete5348lb # [ 88.291474] niks3-server[987]: 2026/09/25 15:00:01 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5349worker2 # [ 88.275778] nix-grpc-daemon[885]: time=2026-09-25T15:00:02.304Z level=INFO msg="Upload complete. (209ms)"5350worker2 # [ 88.281140] nix-grpc-daemon[784]: ts=2026-09-25T15:00:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15351client # copying 1 paths...5352worker1 # [ 88.253626] nix-grpc-daemon[510]: ts=2026-09-25T15:00:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:446605353worker2 # [ 88.295289] nix-daemon[871]: reaped child process 1290, status = succeeded5354worker1 # [ 88.265099] nix-grpc-daemon[510]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5355client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5356worker1 # [ 88.295551] nix-grpc-daemon[510]: ts=2026-09-25T15:00:02Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:446605357worker1 # [ 88.301524] nix-grpc-daemon[510]: ts=2026-09-25T15:00:02Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=0 paths=1 nar_bytes_out=1205358client # [ 88.407958] nix-daemon[1108]: reaped child process 1214, status = succeeded5359client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.45 seconds)5360client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25361client # [ 88.509098] nix-daemon[1108]: accepted connection from pid 1233, user root (trusted)5362client # this derivation will be built:5363client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5364worker1 # [ 88.591959] nix-grpc-daemon[510]: ts=2026-09-25T15:00:02Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:446605365client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5366worker1 # [ 88.598250] nix-grpc-daemon[510]: ts=2026-09-25T15:00:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446605367worker1 # [ 88.604204] nix-grpc-daemon[876]: time=2026-09-25T15:00:02.758Z level=INFO msg="All 1 paths already cached"5368worker2 # [ 88.651189] nix-grpc-daemon[784]: ts=2026-09-25T15:00:02Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05369worker2 # [ 88.656130] nix-grpc-daemon[784]: ts=2026-09-25T15:00:02Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15370worker1 # [ 88.621243] nix-grpc-daemon[510]: ts=2026-09-25T15:00:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446605371worker1 # [ 88.633276] nix-grpc-daemon[510]: ts=2026-09-25T15:00:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446605372worker1 # [ 88.646197] nix-grpc-daemon[876]: time=2026-09-25T15:00:02.801Z level=INFO msg="All 1 paths already cached"5373client # copying 1 paths to 'lb:50051'...5374client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5375worker1 # [ 88.653226] nix-grpc-daemon[510]: ts=2026-09-25T15:00:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446605376worker1 # [ 88.665229] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)5377worker1 # [ 88.765237] nix-daemon[849]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)5378worker1 # [ 88.779307] nix-daemon[849]: reaped child process 1116, status = succeeded5379lb # [ 88.842614] niks3-server[987]: 2026/09/25 15:00:02 INFO Received push request method=POST path=/api/pushes5380worker1 # [ 88.792718] nix-grpc-daemon[876]: time=2026-09-25T15:00:02.948Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5381worker1 # [ 88.794141] nix-grpc-daemon[876]: time=2026-09-25T15:00:02.948Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5382lb # [ 88.872418] niks3-server[987]: 2026/09/25 15:00:02 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign5383lb # [ 88.877782] niks3-server[987]: 2026/09/25 15:00:02 INFO Signed narinfos id=32 count=15384worker1 # [ 88.823643] nix-grpc-daemon[876]: time=2026-09-25T15:00:02.979Z level=INFO msg="Uploading 1 narinfos"5385lb # [ 88.883277] niks3-server[987]: 2026/09/25 15:00:02 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5386lb # [ 88.895649] niks3-server[987]: 2026/09/25 15:00:02 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5387lb # [ 88.906123] niks3-server[987]: 2026/09/25 15:00:02 INFO Received complete push request method=POST path=/api/pushes/32/complete5388lb # [ 88.913547] niks3-server[987]: 2026/09/25 15:00:02 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5389worker1 # [ 88.855678] nix-grpc-daemon[876]: time=2026-09-25T15:00:03.011Z level=INFO msg="Upload complete. (178ms)"5390worker1 # [ 88.860223] nix-grpc-daemon[510]: ts=2026-09-25T15:00:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=0 paths=1 nar_bytes_in=5605391worker1 # [ 88.865208] nix-grpc-daemon[510]: ts=2026-09-25T15:00:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446605392worker1 # [ 88.876132] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)5393worker1 # [ 88.880881] nix-daemon[849]: reaped child process 1111, status = succeeded5394worker1 # [ 88.885835] nix-daemon[849]: accepted connection from pid 510, user nix-grpc-daemon (trusted)5395client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5396client # uses-input-w1onlyc> node-a: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5397client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5398worker1 # [ 89.902170] nix-daemon[849]: reaped child process 1120, status = succeeded5399worker1 # [ 89.975971] nix-daemon[849]: accepted connection from pid 1136, user nix-grpc-daemon (trusted)5400worker1 # [ 89.989183] nix-daemon[849]: reaped child process 1139, status = succeeded5401lb # [ 90.053892] niks3-server[987]: 2026/09/25 15:00:03 INFO Received push request method=POST path=/api/pushes5402worker1 # [ 90.004624] nix-grpc-daemon[876]: time=2026-09-25T15:00:04.160Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5403lb # [ 90.085626] niks3-server[987]: 2026/09/25 15:00:03 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5404lb # [ 90.092672] niks3-server[987]: 2026/09/25 15:00:03 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign5405lb # [ 90.100517] niks3-server[987]: 2026/09/25 15:00:03 INFO Signed narinfos id=33 count=15406worker1 # [ 90.046122] nix-grpc-daemon[876]: time=2026-09-25T15:00:04.201Z level=INFO msg="Uploading 1 narinfos"5407lb # [ 90.104600] niks3-server[987]: 2026/09/25 15:00:03 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5408lb # [ 90.120478] niks3-server[987]: 2026/09/25 15:00:03 INFO Received complete push request method=POST path=/api/pushes/33/complete5409lb # [ 90.129803] niks3-server[987]: 2026/09/25 15:00:03 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5410worker1 # [ 90.076351] nix-grpc-daemon[876]: time=2026-09-25T15:00:04.232Z level=INFO msg="Upload complete. (183ms)"5411client # copying 1 paths...5412worker1 # [ 90.083293] nix-grpc-daemon[510]: ts=2026-09-25T15:00:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44660 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15413worker2 # [ 90.127504] nix-grpc-daemon[784]: ts=2026-09-25T15:00:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:334125414worker1 # [ 90.097766] nix-daemon[849]: reaped child process 1118, status = succeeded5415worker2 # [ 90.136699] nix-grpc-daemon[784]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5416client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5417worker2 # [ 90.164481] nix-grpc-daemon[784]: ts=2026-09-25T15:00:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:334125418worker2 # [ 90.169769] nix-grpc-daemon[784]: ts=2026-09-25T15:00:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_out=1205419client # [ 90.237566] nix-daemon[1108]: reaped child process 1236, status = succeeded5420client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.83 seconds)5421(finished: subtest: an input only one worker has still reaches the builder, in 6.33 seconds)5422subtest: upload whose reference the worker lacks is completed from the cache5423client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5424client # this derivation will be built:5425client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5426client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5427client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.25 seconds)5428client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5429client # this derivation will be built:5430client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5431client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5432client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.24 seconds)5433client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5434client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds)5435worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5436worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.12 seconds)5437worker2 # [ 91.000924] systemd[1]: Stopping Nix worker-protocol over gRPC...5438worker2 # [ 91.788841] nix-grpc-daemon[784]: ts=2026-09-25T15:00:05Z level=info event=signal_exit5439lb # [ 92.017787] niks3-server[987]: 2026/09/25 15:00:05 INFO lead: released remote=192.168.1.4:396165440worker2 # [ 92.999397] nix-grpc-daemon[784]: ts=2026-09-25T15:00:07Z level=info event=scheduler_restarting5441lb # [ 93.018389] envoy[906]: [2026-09-25 15:00:06.324][906][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5442worker2 # [ 93.005490] nix-grpc-daemon[784]: ts=2026-09-25T15:00:07Z level=info event=scheduler_restarting addr=lb:500515443worker1 # [ 92.969398] nix-grpc-daemon[510]: ts=2026-09-25T15:00:07Z level=info event=scheduler_restarting addr=lb:500515444worker2 # [ 93.012359] nix-grpc-daemon[784]: ts=2026-09-25T15:00:07Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:501205445worker2 # [ 93.016314] nix-grpc-daemon[784]: ts=2026-09-25T15:00:07Z level=info event=worker_gone addr=192.168.1.3:500515446worker2 # [ 93.019775] nix-grpc-daemon[784]: ts=2026-09-25T15:00:07Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:501205447worker2 # [ 93.023831] nix-grpc-daemon[784]: ts=2026-09-25T15:00:07Z level=info event=worker_gone addr=192.168.1.4:500515448worker2 # [ 93.047925] nix-daemon[871]: reaped child process 873, status = succeeded5449worker1 # [ 93.169563] nix-grpc-daemon[510]: ts=2026-09-25T15:00:07Z level=info event=scheduler_connected addr=lb:500515450worker1 # [ 93.173926] nix-grpc-daemon[510]: ts=2026-09-25T15:00:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5451worker2 # [ 93.223274] nix-grpc-daemon[784]: ts=2026-09-25T15:00:07Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5452worker1 # [ 93.374895] nix-grpc-daemon[510]: ts=2026-09-25T15:00:07Z level=info event=scheduler_connected addr=lb:500515453worker1 # [ 93.379425] nix-grpc-daemon[510]: ts=2026-09-25T15:00:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5454worker1 # [ 93.579916] nix-grpc-daemon[510]: ts=2026-09-25T15:00:07Z level=info event=scheduler_connected addr=lb:500515455worker1 # [ 93.584312] nix-grpc-daemon[510]: ts=2026-09-25T15:00:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5456worker1 # [ 93.785395] nix-grpc-daemon[510]: ts=2026-09-25T15:00:07Z level=info event=scheduler_connected addr=lb:500515457worker1 # [ 93.789453] nix-grpc-daemon[510]: ts=2026-09-25T15:00:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5458worker2 # [ 93.861996] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5459worker2 # [ 93.865581] systemd[1]: Stopped Nix worker-protocol over gRPC.5460worker2 # [ 93.870069] systemd[1]: nix-grpc-daemon.service: Consumed 2.680s CPU time over 1min 18.246s wall clock time, 69.8M memory peak, 389.7K incoming IP traffic, 319.9K outgoing IP traffic.5461lb: waiting until exactly ['worker1'] are healthy members of x86_64-linux5462lb: must succeed: curl -sf localhost:9901/clusters5463lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5464lb: (finished: waiting until exactly ['worker1'] are healthy members of x86_64-linux, in 0.05 seconds)5465client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25466worker1 # [ 93.991280] nix-grpc-daemon[510]: ts=2026-09-25T15:00:08Z level=info event=scheduler_connected addr=lb:500515467worker1 # [ 93.994810] nix-grpc-daemon[510]: ts=2026-09-25T15:00:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5468worker1 # [ 94.033975] nix-grpc-daemon[510]: ts=2026-09-25T15:00:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:446605469worker1 # [ 94.040991] nix-grpc-daemon[510]: ts=2026-09-25T15:00:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:446605470client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5471client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds)5472worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5473worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5474worker2 # [ 94.165508] systemd[1]: Starting scheduler-token.service...5475worker2 # [ 94.203952] systemd[1]: scheduler-token.service: Deactivated successfully.5476worker2 # [ 94.205543] systemd[1]: Finished scheduler-token.service.5477worker2 # [ 94.207254] systemd[1]: scheduler-token.service: Consumed 18ms CPU time over 40ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5478worker2 # [ 94.211498] systemd[1]: Starting Nix worker-protocol over gRPC...5479worker1 # [ 94.195814] nix-grpc-daemon[510]: ts=2026-09-25T15:00:08Z level=info event=scheduler_connected addr=lb:500515480worker1 # [ 94.199639] nix-grpc-daemon[510]: ts=2026-09-25T15:00:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5481worker2 # [ 94.297424] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:08Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5482worker2 # [ 94.299656] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:08Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5483worker2 # [ 94.303415] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:08Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15484worker2 # [ 94.308058] nix-grpc-daemon[1338]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5485worker2 # [ 94.309392] nix-grpc-daemon[1338]: E0000 00:00:1790348408.335882 1338 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5486worker2 # [ 94.312720] nix-grpc-daemon[1338]: E0000 00:00:1790348408.341330 1338 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5487worker2 # [ 94.318364] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:08Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5488worker2 # [ 94.322141] systemd[1]: Started Nix worker-protocol over gRPC.5489lb # [ 94.346090] niks3-server[987]: 2026/09/25 15:00:07 INFO lead: acquired remote=192.168.1.4:397065490worker2 # [ 94.328077] nix-grpc-daemon[1338]: E0000 00:00:1790348408.354151 1380 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5491worker2 # [ 94.331076] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:08Z level=info event=scheduler_take_over5492worker2 # [ 94.340737] nix-grpc-daemon[1338]: E0000 00:00:1790348408.369431 1380 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5493worker2 # [ 94.344320] nix-grpc-daemon[1338]: E0000 00:00:1790348408.371728 1380 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5494worker2 # [ 94.355255] nix-grpc-daemon[1338]: E0000 00:00:1790348408.383955 1380 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5495worker2 # [ 94.364705] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:08Z level=info event=scheduler_connected addr=lb:500515496worker2 # [ 94.366316] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5497worker1 # [ 94.344705] systemd[1]: Stopping Nix worker-protocol over gRPC...5498worker1 # [ 94.400626] nix-grpc-daemon[510]: ts=2026-09-25T15:00:08Z level=info event=scheduler_connected addr=lb:500515499worker1 # [ 94.404606] nix-grpc-daemon[510]: ts=2026-09-25T15:00:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5500worker2 # [ 94.567932] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:08Z level=info event=scheduler_connected addr=lb:500515501worker2 # [ 94.581142] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:08Z level=info event=oidc_jwks provider=mock keys=15502worker2 # [ 94.585184] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:08Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:521665503worker2 # [ 94.589508] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:08Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05504worker1 # [ 94.609431] nix-grpc-daemon[510]: ts=2026-09-25T15:00:08Z level=info event=scheduler_connected addr=lb:500515505worker2 # [ 94.648502] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:08Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:521665506worker2 # [ 94.652660] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:08Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05507worker1 # [ 94.894302] nix-grpc-daemon[510]: ts=2026-09-25T15:00:09Z level=info event=signal_exit5508worker2 # [ 95.232532] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:09Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:521765509lb # [ 95.381621] envoy[906]: conn peer=[2001:db8:1::1]:49946 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=329 ms=375510lb # [ 95.385326] envoy[906]: conn peer=[2001:db8:1::1]:49944 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=2012 ms=13045511lb # [ 95.388855] envoy[906]: conn peer=[2001:db8:1::1]:49962 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3740 tx=2124 ms=21715512lb # [ 95.392344] envoy[906]: conn peer=[2001:db8:1::1]:49964 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=195513lb # [ 95.395901] envoy[906]: conn peer=[2001:db8:1::1]:49968 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3708 tx=2063 ms=15525514lb # [ 95.399506] envoy[906]: conn peer=[2001:db8:1::1]:49982 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=165515lb # [ 95.402911] envoy[906]: conn peer=192.168.1.4:51298 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3181 tx=1187 ms=622155516lb # [ 95.406456] envoy[906]: conn peer=[2001:db8:1::1]:60352 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=205517worker1 # [ 95.967649] nix-grpc-daemon[510]: ts=2026-09-25T15:00:10Z level=info event=scheduler_restarting5518worker2 # [ 96.008088] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:10Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:521665519worker2 # [ 96.015752] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:10Z level=info event=worker_gone addr=192.168.1.3:500515520worker1 # [ 95.986360] nix-grpc-daemon[510]: ts=2026-09-25T15:00:10Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5521worker1 # [ 96.004654] nix-daemon[849]: reaped child process 851, status = succeeded5522worker1 # [ 97.776671] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5523worker1 # [ 97.779684] systemd[1]: Stopped Nix worker-protocol over gRPC.5524worker1 # [ 97.784278] systemd[1]: nix-grpc-daemon.service: Consumed 1.913s CPU time over 1min 29.196s wall clock time, 61M memory peak, 153.2K incoming IP traffic, 193.4K outgoing IP traffic.5525lb: waiting until exactly ['worker2'] are healthy members of x86_64-linux5526lb: must succeed: curl -sf localhost:9901/clusters5527lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5528lb: (finished: waiting until exactly ['worker2'] are healthy members of x86_64-linux, in 0.02 seconds)5529client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25530worker2 # [ 98.004896] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521805531worker2 # [ 98.017489] nix-daemon[871]: accepted connection from pid 1338, user nix-grpc-daemon (trusted)5532worker2 # [ 98.034639] nix-grpc-daemon[1338]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 1/5)5533worker2 # [ 98.058435] nix-grpc-daemon[1338]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 2/5)5534worker2 # [ 98.124958] nix-grpc-daemon[1338]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 356 ms (attempt 3/5)5535worker2 # [ 98.483244] nix-grpc-daemon[1338]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 223 ms (attempt 4/5)5536worker2 # [ 98.709286] nix-grpc-daemon[1338]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5537worker2 # [ 98.712323] nix-grpc-daemon[1338]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5538client # copying 2 paths...5539client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5540worker2 # [ 98.731133] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:521805541client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5542worker2 # [ 98.744312] nix-daemon[871]: accepted connection from pid 1338, user nix-grpc-daemon (trusted)5543worker2 # [ 98.765710] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:12Z level=info event=niks3_push_started pid=13985544worker2 # [ 98.878337] nix-grpc-daemon[1398]: time=2026-09-25T15:00:12.906Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5545worker2 # [ 98.974101] nix-daemon[871]: accepted connection from pid 1411, user nix-grpc-daemon (trusted)5546worker2 # [ 98.989391] nix-daemon[871]: reaped child process 1414, status = succeeded5547lb # [ 99.017092] niks3-server[987]: 2026/09/25 15:00:12 INFO Received push request method=POST path=/api/pushes5548worker2 # [ 99.002349] nix-grpc-daemon[1398]: time=2026-09-25T15:00:13.030Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5549worker2 # [ 99.003911] nix-grpc-daemon[1398]: time=2026-09-25T15:00:13.032Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5550worker2 # [ 99.006132] nix-grpc-daemon[1398]: time=2026-09-25T15:00:13.034Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5551lb # [ 99.050304] niks3-server[987]: 2026/09/25 15:00:12 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign5552lb # [ 99.055890] niks3-server[987]: 2026/09/25 15:00:12 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5553lb # [ 99.059266] niks3-server[987]: 2026/09/25 15:00:12 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5554lb # [ 99.062913] niks3-server[987]: 2026/09/25 15:00:12 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5555lb # [ 99.065289] niks3-server[987]: 2026/09/25 15:00:12 INFO Signed narinfos id=34 count=25556worker2 # [ 99.045319] nix-grpc-daemon[1398]: time=2026-09-25T15:00:13.073Z level=INFO msg="Uploading 2 narinfos"5557lb # [ 99.067051] niks3-server[987]: 2026/09/25 15:00:12 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5558lb # [ 99.077412] niks3-server[987]: 2026/09/25 15:00:12 INFO Received complete push request method=POST path=/api/pushes/34/complete5559worker2 # [ 99.065687] nix-grpc-daemon[1398]: time=2026-09-25T15:00:13.094Z level=INFO msg="Upload complete. (183ms)"5560lb # [ 99.086676] niks3-server[987]: 2026/09/25 15:00:12 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5561worker2 # [ 99.068737] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52180 duration_s=0 paths=2 nar_bytes_in=3125562lb # [ 99.091208] niks3-server[987]: 2026/09/25 15:00:12 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5563worker2 # [ 99.076635] nix-daemon[871]: reaped child process 1396, status = succeeded5564client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.24 seconds)5565worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5566worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.01 seconds)5567worker1 # [ 99.118801] systemd[1]: Starting Nix worker-protocol over gRPC...5568worker1 # [ 99.207802] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:13Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5569worker1 # [ 99.212241] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:13Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5570worker1 # [ 99.216203] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:13Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15571worker1 # [ 99.218159] nix-grpc-daemon[1162]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5572worker1 # [ 99.219461] nix-grpc-daemon[1162]: E0000 00:00:1790348413.370456 1162 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5573worker1 # [ 99.222224] nix-grpc-daemon[1162]: E0000 00:00:1790348413.377890 1162 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5574worker1 # [ 99.229904] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:13Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5575worker1 # [ 99.234120] systemd[1]: Started Nix worker-protocol over gRPC.5576worker1 # [ 99.234917] nix-grpc-daemon[1162]: E0000 00:00:1790348413.388459 1204 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5577worker1 # [ 99.248069] nix-grpc-daemon[1162]: E0000 00:00:1790348413.403325 1204 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5578worker1 # [ 99.252123] nix-grpc-daemon[1162]: E0000 00:00:1790348413.403465 1204 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5579lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux5580lb: must succeed: curl -sf localhost:9901/clusters5581worker1 # [ 99.262181] nix-grpc-daemon[1162]: E0000 00:00:1790348413.417926 1204 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5582worker1 # [ 99.269841] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:13Z level=info event=scheduler_connected addr=lb:500515583worker2 # [ 99.308797] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:13Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:521665584lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5585worker2 # [ 99.313664] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:13Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05586lb: must succeed: curl -sf localhost:9901/clusters5587lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5588lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 1.07 seconds)5589(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.12 seconds)5590subtest: a niks3 restart does not cost the leader its role5591lb: must succeed: curl -sf localhost:9901/clusters5592lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5593lb: must succeed: curl -sf localhost:9901/clusters5594lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5595lb # [ 100.455499] niks3-server[987]: 2026/09/25 15:00:13 INFO Shutdown signal received, draining in-flight requests timeout=10s5596lb # [ 100.458328] systemd[1]: Stopping niks3 server...5597worker1 # [ 100.401304] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:14Z level=info event=scheduler_lock_lost status=2005598lb # [ 100.462102] niks3-server[987]: 2026/09/25 15:00:13 INFO lead: released remote=192.168.1.4:397065599worker2 # [ 100.442289] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:14Z level=info event=scheduler_lock_lost status=2005600lb # [ 100.474620] systemd[1]: niks3.service: Deactivated successfully.5601lb # [ 100.477054] systemd[1]: Stopped niks3 server.5602lb # [ 100.477862] systemd[1]: niks3.service: Consumed 651ms CPU time over 1min 19.683s wall clock time, 28.6M memory peak, 97.2K incoming IP traffic, 139.2K outgoing IP traffic.5603lb # [ 100.484366] systemd[1]: Starting niks3 server...5604lb # [ 100.532474] niks3-server[1148]: 2026/09/25 15:00:13 goose: no migrations to run. current version: 202609231200005605lb # [ 100.538313] niks3-server[1148]: 2026/09/25 15:00:13 OK 1_commit_pending_closure.sql (4.19ms)5606lb # [ 100.541768] niks3-server[1148]: 2026/09/25 15:00:13 OK 2_object_stats_trigger.sql (3.52ms)5607lb # [ 100.545267] niks3-server[1148]: 2026/09/25 15:00:13 OK 3_commit_push.sql (1.93ms)5608lb # [ 100.546270] niks3-server[1148]: 2026/09/25 15:00:13 goose: up to current file version: 35609lb # [ 100.547572] niks3-server[1148]: 2026/09/25 15:00:13 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5610lb # [ 100.558759] niks3-server[1148]: 2026/09/25 15:00:13 INFO Read proxy enabled — serving cache objects from S35611lb # [ 100.560208] niks3-server[1148]: 2026/09/25 15:00:13 INFO Using socket-activated listener address=0.0.0.0:57515612lb # [ 100.562212] niks3-server[1148]: 2026/09/25 15:00:13 INFO systemd watchdog enabled interval=15s5613lb # [ 100.563576] systemd[1]: Started niks3 server.5614lb # [ 100.564938] niks3-server[1148]: 2026/09/25 15:00:13 INFO Starting HTTP server address=0.0.0.0:57515615lb: waiting for unit niks3.service5616lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5617lb: must succeed: curl -sf localhost:9901/clusters5618lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5619worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5620worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5621lb: must succeed: curl -sf localhost:9901/clusters5622lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5623lb: must succeed: curl -sf localhost:9901/clusters5624lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5625worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5626worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.04 seconds)5627client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25628worker1 # [ 100.838058] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:446245629worker1 # [ 100.849626] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5630client # don't know how to build these paths:5631client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5632worker1 # [ 100.863845] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446245633worker1 # [ 100.874126] nix-grpc-daemon[1162]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 1/5)5634worker1 # [ 100.935361] nix-grpc-daemon[1162]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 114 ms (attempt 2/5)5635worker1 # [ 101.051459] nix-grpc-daemon[1162]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 3/5)5636worker1 # [ 101.074995] nix-grpc-daemon[1162]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 169 ms (attempt 4/5)5637worker1 # [ 101.246754] nix-grpc-daemon[1162]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5638worker1 # [ 101.250385] nix-grpc-daemon[1162]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5639worker2 # [ 101.318275] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:15Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05640worker2 # [ 101.322587] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:15Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15641worker2 # [ 101.331463] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521805642worker2 # [ 101.345752] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521805643client # copying 1 paths to 'lb:50051'...5644client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5645worker2 # [ 101.358445] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:521805646worker2 # [ 101.369338] nix-daemon[871]: accepted connection from pid 1338, user nix-grpc-daemon (trusted)5647lb # [ 101.465193] niks3-server[1148]: 2026/09/25 15:00:14 INFO lead: acquired remote=192.168.1.4:398145648worker2 # [ 101.482511] nix-daemon[871]: accepted connection from pid 1431, user nix-grpc-daemon (trusted)5649worker2 # [ 101.498320] nix-daemon[871]: reaped child process 1436, status = succeeded5650lb # [ 101.526419] niks3-server[1148]: 2026/09/25 15:00:14 INFO Received push request method=POST path=/api/pushes5651worker2 # [ 101.527456] nix-grpc-daemon[1398]: time=2026-09-25T15:00:15.555Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5652worker2 # [ 101.529755] nix-grpc-daemon[1398]: time=2026-09-25T15:00:15.557Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5653lb # [ 101.577500] niks3-server[1148]: 2026/09/25 15:00:14 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign5654lb # [ 101.586126] niks3-server[1148]: 2026/09/25 15:00:14 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5655lb # [ 101.601757] niks3-server[1148]: 2026/09/25 15:00:14 INFO Signed narinfos id=35 count=15656worker2 # [ 101.584221] nix-grpc-daemon[1398]: time=2026-09-25T15:00:15.611Z level=INFO msg="Uploading 1 narinfos"5657lb # [ 101.607598] niks3-server[1148]: 2026/09/25 15:00:14 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5658lb # [ 101.611699] niks3-server[1148]: 2026/09/25 15:00:14 INFO Received complete push request method=POST path=/api/pushes/35/complete5659lb # [ 101.620312] niks3-server[1148]: 2026/09/25 15:00:14 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5660worker2 # [ 101.604316] nix-grpc-daemon[1398]: time=2026-09-25T15:00:15.632Z level=INFO msg="Upload complete. (222ms)"5661worker2 # [ 101.610388] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52180 duration_s=0 paths=1 nar_bytes_in=5365662worker2 # [ 101.616373] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521805663worker2 # [ 101.627255] nix-daemon[871]: accepted connection from pid 1338, user nix-grpc-daemon (trusted)5664worker2 # [ 101.639273] nix-daemon[871]: reaped child process 1429, status = succeeded5665worker2 # [ 101.644988] nix-daemon[871]: accepted connection from pid 1338, user nix-grpc-daemon (trusted)5666client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5667client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5668client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5669client # farm-a> LOG-farm-a-after-niks3-restart5670worker2 # [ 103.083502] nix-daemon[871]: reaped child process 1440, status = succeeded5671worker2 # [ 103.175806] nix-daemon[871]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)5672worker2 # [ 103.191173] nix-daemon[871]: reaped child process 1459, status = succeeded5673lb # [ 103.219786] niks3-server[1148]: 2026/09/25 15:00:16 INFO Received push request method=POST path=/api/pushes5674worker2 # [ 103.211714] nix-grpc-daemon[1398]: time=2026-09-25T15:00:17.240Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5675worker2 # [ 103.213286] nix-grpc-daemon[1398]: time=2026-09-25T15:00:17.241Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5676lb # [ 103.266994] niks3-server[1148]: 2026/09/25 15:00:16 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign5677worker2 # [ 103.250783] nix-grpc-daemon[1398]: time=2026-09-25T15:00:17.279Z level=INFO msg="Uploading 1 narinfos"5678lb # [ 103.271240] niks3-server[1148]: 2026/09/25 15:00:16 INFO Signed narinfos id=36 count=15679lb # [ 103.281553] niks3-server[1148]: 2026/09/25 15:00:16 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5680lb # [ 103.288510] niks3-server[1148]: 2026/09/25 15:00:16 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5681lb # [ 103.298626] niks3-server[1148]: 2026/09/25 15:00:16 INFO Received complete push request method=POST path=/api/pushes/36/complete5682lb # [ 103.300720] niks3-server[1148]: 2026/09/25 15:00:16 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5683lb # [ 103.310160] niks3-server[1148]: 2026/09/25 15:00:16 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5684worker2 # [ 103.293510] nix-grpc-daemon[1398]: time=2026-09-25T15:00:17.322Z level=INFO msg="Upload complete. (238ms)"5685worker2 # [ 103.301663] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52180 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=15686worker2 # [ 103.314736] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:17Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05687worker2 # [ 103.319447] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:17Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=15688worker2 # [ 103.324243] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521805689worker2 # [ 103.334319] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521805690worker2 # [ 103.339252] nix-daemon[871]: reaped child process 1438, status = succeeded5691worker2 # [ 103.342580] nix-grpc-daemon[1398]: time=2026-09-25T15:00:17.371Z level=INFO msg="All 1 paths already cached"5692client # copying 1 paths to 'lb:50051'...5693worker2 # [ 103.348314] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:521805694client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5695worker2 # [ 103.353463] nix-daemon[871]: accepted connection from pid 1338, user nix-grpc-daemon (trusted)5696worker2 # [ 103.448343] nix-daemon[871]: accepted connection from pid 1465, user nix-grpc-daemon (trusted)5697worker2 # [ 103.463100] nix-daemon[871]: reaped child process 1468, status = succeeded5698lb # [ 103.492227] niks3-server[1148]: 2026/09/25 15:00:16 INFO Received push request method=POST path=/api/pushes5699worker2 # [ 103.482191] nix-grpc-daemon[1398]: time=2026-09-25T15:00:17.510Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5700worker2 # [ 103.484344] nix-grpc-daemon[1398]: time=2026-09-25T15:00:17.512Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"5701lb # [ 103.528507] niks3-server[1148]: 2026/09/25 15:00:16 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign5702lb # [ 103.535180] niks3-server[1148]: 2026/09/25 15:00:16 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst5703lb # [ 103.541845] niks3-server[1148]: 2026/09/25 15:00:16 INFO Signed narinfos id=37 count=15704worker2 # [ 103.524751] nix-grpc-daemon[1398]: time=2026-09-25T15:00:17.553Z level=INFO msg="Uploading 1 narinfos"5705lb # [ 103.550515] niks3-server[1148]: 2026/09/25 15:00:16 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls5706lb # [ 103.559495] niks3-server[1148]: 2026/09/25 15:00:16 INFO Received complete push request method=POST path=/api/pushes/37/complete5707lb # [ 103.571462] niks3-server[1148]: 2026/09/25 15:00:16 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo5708worker2 # [ 103.555658] nix-grpc-daemon[1398]: time=2026-09-25T15:00:17.584Z level=INFO msg="Upload complete. (195ms)"5709worker2 # [ 103.560474] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52180 duration_s=0 paths=1 nar_bytes_in=6565710worker2 # [ 103.566560] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521805711worker2 # [ 103.579894] nix-daemon[871]: accepted connection from pid 1338, user nix-grpc-daemon (trusted)5712worker2 # [ 103.587468] nix-daemon[871]: reaped child process 1463, status = succeeded5713client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...5714worker2 # [ 103.594063] nix-daemon[871]: accepted connection from pid 1338, user nix-grpc-daemon (trusted)5715client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5716client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5717client # farm-b> LOG-farm-b-after-niks3-restart5718worker2 # [ 104.633153] nix-daemon[871]: reaped child process 1472, status = succeeded5719worker2 # [ 104.723100] nix-daemon[871]: accepted connection from pid 1488, user nix-grpc-daemon (trusted)5720worker2 # [ 104.736648] nix-daemon[871]: reaped child process 1491, status = succeeded5721lb # [ 104.765713] niks3-server[1148]: 2026/09/25 15:00:18 INFO Received push request method=POST path=/api/pushes5722worker2 # [ 104.754064] nix-grpc-daemon[1398]: time=2026-09-25T15:00:18.782Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5723worker2 # [ 104.755557] nix-grpc-daemon[1398]: time=2026-09-25T15:00:18.784Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"5724lb # [ 104.808627] niks3-server[1148]: 2026/09/25 15:00:18 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5725lb # [ 104.811961] niks3-server[1148]: 2026/09/25 15:00:18 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign5726worker2 # [ 104.797229] nix-grpc-daemon[1398]: time=2026-09-25T15:00:18.825Z level=INFO msg="Uploading 1 narinfos"5727lb # [ 104.816993] niks3-server[1148]: 2026/09/25 15:00:18 INFO Signed narinfos id=38 count=15728lb # [ 104.823246] niks3-server[1148]: 2026/09/25 15:00:18 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls5729lb # [ 104.830175] niks3-server[1148]: 2026/09/25 15:00:18 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst5730lb # [ 104.839296] niks3-server[1148]: 2026/09/25 15:00:18 INFO Received complete push request method=POST path=/api/pushes/38/complete5731worker2 # [ 104.825776] nix-grpc-daemon[1398]: time=2026-09-25T15:00:18.854Z level=INFO msg="Upload complete. (224ms)"5732lb # [ 104.849803] niks3-server[1148]: 2026/09/25 15:00:18 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo5733worker2 # [ 104.837418] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52180 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=15734worker2 # [ 104.846311] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:18Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=05735worker2 # [ 104.849476] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:18Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=15736worker2 # [ 104.852890] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521805737worker2 # [ 104.862319] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:521805738worker2 # [ 104.866130] nix-daemon[871]: reaped child process 1470, status = succeeded5739worker2 # [ 104.871391] nix-grpc-daemon[1398]: time=2026-09-25T15:00:18.900Z level=INFO msg="All 2 paths already cached"5740client # copying 1 paths to 'lb:50051'...5741worker2 # [ 104.876973] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:521805742client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...5743worker2 # [ 104.882605] nix-daemon[871]: accepted connection from pid 1338, user nix-grpc-daemon (trusted)5744worker2 # [ 104.977076] nix-daemon[871]: accepted connection from pid 1497, user nix-grpc-daemon (trusted)5745worker2 # [ 104.992440] nix-daemon[871]: reaped child process 1500, status = succeeded5746lb # [ 105.020799] niks3-server[1148]: 2026/09/25 15:00:18 INFO Received push request method=POST path=/api/pushes5747worker2 # [ 105.008848] nix-grpc-daemon[1398]: time=2026-09-25T15:00:19.037Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5748worker2 # [ 105.010707] nix-grpc-daemon[1398]: time=2026-09-25T15:00:19.039Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"5749lb # [ 105.050822] niks3-server[1148]: 2026/09/25 15:00:18 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign5750lb # [ 105.058109] niks3-server[1148]: 2026/09/25 15:00:18 INFO Signed narinfos id=39 count=15751worker2 # [ 105.040826] nix-grpc-daemon[1398]: time=2026-09-25T15:00:19.069Z level=INFO msg="Uploading 1 narinfos"5752lb # [ 105.066195] niks3-server[1148]: 2026/09/25 15:00:18 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst5753lb # [ 105.075289] niks3-server[1148]: 2026/09/25 15:00:18 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls5754lb # [ 105.083377] niks3-server[1148]: 2026/09/25 15:00:18 INFO Received complete push request method=POST path=/api/pushes/39/complete5755worker2 # [ 105.070948] nix-grpc-daemon[1398]: time=2026-09-25T15:00:19.099Z level=INFO msg="Upload complete. (181ms)"5756lb # [ 105.092586] niks3-server[1148]: 2026/09/25 15:00:18 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo5757worker2 # [ 105.073759] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52180 duration_s=0 paths=1 nar_bytes_in=7845758worker2 # [ 105.079050] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:521805759worker2 # [ 105.090992] nix-daemon[871]: accepted connection from pid 1338, user nix-grpc-daemon (trusted)5760worker2 # [ 105.095194] nix-daemon[871]: reaped child process 1495, status = succeeded5761worker2 # [ 105.100350] nix-daemon[871]: accepted connection from pid 1338, user nix-grpc-daemon (trusted)5762client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...5763client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5764lb # [ 105.382795] envoy[906]: conn peer=[2001:db8:1::3]:54286 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3612 tx=1148 ms=631275765lb # [ 105.386615] envoy[906]: conn peer=[2001:db8:1::1]:60358 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1002 tx=305 ms=10775766client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5767client # farm-top> LOG-farm-top-after-niks3-restart5768worker2 # [ 105.738361] nix-daemon[871]: reaped child process 1504, status = succeeded5769worker2 # [ 105.826775] nix-daemon[871]: accepted connection from pid 1520, user nix-grpc-daemon (trusted)5770worker2 # [ 105.840825] nix-daemon[871]: reaped child process 1523, status = succeeded5771lb # [ 105.870967] niks3-server[1148]: 2026/09/25 15:00:19 INFO Received push request method=POST path=/api/pushes5772worker2 # [ 105.859261] nix-grpc-daemon[1398]: time=2026-09-25T15:00:19.887Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5773worker2 # [ 105.860807] nix-grpc-daemon[1398]: time=2026-09-25T15:00:19.889Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"5774lb # [ 105.908534] niks3-server[1148]: 2026/09/25 15:00:19 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst5775lb # [ 105.918294] niks3-server[1148]: 2026/09/25 15:00:19 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign5776lb # [ 105.923789] niks3-server[1148]: 2026/09/25 15:00:19 INFO Signed narinfos id=40 count=15777worker2 # [ 105.906881] nix-grpc-daemon[1398]: time=2026-09-25T15:00:19.935Z level=INFO msg="Uploading 1 narinfos"5778lb # [ 105.932111] niks3-server[1148]: 2026/09/25 15:00:19 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls5779lb # [ 105.939732] niks3-server[1148]: 2026/09/25 15:00:19 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5780lb # [ 105.949329] niks3-server[1148]: 2026/09/25 15:00:19 INFO Received complete push request method=POST path=/api/pushes/40/complete5781lb # [ 105.954249] niks3-server[1148]: 2026/09/25 15:00:19 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo5782worker2 # [ 105.933395] nix-grpc-daemon[1398]: time=2026-09-25T15:00:19.962Z level=INFO msg="Upload complete. (224ms)"5783worker2 # [ 105.941528] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52180 duration_s=0 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=15784worker2 # [ 105.956701] nix-daemon[871]: reaped child process 1502, status = succeeded5785client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 5.25 seconds)5786client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath5787client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds)5788(finished: subtest: a niks3 restart does not cost the leader its role, in 5.73 seconds)5789subtest: scheduler down: the other node takes the lock and keeps it5790lb: must succeed: curl -sf localhost:9901/clusters5791lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5792lb: must succeed: curl -sf localhost:9901/clusters5793lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5794lb: must succeed: curl -sf localhost:9901/clusters5795lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5796lb: must succeed: curl -sf localhost:9901/clusters5797lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5798worker2 # [ 106.251812] systemd[1]: Stopping Nix worker-protocol over gRPC...5799worker2 # [ 106.328967] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:20Z level=info event=signal_exit5800lb # [ 106.466801] niks3-server[1148]: 2026/09/25 15:00:19 INFO lead: released remote=192.168.1.4:398145801worker2 # [ 107.450156] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:21Z level=info event=scheduler_restarting5802lb # [ 107.468584] envoy[906]: [2026-09-25 15:00:20.775][906][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5803worker1 # [ 107.419959] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:21Z level=info event=scheduler_restarting addr=lb:500515804worker2 # [ 107.457607] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:21Z level=info event=scheduler_restarting addr=lb:500515805worker2 # [ 107.463788] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:21Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:521665806worker2 # [ 107.468192] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:21Z level=info event=worker_gone addr=192.168.1.3:500515807worker2 # [ 107.471462] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:21Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:521665808worker2 # [ 107.475571] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:21Z level=info event=worker_gone addr=192.168.1.4:500515809worker2 # [ 107.500363] nix-daemon[871]: reaped child process 1390, status = succeeded5810worker1 # [ 107.619937] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:21Z level=info event=scheduler_connected addr=lb:500515811worker1 # [ 107.624065] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5812worker2 # [ 107.672424] nix-grpc-daemon[1338]: ts=2026-09-25T15:00:21Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5813worker1 # [ 107.824812] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:21Z level=info event=scheduler_connected addr=lb:500515814worker1 # [ 107.828932] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5815worker1 # [ 108.029664] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:22Z level=info event=scheduler_connected addr=lb:500515816worker1 # [ 108.033785] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5817worker1 # [ 108.234438] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:22Z level=info event=scheduler_connected addr=lb:500515818worker1 # [ 108.238440] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5819worker1 # [ 108.440262] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:22Z level=info event=scheduler_connected addr=lb:500515820worker1 # [ 108.444132] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5821worker2 # [ 108.666995] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5822worker2 # [ 108.670275] systemd[1]: Stopped Nix worker-protocol over gRPC.5823worker2 # [ 108.674283] systemd[1]: nix-grpc-daemon.service: Consumed 1.000s CPU time over 14.454s wall clock time, 37.7M memory peak, 312.6K incoming IP traffic, 110.6K outgoing IP traffic.5824worker1 # [ 108.645136] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:22Z level=info event=scheduler_connected addr=lb:500515825worker1 # [ 108.649116] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5826worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5827worker1 # [ 108.849798] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:23Z level=info event=scheduler_connected addr=lb:500515828worker1 # [ 108.853794] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5829worker1 # [ 109.054832] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:23Z level=info event=scheduler_connected addr=lb:500515830worker1 # [ 109.058862] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5831worker1 # [ 109.259661] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:23Z level=info event=scheduler_connected addr=lb:500515832worker1 # [ 109.263696] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5833worker1 # [ 109.465388] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:23Z level=info event=scheduler_connected addr=lb:500515834worker1 # [ 109.469310] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5835worker1 # [ 109.670076] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:23Z level=info event=scheduler_connected addr=lb:500515836worker1 # [ 109.674134] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5837worker1 # [ 109.875060] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:24Z level=info event=scheduler_connected addr=lb:500515838worker1 # [ 109.879129] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5839worker1 # [ 110.080190] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:24Z level=info event=scheduler_connected addr=lb:500515840worker1 # [ 110.084373] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5841worker1 # [ 110.284922] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:24Z level=info event=scheduler_connected addr=lb:500515842worker1 # [ 110.288879] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5843worker1 # [ 110.490445] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:24Z level=info event=scheduler_connected addr=lb:500515844worker1 # [ 110.494264] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5845worker1 # [ 110.695323] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:24Z level=info event=scheduler_connected addr=lb:500515846worker1 # [ 110.699322] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5847worker1 # [ 110.900312] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:25Z level=info event=scheduler_connected addr=lb:500515848worker1 # [ 110.904410] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5849worker1 # [ 111.105348] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:25Z level=info event=scheduler_connected addr=lb:500515850worker1 # [ 111.109464] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5851worker1 # [ 111.310117] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:25Z level=info event=scheduler_connected addr=lb:500515852worker1 # [ 111.313991] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5853worker1 # [ 111.407087] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:25Z level=info event=scheduler_take_over5854lb # [ 111.465162] niks3-server[1148]: 2026/09/25 15:00:24 INFO lead: acquired remote=192.168.1.3:422305855worker1 # [ 111.515874] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:25Z level=info event=scheduler_connected addr=lb:500515856worker1 # [ 111.520576] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5857worker1 # [ 111.720867] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:25Z level=info event=scheduler_connected addr=lb:500515858worker1 # [ 111.728317] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:25Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:569925859worker1 # [ 111.732627] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:25Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05860worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 3.20 seconds)5861worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5862worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5863client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&25864worker1 # [ 112.028602] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:446245865client # don't know how to build these paths:5866client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5867worker1 # [ 112.038061] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446245868worker1 # [ 112.061614] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:26Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05869worker1 # [ 112.067735] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:26Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15870worker1 # [ 112.072748] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446245871worker1 # [ 112.083842] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446245872client # copying 1 paths to 'lb:50051'...5873client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5874worker1 # [ 112.095816] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446245875worker1 # [ 112.104805] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5876worker1 # [ 112.125065] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:26Z level=info event=niks3_push_started pid=12535877worker1 # [ 112.221565] nix-grpc-daemon[1253]: time=2026-09-25T15:00:26.376Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""5878worker1 # [ 112.309360] nix-daemon[849]: accepted connection from pid 1266, user nix-grpc-daemon (trusted)5879worker1 # [ 112.322670] nix-daemon[849]: reaped child process 1269, status = succeeded5880lb # [ 112.386728] niks3-server[1148]: 2026/09/25 15:00:25 INFO Received push request method=POST path=/api/pushes5881worker1 # [ 112.338254] nix-grpc-daemon[1253]: time=2026-09-25T15:00:26.494Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5882worker1 # [ 112.340138] nix-grpc-daemon[1253]: time=2026-09-25T15:00:26.495Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5883lb # [ 112.424208] niks3-server[1148]: 2026/09/25 15:00:25 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign5884lb # [ 112.430631] niks3-server[1148]: 2026/09/25 15:00:25 INFO Signed narinfos id=41 count=15885worker1 # [ 112.376717] nix-grpc-daemon[1253]: time=2026-09-25T15:00:26.531Z level=INFO msg="Uploading 1 narinfos"5886lb # [ 112.440697] niks3-server[1148]: 2026/09/25 15:00:25 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5887lb # [ 112.453068] niks3-server[1148]: 2026/09/25 15:00:25 INFO Received complete push request method=POST path=/api/pushes/41/complete5888worker1 # [ 112.403462] nix-grpc-daemon[1253]: time=2026-09-25T15:00:26.559Z level=INFO msg="Upload complete. (178ms)"5889worker1 # [ 112.405624] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44624 duration_s=0 paths=1 nar_bytes_in=5205890lb # [ 112.464062] niks3-server[1148]: 2026/09/25 15:00:25 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5891worker1 # [ 112.409218] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446245892lb # [ 112.467786] niks3-server[1148]: 2026/09/25 15:00:25 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5893worker1 # [ 112.416553] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5894worker1 # [ 112.419188] nix-daemon[849]: reaped child process 1251, status = succeeded5895worker1 # [ 112.424188] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5896client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5897client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5898client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5899client # farm-a> LOG-farm-a-on-standby5900worker1 # [ 113.840658] nix-daemon[849]: reaped child process 1273, status = succeeded5901worker1 # [ 113.915079] nix-daemon[849]: accepted connection from pid 1289, user nix-grpc-daemon (trusted)5902worker1 # [ 113.928109] nix-daemon[849]: reaped child process 1292, status = succeeded5903lb # [ 113.992640] niks3-server[1148]: 2026/09/25 15:00:27 INFO Received push request method=POST path=/api/pushes5904worker1 # [ 113.945287] nix-grpc-daemon[1253]: time=2026-09-25T15:00:28.099Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5905worker1 # [ 113.946676] nix-grpc-daemon[1253]: time=2026-09-25T15:00:28.099Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5906lb # [ 114.026301] niks3-server[1148]: 2026/09/25 15:00:27 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5907lb # [ 114.041132] niks3-server[1148]: 2026/09/25 15:00:27 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign5908worker1 # [ 113.990039] nix-grpc-daemon[1253]: time=2026-09-25T15:00:28.145Z level=INFO msg="Uploading 1 narinfos"5909lb # [ 114.047637] niks3-server[1148]: 2026/09/25 15:00:27 INFO Signed narinfos id=42 count=15910lb # [ 114.054675] niks3-server[1148]: 2026/09/25 15:00:27 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5911lb # [ 114.063737] niks3-server[1148]: 2026/09/25 15:00:27 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5912lb # [ 114.068236] niks3-server[1148]: 2026/09/25 15:00:27 INFO Received complete push request method=POST path=/api/pushes/42/complete5913lb # [ 114.075374] niks3-server[1148]: 2026/09/25 15:00:27 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5914worker1 # [ 114.016350] nix-grpc-daemon[1253]: time=2026-09-25T15:00:28.171Z level=INFO msg="Upload complete. (184ms)"5915worker1 # [ 114.025603] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44624 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15916worker1 # [ 114.038474] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:28Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05917worker1 # [ 114.042671] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:28Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15918worker1 # [ 114.046353] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446245919worker1 # [ 114.055311] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446245920worker1 # [ 114.058413] nix-daemon[849]: reaped child process 1271, status = succeeded5921worker1 # [ 114.063238] nix-grpc-daemon[1253]: time=2026-09-25T15:00:28.218Z level=INFO msg="All 1 paths already cached"5922client # copying 1 paths to 'lb:50051'...5923worker1 # [ 114.068854] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446245924client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5925worker1 # [ 114.073947] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5926worker1 # [ 114.158150] nix-daemon[849]: accepted connection from pid 1298, user nix-grpc-daemon (trusted)5927worker1 # [ 114.171151] nix-daemon[849]: reaped child process 1301, status = succeeded5928lb # [ 114.235492] niks3-server[1148]: 2026/09/25 15:00:27 INFO Received push request method=POST path=/api/pushes5929worker1 # [ 114.186290] nix-grpc-daemon[1253]: time=2026-09-25T15:00:28.342Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5930worker1 # [ 114.188314] nix-grpc-daemon[1253]: time=2026-09-25T15:00:28.343Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5931lb # [ 114.269328] niks3-server[1148]: 2026/09/25 15:00:27 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign5932lb # [ 114.279598] niks3-server[1148]: 2026/09/25 15:00:27 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5933lb # [ 114.289236] niks3-server[1148]: 2026/09/25 15:00:27 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5934lb # [ 114.301329] niks3-server[1148]: 2026/09/25 15:00:27 INFO Signed narinfos id=43 count=15935worker1 # [ 114.246270] nix-grpc-daemon[1253]: time=2026-09-25T15:00:28.401Z level=INFO msg="Uploading 1 narinfos"5936lb # [ 114.308141] niks3-server[1148]: 2026/09/25 15:00:27 INFO Received complete push request method=POST path=/api/pushes/43/complete5937lb # [ 114.314570] niks3-server[1148]: 2026/09/25 15:00:27 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5938worker1 # [ 114.262686] nix-grpc-daemon[1253]: time=2026-09-25T15:00:28.418Z level=INFO msg="Upload complete. (182ms)"5939worker1 # [ 114.267537] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44624 duration_s=0 paths=1 nar_bytes_in=6405940worker1 # [ 114.274222] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446245941worker1 # [ 114.283864] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5942worker1 # [ 114.293286] nix-daemon[849]: reaped child process 1296, status = succeeded5943worker1 # [ 114.301336] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5944client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5945client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5946client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5947client # farm-b> LOG-farm-b-on-standby5948worker1 # [ 114.915922] nix-daemon[849]: reaped child process 1305, status = succeeded5949worker1 # [ 114.990327] nix-daemon[849]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5950worker1 # [ 115.003131] nix-daemon[849]: reaped child process 1324, status = succeeded5951lb # [ 115.069917] niks3-server[1148]: 2026/09/25 15:00:28 INFO Received push request method=POST path=/api/pushes5952worker1 # [ 115.024090] nix-grpc-daemon[1253]: time=2026-09-25T15:00:29.178Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5953worker1 # [ 115.025458] nix-grpc-daemon[1253]: time=2026-09-25T15:00:29.178Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5954lb # [ 115.109801] niks3-server[1148]: 2026/09/25 15:00:28 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign5955worker1 # [ 115.060646] nix-grpc-daemon[1253]: time=2026-09-25T15:00:29.216Z level=INFO msg="Uploading 1 narinfos"5956lb # [ 115.119214] niks3-server[1148]: 2026/09/25 15:00:28 INFO Signed narinfos id=44 count=15957lb # [ 115.129054] niks3-server[1148]: 2026/09/25 15:00:28 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5958lb # [ 115.132403] niks3-server[1148]: 2026/09/25 15:00:28 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5959lb # [ 115.140115] niks3-server[1148]: 2026/09/25 15:00:28 INFO Received complete push request method=POST path=/api/pushes/44/complete5960worker1 # [ 115.092259] nix-grpc-daemon[1253]: time=2026-09-25T15:00:29.247Z level=INFO msg="Upload complete. (184ms)"5961lb # [ 115.152100] niks3-server[1148]: 2026/09/25 15:00:28 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5962lb # [ 115.153982] niks3-server[1148]: 2026/09/25 15:00:28 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5963worker1 # [ 115.097368] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44624 duration_s=0 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15964worker1 # [ 115.107995] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:29Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05965worker1 # [ 115.112551] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:29Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=15966worker1 # [ 115.116628] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446245967worker1 # [ 115.125789] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446245968worker1 # [ 115.127949] nix-daemon[849]: reaped child process 1303, status = succeeded5969worker1 # [ 115.133909] nix-grpc-daemon[1253]: time=2026-09-25T15:00:29.289Z level=INFO msg="All 2 paths already cached"5970client # copying 1 paths to 'lb:50051'...5971worker1 # [ 115.139790] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446245972client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...5973worker1 # [ 115.144895] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5974worker1 # [ 115.233265] nix-daemon[849]: accepted connection from pid 1330, user nix-grpc-daemon (trusted)5975worker1 # [ 115.245914] nix-daemon[849]: reaped child process 1333, status = succeeded5976lb # [ 115.312175] niks3-server[1148]: 2026/09/25 15:00:28 INFO Received push request method=POST path=/api/pushes5977worker1 # [ 115.263305] nix-grpc-daemon[1253]: time=2026-09-25T15:00:29.419Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5978worker1 # [ 115.264836] nix-grpc-daemon[1253]: time=2026-09-25T15:00:29.420Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"5979lb # [ 115.348275] niks3-server[1148]: 2026/09/25 15:00:28 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign5980worker1 # [ 115.297202] nix-grpc-daemon[1253]: time=2026-09-25T15:00:29.452Z level=INFO msg="Uploading 1 narinfos"5981lb # [ 115.355312] niks3-server[1148]: 2026/09/25 15:00:28 INFO Signed narinfos id=45 count=15982lb # [ 115.360184] niks3-server[1148]: 2026/09/25 15:00:28 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls5983lb # [ 115.365299] niks3-server[1148]: 2026/09/25 15:00:28 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst5984lb # [ 115.379365] niks3-server[1148]: 2026/09/25 15:00:28 INFO Received complete push request method=POST path=/api/pushes/45/complete5985lb # [ 115.385567] envoy[906]: conn peer=[2001:db8:1::1]:60372 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9735 tx=4451 ms=50795986lb # [ 115.387581] envoy[906]: conn peer=[2001:db8:1::4]:33546 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2597 tx=530 ms=133095987lb # [ 115.393066] niks3-server[1148]: 2026/09/25 15:00:28 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo5988worker1 # [ 115.334565] nix-grpc-daemon[1253]: time=2026-09-25T15:00:29.490Z level=INFO msg="Upload complete. (183ms)"5989worker1 # [ 115.338972] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44624 duration_s=0 paths=1 nar_bytes_in=7685990worker1 # [ 115.345337] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446245991worker1 # [ 115.358107] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5992worker1 # [ 115.365421] nix-daemon[849]: reaped child process 1328, status = succeeded5993worker1 # [ 115.372945] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)5994client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...5995client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5996client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5997client # farm-top> LOG-farm-top-on-standby5998worker1 # [ 116.387204] nix-daemon[849]: reaped child process 1337, status = succeeded5999worker1 # [ 116.467094] nix-daemon[849]: accepted connection from pid 1353, user nix-grpc-daemon (trusted)6000worker1 # [ 116.479125] nix-daemon[849]: reaped child process 1356, status = succeeded6001lb # [ 116.545657] niks3-server[1148]: 2026/09/25 15:00:29 INFO Received push request method=POST path=/api/pushes6002worker1 # [ 116.499271] nix-grpc-daemon[1253]: time=2026-09-25T15:00:30.654Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6003worker1 # [ 116.501844] nix-grpc-daemon[1253]: time=2026-09-25T15:00:30.656Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6004lb # [ 116.584799] niks3-server[1148]: 2026/09/25 15:00:29 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6005lb # [ 116.600478] niks3-server[1148]: 2026/09/25 15:00:29 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6006lb # [ 116.606766] niks3-server[1148]: 2026/09/25 15:00:29 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign6007worker1 # [ 116.553677] nix-grpc-daemon[1253]: time=2026-09-25T15:00:30.709Z level=INFO msg="Uploading 1 narinfos"6008lb # [ 116.611985] niks3-server[1148]: 2026/09/25 15:00:29 INFO Signed narinfos id=46 count=16009lb # [ 116.620649] niks3-server[1148]: 2026/09/25 15:00:29 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6010lb # [ 116.624044] niks3-server[1148]: 2026/09/25 15:00:29 INFO Received complete push request method=POST path=/api/pushes/46/complete6011lb # [ 116.629858] niks3-server[1148]: 2026/09/25 15:00:29 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6012worker1 # [ 116.571254] nix-grpc-daemon[1253]: time=2026-09-25T15:00:30.726Z level=INFO msg="Upload complete. (191ms)"6013worker1 # [ 116.577656] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44624 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16014worker1 # [ 116.592361] nix-daemon[849]: reaped child process 1335, status = succeeded6015client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 4.73 seconds)6016client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6017client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.12 seconds)6018worker2 # [ 116.822271] systemd[1]: Starting scheduler-token.service...6019worker2 # [ 116.868718] systemd[1]: scheduler-token.service: Deactivated successfully.6020worker2 # [ 116.870473] systemd[1]: Finished scheduler-token.service.6021worker2 # [ 116.873221] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 50ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6022worker2 # [ 116.876385] systemd[1]: Starting Nix worker-protocol over gRPC...6023worker1 # [ 116.919158] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:31Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:453346024worker2 # [ 116.965806] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6025worker2 # [ 116.967945] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6026worker2 # [ 116.971614] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16027worker2 # [ 116.976057] nix-grpc-daemon[1544]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6028worker2 # [ 116.978296] nix-grpc-daemon[1544]: E0000 00:00:1790348431.004716 1544 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6029worker2 # [ 116.983737] nix-grpc-daemon[1544]: E0000 00:00:1790348431.010426 1544 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6030worker2 # [ 116.987990] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:31Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6031worker2 # [ 116.993279] systemd[1]: Started Nix worker-protocol over gRPC.6032worker2 # [ 116.998064] nix-grpc-daemon[1544]: E0000 00:00:1790348431.023224 1586 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6033worker2 # [ 117.008859] nix-grpc-daemon[1544]: E0000 00:00:1790348431.037445 1586 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6034worker2 # [ 117.012198] nix-grpc-daemon[1544]: E0000 00:00:1790348431.039712 1586 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6035lb: must succeed: curl -sf localhost:9901/clusters6036worker2 # [ 117.024610] nix-grpc-daemon[1544]: E0000 00:00:1790348431.053242 1586 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6037worker2 # [ 117.036775] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:31Z level=info event=scheduler_connected addr=lb:500516038worker1 # [ 117.009171] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:31Z level=info event=oidc_jwks provider=mock keys=16039worker1 # [ 117.013509] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:31Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:569926040worker1 # [ 117.019980] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06041lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6042worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6043worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6044lb: must succeed: curl -sf localhost:9901/clusters6045lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6046lb: must succeed: curl -sf localhost:9901/clusters6047lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6048worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6049worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6050(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.14 seconds)6051subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6052client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6053client # [ 117.274488] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6054client # Running as unit: rs.service; invocation ID: a6d03a0998b34debbc67529d6099b44e6055client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6056worker2 # [ 117.414182] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:437726057worker2 # [ 117.426647] nix-daemon[871]: accepted connection from pid 1544, user nix-grpc-daemon (trusted)6058client # [ 117.436041] nix[1360]: don't know how to build these paths:6059client # [ 117.436961] nix[1360]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6060worker1 # [ 117.410234] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:31Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06061worker1 # [ 117.415796] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:31Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16062worker1 # [ 117.421752] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446246063worker1 # [ 117.433533] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:446246064client # [ 117.479259] nix[1360]: copying 1 paths to 'lb:50051'...6065client # [ 117.480754] nix[1360]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6066worker1 # [ 117.444421] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:446246067worker1 # [ 117.455994] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)6068worker1 # [ 117.557449] nix-daemon[849]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)6069worker1 # [ 117.570469] nix-daemon[849]: reaped child process 1372, status = succeeded6070lb # [ 117.636989] niks3-server[1148]: 2026/09/25 15:00:30 INFO Received push request method=POST path=/api/pushes6071worker1 # [ 117.587907] nix-grpc-daemon[1253]: time=2026-09-25T15:00:31.743Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6072worker1 # [ 117.589362] nix-grpc-daemon[1253]: time=2026-09-25T15:00:31.743Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6073lb # [ 117.664776] niks3-server[1148]: 2026/09/25 15:00:30 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign6074lb # [ 117.671142] niks3-server[1148]: 2026/09/25 15:00:30 INFO Signed narinfos id=47 count=16075worker1 # [ 117.617037] nix-grpc-daemon[1253]: time=2026-09-25T15:00:31.772Z level=INFO msg="Uploading 1 narinfos"6076lb # [ 117.679125] niks3-server[1148]: 2026/09/25 15:00:30 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6077lb # [ 117.685305] niks3-server[1148]: 2026/09/25 15:00:30 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6078lb # [ 117.698118] niks3-server[1148]: 2026/09/25 15:00:31 INFO Received complete push request method=POST path=/api/pushes/47/complete6079worker1 # [ 117.646674] nix-grpc-daemon[1253]: time=2026-09-25T15:00:31.802Z level=INFO msg="Upload complete. (179ms)"6080lb # [ 117.708324] niks3-server[1148]: 2026/09/25 15:00:31 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6081worker1 # [ 117.650580] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44624 duration_s=0 paths=1 nar_bytes_in=4166082worker1 # [ 117.656268] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:446246083worker1 # [ 117.664572] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)6084worker1 # [ 117.672782] nix-daemon[849]: reaped child process 1367, status = succeeded6085worker1 # [ 117.677431] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)6086client # [ 117.713176] nix[1360]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6087lb: must succeed: curl -sf localhost:9901/clusters6088lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6089lb: must succeed: curl -sf localhost:9901/clusters6090lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6091lb: must succeed: curl -sf localhost:9901/clusters6092lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6093lb: must succeed: curl -sf localhost:9901/clusters6094lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6095worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6096worker2 # [ 119.959012] systemd-journald[285]: Received client request to rotate journal, rotating.6097worker2 # [ 119.981073] systemd-journald[285]: Vacuuming done, freed 0B of archived journals from /var/log/journal/26803de17eb3418f8779cb68ada3d99d.6098worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.07 seconds)6099worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6100worker1 # [ 119.855183] systemd[1]: Stopping Nix worker-protocol over gRPC...6101worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6102client: must succeed: sleep 5.06103worker1 # [ 120.234457] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:34Z level=info event=draining6104client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6105worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6106worker1 # [ 125.100151] nix-daemon[849]: reaped child process 1376, status = succeeded6107worker1 # [ 125.177266] nix-daemon[849]: accepted connection from pid 1417, user nix-grpc-daemon (trusted)6108worker1 # [ 125.190357] nix-daemon[849]: reaped child process 1420, status = succeeded6109lb # [ 125.256561] niks3-server[1148]: 2026/09/25 15:00:38 INFO Received push request method=POST path=/api/pushes6110worker1 # [ 125.207459] nix-grpc-daemon[1253]: time=2026-09-25T15:00:39.362Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6111worker1 # [ 125.208937] nix-grpc-daemon[1253]: time=2026-09-25T15:00:39.364Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6112lb # [ 125.305167] niks3-server[1148]: 2026/09/25 15:00:38 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign6113worker1 # [ 125.251506] nix-grpc-daemon[1253]: time=2026-09-25T15:00:39.407Z level=INFO msg="Uploading 1 narinfos"6114lb # [ 125.309485] niks3-server[1148]: 2026/09/25 15:00:38 INFO Signed narinfos id=48 count=16115lb # [ 125.322288] niks3-server[1148]: 2026/09/25 15:00:38 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6116lb # [ 125.328116] niks3-server[1148]: 2026/09/25 15:00:38 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6117lb # [ 125.340546] niks3-server[1148]: 2026/09/25 15:00:38 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6118lb # [ 125.343584] niks3-server[1148]: 2026/09/25 15:00:38 INFO Received complete push request method=POST path=/api/pushes/48/complete6119lb # [ 125.349508] niks3-server[1148]: 2026/09/25 15:00:38 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6120worker1 # [ 125.290982] nix-grpc-daemon[1253]: time=2026-09-25T15:00:39.446Z level=INFO msg="Upload complete. (200ms)"6121worker1 # [ 125.297647] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44624 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16122worker1 # [ 125.314281] nix-daemon[849]: reaped child process 1374, status = succeeded6123lb # [ 125.383734] envoy[906]: conn peer=[2001:db8:1::1]:57970 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9547 tx=4345 ms=45656124lb # [ 125.385202] envoy[906]: conn peer=[2001:db8:1::1]:34442 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2956 tx=1498 ms=79416125client # [ 125.370922] systemd[1]: rs.service: Deactivated successfully.6126client # [ 125.374791] systemd[1]: rs.service: Consumed 130ms CPU time over 8.099s wall clock time, 43.4M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic.6127worker1 # [ 126.236088] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:40Z level=info event=signal_exit6128lb # [ 126.465848] niks3-server[1148]: 2026/09/25 15:00:39 INFO lead: released remote=192.168.1.3:422306129lb # [ 127.015396] niks3-server[1148]: 2026/09/25 15:00:40 INFO lead: acquired remote=192.168.1.4:583546130worker2 # [ 126.996137] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:41Z level=info event=scheduler_take_over6131worker1 # [ 127.409974] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:41Z level=info event=scheduler_restarting6132lb # [ 127.467158] envoy[906]: [2026-09-25 15:00:40.773][906][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}6133worker2 # [ 127.455278] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:41Z level=info event=scheduler_restarting addr=lb:500516134worker1 # [ 127.418485] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:41Z level=info event=scheduler_restarting addr=lb:500516135worker1 # [ 127.424468] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:569926136worker1 # [ 127.428573] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:41Z level=info event=worker_gone addr=192.168.1.4:500516137worker1 # [ 127.432340] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:569926138worker1 # [ 127.435949] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:41Z level=info event=worker_gone addr=192.168.1.3:500516139worker1 # [ 127.458380] nix-daemon[849]: reaped child process 1214, status = succeeded6140worker2 # [ 127.628444] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:41Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:569246141worker2 # [ 127.656653] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:41Z level=info event=scheduler_connected addr=lb:500516142worker2 # [ 127.669157] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:41Z level=info event=oidc_jwks provider=mock keys=16143worker1 # [ 127.632827] nix-grpc-daemon[1162]: ts=2026-09-25T15:00:41Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6144worker2 # [ 127.673652] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:41Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:569406145worker2 # [ 127.680982] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:41Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06146worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.16 seconds)6147client: waiting for success: ! systemctl is-active rs6148client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6149client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6150client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6151client: must fail: journalctl -u rs | grep -q 'reconnecting'6152client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6153lb: must succeed: curl -sf localhost:9901/clusters6154lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6155worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6156worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6157worker1 # [ 129.122169] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6158worker1 # [ 129.125140] systemd[1]: Stopped Nix worker-protocol over gRPC.6159worker1 # [ 129.128459] systemd[1]: nix-grpc-daemon.service: Consumed 1.041s CPU time over 30.002s wall clock time, 38.1M memory peak, 330.8K incoming IP traffic, 143.3K outgoing IP traffic.6160worker1 # [ 129.142823] systemd[1]: Starting Nix worker-protocol over gRPC...6161worker1 # [ 129.236061] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:43Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6162worker1 # [ 129.238052] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6163worker1 # [ 129.241789] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16164worker1 # [ 129.246058] nix-grpc-daemon[1428]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6165worker1 # [ 129.247429] nix-grpc-daemon[1428]: E0000 00:00:1790348443.401254 1428 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6166worker1 # [ 129.250853] nix-grpc-daemon[1428]: E0000 00:00:1790348443.406513 1428 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6167worker1 # [ 129.255281] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:43Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6168worker1 # [ 129.258766] systemd[1]: Started Nix worker-protocol over gRPC.6169worker1 # [ 129.261566] nix-grpc-daemon[1428]: E0000 00:00:1790348443.416881 1470 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6170worker1 # [ 129.273403] nix-grpc-daemon[1428]: E0000 00:00:1790348443.429102 1470 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6171worker1 # [ 129.278053] nix-grpc-daemon[1428]: E0000 00:00:1790348443.431379 1470 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6172worker1 # [ 129.287531] nix-grpc-daemon[1428]: E0000 00:00:1790348443.443233 1470 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6173worker1 # [ 129.296571] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:43Z level=info event=scheduler_connected addr=lb:500516174worker2 # [ 129.336443] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:569406175worker2 # [ 129.341254] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06176lb: must succeed: curl -sf localhost:9901/clusters6177lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6178worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6179worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6180(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.32 seconds)6181subtest: scheduler restart mid-build: build finishes, no double build6182client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6183client # [ 129.595906] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6184client # Running as unit: mid.service; invocation ID: d97eb0da662246ab8b035ac5291b7ce26185client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6186worker1 # [ 129.701305] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479546187worker1 # [ 129.712154] nix-daemon[849]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)6188client # [ 129.760636] nix[1396]: don't know how to build these paths:6189client # [ 129.761566] nix[1396]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6190worker2 # [ 129.770304] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:43Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06191worker2 # [ 129.776324] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:43Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16192worker2 # [ 129.789914] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569486193worker2 # [ 129.801909] nix-grpc-daemon[1544]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 1/5)6194worker2 # [ 129.899868] nix-grpc-daemon[1544]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 2/5)6195worker2 # [ 129.917852] nix-grpc-daemon[1544]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 213 ms (attempt 3/5)6196worker2 # [ 130.138639] nix-grpc-daemon[1544]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 542 ms (attempt 4/5)6197worker2 # [ 130.690315] nix-grpc-daemon[1544]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6198worker2 # [ 130.693424] nix-grpc-daemon[1544]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6199worker2 # [ 130.711117] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:569486200client # [ 130.719893] nix[1396]: copying 1 paths to 'lb:50051'...6201client # [ 130.724691] nix[1396]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6202worker2 # [ 130.726461] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:569486203worker2 # [ 130.737224] nix-daemon[871]: accepted connection from pid 1544, user nix-grpc-daemon (trusted)6204worker2 # [ 130.758429] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:44Z level=info event=niks3_push_started pid=16726205worker2 # [ 130.889402] nix-grpc-daemon[1672]: time=2026-09-25T15:00:44.917Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6206worker2 # [ 130.987732] nix-daemon[871]: accepted connection from pid 1689, user nix-grpc-daemon (trusted)6207worker2 # [ 131.001656] nix-daemon[871]: reaped child process 1692, status = succeeded6208lb # [ 131.028172] niks3-server[1148]: 2026/09/25 15:00:44 INFO Received push request method=POST path=/api/pushes6209worker2 # [ 131.016293] nix-grpc-daemon[1672]: time=2026-09-25T15:00:45.044Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6210worker2 # [ 131.017940] nix-grpc-daemon[1672]: time=2026-09-25T15:00:45.046Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6211lb # [ 131.068450] niks3-server[1148]: 2026/09/25 15:00:44 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign6212lb # [ 131.075511] niks3-server[1148]: 2026/09/25 15:00:44 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6213lb # [ 131.091902] niks3-server[1148]: 2026/09/25 15:00:44 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6214worker2 # [ 131.076482] nix-grpc-daemon[1672]: time=2026-09-25T15:00:45.105Z level=INFO msg="Uploading 1 narinfos"6215lb # [ 131.096849] niks3-server[1148]: 2026/09/25 15:00:44 INFO Signed narinfos id=49 count=16216lb # [ 131.107656] niks3-server[1148]: 2026/09/25 15:00:44 INFO Received complete push request method=POST path=/api/pushes/49/complete6217lb # [ 131.117972] niks3-server[1148]: 2026/09/25 15:00:44 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6218worker2 # [ 131.102664] nix-grpc-daemon[1672]: time=2026-09-25T15:00:45.131Z level=INFO msg="Upload complete. (210ms)"6219worker2 # [ 131.108137] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56948 duration_s=0 paths=1 nar_bytes_in=4246220worker2 # [ 131.114077] nix-grpc-daemon[1544]: ts=2026-09-25T15:00:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:569486221worker2 # [ 131.126528] nix-daemon[871]: accepted connection from pid 1544, user nix-grpc-daemon (trusted)6222worker2 # [ 131.136351] nix-daemon[871]: reaped child process 1670, status = succeeded6223worker2 # [ 131.141381] nix-daemon[871]: accepted connection from pid 1544, user nix-grpc-daemon (trusted)6224client # [ 131.140577] nix[1396]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6225lb: must succeed: curl -sf localhost:9901/clusters6226lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6227lb: must succeed: curl -sf localhost:9901/clusters6228lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6229worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6230worker2 # [ 133.259316] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1544 (nix-grpc-daemon) on client request.6231worker2 # [ 133.261120] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6232lb # [ 133.286738] niks3-server[1148]: 2026/09/25 15:00:46 INFO lead: released remote=192.168.1.4:583546233worker2 # [ 133.275761] systemd[1]: Starting scheduler-token.service...6234lb # [ 133.291669] envoy[906]: [2026-09-25 15:00:46.596][906][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}6235worker2 # [ 133.279840] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6236worker2 # [ 133.281451] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6237worker2 # [ 133.284732] systemd[1]: nix-grpc-daemon.service: Consumed 443ms CPU time over 16.401s wall clock time, 36.3M memory peak, 279.9K incoming IP traffic, 57.6K outgoing IP traffic.6238client # [ 133.280758] nix[1396]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6239worker1 # [ 133.244529] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:47Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6240client # [ 133.285508] nix[1396]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6241worker2 # [ 133.308570] nix-daemon[871]: reaped child process 1694, status = succeeded6242worker2 # [ 133.309965] nix-daemon[871]: reaped child process 1603, status = succeeded6243worker2 # [ 133.318956] nix-daemon[1696]: unexpected Nix daemon error: error: interrupted by the user6244worker2 # [ 133.326194] nix-daemon[871]: reaped child process 1696, status = failed with exit code 16245worker2 # [ 133.341520] systemd[1]: scheduler-token.service: Deactivated successfully.6246worker2 # [ 133.343231] systemd[1]: Finished scheduler-token.service.6247worker2 # [ 133.344484] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 67ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6248worker2 # [ 133.347416] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6249worker2 # [ 133.350746] systemd[1]: Starting scheduler-token.service...6250worker2 # [ 133.386900] systemd[1]: scheduler-token.service: Deactivated successfully.6251worker2 # [ 133.388666] systemd[1]: Finished scheduler-token.service.6252worker2 # [ 133.390055] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 37ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6253worker2 # [ 133.394955] systemd[1]: Starting Nix worker-protocol over gRPC...6254worker2 # [ 133.476382] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:47Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6255worker2 # [ 133.478483] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:47Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6256worker2 # [ 133.481828] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:47Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16257worker1 # [ 133.445503] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:47Z level=info event=scheduler_connected addr=lb:500516258worker2 # [ 133.486483] nix-grpc-daemon[1739]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6259worker2 # [ 133.489168] nix-grpc-daemon[1739]: E0000 00:00:1790348447.515000 1739 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6260worker2 # [ 133.491895] nix-grpc-daemon[1739]: E0000 00:00:1790348447.520496 1739 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6261worker2 # [ 133.495775] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:47Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6262worker2 # [ 133.499515] systemd[1]: Started Nix worker-protocol over gRPC.6263worker2 # [ 133.505076] nix-grpc-daemon[1739]: E0000 00:00:1790348447.532681 1782 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6264lb # [ 133.525212] niks3-server[1148]: 2026/09/25 15:00:46 INFO lead: acquired remote=192.168.1.4:538026265worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.28 seconds)6266worker2 # [ 133.511115] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:47Z level=info event=scheduler_take_over6267client: must succeed: sleep 5.06268worker2 # [ 133.515497] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:587906269worker2 # [ 133.518109] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06270worker2 # [ 133.526051] nix-grpc-daemon[1739]: E0000 00:00:1790348447.554713 1782 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6271worker2 # [ 133.531056] nix-grpc-daemon[1739]: E0000 00:00:1790348447.557056 1782 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6272worker2 # [ 133.540879] nix-grpc-daemon[1739]: E0000 00:00:1790348447.569516 1782 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6273worker2 # [ 133.549945] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:47Z level=info event=scheduler_connected addr=lb:500516274worker2 # [ 133.554784] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:47Z level=info event=oidc_jwks provider=mock keys=16275worker2 # [ 133.556168] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:47Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:587906276worker2 # [ 133.557836] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:47Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06277worker2 # [ 133.795351] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:47Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06278worker2 # [ 133.800211] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:47Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16279worker1 # [ 133.766378] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479546280worker1 # [ 133.777739] nix-grpc-daemon[1428]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 1/5)6281worker2 # [ 133.827881] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:47Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:588266282worker1 # [ 133.845097] nix-grpc-daemon[1428]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 2/5)6283worker1 # [ 133.915460] nix-grpc-daemon[1428]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 3/5)6284worker1 # [ 134.057751] nix-grpc-daemon[1428]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 400 ms (attempt 4/5)6285worker1 # [ 134.461260] nix-grpc-daemon[1428]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6286worker1 # [ 134.464507] nix-grpc-daemon[1428]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6287worker1 # [ 134.484177] nix-grpc-daemon[1428]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...6288worker1 # [ 134.509372] nix-daemon[849]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)6289worker1 # [ 134.522833] nix-daemon[849]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)6290client # [ 134.558070] nix[1396]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6291lb # [ 135.384605] envoy[906]: conn peer=[2001:db8:1::3]:54596 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=5610 tx=1303 ms=283546292lb # [ 135.388618] envoy[906]: conn peer=[2001:db8:1::4]:58142 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2089 tx=328 ms=162516293client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6294client: waiting for success: ! systemctl is-active mid6295worker1 # [ 138.749839] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:52Z level=info event=niks3_push_started pid=15346296worker1 # [ 138.784456] nix-daemon[849]: reaped child process 1509, status = succeeded6297worker1 # [ 138.865377] nix-grpc-daemon[1534]: time=2026-09-25T15:00:53.020Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6298worker1 # [ 138.953816] nix-daemon[849]: accepted connection from pid 1547, user nix-grpc-daemon (trusted)6299worker1 # [ 138.967779] nix-daemon[849]: reaped child process 1550, status = succeeded6300lb # [ 139.032991] niks3-server[1148]: 2026/09/25 15:00:52 INFO Received push request method=POST path=/api/pushes6301worker1 # [ 138.983701] nix-grpc-daemon[1534]: time=2026-09-25T15:00:53.139Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6302worker1 # [ 138.985297] nix-grpc-daemon[1534]: time=2026-09-25T15:00:53.141Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6303lb # [ 139.082486] niks3-server[1148]: 2026/09/25 15:00:52 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign6304worker1 # [ 139.031477] nix-grpc-daemon[1534]: time=2026-09-25T15:00:53.186Z level=INFO msg="Uploading 1 narinfos"6305lb # [ 139.088266] niks3-server[1148]: 2026/09/25 15:00:52 INFO Signed narinfos id=50 count=16306lb # [ 139.103861] niks3-server[1148]: 2026/09/25 15:00:52 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6307lb # [ 139.107513] niks3-server[1148]: 2026/09/25 15:00:52 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6308lb # [ 139.119977] niks3-server[1148]: 2026/09/25 15:00:52 INFO Received complete push request method=POST path=/api/pushes/50/complete6309worker1 # [ 139.067486] nix-grpc-daemon[1534]: time=2026-09-25T15:00:53.223Z level=INFO msg="Upload complete. (199ms)"6310lb # [ 139.127022] niks3-server[1148]: 2026/09/25 15:00:52 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6311lb # [ 139.130804] niks3-server[1148]: 2026/09/25 15:00:52 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6312worker1 # [ 139.073288] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47954 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16313worker1 # [ 139.087622] nix-daemon[849]: reaped child process 1507, status = succeeded6314client # [ 139.141947] systemd[1]: mid.service: Deactivated successfully.6315client # [ 139.145670] systemd[1]: mid.service: Consumed 148ms CPU time over 9.549s wall clock time, 43.4M memory peak, 8.7K incoming IP traffic, 10.4K outgoing IP traffic.6316client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6317client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6318client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6319(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.24 seconds)6320subtest: interrupting the client stops the build on the worker6321client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6322client # [ 139.811454] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6323client # Running as unit: intr.service; invocation ID: 80130aa26f2b44e89d50f8d8ec0186316324client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6325worker2 # [ 139.936394] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:588406326worker2 # [ 139.941757] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6327client # [ 139.946911] nix[1432]: don't know how to build these paths:6328client # [ 139.947944] nix[1432]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6329worker2 # [ 139.955499] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:53Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06330worker2 # [ 139.957372] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:53Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16331worker2 # [ 139.961442] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588406332worker2 # [ 139.966284] nix-grpc-daemon[1739]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 1/5)6333worker2 # [ 140.029801] nix-grpc-daemon[1739]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 165 ms (attempt 2/5)6334worker2 # [ 140.196295] nix-grpc-daemon[1739]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 216 ms (attempt 3/5)6335worker2 # [ 140.413557] nix-grpc-daemon[1739]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 614 ms (attempt 4/5)6336worker2 # [ 141.056570] nix-grpc-daemon[1739]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6337worker2 # [ 141.060443] nix-grpc-daemon[1739]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6338worker2 # [ 141.076832] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588406339client # [ 141.086934] nix[1432]: copying 1 paths to 'lb:50051'...6340worker2 # [ 141.088769] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588406341client # [ 141.090142] nix[1432]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6342worker2 # [ 141.100660] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6343worker2 # [ 141.124149] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:55Z level=info event=niks3_push_started pid=18196344worker2 # [ 141.227676] nix-grpc-daemon[1819]: time=2026-09-25T15:00:55.254Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6345worker2 # [ 141.313908] nix-daemon[871]: accepted connection from pid 1831, user nix-grpc-daemon (trusted)6346worker2 # [ 141.325633] nix-daemon[871]: reaped child process 1834, status = succeeded6347lb # [ 141.354916] niks3-server[1148]: 2026/09/25 15:00:54 INFO Received push request method=POST path=/api/pushes6348worker2 # [ 141.345631] nix-grpc-daemon[1819]: time=2026-09-25T15:00:55.374Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6349worker2 # [ 141.347202] nix-grpc-daemon[1819]: time=2026-09-25T15:00:55.375Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6350lb # [ 141.397565] niks3-server[1148]: 2026/09/25 15:00:54 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6351lb # [ 141.404784] niks3-server[1148]: 2026/09/25 15:00:54 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign6352lb # [ 141.409744] niks3-server[1148]: 2026/09/25 15:00:54 INFO Signed narinfos id=51 count=16353worker2 # [ 141.392333] nix-grpc-daemon[1819]: time=2026-09-25T15:00:55.421Z level=INFO msg="Uploading 1 narinfos"6354lb # [ 141.423751] niks3-server[1148]: 2026/09/25 15:00:54 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6355lb # [ 141.429835] niks3-server[1148]: 2026/09/25 15:00:54 INFO Received complete push request method=POST path=/api/pushes/51/complete6356lb # [ 141.439946] niks3-server[1148]: 2026/09/25 15:00:54 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6357worker2 # [ 141.423188] nix-grpc-daemon[1819]: time=2026-09-25T15:00:55.451Z level=INFO msg="Upload complete. (196ms)"6358worker2 # [ 141.427265] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=0 paths=1 nar_bytes_in=4246359worker2 # [ 141.432438] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588406360worker2 # [ 141.442696] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6361worker2 # [ 141.450368] nix-daemon[871]: reaped child process 1817, status = succeeded6362worker2 # [ 141.455028] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6363client # [ 141.453314] nix[1432]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6364client: must succeed: systemctl kill -s INT intr6365worker2 # [ 143.225214] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:57Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6366client # [ 143.223452] systemd[1]: intr.service: Sent signal SIGINT to main process 1432 (nix) on client request.6367client # [ 143.228071] nix[1432]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6368client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6369client # [ 143.234710] nix[1432]: error: interrupted by the user6370client # [ 143.262705] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6371client # [ 143.265448] systemd[1]: intr.service: Failed with result 'exit-code'.6372client # [ 143.269458] systemd[1]: intr.service: Consumed 121ms CPU time over 3.456s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6373worker2 # [ 143.273612] nix-daemon[1838]: unexpected Nix daemon error: error: interrupted by the user6374worker2 # [ 143.276326] nix-daemon[871]: reaped child process 1836, status = succeeded6375worker2 # [ 143.279703] nix-daemon[871]: reaped child process 1838, status = failed with exit code 16376(finished: subtest: interrupting the client stops the build on the worker, in 3.51 seconds)6377subtest: a deploy mid-build drains: the build finishes, the new generation takes over6378client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6379client # [ 143.338844] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6380client # Running as unit: sw.service; invocation ID: d66cb82f09124a2cbd6561d0349792a06381client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.04 seconds)6382worker1 # [ 143.423863] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479546383client # [ 143.466064] nix[1453]: don't know how to build these paths:6384client # [ 143.467074] nix[1453]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6385worker2 # [ 143.474659] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:57Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06386worker2 # [ 143.480157] nix-grpc-daemon[1739]: ts=2026-09-25T15:00:57Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16387worker1 # [ 143.445531] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479546388worker1 # [ 143.458340] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479546389client # [ 143.504148] nix[1453]: copying 1 paths to 'lb:50051'...6390client # [ 143.504956] nix[1453]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6391worker1 # [ 143.469436] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:479546392worker1 # [ 143.478680] nix-daemon[849]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)6393worker1 # [ 143.584161] nix-daemon[849]: accepted connection from pid 1578, user nix-grpc-daemon (trusted)6394worker1 # [ 143.598461] nix-daemon[849]: reaped child process 1581, status = succeeded6395lb # [ 143.663846] niks3-server[1148]: 2026/09/25 15:00:56 INFO Received push request method=POST path=/api/pushes6396worker1 # [ 143.615336] nix-grpc-daemon[1534]: time=2026-09-25T15:00:57.770Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6397worker1 # [ 143.617738] nix-grpc-daemon[1534]: time=2026-09-25T15:00:57.772Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6398lb # [ 143.694338] niks3-server[1148]: 2026/09/25 15:00:57 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign6399lb # [ 143.703156] niks3-server[1148]: 2026/09/25 15:00:57 INFO Signed narinfos id=52 count=16400worker1 # [ 143.649666] nix-grpc-daemon[1534]: time=2026-09-25T15:00:57.804Z level=INFO msg="Uploading 1 narinfos"6401lb # [ 143.709157] niks3-server[1148]: 2026/09/25 15:00:57 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6402lb # [ 143.713578] niks3-server[1148]: 2026/09/25 15:00:57 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6403lb # [ 143.725465] niks3-server[1148]: 2026/09/25 15:00:57 INFO Received complete push request method=POST path=/api/pushes/52/complete6404worker1 # [ 143.676980] nix-grpc-daemon[1534]: time=2026-09-25T15:00:57.832Z level=INFO msg="Upload complete. (184ms)"6405lb # [ 143.737525] niks3-server[1148]: 2026/09/25 15:00:57 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6406worker1 # [ 143.681125] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47954 duration_s=0 paths=1 nar_bytes_in=4166407worker1 # [ 143.685973] nix-grpc-daemon[1428]: ts=2026-09-25T15:00:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:479546408worker1 # [ 143.697064] nix-daemon[849]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)6409worker1 # [ 143.701393] nix-daemon[849]: reaped child process 1576, status = succeeded6410worker1 # [ 143.706171] nix-daemon[849]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)6411client # [ 143.741586] nix[1453]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6412lb # [ 145.385727] envoy[906]: conn peer=[2001:db8:1::1]:52556 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3934 tx=1989 ms=93886413lb # [ 145.389978] envoy[906]: conn peer=[2001:db8:1::1]:45140 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2993 tx=853 ms=33156414worker1: must succeed: systemctl reload nix-grpc-daemon.service6415worker1 # [ 145.722144] systemd[1]: Reloading Nix worker-protocol over gRPC...6416worker1 # [ 145.747830] systemd[1]: Reloaded Nix worker-protocol over gRPC.6417worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6418worker1: must succeed: pgrep -f 'read -t [9]0 x'6419worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6420client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26421client # don't know how to build these paths:6422client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6423worker2 # [ 145.974568] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:588406424worker1 # [ 145.942204] nix-grpc-daemon[1428]: ts=2026-09-25T15:01:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:479546425worker2 # [ 146.001549] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:00Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06426worker2 # [ 146.006919] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:00Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16427worker2 # [ 146.014071] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588406428worker2 # [ 146.022674] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588406429client # copying 1 paths to 'lb:50051'...6430client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6431worker2 # [ 146.034751] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588406432worker2 # [ 146.047650] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6433worker2 # [ 146.150186] nix-daemon[871]: accepted connection from pid 1889, user nix-grpc-daemon (trusted)6434worker2 # [ 146.165282] nix-daemon[871]: reaped child process 1892, status = succeeded6435lb # [ 146.192563] niks3-server[1148]: 2026/09/25 15:00:59 INFO Received push request method=POST path=/api/pushes6436worker2 # [ 146.180994] nix-grpc-daemon[1819]: time=2026-09-25T15:01:00.209Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6437worker2 # [ 146.183356] nix-grpc-daemon[1819]: time=2026-09-25T15:01:00.211Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6438lb # [ 146.227619] niks3-server[1148]: 2026/09/25 15:00:59 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign6439worker2 # [ 146.213281] nix-grpc-daemon[1819]: time=2026-09-25T15:01:00.241Z level=INFO msg="Uploading 1 narinfos"6440lb # [ 146.233733] niks3-server[1148]: 2026/09/25 15:00:59 INFO Signed narinfos id=53 count=16441lb # [ 146.240348] niks3-server[1148]: 2026/09/25 15:00:59 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6442lb # [ 146.245712] niks3-server[1148]: 2026/09/25 15:00:59 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6443lb # [ 146.253274] niks3-server[1148]: 2026/09/25 15:00:59 INFO Received complete push request method=POST path=/api/pushes/53/complete6444worker2 # [ 146.239192] nix-grpc-daemon[1819]: time=2026-09-25T15:01:00.267Z level=INFO msg="Upload complete. (179ms)"6445lb # [ 146.260364] niks3-server[1148]: 2026/09/25 15:00:59 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6446worker2 # [ 146.243137] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=0 paths=1 nar_bytes_in=5286447worker2 # [ 146.249131] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588406448worker2 # [ 146.257517] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6449worker2 # [ 146.265147] nix-daemon[871]: reaped child process 1887, status = succeeded6450worker2 # [ 146.268395] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6451client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6452client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6453worker1 # [ 146.259714] nix-grpc-daemon[1428]: ts=2026-09-25T15:01:00Z level=info event=draining6454client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6455client # farm-a> LOG-farm-a-during-drain6456worker2 # [ 147.686238] nix-daemon[871]: reaped child process 1896, status = succeeded6457worker2 # [ 147.764516] nix-daemon[871]: accepted connection from pid 1912, user nix-grpc-daemon (trusted)6458worker2 # [ 147.779393] nix-daemon[871]: reaped child process 1915, status = succeeded6459lb # [ 147.805511] niks3-server[1148]: 2026/09/25 15:01:01 INFO Received push request method=POST path=/api/pushes6460worker2 # [ 147.793930] nix-grpc-daemon[1819]: time=2026-09-25T15:01:01.822Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6461worker2 # [ 147.795572] nix-grpc-daemon[1819]: time=2026-09-25T15:01:01.824Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6462lb # [ 147.843788] niks3-server[1148]: 2026/09/25 15:01:01 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6463lb # [ 147.853896] niks3-server[1148]: 2026/09/25 15:01:01 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6464lb # [ 147.863135] niks3-server[1148]: 2026/09/25 15:01:01 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign6465lb # [ 147.869952] niks3-server[1148]: 2026/09/25 15:01:01 INFO Signed narinfos id=54 count=16466worker2 # [ 147.852239] nix-grpc-daemon[1819]: time=2026-09-25T15:01:01.880Z level=INFO msg="Uploading 1 narinfos"6467lb # [ 147.876486] niks3-server[1148]: 2026/09/25 15:01:01 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6468lb # [ 147.884962] niks3-server[1148]: 2026/09/25 15:01:01 INFO Received complete push request method=POST path=/api/pushes/54/complete6469worker2 # [ 147.867492] nix-grpc-daemon[1819]: time=2026-09-25T15:01:01.896Z level=INFO msg="Upload complete. (191ms)"6470lb # [ 147.889272] niks3-server[1148]: 2026/09/25 15:01:01 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6471worker2 # [ 147.874533] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16472worker2 # [ 147.885506] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:01Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06473worker2 # [ 147.890797] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:01Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16474worker2 # [ 147.894758] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588406475worker2 # [ 147.905311] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588406476worker2 # [ 147.908111] nix-daemon[871]: reaped child process 1894, status = succeeded6477worker2 # [ 147.913729] nix-grpc-daemon[1819]: time=2026-09-25T15:01:01.942Z level=INFO msg="All 1 paths already cached"6478client # copying 1 paths to 'lb:50051'...6479worker2 # [ 147.919207] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588406480client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6481worker2 # [ 147.936289] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6482worker2 # [ 148.027060] nix-daemon[871]: accepted connection from pid 1921, user nix-grpc-daemon (trusted)6483worker2 # [ 148.040142] nix-daemon[871]: reaped child process 1924, status = succeeded6484lb # [ 148.068590] niks3-server[1148]: 2026/09/25 15:01:01 INFO Received push request method=POST path=/api/pushes6485worker2 # [ 148.056122] nix-grpc-daemon[1819]: time=2026-09-25T15:01:02.084Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6486worker2 # [ 148.057610] nix-grpc-daemon[1819]: time=2026-09-25T15:01:02.086Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6487lb # [ 148.101838] niks3-server[1148]: 2026/09/25 15:01:01 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign6488worker2 # [ 148.089753] nix-grpc-daemon[1819]: time=2026-09-25T15:01:02.118Z level=INFO msg="Uploading 1 narinfos"6489lb # [ 148.110228] niks3-server[1148]: 2026/09/25 15:01:01 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6490lb # [ 148.114547] niks3-server[1148]: 2026/09/25 15:01:01 INFO Signed narinfos id=55 count=16491lb # [ 148.121954] niks3-server[1148]: 2026/09/25 15:01:01 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6492lb # [ 148.132413] niks3-server[1148]: 2026/09/25 15:01:01 INFO Received complete push request method=POST path=/api/pushes/55/complete6493worker2 # [ 148.122046] nix-grpc-daemon[1819]: time=2026-09-25T15:01:02.150Z level=INFO msg="Upload complete. (179ms)"6494lb # [ 148.143244] niks3-server[1148]: 2026/09/25 15:01:01 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6495worker2 # [ 148.125481] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=0 paths=1 nar_bytes_in=6406496worker2 # [ 148.131388] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588406497worker2 # [ 148.141239] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6498worker2 # [ 148.149493] nix-daemon[871]: reaped child process 1919, status = succeeded6499worker2 # [ 148.154472] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6500client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6501client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6502client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6503client # farm-b> LOG-farm-b-during-drain6504worker2 # [ 149.205356] nix-daemon[871]: reaped child process 1928, status = succeeded6505worker2 # [ 149.280827] nix-daemon[871]: accepted connection from pid 1944, user nix-grpc-daemon (trusted)6506worker2 # [ 149.294262] nix-daemon[871]: reaped child process 1947, status = succeeded6507lb # [ 149.322822] niks3-server[1148]: 2026/09/25 15:01:02 INFO Received push request method=POST path=/api/pushes6508worker2 # [ 149.310560] nix-grpc-daemon[1819]: time=2026-09-25T15:01:03.339Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6509worker2 # [ 149.312857] nix-grpc-daemon[1819]: time=2026-09-25T15:01:03.340Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6510lb # [ 149.360160] niks3-server[1148]: 2026/09/25 15:01:02 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6511lb # [ 149.369834] niks3-server[1148]: 2026/09/25 15:01:02 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign6512lb # [ 149.374895] niks3-server[1148]: 2026/09/25 15:01:02 INFO Signed narinfos id=56 count=16513worker2 # [ 149.357274] nix-grpc-daemon[1819]: time=2026-09-25T15:01:03.385Z level=INFO msg="Uploading 1 narinfos"6514lb # [ 149.382380] niks3-server[1148]: 2026/09/25 15:01:02 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6515lb # [ 149.387463] niks3-server[1148]: 2026/09/25 15:01:02 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6516lb # [ 149.398974] niks3-server[1148]: 2026/09/25 15:01:02 INFO Received complete push request method=POST path=/api/pushes/56/complete6517lb # [ 149.406592] niks3-server[1148]: 2026/09/25 15:01:02 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6518worker2 # [ 149.390122] nix-grpc-daemon[1819]: time=2026-09-25T15:01:03.418Z level=INFO msg="Upload complete. (228ms)"6519worker2 # [ 149.396960] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16520worker2 # [ 149.408642] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:03Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06521worker2 # [ 149.414205] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:03Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16522worker2 # [ 149.418779] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588406523worker2 # [ 149.430263] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588406524worker2 # [ 149.432449] nix-daemon[871]: reaped child process 1926, status = succeeded6525worker2 # [ 149.438740] nix-grpc-daemon[1819]: time=2026-09-25T15:01:03.467Z level=INFO msg="All 2 paths already cached"6526client # copying 1 paths to 'lb:50051'...6527worker2 # [ 149.444676] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588406528client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6529worker2 # [ 149.457614] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6530worker2 # [ 149.547485] nix-daemon[871]: accepted connection from pid 1953, user nix-grpc-daemon (trusted)6531worker2 # [ 149.562933] nix-daemon[871]: reaped child process 1956, status = succeeded6532lb # [ 149.587670] niks3-server[1148]: 2026/09/25 15:01:02 INFO Received push request method=POST path=/api/pushes6533worker2 # [ 149.575633] nix-grpc-daemon[1819]: time=2026-09-25T15:01:03.604Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6534worker2 # [ 149.577161] nix-grpc-daemon[1819]: time=2026-09-25T15:01:03.605Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6535lb # [ 149.620357] niks3-server[1148]: 2026/09/25 15:01:02 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign6536lb # [ 149.626829] niks3-server[1148]: 2026/09/25 15:01:02 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6537worker2 # [ 149.612433] nix-grpc-daemon[1819]: time=2026-09-25T15:01:03.641Z level=INFO msg="Uploading 1 narinfos"6538lb # [ 149.632707] niks3-server[1148]: 2026/09/25 15:01:02 INFO Signed narinfos id=57 count=16539lb # [ 149.639614] niks3-server[1148]: 2026/09/25 15:01:02 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6540lb # [ 149.651400] niks3-server[1148]: 2026/09/25 15:01:02 INFO Received complete push request method=POST path=/api/pushes/57/complete6541worker2 # [ 149.640396] nix-grpc-daemon[1819]: time=2026-09-25T15:01:03.669Z level=INFO msg="Upload complete. (177ms)"6542lb # [ 149.662487] niks3-server[1148]: 2026/09/25 15:01:02 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6543worker2 # [ 149.643739] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=0 paths=1 nar_bytes_in=7686544worker2 # [ 149.649334] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588406545worker2 # [ 149.663063] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6546worker2 # [ 149.666675] nix-daemon[871]: reaped child process 1951, status = succeeded6547worker2 # [ 149.671117] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6548client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6549client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6550client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6551client # farm-top> LOG-farm-top-during-drain6552worker2 # [ 150.514624] nix-daemon[871]: reaped child process 1960, status = succeeded6553worker2 # [ 150.592269] nix-daemon[871]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6554worker2 # [ 150.607350] nix-daemon[871]: reaped child process 1979, status = succeeded6555lb # [ 150.634145] niks3-server[1148]: 2026/09/25 15:01:03 INFO Received push request method=POST path=/api/pushes6556worker2 # [ 150.621979] nix-grpc-daemon[1819]: time=2026-09-25T15:01:04.650Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6557worker2 # [ 150.623623] nix-grpc-daemon[1819]: time=2026-09-25T15:01:04.652Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6558lb # [ 150.678174] niks3-server[1148]: 2026/09/25 15:01:03 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign6559lb # [ 150.687461] niks3-server[1148]: 2026/09/25 15:01:03 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6560lb # [ 150.693883] niks3-server[1148]: 2026/09/25 15:01:04 INFO Signed narinfos id=58 count=16561worker2 # [ 150.676835] nix-grpc-daemon[1819]: time=2026-09-25T15:01:04.705Z level=INFO msg="Uploading 1 narinfos"6562lb # [ 150.699526] niks3-server[1148]: 2026/09/25 15:01:04 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6563lb # [ 150.704373] niks3-server[1148]: 2026/09/25 15:01:04 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6564lb # [ 150.715571] niks3-server[1148]: 2026/09/25 15:01:04 INFO Received complete push request method=POST path=/api/pushes/58/complete6565lb # [ 150.722083] niks3-server[1148]: 2026/09/25 15:01:04 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6566worker2 # [ 150.700965] nix-grpc-daemon[1819]: time=2026-09-25T15:01:04.729Z level=INFO msg="Upload complete. (222ms)"6567worker2 # [ 150.708174] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16568worker2 # [ 150.724229] nix-daemon[871]: reaped child process 1958, status = succeeded6569client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 4.92 seconds)6570client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6571client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.11 seconds)6572worker1: must succeed: pkill -f 'read -t [9]0 x'6573worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6574client: waiting for success: ! systemctl is-active sw6575worker1 # [ 150.929505] nix-daemon[849]: reaped child process 1585, status = succeeded6576worker1 # [ 151.005215] nix-daemon[849]: accepted connection from pid 1627, user nix-grpc-daemon (trusted)6577worker1 # [ 151.017496] nix-daemon[849]: reaped child process 1630, status = succeeded6578lb # [ 151.084208] niks3-server[1148]: 2026/09/25 15:01:04 INFO Received push request method=POST path=/api/pushes6579worker1 # [ 151.034725] nix-grpc-daemon[1534]: time=2026-09-25T15:01:05.190Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6580worker1 # [ 151.036299] nix-grpc-daemon[1534]: time=2026-09-25T15:01:05.192Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6581lb # [ 151.124981] niks3-server[1148]: 2026/09/25 15:01:04 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign6582lb # [ 151.129977] niks3-server[1148]: 2026/09/25 15:01:04 INFO Signed narinfos id=59 count=16583worker1 # [ 151.076186] nix-grpc-daemon[1534]: time=2026-09-25T15:01:05.231Z level=INFO msg="Uploading 1 narinfos"6584lb # [ 151.137223] niks3-server[1148]: 2026/09/25 15:01:04 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6585lb # [ 151.144613] niks3-server[1148]: 2026/09/25 15:01:04 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6586lb # [ 151.156112] niks3-server[1148]: 2026/09/25 15:01:04 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6587lb # [ 151.162231] niks3-server[1148]: 2026/09/25 15:01:04 INFO Received complete push request method=POST path=/api/pushes/59/complete6588lb # [ 151.169020] niks3-server[1148]: 2026/09/25 15:01:04 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6589worker1 # [ 151.115274] nix-grpc-daemon[1534]: time=2026-09-25T15:01:05.270Z level=INFO msg="Upload complete. (195ms)"6590worker1 # [ 151.122747] nix-grpc-daemon[1428]: ts=2026-09-25T15:01:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47954 duration_s=7 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16591worker1 # [ 151.137818] nix-daemon[849]: reaped child process 1583, status = succeeded6592client # [ 151.188484] systemd[1]: sw.service: Deactivated successfully.6593client # [ 151.193189] systemd[1]: sw.service: Consumed 123ms CPU time over 7.852s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic.6594worker1 # [ 151.262356] nix-grpc-daemon[1428]: ts=2026-09-25T15:01:05Z level=info event=signal_exit6595client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6596client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6597client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6598worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6599worker2 # [ 152.310279] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:06Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:587906600worker2 # [ 152.317287] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:06Z level=info event=worker_gone addr=192.168.1.3:500516601worker1 # [ 152.273951] nix-grpc-daemon[1428]: ts=2026-09-25T15:01:06Z level=info event=scheduler_restarting6602worker1 # [ 152.288982] nix-grpc-daemon[1428]: ts=2026-09-25T15:01:06Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6603worker1 # [ 152.309787] nix-daemon[849]: reaped child process 1484, status = succeeded6604worker1 # [ 153.291700] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6605worker1 # [ 153.295669] systemd[1]: nix-grpc-daemon.service: Consumed 586ms CPU time over 24.151s wall clock time, 34.8M memory peak, 47.9K incoming IP traffic, 54.6K outgoing IP traffic.6606worker1 # [ 153.308689] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6607worker1 # [ 153.318646] systemd[1]: Starting Nix worker-protocol over gRPC...6608worker1 # [ 153.411354] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6609worker1 # [ 153.413335] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6610worker1 # [ 153.416646] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16611worker1 # [ 153.421040] nix-grpc-daemon[1645]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6612worker1 # [ 153.422355] nix-grpc-daemon[1645]: E0000 00:00:1790348467.576588 1645 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6613worker1 # [ 153.427038] nix-grpc-daemon[1645]: E0000 00:00:1790348467.582017 1645 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6614worker1 # [ 153.430795] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:07Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6615worker1 # [ 153.435337] systemd[1]: Started Nix worker-protocol over gRPC.6616worker1 # [ 153.439951] nix-grpc-daemon[1645]: E0000 00:00:1790348467.594175 1687 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6617worker1 # [ 153.452393] nix-grpc-daemon[1645]: E0000 00:00:1790348467.608123 1687 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6618worker1 # [ 153.456132] nix-grpc-daemon[1645]: E0000 00:00:1790348467.610453 1687 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6619worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.51 seconds)6620lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux6621lb: must succeed: curl -sf localhost:9901/clusters6622worker1 # [ 153.467706] nix-grpc-daemon[1645]: E0000 00:00:1790348467.623398 1687 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6623worker1 # [ 153.479901] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:07Z level=info event=scheduler_connected addr=lb:500516624worker2 # [ 153.519903] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:07Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:587906625worker2 # [ 153.527460] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:07Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06626lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6627lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 0.05 seconds)6628lb: must succeed: curl -sf localhost:9901/clusters6629lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6630worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6631worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6632(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 10.31 seconds)6633subtest: queue gauges show a burst while every build blocks6634lb: must succeed: curl -sf localhost:9901/clusters6635lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6636lb: must succeed: curl -sf localhost:9901/clusters6637lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6638worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6639worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6640client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6641client # [ 153.764959] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6642client # Running as unit: burst.service; invocation ID: 9f2ed333318b40eeaab23734b6c632476643client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.06 seconds)6644lb: must succeed: curl -sf localhost:9901/clusters6645lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6646lb: must succeed: curl -sf localhost:9901/clusters6647lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6648worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6649worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6650worker1 # [ 153.871980] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376146651worker1 # [ 153.883763] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6652client # [ 153.932439] nix[1509]: don't know how to build these paths:6653client # [ 153.933373] nix[1509]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6654client # [ 153.934302] nix[1509]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6655client # [ 153.936050] nix[1509]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6656client # [ 153.937176] nix[1509]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6657worker2 # [ 153.945226] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:07Z level=debug event=want client=5 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=06658worker2 # [ 153.951264] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:07Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=16659worker1 # [ 153.916149] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376146660worker2 # [ 153.957966] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:07Z level=debug event=want client=5 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=06661worker2 # [ 153.962380] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:07Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=16662worker1 # [ 153.927260] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 1/5)6663worker2 # [ 153.967943] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:07Z level=debug event=want client=5 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=06664worker2 # [ 153.972867] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:07Z level=debug event=want client=5 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=06665worker2 # [ 153.977948] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588406666worker2 # [ 153.984723] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588406667client # [ 153.993173] nix[1509]: copying 1 paths to 'lb:50051'...6668worker2 # [ 153.995233] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588406669client # [ 153.995177] nix[1509]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'...6670worker2 # [ 154.007403] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6671worker1 # [ 153.973904] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 178 ms (attempt 2/5)6672worker2 # [ 154.101863] nix-daemon[871]: accepted connection from pid 1994, user nix-grpc-daemon (trusted)6673worker2 # [ 154.115114] nix-daemon[871]: reaped child process 1997, status = succeeded6674lb # [ 154.144151] niks3-server[1148]: 2026/09/25 15:01:07 INFO Received push request method=POST path=/api/pushes6675worker2 # [ 154.133991] nix-grpc-daemon[1819]: time=2026-09-25T15:01:08.162Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6676worker2 # [ 154.135566] nix-grpc-daemon[1819]: time=2026-09-25T15:01:08.162Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)"6677lb # [ 154.172527] niks3-server[1148]: 2026/09/25 15:01:07 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign6678lb # [ 154.179434] niks3-server[1148]: 2026/09/25 15:01:07 INFO Signed narinfos id=60 count=16679worker2 # [ 154.162350] nix-grpc-daemon[1819]: time=2026-09-25T15:01:08.190Z level=INFO msg="Uploading 1 narinfos"6680lb # [ 154.188143] niks3-server[1148]: 2026/09/25 15:01:07 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst6681lb # [ 154.200911] niks3-server[1148]: 2026/09/25 15:01:07 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls6682lb # [ 154.206779] niks3-server[1148]: 2026/09/25 15:01:07 INFO Received complete push request method=POST path=/api/pushes/60/complete6683lb # [ 154.215529] niks3-server[1148]: 2026/09/25 15:01:07 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo6684worker1 # [ 154.154289] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 392 ms (attempt 3/5)6685worker2 # [ 154.199588] nix-grpc-daemon[1819]: time=2026-09-25T15:01:08.228Z level=INFO msg="Upload complete. (183ms)"6686worker2 # [ 154.204712] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=0 paths=1 nar_bytes_in=4166687worker2 # [ 154.211345] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588406688worker2 # [ 154.222534] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6689worker2 # [ 154.231822] nix-daemon[871]: reaped child process 1992, status = succeeded6690worker2 # [ 154.236320] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6691client # [ 154.234634] nix[1509]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051...6692worker1 # [ 154.549269] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 406 ms (attempt 4/5)6693lb: must succeed: curl -sf localhost:9901/clusters6694lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6695lb: must succeed: curl -sf localhost:9901/clusters6696lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6697worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6698worker1 # [ 154.956829] nix-grpc-daemon[1645]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6699worker1 # [ 154.960323] nix-grpc-daemon[1645]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6700worker1 # [ 154.975085] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376146701client # [ 155.020872] nix[1509]: copying 1 paths to 'lb:50051'...6702client # [ 155.024494] nix[1509]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'...6703worker1 # [ 154.988894] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376146704worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6705worker1 # [ 155.001626] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6706worker1 # [ 155.029965] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:09Z level=info event=niks3_push_started pid=17096707worker1 # [ 155.134555] nix-grpc-daemon[1709]: time=2026-09-25T15:01:09.289Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6708worker1 # [ 155.224333] nix-daemon[849]: accepted connection from pid 1722, user nix-grpc-daemon (trusted)6709worker1 # [ 155.237275] nix-daemon[849]: reaped child process 1725, status = succeeded6710lb # [ 155.304866] niks3-server[1148]: 2026/09/25 15:01:08 INFO Received push request method=POST path=/api/pushes6711worker1 # [ 155.256042] nix-grpc-daemon[1709]: time=2026-09-25T15:01:09.411Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6712worker1 # [ 155.257916] nix-grpc-daemon[1709]: time=2026-09-25T15:01:09.413Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)"6713lb # [ 155.346397] niks3-server[1148]: 2026/09/25 15:01:08 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign6714lb # [ 155.351102] niks3-server[1148]: 2026/09/25 15:01:08 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst6715lb # [ 155.359359] niks3-server[1148]: 2026/09/25 15:01:08 INFO Signed narinfos id=61 count=16716worker1 # [ 155.304623] nix-grpc-daemon[1709]: time=2026-09-25T15:01:09.460Z level=INFO msg="Uploading 1 narinfos"6717lb # [ 155.367324] niks3-server[1148]: 2026/09/25 15:01:08 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls6718lb # [ 155.379221] niks3-server[1148]: 2026/09/25 15:01:08 INFO Received complete push request method=POST path=/api/pushes/61/complete6719lb # [ 155.383746] niks3-server[1148]: 2026/09/25 15:01:08 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo6720lb # [ 155.385851] envoy[906]: conn peer=[2001:db8:1::1]:55984 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9525 tx=4364 ms=47576721lb # [ 155.388087] envoy[906]: conn peer=[2001:db8:1::1]:55974 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2945 tx=1498 ms=77096722lb # [ 155.389609] envoy[906]: conn peer=[2001:db8:1::3]:35016 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1148 tx=459 ms=229836723worker1 # [ 155.334098] nix-grpc-daemon[1709]: time=2026-09-25T15:01:09.489Z level=INFO msg="Upload complete. (195ms)"6724worker1 # [ 155.338096] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=0 paths=1 nar_bytes_in=4166725worker1 # [ 155.343294] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376146726worker1 # [ 155.356065] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6727worker1 # [ 155.358648] nix-daemon[849]: reaped child process 1704, status = succeeded6728worker1 # [ 155.363876] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6729client # [ 155.399417] nix[1509]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051...6730worker2 # [ 156.483128] nix-daemon[871]: reaped child process 2001, status = succeeded6731worker2 # [ 156.561452] nix-daemon[871]: accepted connection from pid 2030, user nix-grpc-daemon (trusted)6732worker2 # [ 156.574718] nix-daemon[871]: reaped child process 2033, status = succeeded6733lb # [ 156.603498] niks3-server[1148]: 2026/09/25 15:01:09 INFO Received push request method=POST path=/api/pushes6734worker2 # [ 156.592702] nix-grpc-daemon[1819]: time=2026-09-25T15:01:10.621Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6735worker2 # [ 156.594312] nix-grpc-daemon[1819]: time=2026-09-25T15:01:10.623Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)"6736lb # [ 156.645061] niks3-server[1148]: 2026/09/25 15:01:09 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign6737worker2 # [ 156.629338] nix-grpc-daemon[1819]: time=2026-09-25T15:01:10.657Z level=INFO msg="Uploading 1 narinfos"6738lb # [ 156.650324] niks3-server[1148]: 2026/09/25 15:01:09 INFO Signed narinfos id=62 count=16739lb # [ 156.657338] niks3-server[1148]: 2026/09/25 15:01:09 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6740lb # [ 156.666237] niks3-server[1148]: 2026/09/25 15:01:09 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6741lb # [ 156.670053] niks3-server[1148]: 2026/09/25 15:01:09 INFO Received complete push request method=POST path=/api/pushes/62/complete6742lb # [ 156.673726] niks3-server[1148]: 2026/09/25 15:01:09 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls6743worker2 # [ 156.656624] nix-grpc-daemon[1819]: time=2026-09-25T15:01:10.685Z level=INFO msg="Upload complete. (211ms)"6744lb # [ 156.679456] niks3-server[1148]: 2026/09/25 15:01:09 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo6745worker2 # [ 156.666972] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=2 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=7 outputs=16746worker2 # [ 156.672429] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:10Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=16747worker2 # [ 156.677424] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588406748worker2 # [ 156.688053] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588406749worker2 # [ 156.690461] nix-daemon[871]: reaped child process 1999, status = succeeded6750worker2 # [ 156.693964] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588406751client # [ 156.693166] nix[1509]: copying 1 paths to 'lb:50051'...6752client # [ 156.696401] nix[1509]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'...6753worker2 # [ 156.699914] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6754worker2 # [ 156.790272] nix-daemon[871]: accepted connection from pid 2037, user nix-grpc-daemon (trusted)6755worker2 # [ 156.803756] nix-daemon[871]: reaped child process 2040, status = succeeded6756lb # [ 156.831785] niks3-server[1148]: 2026/09/25 15:01:10 INFO Received push request method=POST path=/api/pushes6757worker2 # [ 156.819836] nix-grpc-daemon[1819]: time=2026-09-25T15:01:10.848Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6758worker2 # [ 156.821881] nix-grpc-daemon[1819]: time=2026-09-25T15:01:10.850Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)"6759lb # [ 156.863642] niks3-server[1148]: 2026/09/25 15:01:10 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign6760worker2 # [ 156.847341] nix-grpc-daemon[1819]: time=2026-09-25T15:01:10.875Z level=INFO msg="Uploading 1 narinfos"6761lb # [ 156.867372] niks3-server[1148]: 2026/09/25 15:01:10 INFO Signed narinfos id=63 count=16762lb # [ 156.873481] niks3-server[1148]: 2026/09/25 15:01:10 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls6763lb # [ 156.881068] niks3-server[1148]: 2026/09/25 15:01:10 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst6764lb # [ 156.891352] niks3-server[1148]: 2026/09/25 15:01:10 INFO Received complete push request method=POST path=/api/pushes/63/complete6765lb # [ 156.895454] niks3-server[1148]: 2026/09/25 15:01:10 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo6766worker2 # [ 156.877834] nix-grpc-daemon[1819]: time=2026-09-25T15:01:10.906Z level=INFO msg="Upload complete. (171ms)"6767worker2 # [ 156.880083] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=0 paths=1 nar_bytes_in=4166768worker2 # [ 156.882920] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588406769worker2 # [ 156.890557] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6770worker2 # [ 156.893651] nix-daemon[871]: reaped child process 2035, status = succeeded6771worker2 # [ 156.899726] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6772client # [ 156.896958] nix[1509]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051...6773worker1 # [ 157.379405] nix-daemon[849]: reaped child process 1729, status = succeeded6774worker1 # [ 157.458906] nix-daemon[849]: accepted connection from pid 1754, user nix-grpc-daemon (trusted)6775worker1 # [ 157.472741] nix-daemon[849]: reaped child process 1757, status = succeeded6776lb # [ 157.536650] niks3-server[1148]: 2026/09/25 15:01:10 INFO Received push request method=POST path=/api/pushes6777worker1 # [ 157.488365] nix-grpc-daemon[1709]: time=2026-09-25T15:01:11.643Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6778worker1 # [ 157.489845] nix-grpc-daemon[1709]: time=2026-09-25T15:01:11.645Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)"6779lb # [ 157.578626] niks3-server[1148]: 2026/09/25 15:01:10 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign6780lb # [ 157.584542] niks3-server[1148]: 2026/09/25 15:01:10 INFO Signed narinfos id=64 count=16781worker1 # [ 157.529519] nix-grpc-daemon[1709]: time=2026-09-25T15:01:11.684Z level=INFO msg="Uploading 1 narinfos"6782lb # [ 157.591054] niks3-server[1148]: 2026/09/25 15:01:10 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6783lb # [ 157.601069] niks3-server[1148]: 2026/09/25 15:01:10 INFO Received complete push request method=POST path=/api/pushes/64/complete6784worker1 # [ 157.549390] nix-grpc-daemon[1709]: time=2026-09-25T15:01:11.705Z level=INFO msg="Upload complete. (176ms)"6785lb # [ 157.607916] niks3-server[1148]: 2026/09/25 15:01:10 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls6786worker1 # [ 157.552074] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=2 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=8 outputs=16787lb # [ 157.611512] niks3-server[1148]: 2026/09/25 15:01:10 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo6788lb # [ 157.613609] niks3-server[1148]: 2026/09/25 15:01:10 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6789worker2 # [ 157.590877] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:11Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.3:50051 clients=16790worker1 # [ 157.558730] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376146791worker1 # [ 157.563386] nix-daemon[849]: reaped child process 1727, status = succeeded6792worker1 # [ 157.566751] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376146793client # [ 157.607480] nix[1509]: copying 1 paths to 'lb:50051'...6794worker1 # [ 157.572215] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376146795client # [ 157.610138] nix[1509]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'...6796worker1 # [ 157.580668] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6797worker1 # [ 157.668983] nix-daemon[849]: accepted connection from pid 1761, user nix-grpc-daemon (trusted)6798worker1 # [ 157.682252] nix-daemon[849]: reaped child process 1764, status = succeeded6799lb # [ 157.746902] niks3-server[1148]: 2026/09/25 15:01:11 INFO Received push request method=POST path=/api/pushes6800worker1 # [ 157.697444] nix-grpc-daemon[1709]: time=2026-09-25T15:01:11.853Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6801worker1 # [ 157.699511] nix-grpc-daemon[1709]: time=2026-09-25T15:01:11.854Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)"6802lb # [ 157.776959] niks3-server[1148]: 2026/09/25 15:01:11 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign6803lb # [ 157.782422] niks3-server[1148]: 2026/09/25 15:01:11 INFO Signed narinfos id=65 count=16804worker1 # [ 157.727580] nix-grpc-daemon[1709]: time=2026-09-25T15:01:11.883Z level=INFO msg="Uploading 1 narinfos"6805lb # [ 157.788869] niks3-server[1148]: 2026/09/25 15:01:11 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst6806lb # [ 157.794936] niks3-server[1148]: 2026/09/25 15:01:11 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls6807lb # [ 157.803620] niks3-server[1148]: 2026/09/25 15:01:11 INFO Received complete push request method=POST path=/api/pushes/65/complete6808worker1 # [ 157.752213] nix-grpc-daemon[1709]: time=2026-09-25T15:01:11.907Z level=INFO msg="Upload complete. (165ms)"6809lb # [ 157.810622] niks3-server[1148]: 2026/09/25 15:01:11 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo6810worker1 # [ 157.754349] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=0 paths=1 nar_bytes_in=4166811worker1 # [ 157.757816] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376146812worker1 # [ 157.765966] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6813worker1 # [ 157.772181] nix-daemon[849]: reaped child process 1759, status = succeeded6814worker1 # [ 157.775176] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6815client # [ 157.811434] nix[1509]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051...6816worker2 # [ 158.745492] nix-daemon[871]: reaped child process 2044, status = succeeded6817worker2 # [ 158.822442] nix-daemon[871]: accepted connection from pid 2069, user nix-grpc-daemon (trusted)6818worker2 # [ 158.835450] nix-daemon[871]: reaped child process 2072, status = succeeded6819lb # [ 158.863792] niks3-server[1148]: 2026/09/25 15:01:12 INFO Received push request method=POST path=/api/pushes6820worker2 # [ 158.851209] nix-grpc-daemon[1819]: time=2026-09-25T15:01:12.879Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6821worker2 # [ 158.857062] nix-grpc-daemon[1819]: time=2026-09-25T15:01:12.881Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)"6822lb # [ 158.908259] niks3-server[1148]: 2026/09/25 15:01:12 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign6823lb # [ 158.917230] niks3-server[1148]: 2026/09/25 15:01:12 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6824lb # [ 158.925294] niks3-server[1148]: 2026/09/25 15:01:12 INFO Signed narinfos id=66 count=16825worker2 # [ 158.908222] nix-grpc-daemon[1819]: time=2026-09-25T15:01:12.936Z level=INFO msg="Uploading 1 narinfos"6826lb # [ 158.928992] niks3-server[1148]: 2026/09/25 15:01:12 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6827lb # [ 158.933369] niks3-server[1148]: 2026/09/25 15:01:12 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls6828lb # [ 158.944687] niks3-server[1148]: 2026/09/25 15:01:12 INFO Received complete push request method=POST path=/api/pushes/66/complete6829worker2 # [ 158.931475] nix-grpc-daemon[1819]: time=2026-09-25T15:01:12.960Z level=INFO msg="Upload complete. (224ms)"6830lb # [ 158.953404] niks3-server[1148]: 2026/09/25 15:01:12 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo6831worker2 # [ 158.938458] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=2 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=9 outputs=16832worker2 # [ 158.952975] nix-daemon[871]: reaped child process 2042, status = succeeded6833worker1 # [ 159.817868] nix-daemon[849]: reaped child process 1768, status = succeeded6834worker1 # [ 159.896391] nix-daemon[849]: accepted connection from pid 1793, user nix-grpc-daemon (trusted)6835worker1 # [ 159.909082] nix-daemon[849]: reaped child process 1796, status = succeeded6836lb # [ 159.975531] niks3-server[1148]: 2026/09/25 15:01:13 INFO Received push request method=POST path=/api/pushes6837worker1 # [ 159.926368] nix-grpc-daemon[1709]: time=2026-09-25T15:01:14.081Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6838worker1 # [ 159.927870] nix-grpc-daemon[1709]: time=2026-09-25T15:01:14.083Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)"6839lb # [ 160.011995] niks3-server[1148]: 2026/09/25 15:01:13 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls6840lb # [ 160.023793] niks3-server[1148]: 2026/09/25 15:01:13 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign6841worker1 # [ 159.973286] nix-grpc-daemon[1709]: time=2026-09-25T15:01:14.128Z level=INFO msg="Uploading 1 narinfos"6842lb # [ 160.030677] niks3-server[1148]: 2026/09/25 15:01:13 INFO Signed narinfos id=67 count=16843lb # [ 160.038191] niks3-server[1148]: 2026/09/25 15:01:13 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6844lb # [ 160.044116] niks3-server[1148]: 2026/09/25 15:01:13 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6845lb # [ 160.055169] niks3-server[1148]: 2026/09/25 15:01:13 INFO Received complete push request method=POST path=/api/pushes/67/complete6846worker1 # [ 160.000456] nix-grpc-daemon[1709]: time=2026-09-25T15:01:14.156Z level=INFO msg="Upload complete. (217ms)"6847lb # [ 160.060231] niks3-server[1148]: 2026/09/25 15:01:13 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo6848worker1 # [ 160.006645] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=2 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=10 outputs=16849worker1 # [ 160.021664] nix-daemon[849]: reaped child process 1766, status = succeeded6850client # [ 160.074135] systemd[1]: burst.service: Deactivated successfully.6851client # [ 160.078448] systemd[1]: burst.service: Consumed 167ms CPU time over 6.313s wall clock time, 51.9M memory peak, 14.1K incoming IP traffic, 18.9K outgoing IP traffic.6852client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6853client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds)6854lb: must succeed: curl -sf localhost:9901/clusters6855lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6856lb: must succeed: curl -sf localhost:9901/clusters6857lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6858worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6859worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6860(finished: subtest: queue gauges show a burst while every build blocks, in 7.42 seconds)6861subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6862client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16863worker1 # [ 161.143845] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:376146864worker1 # [ 161.151481] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376146865worker2 # [ 161.227917] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:15Z level=debug event=want client=6 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06866worker1 # [ 161.192944] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376146867worker2 # [ 161.232594] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:15Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16868worker2 # [ 161.236979] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:15Z level=debug event=want client=6 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06869worker2 # [ 161.241180] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:15Z level=debug event=want client=6 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06870worker1 # [ 161.204305] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376146871worker1 # [ 161.215477] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376146872worker1 # [ 161.226847] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6873worker1 # [ 161.329222] nix-daemon[849]: accepted connection from pid 1804, user nix-grpc-daemon (trusted)6874worker1 # [ 161.342164] nix-daemon[849]: reaped child process 1807, status = succeeded6875lb # [ 161.408368] niks3-server[1148]: 2026/09/25 15:01:14 INFO Received push request method=POST path=/api/pushes6876worker1 # [ 161.358914] nix-grpc-daemon[1709]: time=2026-09-25T15:01:15.514Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6877worker1 # [ 161.360362] nix-grpc-daemon[1709]: time=2026-09-25T15:01:15.514Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6878lb # [ 161.436245] niks3-server[1148]: 2026/09/25 15:01:14 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign6879lb # [ 161.440069] niks3-server[1148]: 2026/09/25 15:01:14 INFO Signed narinfos id=68 count=16880worker1 # [ 161.388185] nix-grpc-daemon[1709]: time=2026-09-25T15:01:15.543Z level=INFO msg="Uploading 1 narinfos"6881lb # [ 161.449517] niks3-server[1148]: 2026/09/25 15:01:14 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6882lb # [ 161.455115] niks3-server[1148]: 2026/09/25 15:01:14 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6883lb # [ 161.470667] niks3-server[1148]: 2026/09/25 15:01:14 INFO Received complete push request method=POST path=/api/pushes/68/complete6884worker1 # [ 161.419377] nix-grpc-daemon[1709]: time=2026-09-25T15:01:15.574Z level=INFO msg="Upload complete. (180ms)"6885lb # [ 161.477754] niks3-server[1148]: 2026/09/25 15:01:14 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6886worker1 # [ 161.423990] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=0 paths=1 nar_bytes_in=5046887worker1 # [ 161.428808] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376146888worker1 # [ 161.439081] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6889worker1 # [ 161.444187] nix-daemon[849]: reaped child process 1802, status = succeeded6890worker1 # [ 161.448879] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6891worker1 # [ 162.886579] nix-daemon[849]: reaped child process 1811, status = succeeded6892worker1 # [ 162.971133] nix-daemon[849]: accepted connection from pid 1827, user nix-grpc-daemon (trusted)6893worker1 # [ 162.984892] nix-daemon[849]: reaped child process 1830, status = succeeded6894lb # [ 163.049254] niks3-server[1148]: 2026/09/25 15:01:16 INFO Received push request method=POST path=/api/pushes6895worker1 # [ 163.001607] nix-grpc-daemon[1709]: time=2026-09-25T15:01:17.157Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6896worker1 # [ 163.003195] nix-grpc-daemon[1709]: time=2026-09-25T15:01:17.158Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6897lb # [ 163.091818] niks3-server[1148]: 2026/09/25 15:01:16 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6898lb # [ 163.098919] niks3-server[1148]: 2026/09/25 15:01:16 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign6899worker1 # [ 163.047642] nix-grpc-daemon[1709]: time=2026-09-25T15:01:17.203Z level=INFO msg="Uploading 1 narinfos"6900lb # [ 163.104908] niks3-server[1148]: 2026/09/25 15:01:16 INFO Signed narinfos id=69 count=16901lb # [ 163.111536] niks3-server[1148]: 2026/09/25 15:01:16 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6902lb # [ 163.117758] niks3-server[1148]: 2026/09/25 15:01:16 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6903lb # [ 163.126881] niks3-server[1148]: 2026/09/25 15:01:16 INFO Received complete push request method=POST path=/api/pushes/69/complete6904lb # [ 163.133519] niks3-server[1148]: 2026/09/25 15:01:16 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6905worker1 # [ 163.074431] nix-grpc-daemon[1709]: time=2026-09-25T15:01:17.230Z level=INFO msg="Upload complete. (216ms)"6906worker1 # [ 163.080872] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=11 outputs=16907worker2 # [ 163.120916] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:17Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16908worker1 # [ 163.089647] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376146909worker2 # [ 163.128764] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:17Z level=debug event=want client=6 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06910worker1 # [ 163.104662] nix-daemon[849]: reaped child process 1809, status = succeeded6911worker1 # [ 163.106857] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376146912worker1 # [ 163.115623] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376146913worker1 # [ 163.124179] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6914worker1 # [ 163.210132] nix-daemon[849]: accepted connection from pid 1834, user nix-grpc-daemon (trusted)6915worker1 # [ 163.222471] nix-daemon[849]: reaped child process 1837, status = succeeded6916lb # [ 163.288775] niks3-server[1148]: 2026/09/25 15:01:16 INFO Received push request method=POST path=/api/pushes6917worker1 # [ 163.239309] nix-grpc-daemon[1709]: time=2026-09-25T15:01:17.395Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6918worker1 # [ 163.240746] nix-grpc-daemon[1709]: time=2026-09-25T15:01:17.396Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6919lb # [ 163.318863] niks3-server[1148]: 2026/09/25 15:01:16 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign6920worker1 # [ 163.268100] nix-grpc-daemon[1709]: time=2026-09-25T15:01:17.423Z level=INFO msg="Uploading 1 narinfos"6921lb # [ 163.325995] niks3-server[1148]: 2026/09/25 15:01:16 INFO Signed narinfos id=70 count=16922lb # [ 163.333805] niks3-server[1148]: 2026/09/25 15:01:16 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6923lb # [ 163.339332] niks3-server[1148]: 2026/09/25 15:01:16 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6924lb # [ 163.351131] niks3-server[1148]: 2026/09/25 15:01:16 INFO Received complete push request method=POST path=/api/pushes/70/complete6925worker1 # [ 163.298940] nix-grpc-daemon[1709]: time=2026-09-25T15:01:17.454Z level=INFO msg="Upload complete. (169ms)"6926lb # [ 163.357641] niks3-server[1148]: 2026/09/25 15:01:16 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6927worker1 # [ 163.301837] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=0 paths=1 nar_bytes_in=5046928worker1 # [ 163.306902] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376146929worker1 # [ 163.315122] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6930worker1 # [ 163.323534] nix-daemon[849]: reaped child process 1832, status = succeeded6931worker1 # [ 163.327623] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6932worker1 # [ 164.343892] nix-daemon[849]: reaped child process 1841, status = succeeded6933worker1 # [ 164.418080] nix-daemon[849]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6934worker1 # [ 164.430668] nix-daemon[849]: reaped child process 1860, status = succeeded6935lb # [ 164.496677] niks3-server[1148]: 2026/09/25 15:01:17 INFO Received push request method=POST path=/api/pushes6936worker1 # [ 164.447350] nix-grpc-daemon[1709]: time=2026-09-25T15:01:18.602Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6937worker1 # [ 164.452211] nix-grpc-daemon[1709]: time=2026-09-25T15:01:18.602Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6938lb # [ 164.540180] niks3-server[1148]: 2026/09/25 15:01:17 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign6939worker1 # [ 164.490746] nix-grpc-daemon[1709]: time=2026-09-25T15:01:18.646Z level=INFO msg="Uploading 1 narinfos"6940lb # [ 164.549178] niks3-server[1148]: 2026/09/25 15:01:17 INFO Signed narinfos id=71 count=16941lb # [ 164.554237] niks3-server[1148]: 2026/09/25 15:01:17 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6942lb # [ 164.561751] niks3-server[1148]: 2026/09/25 15:01:17 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6943lb # [ 164.564667] niks3-server[1148]: 2026/09/25 15:01:17 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6944lb # [ 164.576220] niks3-server[1148]: 2026/09/25 15:01:17 INFO Received complete push request method=POST path=/api/pushes/71/complete6945worker1 # [ 164.525292] nix-grpc-daemon[1709]: time=2026-09-25T15:01:18.680Z level=INFO msg="Upload complete. (190ms)"6946lb # [ 164.584320] niks3-server[1148]: 2026/09/25 15:01:17 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6947worker1 # [ 164.530726] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=12 outputs=16948worker2 # [ 164.571859] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:18Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16949worker1 # [ 164.539360] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376146950worker2 # [ 164.578478] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:18Z level=debug event=want client=6 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06951worker1 # [ 164.550149] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376146952worker1 # [ 164.559108] nix-daemon[849]: reaped child process 1839, status = succeeded6953worker1 # [ 164.561143] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376146954worker1 # [ 164.565501] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6955worker1 # [ 164.651307] nix-daemon[849]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)6956worker1 # [ 164.664727] nix-daemon[849]: reaped child process 1867, status = succeeded6957lb # [ 164.728716] niks3-server[1148]: 2026/09/25 15:01:18 INFO Received push request method=POST path=/api/pushes6958worker1 # [ 164.678541] nix-grpc-daemon[1709]: time=2026-09-25T15:01:18.834Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6959worker1 # [ 164.680474] nix-grpc-daemon[1709]: time=2026-09-25T15:01:18.835Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6960lb # [ 164.755644] niks3-server[1148]: 2026/09/25 15:01:18 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign6961lb # [ 164.760629] niks3-server[1148]: 2026/09/25 15:01:18 INFO Signed narinfos id=72 count=16962worker1 # [ 164.706096] nix-grpc-daemon[1709]: time=2026-09-25T15:01:18.861Z level=INFO msg="Uploading 1 narinfos"6963lb # [ 164.768667] niks3-server[1148]: 2026/09/25 15:01:18 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6964lb # [ 164.776967] niks3-server[1148]: 2026/09/25 15:01:18 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6965lb # [ 164.785524] niks3-server[1148]: 2026/09/25 15:01:18 INFO Received complete push request method=POST path=/api/pushes/72/complete6966worker1 # [ 164.736325] nix-grpc-daemon[1709]: time=2026-09-25T15:01:18.892Z level=INFO msg="Upload complete. (166ms)"6967lb # [ 164.795640] niks3-server[1148]: 2026/09/25 15:01:18 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6968worker1 # [ 164.739812] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=0 paths=1 nar_bytes_in=5046969worker1 # [ 164.744839] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376146970worker1 # [ 164.752539] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6971worker1 # [ 164.758356] nix-daemon[849]: reaped child process 1862, status = succeeded6972worker1 # [ 164.763178] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6973lb # [ 165.386495] envoy[906]: conn peer=[2001:db8:1::1]:48172 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9677 tx=5232 ms=61536974worker1 # [ 166.006962] nix-daemon[849]: reaped child process 1871, status = succeeded6975worker1 # [ 166.082732] nix-daemon[849]: accepted connection from pid 1887, user nix-grpc-daemon (trusted)6976worker1 # [ 166.095539] nix-daemon[849]: reaped child process 1890, status = succeeded6977lb # [ 166.160432] niks3-server[1148]: 2026/09/25 15:01:19 INFO Received push request method=POST path=/api/pushes6978worker1 # [ 166.110819] nix-grpc-daemon[1709]: time=2026-09-25T15:01:20.266Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6979worker1 # [ 166.112392] nix-grpc-daemon[1709]: time=2026-09-25T15:01:20.268Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6980lb # [ 166.199563] niks3-server[1148]: 2026/09/25 15:01:19 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign6981lb # [ 166.207097] niks3-server[1148]: 2026/09/25 15:01:19 INFO Signed narinfos id=73 count=16982worker1 # [ 166.152788] nix-grpc-daemon[1709]: time=2026-09-25T15:01:20.308Z level=INFO msg="Uploading 1 narinfos"6983lb # [ 166.216057] niks3-server[1148]: 2026/09/25 15:01:19 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6984lb # [ 166.221598] niks3-server[1148]: 2026/09/25 15:01:19 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6985lb # [ 166.232137] niks3-server[1148]: 2026/09/25 15:01:19 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6986lb # [ 166.242491] niks3-server[1148]: 2026/09/25 15:01:19 INFO Received complete push request method=POST path=/api/pushes/73/complete6987lb # [ 166.247045] niks3-server[1148]: 2026/09/25 15:01:19 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6988worker1 # [ 166.188080] nix-grpc-daemon[1709]: time=2026-09-25T15:01:20.343Z level=INFO msg="Upload complete. (217ms)"6989worker1 # [ 166.194623] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=13 outputs=16990worker2 # [ 166.232906] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:20Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16991worker1 # [ 166.203748] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376146992worker2 # [ 166.242279] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:20Z level=debug event=want client=6 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06993worker1 # [ 166.214179] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376146994worker1 # [ 166.223776] nix-daemon[849]: reaped child process 1869, status = succeeded6995worker1 # [ 166.226749] nix-grpc-daemon[1709]: time=2026-09-25T15:01:20.381Z level=INFO msg="All 1 paths already cached"6996worker1 # [ 166.230129] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376146997worker1 # [ 166.235381] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)6998worker1 # [ 166.317940] nix-daemon[849]: accepted connection from pid 1894, user nix-grpc-daemon (trusted)6999worker1 # [ 166.330770] nix-daemon[849]: reaped child process 1897, status = succeeded7000lb # [ 166.395461] niks3-server[1148]: 2026/09/25 15:01:19 INFO Received push request method=POST path=/api/pushes7001worker1 # [ 166.346350] nix-grpc-daemon[1709]: time=2026-09-25T15:01:20.501Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7002worker1 # [ 166.347788] nix-grpc-daemon[1709]: time=2026-09-25T15:01:20.501Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7003lb # [ 166.424358] niks3-server[1148]: 2026/09/25 15:01:19 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign7004lb # [ 166.433244] niks3-server[1148]: 2026/09/25 15:01:19 INFO Signed narinfos id=74 count=17005worker1 # [ 166.378909] nix-grpc-daemon[1709]: time=2026-09-25T15:01:20.534Z level=INFO msg="Uploading 1 narinfos"7006lb # [ 166.437870] niks3-server[1148]: 2026/09/25 15:01:19 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7007lb # [ 166.446467] niks3-server[1148]: 2026/09/25 15:01:19 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7008lb # [ 166.457536] niks3-server[1148]: 2026/09/25 15:01:19 INFO Received complete push request method=POST path=/api/pushes/74/complete7009lb # [ 166.467059] niks3-server[1148]: 2026/09/25 15:01:19 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7010worker1 # [ 166.408750] nix-grpc-daemon[1709]: time=2026-09-25T15:01:20.564Z level=INFO msg="Upload complete. (168ms)"7011worker1 # [ 166.412753] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=0 paths=1 nar_bytes_in=6247012worker1 # [ 166.417737] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376147013worker1 # [ 166.427460] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7014worker1 # [ 166.433515] nix-daemon[849]: reaped child process 1892, status = succeeded7015worker1 # [ 166.437883] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7016worker1 # [ 167.665691] nix-daemon[849]: reaped child process 1901, status = succeeded7017worker1 # [ 167.735355] nix-daemon[849]: accepted connection from pid 1917, user nix-grpc-daemon (trusted)7018worker1 # [ 167.748390] nix-daemon[849]: reaped child process 1920, status = succeeded7019lb # [ 167.813464] niks3-server[1148]: 2026/09/25 15:01:21 INFO Received push request method=POST path=/api/pushes7020worker1 # [ 167.763821] nix-grpc-daemon[1709]: time=2026-09-25T15:01:21.919Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7021worker1 # [ 167.765327] nix-grpc-daemon[1709]: time=2026-09-25T15:01:21.921Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7022lb # [ 167.856165] niks3-server[1148]: 2026/09/25 15:01:21 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7023lb # [ 167.860733] niks3-server[1148]: 2026/09/25 15:01:21 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7024lb # [ 167.866265] niks3-server[1148]: 2026/09/25 15:01:21 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign7025lb # [ 167.870879] niks3-server[1148]: 2026/09/25 15:01:21 INFO Signed narinfos id=75 count=17026worker1 # [ 167.815729] nix-grpc-daemon[1709]: time=2026-09-25T15:01:21.971Z level=INFO msg="Uploading 1 narinfos"7027lb # [ 167.874873] niks3-server[1148]: 2026/09/25 15:01:21 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7028lb # [ 167.886890] niks3-server[1148]: 2026/09/25 15:01:21 INFO Received complete push request method=POST path=/api/pushes/75/complete7029lb # [ 167.890260] niks3-server[1148]: 2026/09/25 15:01:21 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7030worker1 # [ 167.831783] nix-grpc-daemon[1709]: time=2026-09-25T15:01:21.987Z level=INFO msg="Upload complete. (186ms)"7031worker1 # [ 167.836719] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=14 outputs=17032worker2 # [ 167.877180] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:21Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17033worker1 # [ 167.843604] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376147034worker2 # [ 167.884927] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:21Z level=debug event=want client=6 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07035worker1 # [ 167.852931] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376147036worker1 # [ 167.860787] nix-daemon[849]: reaped child process 1899, status = succeeded7037worker1 # [ 167.862323] nix-grpc-daemon[1709]: time=2026-09-25T15:01:22.018Z level=INFO msg="All 1 paths already cached"7038worker1 # [ 167.866268] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376147039worker1 # [ 167.871101] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7040worker1 # [ 167.953690] nix-daemon[849]: accepted connection from pid 1924, user nix-grpc-daemon (trusted)7041worker1 # [ 167.967101] nix-daemon[849]: reaped child process 1927, status = succeeded7042lb # [ 168.030387] niks3-server[1148]: 2026/09/25 15:01:21 INFO Received push request method=POST path=/api/pushes7043worker1 # [ 167.980911] nix-grpc-daemon[1709]: time=2026-09-25T15:01:22.136Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7044worker1 # [ 167.982917] nix-grpc-daemon[1709]: time=2026-09-25T15:01:22.138Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7045lb # [ 168.059060] niks3-server[1148]: 2026/09/25 15:01:21 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign7046lb # [ 168.064514] niks3-server[1148]: 2026/09/25 15:01:21 INFO Signed narinfos id=76 count=17047worker1 # [ 168.010222] nix-grpc-daemon[1709]: time=2026-09-25T15:01:22.165Z level=INFO msg="Uploading 1 narinfos"7048lb # [ 168.073135] niks3-server[1148]: 2026/09/25 15:01:21 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7049lb # [ 168.084998] niks3-server[1148]: 2026/09/25 15:01:21 INFO Received complete push request method=POST path=/api/pushes/76/complete7050worker1 # [ 168.035593] nix-grpc-daemon[1709]: time=2026-09-25T15:01:22.191Z level=INFO msg="Upload complete. (159ms)"7051lb # [ 168.094286] niks3-server[1148]: 2026/09/25 15:01:21 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7052worker1 # [ 168.037720] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=0 paths=1 nar_bytes_in=6247053lb # [ 168.099100] niks3-server[1148]: 2026/09/25 15:01:21 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7054worker1 # [ 168.045427] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376147055worker1 # [ 168.050181] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7056worker1 # [ 168.052555] nix-daemon[849]: reaped child process 1922, status = succeeded7057worker1 # [ 168.057771] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7058worker1 # [ 168.673240] nix-daemon[849]: reaped child process 1931, status = succeeded7059worker1 # [ 168.745060] nix-daemon[849]: accepted connection from pid 1947, user nix-grpc-daemon (trusted)7060worker1 # [ 168.758844] nix-daemon[849]: reaped child process 1950, status = succeeded7061lb # [ 168.822659] niks3-server[1148]: 2026/09/25 15:01:22 INFO Received push request method=POST path=/api/pushes7062worker1 # [ 168.773227] nix-grpc-daemon[1709]: time=2026-09-25T15:01:22.928Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7063worker1 # [ 168.774757] nix-grpc-daemon[1709]: time=2026-09-25T15:01:22.930Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7064lb # [ 168.864153] niks3-server[1148]: 2026/09/25 15:01:22 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign7065worker1 # [ 168.815853] nix-grpc-daemon[1709]: time=2026-09-25T15:01:22.971Z level=INFO msg="Uploading 1 narinfos"7066lb # [ 168.873159] niks3-server[1148]: 2026/09/25 15:01:22 INFO Signed narinfos id=77 count=17067lb # [ 168.878271] niks3-server[1148]: 2026/09/25 15:01:22 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7068lb # [ 168.882304] niks3-server[1148]: 2026/09/25 15:01:22 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7069lb # [ 168.889581] niks3-server[1148]: 2026/09/25 15:01:22 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7070lb # [ 168.898633] niks3-server[1148]: 2026/09/25 15:01:22 INFO Received complete push request method=POST path=/api/pushes/77/complete7071lb # [ 168.907638] niks3-server[1148]: 2026/09/25 15:01:22 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7072worker1 # [ 168.849366] nix-grpc-daemon[1709]: time=2026-09-25T15:01:23.005Z level=INFO msg="Upload complete. (185ms)"7073worker1 # [ 168.854753] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=0 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=15 outputs=17074worker2 # [ 168.894974] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:22Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17075worker1 # [ 168.864166] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376147076worker2 # [ 168.902701] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:22Z level=debug event=want client=6 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07077worker1 # [ 168.873406] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376147078worker1 # [ 168.882954] nix-daemon[849]: reaped child process 1929, status = succeeded7079worker1 # [ 168.884320] nix-grpc-daemon[1709]: time=2026-09-25T15:01:23.040Z level=INFO msg="All 1 paths already cached"7080worker1 # [ 168.888217] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376147081worker1 # [ 168.893075] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7082worker1 # [ 168.977806] nix-daemon[849]: accepted connection from pid 1954, user nix-grpc-daemon (trusted)7083worker1 # [ 168.992247] nix-daemon[849]: reaped child process 1957, status = succeeded7084lb # [ 169.054285] niks3-server[1148]: 2026/09/25 15:01:22 INFO Received push request method=POST path=/api/pushes7085worker1 # [ 169.004354] nix-grpc-daemon[1709]: time=2026-09-25T15:01:23.160Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7086worker1 # [ 169.006275] nix-grpc-daemon[1709]: time=2026-09-25T15:01:23.161Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7087lb # [ 169.084397] niks3-server[1148]: 2026/09/25 15:01:22 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign7088worker1 # [ 169.033355] nix-grpc-daemon[1709]: time=2026-09-25T15:01:23.188Z level=INFO msg="Uploading 1 narinfos"7089lb # [ 169.091180] niks3-server[1148]: 2026/09/25 15:01:22 INFO Signed narinfos id=78 count=17090lb # [ 169.098108] niks3-server[1148]: 2026/09/25 15:01:22 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7091lb # [ 169.102941] niks3-server[1148]: 2026/09/25 15:01:22 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7092lb # [ 169.116516] niks3-server[1148]: 2026/09/25 15:01:22 INFO Received complete push request method=POST path=/api/pushes/78/complete7093worker1 # [ 169.066244] nix-grpc-daemon[1709]: time=2026-09-25T15:01:23.221Z level=INFO msg="Upload complete. (168ms)"7094lb # [ 169.126342] niks3-server[1148]: 2026/09/25 15:01:22 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7095worker1 # [ 169.069403] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=0 paths=1 nar_bytes_in=6247096worker1 # [ 169.075603] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376147097worker1 # [ 169.087563] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7098worker1 # [ 169.090527] nix-daemon[849]: reaped child process 1952, status = succeeded7099worker1 # [ 169.095182] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7100worker1 # [ 169.936180] nix-daemon[849]: reaped child process 1961, status = succeeded7101worker1 # [ 170.012483] nix-daemon[849]: accepted connection from pid 1977, user nix-grpc-daemon (trusted)7102worker1 # [ 170.023903] nix-daemon[849]: reaped child process 1980, status = succeeded7103lb # [ 170.089535] niks3-server[1148]: 2026/09/25 15:01:23 INFO Received push request method=POST path=/api/pushes7104worker1 # [ 170.039661] nix-grpc-daemon[1709]: time=2026-09-25T15:01:24.195Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7105worker1 # [ 170.041386] nix-grpc-daemon[1709]: time=2026-09-25T15:01:24.197Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7106lb # [ 170.130908] niks3-server[1148]: 2026/09/25 15:01:23 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign7107worker1 # [ 170.082975] nix-grpc-daemon[1709]: time=2026-09-25T15:01:24.238Z level=INFO msg="Uploading 1 narinfos"7108lb # [ 170.141303] niks3-server[1148]: 2026/09/25 15:01:23 INFO Signed narinfos id=79 count=17109lb # [ 170.145994] niks3-server[1148]: 2026/09/25 15:01:23 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7110lb # [ 170.151628] niks3-server[1148]: 2026/09/25 15:01:23 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7111lb # [ 170.156340] niks3-server[1148]: 2026/09/25 15:01:23 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7112lb # [ 170.166632] niks3-server[1148]: 2026/09/25 15:01:23 INFO Received complete push request method=POST path=/api/pushes/79/complete7113lb # [ 170.173978] niks3-server[1148]: 2026/09/25 15:01:23 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7114worker1 # [ 170.119977] nix-grpc-daemon[1709]: time=2026-09-25T15:01:24.275Z level=INFO msg="Upload complete. (217ms)"7115worker2 # [ 170.165912] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:24Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17116worker1 # [ 170.127536] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=16 outputs=17117worker1 # [ 170.134400] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376147118worker2 # [ 170.173542] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:24Z level=debug event=want client=6 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07119worker1 # [ 170.145798] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376147120worker1 # [ 170.155112] nix-daemon[849]: reaped child process 1959, status = succeeded7121worker1 # [ 170.158645] nix-grpc-daemon[1709]: time=2026-09-25T15:01:24.313Z level=INFO msg="All 2 paths already cached"7122worker1 # [ 170.163249] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376147123worker1 # [ 170.167473] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7124worker1 # [ 170.250913] nix-daemon[849]: accepted connection from pid 1984, user nix-grpc-daemon (trusted)7125worker1 # [ 170.264464] nix-daemon[849]: reaped child process 1987, status = succeeded7126lb # [ 170.327607] niks3-server[1148]: 2026/09/25 15:01:23 INFO Received push request method=POST path=/api/pushes7127worker1 # [ 170.277951] nix-grpc-daemon[1709]: time=2026-09-25T15:01:24.433Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7128worker1 # [ 170.280026] nix-grpc-daemon[1709]: time=2026-09-25T15:01:24.435Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7129lb # [ 170.357231] niks3-server[1148]: 2026/09/25 15:01:23 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7130lb # [ 170.365107] niks3-server[1148]: 2026/09/25 15:01:23 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign7131worker1 # [ 170.314742] nix-grpc-daemon[1709]: time=2026-09-25T15:01:24.470Z level=INFO msg="Uploading 1 narinfos"7132lb # [ 170.372455] niks3-server[1148]: 2026/09/25 15:01:23 INFO Signed narinfos id=80 count=17133lb # [ 170.377140] niks3-server[1148]: 2026/09/25 15:01:23 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7134lb # [ 170.390329] niks3-server[1148]: 2026/09/25 15:01:23 INFO Received complete push request method=POST path=/api/pushes/80/complete7135worker1 # [ 170.340400] nix-grpc-daemon[1709]: time=2026-09-25T15:01:24.496Z level=INFO msg="Upload complete. (168ms)"7136lb # [ 170.399611] niks3-server[1148]: 2026/09/25 15:01:23 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7137worker1 # [ 170.344262] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=0 paths=1 nar_bytes_in=7527138worker1 # [ 170.349443] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376147139worker1 # [ 170.358104] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7140worker1 # [ 170.365953] nix-daemon[849]: reaped child process 1982, status = succeeded7141worker1 # [ 170.368864] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7142worker1 # [ 170.801569] nix-daemon[849]: reaped child process 1991, status = succeeded7143worker1 # [ 170.876854] nix-daemon[849]: accepted connection from pid 2007, user nix-grpc-daemon (trusted)7144worker1 # [ 170.888931] nix-daemon[849]: reaped child process 2010, status = succeeded7145lb # [ 170.952858] niks3-server[1148]: 2026/09/25 15:01:24 INFO Received push request method=POST path=/api/pushes7146worker1 # [ 170.903088] nix-grpc-daemon[1709]: time=2026-09-25T15:01:25.058Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7147worker1 # [ 170.904564] nix-grpc-daemon[1709]: time=2026-09-25T15:01:25.060Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7148lb # [ 170.993894] niks3-server[1148]: 2026/09/25 15:01:24 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign7149worker1 # [ 170.941525] nix-grpc-daemon[1709]: time=2026-09-25T15:01:25.097Z level=INFO msg="Uploading 1 narinfos"7150lb # [ 170.998930] niks3-server[1148]: 2026/09/25 15:01:24 INFO Signed narinfos id=81 count=17151lb # [ 171.007182] niks3-server[1148]: 2026/09/25 15:01:24 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7152lb # [ 171.013125] niks3-server[1148]: 2026/09/25 15:01:24 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7153lb # [ 171.025107] niks3-server[1148]: 2026/09/25 15:01:24 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7154lb # [ 171.030279] niks3-server[1148]: 2026/09/25 15:01:24 INFO Received complete push request method=POST path=/api/pushes/81/complete7155worker1 # [ 170.980126] nix-grpc-daemon[1709]: time=2026-09-25T15:01:25.135Z level=INFO msg="Upload complete. (205ms)"7156lb # [ 171.039388] niks3-server[1148]: 2026/09/25 15:01:24 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7157worker2 # [ 171.024173] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:25Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17158worker1 # [ 170.985840] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=0 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=17 outputs=17159worker1 # [ 170.994189] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376147160worker1 # [ 171.006519] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376147161worker1 # [ 171.013568] nix-daemon[849]: reaped child process 1989, status = succeeded7162worker1 # [ 171.019595] nix-grpc-daemon[1709]: time=2026-09-25T15:01:25.175Z level=INFO msg="All 2 paths already cached"7163worker1 # [ 171.026441] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376147164worker1 # [ 171.034182] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7165worker1 # [ 171.118958] nix-daemon[849]: accepted connection from pid 2014, user nix-grpc-daemon (trusted)7166worker1 # [ 171.131677] nix-daemon[849]: reaped child process 2017, status = succeeded7167lb # [ 171.197304] niks3-server[1148]: 2026/09/25 15:01:24 INFO Received push request method=POST path=/api/pushes7168worker1 # [ 171.148433] nix-grpc-daemon[1709]: time=2026-09-25T15:01:25.303Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7169worker1 # [ 171.150388] nix-grpc-daemon[1709]: time=2026-09-25T15:01:25.305Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7170lb # [ 171.227799] niks3-server[1148]: 2026/09/25 15:01:24 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7171lb # [ 171.233614] niks3-server[1148]: 2026/09/25 15:01:24 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign7172worker1 # [ 171.183320] nix-grpc-daemon[1709]: time=2026-09-25T15:01:25.338Z level=INFO msg="Uploading 1 narinfos"7173lb # [ 171.241181] niks3-server[1148]: 2026/09/25 15:01:24 INFO Signed narinfos id=82 count=17174lb # [ 171.248071] niks3-server[1148]: 2026/09/25 15:01:24 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7175lb # [ 171.259817] niks3-server[1148]: 2026/09/25 15:01:24 INFO Received complete push request method=POST path=/api/pushes/82/complete7176worker1 # [ 171.210565] nix-grpc-daemon[1709]: time=2026-09-25T15:01:25.366Z level=INFO msg="Upload complete. (170ms)"7177lb # [ 171.269469] niks3-server[1148]: 2026/09/25 15:01:24 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7178worker1 # [ 171.216367] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=0 paths=1 nar_bytes_in=7527179worker1 # [ 171.222286] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376147180worker1 # [ 171.230443] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7181worker1 # [ 171.238221] nix-daemon[849]: reaped child process 2012, status = succeeded7182worker1 # [ 171.242844] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7183worker1 # [ 172.885402] nix-daemon[849]: reaped child process 2021, status = succeeded7184worker1 # [ 172.958904] nix-daemon[849]: accepted connection from pid 2037, user nix-grpc-daemon (trusted)7185worker1 # [ 172.972311] nix-daemon[849]: reaped child process 2040, status = succeeded7186lb # [ 173.037178] niks3-server[1148]: 2026/09/25 15:01:26 INFO Received push request method=POST path=/api/pushes7187worker1 # [ 172.987937] nix-grpc-daemon[1709]: time=2026-09-25T15:01:27.143Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7188worker1 # [ 172.989412] nix-grpc-daemon[1709]: time=2026-09-25T15:01:27.145Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7189lb # [ 173.076806] niks3-server[1148]: 2026/09/25 15:01:26 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7190lb # [ 173.085454] niks3-server[1148]: 2026/09/25 15:01:26 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7191lb # [ 173.095067] niks3-server[1148]: 2026/09/25 15:01:26 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign7192lb # [ 173.099064] niks3-server[1148]: 2026/09/25 15:01:26 INFO Signed narinfos id=83 count=17193worker1 # [ 173.044099] nix-grpc-daemon[1709]: time=2026-09-25T15:01:27.199Z level=INFO msg="Uploading 1 narinfos"7194lb # [ 173.105712] niks3-server[1148]: 2026/09/25 15:01:26 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7195lb # [ 173.115954] niks3-server[1148]: 2026/09/25 15:01:26 INFO Received complete push request method=POST path=/api/pushes/83/complete7196lb # [ 173.120694] niks3-server[1148]: 2026/09/25 15:01:26 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7197worker1 # [ 173.062322] nix-grpc-daemon[1709]: time=2026-09-25T15:01:27.217Z level=INFO msg="Upload complete. (210ms)"7198worker1 # [ 173.068571] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=18 outputs=17199worker2 # [ 173.106834] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:27Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17200worker1 # [ 173.077342] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376147201worker1 # [ 173.089571] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376147202worker1 # [ 173.097606] nix-daemon[849]: reaped child process 2019, status = succeeded7203worker1 # [ 173.100376] nix-grpc-daemon[1709]: time=2026-09-25T15:01:27.255Z level=INFO msg="All 2 paths already cached"7204worker1 # [ 173.106575] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:376147205worker1 # [ 173.112217] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7206worker1 # [ 173.196396] nix-daemon[849]: accepted connection from pid 2044, user nix-grpc-daemon (trusted)7207worker1 # [ 173.209518] nix-daemon[849]: reaped child process 2047, status = succeeded7208lb # [ 173.274181] niks3-server[1148]: 2026/09/25 15:01:26 INFO Received push request method=POST path=/api/pushes7209worker1 # [ 173.224321] nix-grpc-daemon[1709]: time=2026-09-25T15:01:27.380Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7210worker1 # [ 173.226338] nix-grpc-daemon[1709]: time=2026-09-25T15:01:27.381Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7211lb # [ 173.305764] niks3-server[1148]: 2026/09/25 15:01:26 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign7212lb # [ 173.311123] niks3-server[1148]: 2026/09/25 15:01:26 INFO Signed narinfos id=84 count=17213worker1 # [ 173.256508] nix-grpc-daemon[1709]: time=2026-09-25T15:01:27.412Z level=INFO msg="Uploading 1 narinfos"7214lb # [ 173.317392] niks3-server[1148]: 2026/09/25 15:01:26 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7215lb # [ 173.332310] niks3-server[1148]: 2026/09/25 15:01:26 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7216lb # [ 173.338282] niks3-server[1148]: 2026/09/25 15:01:26 INFO Received complete push request method=POST path=/api/pushes/84/complete7217worker1 # [ 173.287723] nix-grpc-daemon[1709]: time=2026-09-25T15:01:27.443Z level=INFO msg="Upload complete. (169ms)"7218lb # [ 173.347158] niks3-server[1148]: 2026/09/25 15:01:26 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7219worker1 # [ 173.291494] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=0 paths=1 nar_bytes_in=7527220worker1 # [ 173.296799] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:376147221worker1 # [ 173.304147] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7222worker1 # [ 173.313272] nix-daemon[849]: reaped child process 2042, status = succeeded7223worker1 # [ 173.318189] nix-daemon[849]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)7224worker1 # [ 174.362439] nix-daemon[849]: reaped child process 2051, status = succeeded7225worker1 # [ 174.439181] nix-daemon[849]: accepted connection from pid 2067, user nix-grpc-daemon (trusted)7226worker1 # [ 174.453045] nix-daemon[849]: reaped child process 2070, status = succeeded7227lb # [ 174.517205] niks3-server[1148]: 2026/09/25 15:01:27 INFO Received push request method=POST path=/api/pushes7228worker1 # [ 174.469118] nix-grpc-daemon[1709]: time=2026-09-25T15:01:28.624Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7229worker1 # [ 174.470554] nix-grpc-daemon[1709]: time=2026-09-25T15:01:28.626Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7230lb # [ 174.561585] niks3-server[1148]: 2026/09/25 15:01:27 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign7231lb # [ 174.567598] niks3-server[1148]: 2026/09/25 15:01:27 INFO Signed narinfos id=85 count=17232worker1 # [ 174.516093] nix-grpc-daemon[1709]: time=2026-09-25T15:01:28.671Z level=INFO msg="Uploading 1 narinfos"7233lb # [ 174.576109] niks3-server[1148]: 2026/09/25 15:01:27 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7234lb # [ 174.583858] niks3-server[1148]: 2026/09/25 15:01:27 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7235lb # [ 174.591114] niks3-server[1148]: 2026/09/25 15:01:27 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7236lb # [ 174.603253] niks3-server[1148]: 2026/09/25 15:01:27 INFO Received complete push request method=POST path=/api/pushes/85/complete7237lb # [ 174.612431] niks3-server[1148]: 2026/09/25 15:01:27 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7238worker1 # [ 174.553608] nix-grpc-daemon[1709]: time=2026-09-25T15:01:28.709Z level=INFO msg="Upload complete. (227ms)"7239worker1 # [ 174.559962] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37614 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=19 outputs=17240worker1 # [ 174.575806] nix-daemon[849]: reaped child process 2049, status = succeeded7241client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 13.61 seconds)7242worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7243worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7244client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7245client # [ 174.724789] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7246client # Running as unit: gpu.service; invocation ID: 73bcaf9280dc4c0d9416e594def375397247client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7248lb: must succeed: curl -sf localhost:9901/clusters7249lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7250lb: must succeed: curl -sf localhost:9901/clusters7251lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7252worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7253client # [ 174.857567] nix[1583]: don't know how to build these paths:7254client # [ 174.858991] nix[1583]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7255worker2 # [ 174.856971] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:588407256worker1 # [ 174.824912] nix-grpc-daemon[1645]: ts=2026-09-25T15:01:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:376147257worker2 # [ 174.882413] nix-grpc-daemon[1739]: ts=2026-09-25T15:01:28Z level=debug event=want client=7 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07258client # [ 174.881714] nix[1583]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7259worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7260lb # [ 175.386264] envoy[906]: conn peer=[2001:db8:1::1]:48186 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27051 tx=12249 ms=134327261lb: must succeed: curl -sf localhost:9901/clusters7262lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7263lb: must succeed: curl -sf localhost:9901/clusters7264lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7265worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7266worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7267client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7268client # [ 204.884902] nix[1583]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7269client # [ 204.892578] nix[1583]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7270client # [ 204.925847] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7271client # [ 204.929420] systemd[1]: gpu.service: Failed with result 'exit-code'.7272client # [ 204.934148] systemd[1]: gpu.service: Consumed 140ms CPU time over 30.207s wall clock time, 35.6M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.7273lb # [ 205.389605] envoy[906]: conn peer=[2001:db8:1::1]:40162 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300477274client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.45 seconds)7275client: must succeed: journalctl -u gpu -o cat7276client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7277lb: must succeed: curl -sf localhost:9901/clusters7278lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7279lb: must succeed: curl -sf localhost:9901/clusters7280lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7281worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7282worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7283(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 44.57 seconds)7284subtest: low disk drains a worker and builds go to the other7285worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7286worker1 # [ 206.452208] nix-grpc-daemon[1645]: ts=2026-09-25T15:02:00Z level=info event=unhealthy reason=min_free7287worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.95 seconds)7288worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7289worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.03 seconds)7290client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27291client # don't know how to build these paths:7292client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7293worker2 # [ 206.734960] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:588407294worker2 # [ 206.740497] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588407295worker2 # [ 206.760711] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:00Z level=debug event=want client=8 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07296worker2 # [ 206.765731] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:00Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17297worker2 # [ 206.773515] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588407298worker2 # [ 206.785518] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588407299client # copying 1 paths to 'lb:50051'...7300client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7301worker2 # [ 206.798842] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588407302worker2 # [ 206.811285] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)7303worker2 # [ 206.911671] nix-daemon[871]: accepted connection from pid 2116, user nix-grpc-daemon (trusted)7304worker2 # [ 206.926057] nix-daemon[871]: reaped child process 2119, status = succeeded7305lb # [ 206.954602] niks3-server[1148]: 2026/09/25 15:02:00 INFO Received push request method=POST path=/api/pushes7306worker2 # [ 206.942401] nix-grpc-daemon[1819]: time=2026-09-25T15:02:00.970Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7307worker2 # [ 206.944243] nix-grpc-daemon[1819]: time=2026-09-25T15:02:00.972Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7308lb # [ 207.004122] niks3-server[1148]: 2026/09/25 15:02:00 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7309lb # [ 207.013757] niks3-server[1148]: 2026/09/25 15:02:00 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign7310lb # [ 207.019597] niks3-server[1148]: 2026/09/25 15:02:00 INFO Signed narinfos id=86 count=17311worker2 # [ 207.002480] nix-grpc-daemon[1819]: time=2026-09-25T15:02:01.030Z level=INFO msg="Uploading 1 narinfos"7312lb # [ 207.026663] niks3-server[1148]: 2026/09/25 15:02:00 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7313lb # [ 207.032426] niks3-server[1148]: 2026/09/25 15:02:00 INFO Received complete push request method=POST path=/api/pushes/86/complete7314lb # [ 207.037466] niks3-server[1148]: 2026/09/25 15:02:00 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7315worker2 # [ 207.016521] nix-grpc-daemon[1819]: time=2026-09-25T15:02:01.045Z level=INFO msg="Upload complete. (193ms)"7316worker2 # [ 207.022613] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=0 paths=1 nar_bytes_in=5127317worker2 # [ 207.029326] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588407318worker2 # [ 207.042247] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)7319worker2 # [ 207.050722] nix-daemon[871]: reaped child process 2114, status = succeeded7320worker2 # [ 207.058648] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)7321client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7322client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7323client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7324client # farm-a> LOG-farm-a-drain7325worker2 # [ 208.303717] nix-daemon[871]: reaped child process 2123, status = succeeded7326worker2 # [ 208.382642] nix-daemon[871]: accepted connection from pid 2139, user nix-grpc-daemon (trusted)7327worker2 # [ 208.395522] nix-daemon[871]: reaped child process 2142, status = succeeded7328lb # [ 208.423732] niks3-server[1148]: 2026/09/25 15:02:01 INFO Received push request method=POST path=/api/pushes7329worker2 # [ 208.414376] nix-grpc-daemon[1819]: time=2026-09-25T15:02:02.442Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7330worker2 # [ 208.416039] nix-grpc-daemon[1819]: time=2026-09-25T15:02:02.444Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7331lb # [ 208.466097] niks3-server[1148]: 2026/09/25 15:02:01 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign7332worker2 # [ 208.449826] nix-grpc-daemon[1819]: time=2026-09-25T15:02:02.478Z level=INFO msg="Uploading 1 narinfos"7333lb # [ 208.470948] niks3-server[1148]: 2026/09/25 15:02:01 INFO Signed narinfos id=87 count=17334lb # [ 208.475824] niks3-server[1148]: 2026/09/25 15:02:01 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7335lb # [ 208.480777] niks3-server[1148]: 2026/09/25 15:02:01 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7336lb # [ 208.488417] niks3-server[1148]: 2026/09/25 15:02:01 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7337lb # [ 208.492143] niks3-server[1148]: 2026/09/25 15:02:01 INFO Received complete push request method=POST path=/api/pushes/87/complete7338worker2 # [ 208.477263] nix-grpc-daemon[1819]: time=2026-09-25T15:02:02.505Z level=INFO msg="Upload complete. (210ms)"7339lb # [ 208.498858] niks3-server[1148]: 2026/09/25 15:02:01 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7340worker2 # [ 208.485133] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=20 outputs=17341worker2 # [ 208.493539] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:02Z level=debug event=want client=8 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07342worker2 # [ 208.497904] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:02Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17343worker2 # [ 208.501936] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588407344worker2 # [ 208.509807] nix-daemon[871]: reaped child process 2121, status = succeeded7345worker2 # [ 208.511671] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588407346worker2 # [ 208.517577] nix-grpc-daemon[1819]: time=2026-09-25T15:02:02.546Z level=INFO msg="All 1 paths already cached"7347client # copying 1 paths to 'lb:50051'...7348worker2 # [ 208.522750] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588407349client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7350worker2 # [ 208.529058] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)7351worker2 # [ 208.613833] nix-daemon[871]: accepted connection from pid 2148, user nix-grpc-daemon (trusted)7352worker2 # [ 208.627093] nix-daemon[871]: reaped child process 2151, status = succeeded7353lb # [ 208.654496] niks3-server[1148]: 2026/09/25 15:02:01 INFO Received push request method=POST path=/api/pushes7354worker2 # [ 208.643055] nix-grpc-daemon[1819]: time=2026-09-25T15:02:02.671Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7355worker2 # [ 208.645071] nix-grpc-daemon[1819]: time=2026-09-25T15:02:02.673Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7356lb # [ 208.686299] niks3-server[1148]: 2026/09/25 15:02:01 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign7357lb # [ 208.690351] niks3-server[1148]: 2026/09/25 15:02:01 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7358worker2 # [ 208.676495] nix-grpc-daemon[1819]: time=2026-09-25T15:02:02.705Z level=INFO msg="Uploading 1 narinfos"7359lb # [ 208.696664] niks3-server[1148]: 2026/09/25 15:02:02 INFO Signed narinfos id=88 count=17360lb # [ 208.704530] niks3-server[1148]: 2026/09/25 15:02:02 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7361lb # [ 208.712286] niks3-server[1148]: 2026/09/25 15:02:02 INFO Received complete push request method=POST path=/api/pushes/88/complete7362worker2 # [ 208.696531] nix-grpc-daemon[1819]: time=2026-09-25T15:02:02.725Z level=INFO msg="Upload complete. (162ms)"7363lb # [ 208.718161] niks3-server[1148]: 2026/09/25 15:02:02 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7364worker2 # [ 208.700549] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=0 paths=1 nar_bytes_in=6247365worker2 # [ 208.705263] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588407366worker2 # [ 208.713579] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)7367worker2 # [ 208.718314] nix-daemon[871]: reaped child process 2146, status = succeeded7368worker2 # [ 208.722676] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)7369client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7370client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7371client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7372client # farm-b> LOG-farm-b-drain7373worker2 # [ 209.753773] nix-daemon[871]: reaped child process 2155, status = succeeded7374worker2 # [ 209.828157] nix-daemon[871]: accepted connection from pid 2171, user nix-grpc-daemon (trusted)7375worker2 # [ 209.841232] nix-daemon[871]: reaped child process 2174, status = succeeded7376lb # [ 209.870577] niks3-server[1148]: 2026/09/25 15:02:03 INFO Received push request method=POST path=/api/pushes7377worker2 # [ 209.858628] nix-grpc-daemon[1819]: time=2026-09-25T15:02:03.887Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7378worker2 # [ 209.860186] nix-grpc-daemon[1819]: time=2026-09-25T15:02:03.888Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7379lb # [ 209.915729] niks3-server[1148]: 2026/09/25 15:02:03 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7380lb # [ 209.918839] niks3-server[1148]: 2026/09/25 15:02:03 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign7381lb # [ 209.924276] niks3-server[1148]: 2026/09/25 15:02:03 INFO Signed narinfos id=89 count=17382worker2 # [ 209.907221] nix-grpc-daemon[1819]: time=2026-09-25T15:02:03.935Z level=INFO msg="Uploading 1 narinfos"7383lb # [ 209.932968] niks3-server[1148]: 2026/09/25 15:02:03 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7384lb # [ 209.943246] niks3-server[1148]: 2026/09/25 15:02:03 INFO Received complete push request method=POST path=/api/pushes/89/complete7385worker2 # [ 209.929742] nix-grpc-daemon[1819]: time=2026-09-25T15:02:03.958Z level=INFO msg="Upload complete. (200ms)"7386lb # [ 209.953348] niks3-server[1148]: 2026/09/25 15:02:03 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7387lb # [ 209.955976] niks3-server[1148]: 2026/09/25 15:02:03 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7388worker2 # [ 209.936510] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=21 outputs=17389worker2 # [ 209.947609] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:03Z level=debug event=want client=8 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07390worker2 # [ 209.952234] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:03Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17391worker2 # [ 209.957212] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588407392worker2 # [ 209.966394] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:588407393worker2 # [ 209.968567] nix-daemon[871]: reaped child process 2153, status = succeeded7394worker2 # [ 209.974918] nix-grpc-daemon[1819]: time=2026-09-25T15:02:04.003Z level=INFO msg="All 2 paths already cached"7395client # copying 1 paths to 'lb:50051'...7396worker2 # [ 209.981367] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:588407397client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7398worker2 # [ 209.987903] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)7399worker2 # [ 210.073428] nix-daemon[871]: accepted connection from pid 2180, user nix-grpc-daemon (trusted)7400worker2 # [ 210.086267] nix-daemon[871]: reaped child process 2183, status = succeeded7401lb # [ 210.114815] niks3-server[1148]: 2026/09/25 15:02:03 INFO Received push request method=POST path=/api/pushes7402worker2 # [ 210.101720] nix-grpc-daemon[1819]: time=2026-09-25T15:02:04.130Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7403worker2 # [ 210.103711] nix-grpc-daemon[1819]: time=2026-09-25T15:02:04.131Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7404lb # [ 210.140834] niks3-server[1148]: 2026/09/25 15:02:03 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign7405lb # [ 210.146411] niks3-server[1148]: 2026/09/25 15:02:03 INFO Signed narinfos id=90 count=17406worker2 # [ 210.131708] nix-grpc-daemon[1819]: time=2026-09-25T15:02:04.160Z level=INFO msg="Uploading 1 narinfos"7407lb # [ 210.155403] niks3-server[1148]: 2026/09/25 15:02:03 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7408lb # [ 210.162654] niks3-server[1148]: 2026/09/25 15:02:03 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7409lb # [ 210.173334] niks3-server[1148]: 2026/09/25 15:02:03 INFO Received complete push request method=POST path=/api/pushes/90/complete7410worker2 # [ 210.160441] nix-grpc-daemon[1819]: time=2026-09-25T15:02:04.189Z level=INFO msg="Upload complete. (167ms)"7411lb # [ 210.182893] niks3-server[1148]: 2026/09/25 15:02:03 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7412worker2 # [ 210.163489] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=0 paths=1 nar_bytes_in=7527413worker2 # [ 210.169187] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:588407414worker2 # [ 210.178221] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)7415worker2 # [ 210.183376] nix-daemon[871]: reaped child process 2178, status = succeeded7416worker2 # [ 210.188141] nix-daemon[871]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)7417client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7418client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7419client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7420client # farm-top> LOG-farm-top-drain7421worker2 # [ 211.003526] nix-daemon[871]: reaped child process 2187, status = succeeded7422worker2 # [ 211.078810] nix-daemon[871]: accepted connection from pid 2203, user nix-grpc-daemon (trusted)7423worker2 # [ 211.091812] nix-daemon[871]: reaped child process 2206, status = succeeded7424lb # [ 211.120265] niks3-server[1148]: 2026/09/25 15:02:04 INFO Received push request method=POST path=/api/pushes7425worker2 # [ 211.107917] nix-grpc-daemon[1819]: time=2026-09-25T15:02:05.136Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7426worker2 # [ 211.110367] nix-grpc-daemon[1819]: time=2026-09-25T15:02:05.138Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7427lb # [ 211.157295] niks3-server[1148]: 2026/09/25 15:02:04 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign7428worker2 # [ 211.143467] nix-grpc-daemon[1819]: time=2026-09-25T15:02:05.172Z level=INFO msg="Uploading 1 narinfos"7429lb # [ 211.163852] niks3-server[1148]: 2026/09/25 15:02:04 INFO Signed narinfos id=91 count=17430lb # [ 211.173219] niks3-server[1148]: 2026/09/25 15:02:04 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7431lb # [ 211.179253] niks3-server[1148]: 2026/09/25 15:02:04 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7432lb # [ 211.186879] niks3-server[1148]: 2026/09/25 15:02:04 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7433lb # [ 211.194131] niks3-server[1148]: 2026/09/25 15:02:04 INFO Received complete push request method=POST path=/api/pushes/91/complete7434worker2 # [ 211.180796] nix-grpc-daemon[1819]: time=2026-09-25T15:02:05.209Z level=INFO msg="Upload complete. (186ms)"7435lb # [ 211.203150] niks3-server[1148]: 2026/09/25 15:02:04 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7436worker2 # [ 211.189677] nix-grpc-daemon[1739]: ts=2026-09-25T15:02:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58840 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=22 outputs=17437worker2 # [ 211.202950] nix-daemon[871]: reaped child process 2185, status = succeeded7438client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 4.64 seconds)7439client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7440client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds)7441worker1: must succeed: rm /nix/.rw-store/fill7442worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7443worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7444worker1 # [ 212.455835] nix-grpc-daemon[1645]: ts=2026-09-25T15:02:06Z level=info event=healthy reason=min_free7445worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.08 seconds)7446(finished: subtest: low disk drains a worker and builds go to the other, in 6.99 seconds)7447(finished: run the VM test script, in 213.66 seconds)7448test script finished in 213.83s7449cleanup7450kill QemuMachine (pid 49)7451client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7452client # [2026-09-25T15:02:07Z INFO virtiofsd] Client disconnected, shutting down7453client # [2026-09-25T15:02:07Z INFO virtiofsd] Client disconnected, shutting down7454client # [2026-09-25T15:02:07Z INFO virtiofsd] Client disconnected, shutting down7455kill QemuMachine (pid 50)7456lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7457lb # [2026-09-25T15:02:07Z INFO virtiofsd] Client disconnected, shutting down7458lb # [2026-09-25T15:02:07Z INFO virtiofsd] Client disconnected, shutting down7459lb # [2026-09-25T15:02:07Z INFO virtiofsd] Client disconnected, shutting down7460kill QemuMachine (pid 51)7461worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7462worker1 # [2026-09-25T15:02:07Z INFO virtiofsd] Client disconnected, shutting down7463worker1 # [2026-09-25T15:02:08Z INFO virtiofsd] Client disconnected, shutting down7464worker1 # [2026-09-25T15:02:08Z INFO virtiofsd] Client disconnected, shutting down7465kill QemuMachine (pid 52)7466worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7467worker2 # [2026-09-25T15:02:08Z INFO virtiofsd] Client disconnected, shutting down7468worker2 # [2026-09-25T15:02:08Z INFO virtiofsd] Client disconnected, shutting down7469worker2 # [2026-09-25T15:02:08Z INFO virtiofsd] Client disconnected, shutting down7470(finished: cleanup, in 1.14 seconds)7471additionally exposed symbols:7472 client, lb, worker1, worker2,7473 vlan1,7474 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh