vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #560
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker1 # Formatting '/build/vm-state-worker1/tmp.9b4ivb9QXN', fmt=raw size=107374182414worker1: QEMU running (pid 52)15client # Disk image does not exist, creating the virtualisation disk image...16client: QEMU running (pid 49)17lb # Disk image does not exist, creating the virtualisation disk image...18lb: QEMU running (pid 50)19worker1 # mke2fs 1.47.4 (6-Mar-2025)20worker2 # Disk image does not exist, creating the virtualisation disk image...21client # Formatting '/build/vm-state-client/tmp.LYgynqv5oX', fmt=raw size=107374182422worker1 # Discarding device blocks: 0/262144 done23worker2 # Formatting '/build/vm-state-worker2/tmp.mFRHB17Etg', fmt=raw size=107374182424client # mke2fs 1.47.4 (6-Mar-2025)25worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes26worker2 # mke2fs 1.47.4 (6-Mar-2025)27client # Discarding device blocks: 0/262144 done28worker1 # Filesystem UUID: 0dc23d11-6b60-459e-b53a-ac64a4fdb07229worker2 # Discarding device blocks: 0/262144 done30client # Creating filesystem with 262144 4k blocks and 65536 inodes31worker1 # Superblock backups stored on blocks:32worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes33client # Filesystem UUID: e6411231-0724-4220-a72b-86186914ad0234worker1 # 32768, 98304, 163840, 22937635worker2 # Filesystem UUID: b3acbe3a-80b0-4334-9956-ac15f5c1674b36client # Superblock backups stored on blocks:37worker1 # 38worker2 # Superblock backups stored on blocks:39client # 32768, 98304, 163840, 22937640worker1 # Allocating group tables: 0/8 done41worker2 # 32768, 98304, 163840, 22937642client # 43worker1 # Writing inode tables: 0/8 done44worker2 # 45client # Allocating group tables: 0/8 done46worker1 # Creating journal (8192 blocks): done47worker2 # Allocating group tables: 0/8 done48client # Writing inode tables: 0/8 done49worker1 # Writing superblocks and filesystem accounting information: 0/8 done50worker2 # Writing inode tables: 0/8 done51client # Creating journal (8192 blocks): done52worker1 # 53worker2 # Creating journal (8192 blocks): done54client # Writing superblocks and filesystem accounting information: 0/8 done55worker1 # Virtualisation disk image created.56worker2 # Writing superblocks and filesystem accounting information: 0/8 done57client # 58worker1 # Starting virtiofs daemons...59worker2 # 60client # Virtualisation disk image created.61worker1 # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)62worker2 # Virtualisation disk image created.63client # Starting virtiofs daemons...64worker1 # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether65worker2 # Starting virtiofs daemons...66client # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)67worker1 # [2026-09-25T16:49:44Z INFO virtiofsd] Waiting for vhost-user socket connection...68worker2 # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)69client # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether70worker1 # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)71worker2 # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether72client # [2026-09-25T16:49:44Z INFO virtiofsd] Waiting for vhost-user socket connection...73worker1 # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether74worker2 # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)75client # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76worker1 # [2026-09-25T16:49:44Z INFO virtiofsd] Waiting for vhost-user socket connection...77worker2 # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether78client # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79worker1 # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)80worker2 # [2026-09-25T16:49:44Z INFO virtiofsd] Waiting for vhost-user socket connection...81client # [2026-09-25T16:49:44Z INFO virtiofsd] Waiting for vhost-user socket connection...82worker1 # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether83worker2 # [2026-09-25T16:49:44Z INFO virtiofsd] Waiting for vhost-user socket connection...84client # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)85worker1 # [2026-09-25T16:49:44Z INFO virtiofsd] Waiting for vhost-user socket connection...86worker2 # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)87client # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether88worker1 # [2026-09-25T16:49:44Z INFO virtiofsd] Client connected, servicing requests89worker2 # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether90client # [2026-09-25T16:49:44Z INFO virtiofsd] Waiting for vhost-user socket connection...91worker1 # [2026-09-25T16:49:44Z INFO virtiofsd] Client connected, servicing requests92worker2 # [2026-09-25T16:49:44Z INFO virtiofsd] Waiting for vhost-user socket connection...93client # [2026-09-25T16:49:44Z INFO virtiofsd] Client connected, servicing requests94worker1 # [2026-09-25T16:49:44Z INFO virtiofsd] Client connected, servicing requests95worker2 # [2026-09-25T16:49:44Z INFO virtiofsd] Client connected, servicing requests96client # [2026-09-25T16:49:44Z INFO virtiofsd] Client connected, servicing requests97worker2: QEMU running (pid 51)98lb # Formatting '/build/vm-state-lb/tmp.nciR8Dr9Zw', fmt=raw size=107374182499worker2 # [2026-09-25T16:49:44Z INFO virtiofsd] Client connected, servicing requests100lb # mke2fs 1.47.4 (6-Mar-2025)101worker2 # [2026-09-25T16:49:44Z INFO virtiofsd] Client connected, servicing requests102lb # Discarding device blocks: 0/262144 done103(finished: start all VMs, in 0.44 seconds)104client # [2026-09-25T16:49:44Z INFO virtiofsd] Client connected, servicing requests105client: waiting for unit network-addresses-eth1.service106lb # Creating filesystem with 262144 4k blocks and 65536 inodes107client: waiting for the VM to finish booting108lb # Filesystem UUID: ebf4026a-8e21-4f76-8cc0-419079b9c8cc109lb # Superblock backups stored on blocks:110lb # 32768, 98304, 163840, 229376111lb # 112lb # Allocating group tables: 0/8 done113lb # Writing inode tables: 0/8 done114lb # Creating journal (8192 blocks): done115lb # Writing superblocks and filesystem accounting information: 0/8 done116lb # 117lb # Virtualisation disk image created.118lb # Starting virtiofs daemons...119lb # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120lb # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121lb # [2026-09-25T16:49:44Z INFO virtiofsd] Waiting for vhost-user socket connection...122lb # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123lb # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124lb # [2026-09-25T16:49:44Z INFO virtiofsd] Waiting for vhost-user socket connection...125lb # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126lb # [2026-09-25T16:49:44Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127lb # [2026-09-25T16:49:44Z INFO virtiofsd] Waiting for vhost-user socket connection...128lb # [2026-09-25T16:49:44Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-25T16:49:44Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-25T16:49:44Z INFO virtiofsd] Client connected, servicing requests131worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133worker1 # [ 0.000000] KASLR enabled134worker1 # [ 0.000000] random: crng init done135worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]136worker1 # [ 0.000000] Machine model: linux,dummy-virt137worker1 # [ 0.000000] efi: UEFI not found.138worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026139worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT140worker2 # [ 0.000000] KASLR enabled141worker2 # [ 0.000000] random: crng init done142worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]143worker2 # [ 0.000000] Machine model: linux,dummy-virt144worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]145worker2 # [ 0.000000] efi: UEFI not found.146worker1 # [ 0.000000] Zone ranges:147worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT148worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]149worker1 # [ 0.000000] DMA32 empty150worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]151worker1 # [ 0.000000] Normal empty152worker1 # [ 0.000000] Device empty153worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]154worker1 # [ 0.000000] Movable zone start for each node155worker2 # [ 0.000000] Zone ranges:156worker1 # [ 0.000000] Early memory node ranges157worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]158worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]159worker2 # [ 0.000000] DMA32 empty160worker2 # [ 0.000000] Normal empty161worker2 # [ 0.000000] Device empty162worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]163worker2 # [ 0.000000] Movable zone start for each node164worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000165worker2 # [ 0.000000] Early memory node ranges166worker1 # [ 0.000000] psci: probing for conduit method from DT.167worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]168worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.169worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs170worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]171worker1 # [ 0.000000] psci: Trusted OS migration not required172worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000173worker1 # [ 0.000000] psci: SMC Calling Convention v1.1174worker2 # [ 0.000000] psci: probing for conduit method from DT.175worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.176worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)177worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs178worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296179worker2 # [ 0.000000] psci: Trusted OS migration not required180worker1 # [ 0.000000] Detected PIPT I-cache on CPU0181worker2 # [ 0.000000] psci: SMC Calling Convention v1.1182worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)183worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)184worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface185worker1 # [ 0.000000] CPU features: detected: Spectre-v4186worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296187worker1 # [ 0.000000] CPU features: detected: Spectre-BHB188client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]189worker2 # [ 0.000000] Detected PIPT I-cache on CPU0190worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38191lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]192worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)193worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23194client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026195worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface196worker1 # [ 0.000000] alternatives: applying boot alternatives197client # [ 0.000000] KASLR enabled198worker2 # [ 0.000000] CPU features: detected: Spectre-v4199client # [ 0.000000] random: crng init done200lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026201worker2 # [ 0.000000] CPU features: detected: Spectre-BHB202client # [ 0.000000] Machine model: linux,dummy-virt203lb # [ 0.000000] KASLR enabled204client # [ 0.000000] efi: UEFI not found.205lb # [ 0.000000] random: crng init done206worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38207lb # [ 0.000000] Machine model: linux,dummy-virt208client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT209lb # [ 0.000000] efi: UEFI not found.210worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23211worker2 # [ 0.000000] alternatives: applying boot alternatives212client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]213lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT214client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]215worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ba0nmjmsbhgbfdzhzicck71gpd895w8k-nixos-system-worker1-test/init regInfo=/nix/store/1q4l380nqnpf2xw3l92zimnz7hcmn7cb-closure-info/registration console=ttyAMA0,115200n8 console=tty0216lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]217client # [ 0.000000] Zone ranges:218lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]219client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]220lb # [ 0.000000] Zone ranges:221client # [ 0.000000] DMA32 empty222client # [ 0.000000] Normal empty223lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]224client # [ 0.000000] Device empty225lb # [ 0.000000] DMA32 empty226worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/1q4l380nqnpf2xw3l92zimnz7hcmn7cb-closure-info/registration", will be passed to user space.227lb # [ 0.000000] Normal empty228client # [ 0.000000] Movable zone start for each node229worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/i80vw3564hcqphgl1py5adb0i6j2chim-nixos-system-worker2-test/init regInfo=/nix/store/nq7ncka0vcy66qxi29xd2wv7rqxnmdx3-closure-info/registration console=ttyAMA0,115200n8 console=tty0230client # [ 0.000000] Early memory node ranges231lb # [ 0.000000] Device empty232worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes233lb # [ 0.000000] Movable zone start for each node234client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]235lb # [ 0.000000] Early memory node ranges236worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)237worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/nq7ncka0vcy66qxi29xd2wv7rqxnmdx3-closure-info/registration", will be passed to user space.238client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]239lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]240worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)241client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000242worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes243lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]244client # [ 0.000000] psci: probing for conduit method from DT.245worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB246lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000247worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)248client # [ 0.000000] psci: PSCIv1.3 detected in firmware.249worker1 # [ 0.000000] software IO TLB: area num 1.250lb # [ 0.000000] psci: probing for conduit method from DT.251client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs252lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.253worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)254worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB255client # [ 0.000000] psci: Trusted OS migration not required256lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs257worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB258client # [ 0.000000] psci: SMC Calling Convention v1.1259worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)260lb # [ 0.000000] psci: Trusted OS migration not required261worker2 # [ 0.000000] software IO TLB: area num 1.262worker1 # [ 0.000000] Fallback order for Node 0: 0263lb # [ 0.000000] psci: SMC Calling Convention v1.1264client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)265worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB266worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216267worker1 # [ 0.000000] Policy zone: DMA268lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)269client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296270worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)271client # [ 0.000000] Detected PIPT I-cache on CPU0272worker2 # [ 0.000000] Fallback order for Node 0: 0273worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off274lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296275lb # [ 0.000000] Detected PIPT I-cache on CPU0276client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)277worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216278worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1279worker2 # [ 0.000000] Policy zone: DMA280worker1 # [ 0.000000] allocated 3145728 bytes of page_ext281client # [ 0.000000] CPU features: detected: GICv3 CPU interface282lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)283client # [ 0.000000] CPU features: detected: Spectre-v4284worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off285worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages286lb # [ 0.000000] CPU features: detected: GICv3 CPU interface287client # [ 0.000000] CPU features: detected: Spectre-BHB288worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups289worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1290lb # [ 0.000000] CPU features: detected: Spectre-v4291worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.292client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38293worker2 # [ 0.000000] allocated 3145728 bytes of page_ext294lb # [ 0.000000] CPU features: detected: Spectre-BHB295worker1 # [ 0.000000] rcu: RCU event tracing is enabled.296client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23297worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages298lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38299worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.300worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups301client # [ 0.000000] alternatives: applying boot alternatives302lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23303worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.304worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.305lb # [ 0.000000] alternatives: applying boot alternatives306worker2 # [ 0.000000] rcu: RCU event tracing is enabled.307worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.308worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.309worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.310worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.311worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.312worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.313worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1314worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.315client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/h9dnvjlwhvqgr67r3c1v5biw7c9s8za9-nixos-system-client-test/init regInfo=/nix/store/smslndp4zdg0b5gg48lp2k1sffc61w0f-closure-info/registration console=ttyAMA0,115200n8 console=tty0316worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.317worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.318lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5hmvppg1fs8xkiccvl0m50n4dxjz92fr-nixos-system-lb-test/init regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration console=ttyAMA0,115200n8 console=tty0319worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1320worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.321client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/smslndp4zdg0b5gg48lp2k1sffc61w0f-closure-info/registration", will be passed to user space.322worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.323worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.324client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes325lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration", will be passed to user space.326worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0327worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.328lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes329worker1 # [ 0.000000] GICv3: 256 SPIs implemented330client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)331worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented332worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.333lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)334client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)335worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq336worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0337worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI338worker2 # [ 0.000000] GICv3: 256 SPIs implemented339client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB340lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)341worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0342worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented343client # [ 0.000000] software IO TLB: area num 1.344lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB345worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq346worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000347client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB348lb # [ 0.000000] software IO TLB: area num 1.349worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI350worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]351lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB352worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0353client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)354client # [ 0.000000] Fallback order for Node 0: 0355worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)356worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000357lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)358client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216359worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]360lb # [ 0.000000] Fallback order for Node 0: 0361client # [ 0.000000] Policy zone: DMA362worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)363lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216364worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)365client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off366lb # [ 0.000000] Policy zone: DMA367worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000368client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1369lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off370worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000371worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)372client # [ 0.000000] allocated 3145728 bytes of page_ext373lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1374worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000375worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.376client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages377lb # [ 0.000000] allocated 3145728 bytes of page_ext378client # [ 0.000000] ftrace: allocated 294 pages with 4 groups379worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000380lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages381worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns382client # [ 0.000000] rcu: Hierarchical RCU implementation.383lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups384worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.385client # [ 0.000000] rcu: RCU event tracing is enabled.386worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).387lb # [ 0.000000] rcu: Hierarchical RCU implementation.388client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.389lb # [ 0.000000] rcu: RCU event tracing is enabled.390worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns391worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns392client # [ 0.000000] Trampoline variant of Tasks RCU enabled.393lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.394worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).395client # [ 0.000000] Rude variant of Tasks RCU enabled.396lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.397worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns398client # [ 0.000000] Tracing variant of Tasks RCU enabled.399worker1 # [ 0.000033] arm-pv: using stolen time PV400lb # [ 0.000000] Rude variant of Tasks RCU enabled.401worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns402lb # [ 0.000000] Tracing variant of Tasks RCU enabled.403client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.404worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns405worker1 # [ 0.000535] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)406client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1407lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.408worker2 # [ 0.000032] arm-pv: using stolen time PV409worker1 # [ 0.000697] Console: colour dummy device 80x25410worker1 # [ 0.000705] printk: legacy console [tty0] enabled411lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1412client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.413worker2 # [ 0.000527] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)414worker2 # [ 0.000726] Console: colour dummy device 80x25415lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.416client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.417worker1 # [ 0.000908] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)418worker2 # [ 0.000733] printk: legacy console [tty0] enabled419worker1 # [ 0.000915] pid_max: default: 32768 minimum: 301420lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.421client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.422worker1 # [ 0.001009] LSM: initializing lsm=capability,landlock,yama,bpf,ima423worker2 # [ 0.000932] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)424client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0425worker1 # [ 0.001175] landlock: Up and running.426lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.427worker2 # [ 0.000939] pid_max: default: 32768 minimum: 301428worker1 # [ 0.001178] Yama: becoming mindful.429client # [ 0.000000] GICv3: 256 SPIs implemented430lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0431worker1 # [ 0.001757] LSM support for eBPF active432client # [ 0.000000] GICv3: 0 Extended SPIs implemented433worker2 # [ 0.001017] LSM: initializing lsm=capability,landlock,yama,bpf,ima434lb # [ 0.000000] GICv3: 256 SPIs implemented435client # [ 0.000000] Root IRQ handler: gic_handle_irq436worker2 # [ 0.001172] landlock: Up and running.437worker1 # [ 0.001908] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)438lb # [ 0.000000] GICv3: 0 Extended SPIs implemented439worker2 # [ 0.001175] Yama: becoming mindful.440client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI441lb # [ 0.000000] Root IRQ handler: gic_handle_irq442worker2 # [ 0.001719] LSM support for eBPF active443client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0444worker1 # [ 0.001944] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)445lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI446worker2 # [ 0.001885] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)447worker1 # [ 0.003899] rcu: Hierarchical SRCU implementation.448lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0449client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000450client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]451worker1 # [ 0.003905] rcu: Max phase no-delay instances is 1000.452worker2 # [ 0.001925] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)453lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000454worker1 # [ 0.005202] fsl-mc MSI: its@8080000 domain created455lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]456worker2 # [ 0.003884] rcu: Hierarchical SRCU implementation.457client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)458worker1 # [ 0.005294] EFI services will not be available.459worker2 # [ 0.003889] rcu: Max phase no-delay instances is 1000.460worker1 # [ 0.005424] smp: Bringing up secondary CPUs ...461lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)462worker2 # [ 0.005386] fsl-mc MSI: its@8080000 domain created463worker1 # [ 0.005432] smp: Brought up 1 node, 1 CPU464client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)465worker2 # [ 0.005479] EFI services will not be available.466worker1 # [ 0.005435] SMP: Total of 1 processors activated.467client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000468lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)469worker2 # [ 0.005561] smp: Bringing up secondary CPUs ...470worker1 # [ 0.005440] CPU: All CPU(s) started at EL1471worker2 # [ 0.005570] smp: Brought up 1 node, 1 CPU472lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000473client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000474worker1 # [ 0.005451] CPU features: detected: Branch Target Identification475worker2 # [ 0.005573] SMP: Total of 1 processors activated.476lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000477client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.478worker2 # [ 0.005578] CPU: All CPU(s) started at EL1479worker1 # [ 0.005455] CPU features: detected: ARMv8.4 Translation Table Level480worker2 # [ 0.005590] CPU features: detected: Branch Target Identification481lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.482client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns483worker1 # [ 0.005458] CPU features: detected: Instruction cache invalidation not required for I/D coherence484worker2 # [ 0.005595] CPU features: detected: ARMv8.4 Translation Table Level485client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).486lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns487worker1 # [ 0.005462] CPU features: detected: Data cache clean to the PoU not required for I/D coherence488worker2 # [ 0.005598] CPU features: detected: Instruction cache invalidation not required for I/D coherence489lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).490worker1 # [ 0.005466] CPU features: detected: Common not Private translations491client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns492worker2 # [ 0.005601] CPU features: detected: Data cache clean to the PoU not required for I/D coherence493worker1 # [ 0.005469] CPU features: detected: CRC32 instructions494lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns495client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns496worker2 # [ 0.005605] CPU features: detected: Common not Private translations497worker1 # [ 0.005472] CPU features: detected: Data cache clean to Point of Deep Persistence498client # [ 0.000029] arm-pv: using stolen time PV499lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns500worker2 # [ 0.005608] CPU features: detected: CRC32 instructions501lb # [ 0.000031] arm-pv: using stolen time PV502worker1 # [ 0.005475] CPU features: detected: Data cache clean to Point of Persistence503client # [ 0.000379] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)504worker2 # [ 0.005611] CPU features: detected: Data cache clean to Point of Deep Persistence505worker1 # [ 0.005479] CPU features: detected: Data independent timing control (DIT)506client # [ 0.000570] Console: colour dummy device 80x25507lb # [ 0.000415] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)508worker1 # [ 0.005482] CPU features: detected: E0PD509worker2 # [ 0.005614] CPU features: detected: Data cache clean to Point of Persistence510client # [ 0.000578] printk: legacy console [tty0] enabled511lb # [ 0.000617] Console: colour dummy device 80x25512worker1 # [ 0.005485] CPU features: detected: Enhanced Counter Virtualization513worker2 # [ 0.005617] CPU features: detected: Data independent timing control (DIT)514lb # [ 0.000625] printk: legacy console [tty0] enabled515worker2 # [ 0.005620] CPU features: detected: E0PD516client # [ 0.000787] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)517worker1 # [ 0.005488] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)518worker2 # [ 0.005623] CPU features: detected: Enhanced Counter Virtualization519client # [ 0.000793] pid_max: default: 32768 minimum: 301520lb # [ 0.000820] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)521worker1 # [ 0.005491] CPU features: detected: Enhanced Virtualization Traps522client # [ 0.000878] LSM: initializing lsm=capability,landlock,yama,bpf,ima523lb # [ 0.000826] pid_max: default: 32768 minimum: 301524worker2 # [ 0.005626] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)525worker1 # [ 0.005495] CPU features: detected: Fine Grained Traps526client # [ 0.001011] landlock: Up and running.527lb # [ 0.000902] LSM: initializing lsm=capability,landlock,yama,bpf,ima528worker2 # [ 0.005629] CPU features: detected: Enhanced Virtualization Traps529client # [ 0.001014] Yama: becoming mindful.530lb # [ 0.001033] landlock: Up and running.531worker1 # [ 0.005499] CPU features: detected: Generic authentication (architected QARMA5 algorithm)532client # [ 0.001437] LSM support for eBPF active533worker2 # [ 0.005633] CPU features: detected: Fine Grained Traps534lb # [ 0.001036] Yama: becoming mindful.535worker1 # [ 0.005504] CPU features: detected: RCpc load-acquire (LDAPR)536lb # [ 0.001484] LSM support for eBPF active537client # [ 0.001595] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)538worker2 # [ 0.005636] CPU features: detected: Generic authentication (architected QARMA5 algorithm)539worker1 # [ 0.005506] CPU features: detected: LSE atomic instructions540lb # [ 0.001606] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)541worker2 # [ 0.005641] CPU features: detected: RCpc load-acquire (LDAPR)542client # [ 0.001629] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)543worker1 # [ 0.005509] CPU features: detected: Privileged Access Never544client # [ 0.003520] rcu: Hierarchical SRCU implementation.545worker1 # [ 0.005512] CPU features: detected: PMUv3546lb # [ 0.001640] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)547client # [ 0.003525] rcu: Max phase no-delay instances is 1000.548lb # [ 0.003579] rcu: Hierarchical SRCU implementation.549worker1 # [ 0.005515] CPU features: detected: RAS Extension Support550client # [ 0.004794] fsl-mc MSI: its@8080000 domain created551lb # [ 0.003584] rcu: Max phase no-delay instances is 1000.552worker1 # [ 0.005517] CPU features: detected: RASv1p1 Extension Support553client # [ 0.004884] EFI services will not be available.554lb # [ 0.004850] fsl-mc MSI: its@8080000 domain created555worker1 # [ 0.005520] CPU features: detected: Random Number Generator556client # [ 0.004950] smp: Bringing up secondary CPUs ...557lb # [ 0.004945] EFI services will not be available.558client # [ 0.004958] smp: Brought up 1 node, 1 CPU559worker1 # [ 0.005523] CPU features: detected: Speculation barrier (SB)560lb # [ 0.005011] smp: Bringing up secondary CPUs ...561client # [ 0.004961] SMP: Total of 1 processors activated.562lb # [ 0.005019] smp: Brought up 1 node, 1 CPU563worker1 # [ 0.005526] CPU features: detected: Stage-2 Force Write-Back564client # [ 0.004966] CPU: All CPU(s) started at EL1565lb # [ 0.005022] SMP: Total of 1 processors activated.566worker1 # [ 0.005528] CPU features: detected: TLB range maintenance instructions567client # [ 0.004976] CPU features: detected: Branch Target Identification568lb # [ 0.005027] CPU: All CPU(s) started at EL1569worker1 # [ 0.005534] CPU features: detected: Speculative Store Bypassing Safe (SSBS)570client # [ 0.004980] CPU features: detected: ARMv8.4 Translation Table Level571lb # [ 0.005038] CPU features: detected: Branch Target Identification572worker1 # [ 0.005576] alternatives: applying system-wide alternatives573lb # [ 0.005042] CPU features: detected: ARMv8.4 Translation Table Level574client # [ 0.004983] CPU features: detected: Instruction cache invalidation not required for I/D coherence575worker1 # [ 0.008597] CPU features: detected: BBM Level 2 without TLB conflict abort576lb # [ 0.005045] CPU features: detected: Instruction cache invalidation not required for I/D coherence577client # [ 0.004987] CPU features: detected: Data cache clean to the PoU not required for I/D coherence578worker2 # [ 0.005644] CPU features: detected: LSE atomic instructions579lb # [ 0.005049] CPU features: detected: Data cache clean to the PoU not required for I/D coherence580client # [ 0.004991] CPU features: detected: Common not Private translations581worker2 # [ 0.005647] CPU features: detected: Privileged Access Never582worker1 # [ 0.008800] Memory: 1405672K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)583worker2 # [ 0.005650] CPU features: detected: PMUv3584client # [ 0.004994] CPU features: detected: CRC32 instructions585lb # [ 0.005053] CPU features: detected: Common not Private translations586worker2 # [ 0.005652] CPU features: detected: RAS Extension Support587lb # [ 0.005056] CPU features: detected: CRC32 instructions588client # [ 0.004997] CPU features: detected: Data cache clean to Point of Deep Persistence589worker2 # [ 0.005655] CPU features: detected: RASv1p1 Extension Support590lb # [ 0.005059] CPU features: detected: Data cache clean to Point of Deep Persistence591client # [ 0.005000] CPU features: detected: Data cache clean to Point of Persistence592worker2 # [ 0.005658] CPU features: detected: Random Number Generator593lb # [ 0.005062] CPU features: detected: Data cache clean to Point of Persistence594client # [ 0.005004] CPU features: detected: Data independent timing control (DIT)595worker2 # [ 0.005660] CPU features: detected: Speculation barrier (SB)596client # [ 0.005007] CPU features: detected: E0PD597worker2 # [ 0.005663] CPU features: detected: Stage-2 Force Write-Back598lb # [ 0.005065] CPU features: detected: Data independent timing control (DIT)599client # [ 0.005009] CPU features: detected: Enhanced Counter Virtualization600lb # [ 0.005068] CPU features: detected: E0PD601worker2 # [ 0.005666] CPU features: detected: TLB range maintenance instructions602client # [ 0.005012] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)603lb # [ 0.005071] CPU features: detected: Enhanced Counter Virtualization604worker2 # [ 0.005671] CPU features: detected: Speculative Store Bypassing Safe (SSBS)605client # [ 0.005016] CPU features: detected: Enhanced Virtualization Traps606lb # [ 0.005074] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)607worker2 # [ 0.005711] alternatives: applying system-wide alternatives608client # [ 0.005019] CPU features: detected: Fine Grained Traps609lb # [ 0.005078] CPU features: detected: Enhanced Virtualization Traps610worker2 # [ 0.008681] CPU features: detected: BBM Level 2 without TLB conflict abort611lb # [ 0.005080] CPU features: detected: Fine Grained Traps612client # [ 0.005022] CPU features: detected: Generic authentication (architected QARMA5 algorithm)613client # [ 0.005027] CPU features: detected: RCpc load-acquire (LDAPR)614worker1 # [ 0.009245] devtmpfs: initialized615lb # [ 0.005084] CPU features: detected: Generic authentication (architected QARMA5 algorithm)616worker2 # [ 0.008894] Memory: 1405660K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)617client # [ 0.005030] CPU features: detected: LSE atomic instructions618worker2 # [ 0.009396] devtmpfs: initialized619worker1 # [ 0.011048] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)620lb # [ 0.005088] CPU features: detected: RCpc load-acquire (LDAPR)621client # [ 0.005033] CPU features: detected: Privileged Access Never622lb # [ 0.005091] CPU features: detected: LSE atomic instructions623worker2 # [ 0.011179] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)624client # [ 0.005036] CPU features: detected: PMUv3625worker1 # [ 0.011071] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).626lb # [ 0.005094] CPU features: detected: Privileged Access Never627client # [ 0.005039] CPU features: detected: RAS Extension Support628worker1 # [ 0.011243] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL629worker2 # [ 0.011201] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).630lb # [ 0.005097] CPU features: detected: PMUv3631client # [ 0.005041] CPU features: detected: RASv1p1 Extension Support632worker1 # [ 0.011247] 0 pages in range for non-PLT usage633lb # [ 0.005100] CPU features: detected: RAS Extension Support634worker2 # [ 0.011382] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL635worker1 # [ 0.011248] 508272 pages in range for PLT usage636client # [ 0.005044] CPU features: detected: Random Number Generator637worker2 # [ 0.011386] 0 pages in range for non-PLT usage638lb # [ 0.005103] CPU features: detected: RASv1p1 Extension Support639worker1 # [ 0.011343] pinctrl core: initialized pinctrl subsystem640client # [ 0.005047] CPU features: detected: Speculation barrier (SB)641worker2 # [ 0.011387] 508272 pages in range for PLT usage642lb # [ 0.005105] CPU features: detected: Random Number Generator643worker1 # [ 0.012261] DMI not present or invalid.644worker2 # [ 0.011499] pinctrl core: initialized pinctrl subsystem645client # [ 0.005049] CPU features: detected: Stage-2 Force Write-Back646lb # [ 0.005108] CPU features: detected: Speculation barrier (SB)647worker1 # [ 0.015508] NET: Registered PF_NETLINK/PF_ROUTE protocol family648worker2 # [ 0.012271] DMI not present or invalid.649client # [ 0.005052] CPU features: detected: TLB range maintenance instructions650lb # [ 0.005110] CPU features: detected: Stage-2 Force Write-Back651worker2 # [ 0.015648] NET: Registered PF_NETLINK/PF_ROUTE protocol family652worker1 # [ 0.017796] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations653lb # [ 0.005113] CPU features: detected: TLB range maintenance instructions654client # [ 0.005057] CPU features: detected: Speculative Store Bypassing Safe (SSBS)655worker2 # [ 0.017923] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations656worker1 # [ 0.017993] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations657client # [ 0.005092] alternatives: applying system-wide alternatives658lb # [ 0.005118] CPU features: detected: Speculative Store Bypassing Safe (SSBS)659worker2 # [ 0.018120] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations660worker1 # [ 0.018192] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations661client # [ 0.008115] CPU features: detected: BBM Level 2 without TLB conflict abort662lb # [ 0.005154] alternatives: applying system-wide alternatives663worker1 # [ 0.018213] audit: initializing netlink subsys (disabled)664worker2 # [ 0.018319] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations665worker1 # [ 0.018768] thermal_sys: Registered thermal governor 'fair_share'666worker2 # [ 0.018342] audit: initializing netlink subsys (disabled)667worker1 # [ 0.018770] thermal_sys: Registered thermal governor 'bang_bang'668worker2 # [ 0.018871] thermal_sys: Registered thermal governor 'fair_share'669worker1 # [ 0.018774] thermal_sys: Registered thermal governor 'step_wise'670worker2 # [ 0.018873] thermal_sys: Registered thermal governor 'bang_bang'671worker1 # [ 0.018777] thermal_sys: Registered thermal governor 'user_space'672worker2 # [ 0.018877] thermal_sys: Registered thermal governor 'step_wise'673worker2 # [ 0.018880] thermal_sys: Registered thermal governor 'user_space'674worker1 # [ 0.018782] thermal_sys: Registered thermal governor 'power_allocator'675worker2 # [ 0.018884] thermal_sys: Registered thermal governor 'power_allocator'676worker1 # [ 0.018812] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1677worker1 # [ 0.018820] cpuidle: using governor ladder678worker2 # [ 0.018910] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1679worker1 # [ 0.018826] cpuidle: using governor menu680worker2 # [ 0.018918] cpuidle: using governor ladder681worker2 # [ 0.018924] cpuidle: using governor menu682worker1 # [ 0.019021] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.683worker1 # [ 0.019036] ASID allocator initialised with 65536 entries684worker2 # [ 0.019109] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.685worker1 # [ 0.020267] Serial: AMBA PL011 UART driver686worker2 # [ 0.019124] ASID allocator initialised with 65536 entries687lb # [ 0.008120] CPU features: detected: BBM Level 2 without TLB conflict abort688worker2 # [ 0.020362] Serial: AMBA PL011 UART driver689worker1 # [ 0.025487] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1690client # [ 0.008278] Memory: 1405648K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)691client # [ 0.008656] devtmpfs: initialized692worker1 # [ 0.025637] printk: console [ttyAMA0] enabled693worker2 # [ 0.025618] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1694lb # [ 0.008281] Memory: 1405880K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)695worker2 # [ 0.025737] printk: console [ttyAMA0] enabled696lb # [ 0.008674] devtmpfs: initialized697client # [ 0.010404] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)698worker1 # [ 0.153565] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages699lb # [ 0.010573] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)700client # [ 0.010427] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).701worker2 # [ 0.151809] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages702worker1 # [ 0.153585] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page703client # [ 0.010627] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL704lb # [ 0.010597] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).705worker2 # [ 0.151827] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page706worker1 # [ 0.153591] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages707client # [ 0.010631] 0 pages in range for non-PLT usage708lb # [ 0.010808] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL709worker2 # [ 0.151832] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages710client # [ 0.010632] 508272 pages in range for PLT usage711worker1 # [ 0.153595] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page712lb # [ 0.010813] 0 pages in range for non-PLT usage713worker2 # [ 0.151837] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page714client # [ 0.010760] pinctrl core: initialized pinctrl subsystem715lb # [ 0.010814] 508272 pages in range for PLT usage716worker1 # [ 0.153600] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages717client # [ 0.011523] DMI not present or invalid.718worker2 # [ 0.151841] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages719lb # [ 0.010917] pinctrl core: initialized pinctrl subsystem720worker1 # [ 0.153604] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page721client # [ 0.014845] NET: Registered PF_NETLINK/PF_ROUTE protocol family722lb # [ 0.011646] DMI not present or invalid.723worker2 # [ 0.151846] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page724worker1 # [ 0.153609] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages725lb # [ 0.015046] NET: Registered PF_NETLINK/PF_ROUTE protocol family726client # [ 0.017120] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations727worker2 # [ 0.151850] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages728worker1 # [ 0.153613] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page729lb # [ 0.017294] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations730client # [ 0.017294] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations731worker2 # [ 0.151854] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page732lb # [ 0.017487] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations733client # [ 0.017492] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations734client # [ 0.017513] audit: initializing netlink subsys (disabled)735lb # [ 0.017684] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations736worker1 # [ 0.161303] fbcon: Taking over console737worker2 # [ 0.159355] fbcon: Taking over console738client # [ 0.018128] thermal_sys: Registered thermal governor 'fair_share'739lb # [ 0.017705] audit: initializing netlink subsys (disabled)740worker1 # [ 0.161317] ACPI: Interpreter disabled.741worker2 # [ 0.159371] ACPI: Interpreter disabled.742client # [ 0.018130] thermal_sys: Registered thermal governor 'bang_bang'743worker1 # [ 0.163187] iommu: Default domain type: Translated744lb # [ 0.018310] thermal_sys: Registered thermal governor 'fair_share'745client # [ 0.018134] thermal_sys: Registered thermal governor 'step_wise'746worker1 # [ 0.163198] iommu: DMA domain TLB invalidation policy: strict mode747lb # [ 0.018312] thermal_sys: Registered thermal governor 'bang_bang'748client # [ 0.018137] thermal_sys: Registered thermal governor 'user_space'749lb # [ 0.018316] thermal_sys: Registered thermal governor 'step_wise'750worker1 # [ 0.165009] SCSI subsystem initialized751client # [ 0.018142] thermal_sys: Registered thermal governor 'power_allocator'752lb # [ 0.018319] thermal_sys: Registered thermal governor 'user_space'753lb # [ 0.018324] thermal_sys: Registered thermal governor 'power_allocator'754client # [ 0.018168] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1755client # [ 0.018176] cpuidle: using governor ladder756lb # [ 0.018350] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1757client # [ 0.018182] cpuidle: using governor menu758lb # [ 0.018359] cpuidle: using governor ladder759client # [ 0.018358] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.760lb # [ 0.018364] cpuidle: using governor menu761client # [ 0.018372] ASID allocator initialised with 65536 entries762lb # [ 0.018551] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.763client # [ 0.019602] Serial: AMBA PL011 UART driver764lb # [ 0.018565] ASID allocator initialised with 65536 entries765lb # [ 0.019750] Serial: AMBA PL011 UART driver766client # [ 0.024883] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1767worker2 # [ 0.168292] iommu: Default domain type: Translated768client # [ 0.025009] printk: console [ttyAMA0] enabled769worker2 # [ 0.168303] iommu: DMA domain TLB invalidation policy: strict mode770lb # [ 0.025023] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1771lb # [ 0.025184] printk: console [ttyAMA0] enabled772worker2 # [ 0.170148] SCSI subsystem initialized773client # [ 0.151418] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages774worker1 # [ 0.172816] usbcore: registered new interface driver usbfs775client # [ 0.151437] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page776lb # [ 0.151548] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages777worker2 # [ 0.170835] usbcore: registered new interface driver usbfs778worker1 # [ 0.172842] usbcore: registered new interface driver hub779lb # [ 0.151568] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page780client # [ 0.151443] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages781worker2 # [ 0.170868] usbcore: registered new interface driver hub782worker1 # [ 0.172866] usbcore: registered new device driver usb783worker2 # [ 0.170892] usbcore: registered new device driver usb784client # [ 0.151447] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page785lb # [ 0.151574] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages786worker1 # [ 0.173142] pps_core: LinuxPPS API ver. 1 registered787worker2 # [ 0.171183] pps_core: LinuxPPS API ver. 1 registered788lb # [ 0.151578] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page789client # [ 0.151452] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages790worker1 # [ 0.173149] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>791worker2 # [ 0.171189] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>792lb # [ 0.151582] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages793client # [ 0.151456] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page794worker1 # [ 0.173159] PTP clock support registered795worker1 # [ 0.173208] EDAC MC: Ver: 3.0.0796worker2 # [ 0.171200] PTP clock support registered797lb # [ 0.151587] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page798client # [ 0.151460] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages799worker2 # [ 0.171242] EDAC MC: Ver: 3.0.0800worker1 # [ 0.177980] scmi_core: SCMI protocol bus registered801lb # [ 0.151592] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages802client # [ 0.151465] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page803worker2 # [ 0.175937] scmi_core: SCMI protocol bus registered804lb # [ 0.151596] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page805worker1 # [ 0.178941] FPGA manager framework806worker1 # [ 0.179977] vgaarb: loaded807worker2 # [ 0.176964] FPGA manager framework808worker2 # [ 0.177983] vgaarb: loaded809client # [ 0.159187] fbcon: Taking over console810client # [ 0.159204] ACPI: Interpreter disabled.811worker1 # [ 0.180626] clocksource: Switched to clocksource arch_sys_counter812lb # [ 0.159149] fbcon: Taking over console813lb # [ 0.159164] ACPI: Interpreter disabled.814worker2 # [ 0.178614] clocksource: Switched to clocksource arch_sys_counter815worker1 # [ 0.182046] VFS: Disk quotas dquot_6.6.0816client # [ 0.161150] iommu: Default domain type: Translated817worker2 # [ 0.179225] VFS: Disk quotas dquot_6.6.0818lb # [ 0.161095] iommu: Default domain type: Translated819worker1 # [ 0.182076] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)820client # [ 0.161160] iommu: DMA domain TLB invalidation policy: strict mode821worker2 # [ 0.179257] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)822lb # [ 0.161105] iommu: DMA domain TLB invalidation policy: strict mode823worker1 # [ 0.183744] netfs: FS-Cache loaded824client # [ 0.162895] SCSI subsystem initialized825worker1 # [ 0.183866] pnp: PnP ACPI: disabled826lb # [ 0.162825] SCSI subsystem initialized827worker2 # [ 0.183814] netfs: FS-Cache loaded828worker2 # [ 0.183937] pnp: PnP ACPI: disabled829client # [ 0.168019] usbcore: registered new interface driver usbfs830lb # [ 0.167956] usbcore: registered new interface driver usbfs831client # [ 0.168059] usbcore: registered new interface driver hub832client # [ 0.168083] usbcore: registered new device driver usb833lb # [ 0.167988] usbcore: registered new interface driver hub834client # [ 0.168380] pps_core: LinuxPPS API ver. 1 registered835lb # [ 0.168004] usbcore: registered new device driver usb836lb # [ 0.168284] pps_core: LinuxPPS API ver. 1 registered837worker1 # [ 0.189963] NET: Registered PF_INET protocol family838client # [ 0.168387] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>839worker2 # [ 0.187880] NET: Registered PF_INET protocol family840client # [ 0.168397] PTP clock support registered841lb # [ 0.168291] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>842worker1 # [ 0.190233] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)843client # [ 0.168446] EDAC MC: Ver: 3.0.0844lb # [ 0.168301] PTP clock support registered845worker2 # [ 0.188158] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)846lb # [ 0.168357] EDAC MC: Ver: 3.0.0847client # [ 0.173313] scmi_core: SCMI protocol bus registered848lb # [ 0.173287] scmi_core: SCMI protocol bus registered849client # [ 0.174302] FPGA manager framework850client # [ 0.175350] vgaarb: loaded851lb # [ 0.174323] FPGA manager framework852lb # [ 0.175315] vgaarb: loaded853client # [ 0.175994] clocksource: Switched to clocksource arch_sys_counter854lb # [ 0.175962] clocksource: Switched to clocksource arch_sys_counter855client # [ 0.182272] VFS: Disk quotas dquot_6.6.0856lb # [ 0.182274] VFS: Disk quotas dquot_6.6.0857client # [ 0.182310] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)858lb # [ 0.182312] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)859client # [ 0.184098] netfs: FS-Cache loaded860client # [ 0.184223] pnp: PnP ACPI: disabled861lb # [ 0.184030] netfs: FS-Cache loaded862lb # [ 0.184132] pnp: PnP ACPI: disabled863client # [ 0.188246] NET: Registered PF_INET protocol family864lb # [ 0.188052] NET: Registered PF_INET protocol family865client # [ 0.188516] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)866lb # [ 0.188324] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)867worker1 # [ 0.219766] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)868worker2 # [ 0.217606] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)869worker1 # [ 0.219809] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)870worker2 # [ 0.217650] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)871worker1 # [ 0.219840] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)872worker2 # [ 0.217676] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)873worker1 # [ 0.219911] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)874worker2 # [ 0.217746] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)875worker1 # [ 0.220055] TCP: Hash tables configured (established 16384 bind 16384)876worker2 # [ 0.217888] TCP: Hash tables configured (established 16384 bind 16384)877worker1 # [ 0.220137] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)878worker2 # [ 0.217968] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)879worker1 # [ 0.220172] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)880worker2 # [ 0.218004] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)881worker1 # [ 0.220208] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)882worker2 # [ 0.218040] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)883worker1 # [ 0.220318] NET: Registered PF_UNIX/PF_LOCAL protocol family884worker2 # [ 0.218141] NET: Registered PF_UNIX/PF_LOCAL protocol family885worker1 # [ 0.220339] NET: Registered PF_XDP protocol family886worker2 # [ 0.218183] NET: Registered PF_XDP protocol family887worker1 # [ 0.220357] PCI: CLS 0 bytes, default 64888worker2 # [ 0.218201] PCI: CLS 0 bytes, default 64889worker1 # [ 0.220606] Trying to unpack rootfs image as initramfs...890worker2 # [ 0.218470] Trying to unpack rootfs image as initramfs...891worker1 # [ 0.238891] kvm [1]: HYP mode not available892worker2 # [ 0.236750] kvm [1]: HYP mode not available893client # [ 0.218271] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)894client # [ 0.218315] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)895client # [ 0.218344] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)896client # [ 0.218413] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)897client # [ 0.218556] TCP: Hash tables configured (established 16384 bind 16384)898client # [ 0.218639] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)899lb # [ 0.218974] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)900client # [ 0.218703] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)901lb # [ 0.219017] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)902client # [ 0.218740] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)903lb # [ 0.219047] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)904client # [ 0.218833] NET: Registered PF_UNIX/PF_LOCAL protocol family905lb # [ 0.219116] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)906client # [ 0.218861] NET: Registered PF_XDP protocol family907client # [ 0.218879] PCI: CLS 0 bytes, default 64908lb # [ 0.219259] TCP: Hash tables configured (established 16384 bind 16384)909client # [ 0.219113] Trying to unpack rootfs image as initramfs...910lb # [ 0.219342] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)911lb # [ 0.219378] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)912lb # [ 0.219414] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)913client # [ 0.234151] kvm [1]: HYP mode not available914lb # [ 0.219519] NET: Registered PF_UNIX/PF_LOCAL protocol family915lb # [ 0.219559] NET: Registered PF_XDP protocol family916lb # [ 0.219577] PCI: CLS 0 bytes, default 64917lb # [ 0.219833] Trying to unpack rootfs image as initramfs...918lb # [ 0.238270] kvm [1]: HYP mode not available919worker2 # [ 0.323300] Initialise system trusted keyrings920worker2 # [ 0.324057] workingset: timestamp_bits=42 max_order=19 bucket_order=0921worker2 # [ 0.325316] squashfs: version 4.0 (2009/01/31) Phillip Lougher922worker2 # [ 0.326115] 9p: Installing v9fs 9p2000 file system support923worker1 # [ 0.333197] Initialise system trusted keyrings924worker1 # [ 0.333954] workingset: timestamp_bits=42 max_order=19 bucket_order=0925worker1 # [ 0.335214] squashfs: version 4.0 (2009/01/31) Phillip Lougher926worker1 # [ 0.336042] 9p: Installing v9fs 9p2000 file system support927client # [ 0.328933] Initialise system trusted keyrings928client # [ 0.329696] workingset: timestamp_bits=42 max_order=19 bucket_order=0929client # [ 0.330963] squashfs: version 4.0 (2009/01/31) Phillip Lougher930worker2 # [ 0.350159] Key type asymmetric registered931client # [ 0.331761] 9p: Installing v9fs 9p2000 file system support932worker2 # [ 0.350189] Asymmetric key parser 'x509' registered933lb # [ 0.332507] Initialise system trusted keyrings934worker2 # [ 0.350263] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)935lb # [ 0.333282] workingset: timestamp_bits=42 max_order=19 bucket_order=0936lb # [ 0.334634] squashfs: version 4.0 (2009/01/31) Phillip Lougher937lb # [ 0.335436] 9p: Installing v9fs 9p2000 file system support938worker1 # [ 0.364913] Key type asymmetric registered939worker2 # [ 0.358661] io scheduler mq-deadline registered940worker2 # [ 0.358681] io scheduler kyber registered941worker1 # [ 0.364941] Asymmetric key parser 'x509' registered942worker1 # [ 0.365019] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)943worker2 # [ 0.363956] pl061_gpio 9030000.pl061: PL061 GPIO chip registered944worker1 # [ 0.367239] io scheduler mq-deadline registered945worker1 # [ 0.367252] io scheduler kyber registered946worker2 # [ 0.365336] ledtrig-cpu: registered to indicate activity on CPUs947worker2 # [ 0.365720] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:948worker2 # [ 0.365738] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000949worker2 # [ 0.365752] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000950worker2 # [ 0.365761] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000951worker2 # [ 0.365793] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits952worker2 # [ 0.365817] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]953worker2 # [ 0.365894] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00954worker2 # [ 0.365904] pci_bus 0000:00: root bus resource [bus 00-ff]955worker1 # [ 0.376763] pl061_gpio 9030000.pl061: PL061 GPIO chip registered956worker2 # [ 0.365911] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]957worker2 # [ 0.365916] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]958worker2 # [ 0.365921] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]959worker2 # [ 0.365981] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint960worker1 # [ 0.378237] ledtrig-cpu: registered to indicate activity on CPUs961client # [ 0.360588] Key type asymmetric registered962worker2 # [ 0.366437] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint963worker1 # [ 0.378606] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:964client # [ 0.360616] Asymmetric key parser 'x509' registered965worker1 # [ 0.378624] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000966client # [ 0.360685] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)967client # [ 0.362930] io scheduler mq-deadline registered968worker1 # [ 0.378637] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000969client # [ 0.362942] io scheduler kyber registered970worker1 # [ 0.378646] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000971lb # [ 0.364284] Key type asymmetric registered972worker1 # [ 0.378681] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits973lb # [ 0.364311] Asymmetric key parser 'x509' registered974worker1 # [ 0.378707] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]975lb # [ 0.364374] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)976worker1 # [ 0.378802] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00977lb # [ 0.366612] io scheduler mq-deadline registered978lb # [ 0.366624] io scheduler kyber registered979worker1 # [ 0.378812] pci_bus 0000:00: root bus resource [bus 00-ff]980worker1 # [ 0.378818] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]981worker1 # [ 0.378823] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]982worker1 # [ 0.378829] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]983worker1 # [ 0.378883] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint984worker1 # [ 0.379344] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint985client # [ 0.372138] pl061_gpio 9030000.pl061: PL061 GPIO chip registered986worker1 # [ 0.379537] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]987worker1 # [ 0.379554] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]988worker1 # [ 0.379584] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]989worker1 # [ 0.379603] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]990worker2 # [ 0.386698] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]991worker2 # [ 0.386728] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]992worker1 # [ 0.380098] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint993lb # [ 0.376087] pl061_gpio 9030000.pl061: PL061 GPIO chip registered994worker1 # [ 0.380290] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]995worker2 # [ 0.386762] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]996worker1 # [ 0.380306] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]997worker2 # [ 0.386779] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]998worker1 # [ 0.380337] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]999worker2 # [ 0.387291] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1000client # [ 0.373612] ledtrig-cpu: registered to indicate activity on CPUs1001worker2 # [ 0.387486] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1002lb # [ 0.377509] ledtrig-cpu: registered to indicate activity on CPUs1003worker2 # [ 0.387502] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1004client # [ 0.373977] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1005lb # [ 0.377898] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1006worker2 # [ 0.387533] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1007client # [ 0.373995] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001008worker2 # [ 0.388012] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1009lb # [ 0.377983] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001010client # [ 0.374008] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001011worker2 # [ 0.388200] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1012lb # [ 0.378007] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001013worker1 # [ 0.401180] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1014worker2 # [ 0.388217] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1015client # [ 0.374017] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001016worker1 # [ 0.401378] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1017lb # [ 0.378017] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001018worker2 # [ 0.388247] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1019worker1 # [ 0.401394] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1020client # [ 0.374043] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1021worker2 # [ 0.388708] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1022lb # [ 0.378050] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1023worker1 # [ 0.401425] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1024client # [ 0.374066] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1025worker2 # [ 0.388900] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1026lb # [ 0.378074] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1027worker1 # [ 0.401893] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1028client # [ 0.374151] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001029worker2 # [ 0.388916] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1030worker1 # [ 0.402086] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1031lb # [ 0.378163] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001032client # [ 0.374161] pci_bus 0000:00: root bus resource [bus 00-ff]1033worker2 # [ 0.388946] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1034worker1 # [ 0.402102] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1035lb # [ 0.378174] pci_bus 0000:00: root bus resource [bus 00-ff]1036client # [ 0.374167] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1037worker2 # [ 0.388966] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1038worker1 # [ 0.402132] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1039lb # [ 0.378180] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1040client # [ 0.374172] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1041worker2 # [ 0.389505] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1042worker1 # [ 0.402151] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1043lb # [ 0.378186] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1044client # [ 0.374177] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1045worker2 # [ 0.389698] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1046lb # [ 0.378192] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1047worker1 # [ 0.402624] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1048client # [ 0.374232] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1049worker2 # [ 0.389730] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1050worker1 # [ 0.402819] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1051lb # [ 0.378278] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1052client # [ 0.374755] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1053worker2 # [ 0.390198] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1054worker1 # [ 0.402850] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1055lb # [ 0.378728] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1056client # [ 0.374950] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1057worker2 # [ 0.390393] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1058lb # [ 0.378919] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1059worker1 # [ 0.403352] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1060client # [ 0.374967] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1061worker2 # [ 0.390424] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1062worker1 # [ 0.403550] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1063lb # [ 0.378936] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1064client # [ 0.374997] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1065worker2 # [ 0.390877] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1066lb # [ 0.378967] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1067worker1 # [ 0.403581] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1068client # [ 0.375013] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1069worker2 # [ 0.391064] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1070lb # [ 0.378983] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1071worker1 # [ 0.404045] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1072client # [ 0.375486] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1073worker2 # [ 0.391319] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1074worker1 # [ 0.404238] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1075lb # [ 0.379453] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1076client # [ 0.375677] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1077worker2 # [ 0.391512] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1078lb # [ 0.379641] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1079client # [ 0.375693] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1080worker1 # [ 0.404497] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1081worker2 # [ 0.391543] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1082lb # [ 0.379657] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1083client # [ 0.375722] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1084lb # [ 0.379688] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1085worker2 # [ 0.392014] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1086client # [ 0.376208] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1087worker2 # [ 0.392207] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1088client # [ 0.376396] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1089lb # [ 0.399549] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1090worker2 # [ 0.392237] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1091client # [ 0.376411] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1092lb # [ 0.399742] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1093client # [ 0.376442] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1094worker2 # [ 0.392699] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1095lb # [ 0.399758] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1096worker2 # [ 0.392894] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1097client # [ 0.376906] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1098lb # [ 0.399789] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1099worker2 # [ 0.392925] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100client # [ 0.377095] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1101client # [ 0.377111] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1102worker2 # [ 0.393401] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1103client # [ 0.377141] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1104worker2 # [ 0.393691] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1105worker2 # [ 0.393709] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1106client # [ 0.377159] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1107worker2 # [ 0.393739] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1108client # [ 0.377631] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1109worker1 # [ 0.424759] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1110client # [ 0.377824] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1111worker2 # [ 0.394209] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1112worker1 # [ 0.424802] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1113worker2 # [ 0.394398] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1114client # [ 0.377854] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1115worker2 # [ 0.394414] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1116worker1 # [ 0.425308] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1117client # [ 0.378320] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1118worker1 # [ 0.425512] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1119worker2 # [ 0.394444] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1120client # [ 0.378514] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1121worker1 # [ 0.425542] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1122client # [ 0.378544] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1123worker1 # [ 0.426012] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1124client # [ 0.378999] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1125lb # [ 0.408352] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1126worker1 # [ 0.426207] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1127client # [ 0.379188] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1128lb # [ 0.408563] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1129worker1 # [ 0.426238] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1130client # [ 0.379434] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1131lb # [ 0.408580] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1132worker1 # [ 0.426719] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1133client # [ 0.379628] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1134lb # [ 0.408611] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1135worker1 # [ 0.426998] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1136client # [ 0.379658] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1137lb # [ 0.408630] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1138worker1 # [ 0.427017] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1139lb # [ 0.409099] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1140worker1 # [ 0.427047] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1141lb # [ 0.409292] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1142worker1 # [ 0.427522] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1143lb # [ 0.409323] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1144worker1 # [ 0.427721] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1145lb # [ 0.409790] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1146worker1 # [ 0.427738] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1147worker2 # [ 0.435128] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1148lb # [ 0.410046] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1149worker1 # [ 0.427768] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1150worker2 # [ 0.435151] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1151lb # [ 0.410078] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1152worker1 # [ 0.428346] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1153worker2 # [ 0.435157] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1154worker1 # [ 0.428357] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1155lb # [ 0.410478] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1156lb # [ 0.410666] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1157worker2 # [ 0.435209] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1158worker1 # [ 0.428363] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1159lb # [ 0.410955] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1160worker2 # [ 0.435258] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1161worker1 # [ 0.428411] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1162lb # [ 0.411150] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1163worker2 # [ 0.435307] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1164worker1 # [ 0.428459] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1165lb # [ 0.411181] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1166worker2 # [ 0.435355] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1167worker1 # [ 0.428508] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1168lb # [ 0.411643] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1169worker2 # [ 0.435403] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1170worker1 # [ 0.428557] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1171lb # [ 0.411836] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1172worker2 # [ 0.435451] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1173lb # [ 0.411866] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1174worker1 # [ 0.428606] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1175worker2 # [ 0.435499] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1176worker2 # [ 0.435546] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1177worker2 # [ 0.435593] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1178worker2 # [ 0.435702] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1179lb # [ 0.429055] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1180worker2 # [ 0.435751] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1181lb # [ 0.429252] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1182worker2 # [ 0.435773] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1183lb # [ 0.429283] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1184worker2 # [ 0.435795] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1185client # [ 0.428255] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1186lb # [ 0.429759] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1187worker2 # [ 0.435817] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1188client # [ 0.428472] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1189lb # [ 0.430055] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1190worker2 # [ 0.435838] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1191lb # [ 0.430073] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1192client # [ 0.428505] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1193worker2 # [ 0.435860] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1194lb # [ 0.430103] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1195client # [ 0.429002] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1196worker2 # [ 0.435882] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1197client # [ 0.429197] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1198lb # [ 0.430572] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1199worker2 # [ 0.435907] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1200lb # [ 0.430760] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1201worker1 # [ 0.456737] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1202client # [ 0.429228] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1203worker2 # [ 0.435930] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1204lb # [ 0.430776] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1205worker1 # [ 0.456798] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1206client # [ 0.429718] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1207worker2 # [ 0.435952] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1208lb # [ 0.430807] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1209client # [ 0.430024] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1210worker1 # [ 0.456847] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1211worker2 # [ 0.435975] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1212lb # [ 0.431385] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1213client # [ 0.430041] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1214worker2 # [ 0.435997] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1215worker1 # [ 0.456897] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1216lb # [ 0.431396] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1217client # [ 0.430072] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1218worker2 # [ 0.436019] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1219worker1 # [ 0.457017] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1220lb # [ 0.431402] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1221client # [ 0.430551] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1222worker2 # [ 0.436040] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1223client # [ 0.430797] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1224worker1 # [ 0.457070] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1225worker2 # [ 0.436061] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1226lb # [ 0.431448] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1227client # [ 0.430814] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1228worker1 # [ 0.457094] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1229worker2 # [ 0.436082] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1230lb # [ 0.431495] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1231worker1 # [ 0.457117] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1232client # [ 0.430844] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1233worker2 # [ 0.436103] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1234lb # [ 0.431544] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1235worker2 # [ 0.436125] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1236worker1 # [ 0.457141] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1237client # [ 0.431427] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1238worker2 # [ 0.436151] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1239lb # [ 0.431592] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1240worker1 # [ 0.457164] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1241client # [ 0.431438] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1242worker2 # [ 0.436160] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1243worker1 # [ 0.457187] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1244lb # [ 0.431639] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1245client # [ 0.431444] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1246worker2 # [ 0.436164] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1247worker1 # [ 0.457209] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1248lb # [ 0.431686] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1249client # [ 0.431491] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1250worker2 # [ 0.437001] pci 0000:00:07.0: enabling device (0000 -> 0002)1251worker1 # [ 0.457238] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1252lb # [ 0.431733] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1253client # [ 0.431538] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1254worker1 # [ 0.457260] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1255lb # [ 0.431780] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1256client # [ 0.431587] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1257worker1 # [ 0.457283] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1258lb # [ 0.431828] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1259worker1 # [ 0.457306] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1260client # [ 0.431636] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1261lb # [ 0.431899] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1262worker1 # [ 0.457328] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1263client # [ 0.431684] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1264worker1 # [ 0.457350] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1265lb # [ 0.431946] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1266client # [ 0.431733] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1267worker1 # [ 0.457372] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1268client # [ 0.431781] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1269worker1 # [ 0.457393] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1270worker1 # [ 0.457415] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1271client # [ 0.431828] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1272worker1 # [ 0.457436] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1273client # [ 0.431874] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1274worker1 # [ 0.457458] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1275client # [ 0.431946] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1276worker1 # [ 0.457488] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1277client # [ 0.431993] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1278worker1 # [ 0.457498] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1279client # [ 0.432040] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1280worker1 # [ 0.457502] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1281worker1 # [ 0.458375] pci 0000:00:07.0: enabling device (0000 -> 0002)1282client # [ 0.432062] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1283client # [ 0.432084] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1284client # [ 0.432106] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1285lb # [ 0.460019] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1286client # [ 0.432127] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1287lb # [ 0.460056] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1288client # [ 0.432149] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1289lb # [ 0.460081] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1290client # [ 0.432176] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1291lb # [ 0.460104] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1292client # [ 0.432198] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1293lb # [ 0.460128] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1294client # [ 0.432221] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1295lb # [ 0.460151] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1296client # [ 0.432243] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1297lb # [ 0.460177] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1298client # [ 0.432265] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1299client # [ 0.432287] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1300lb # [ 0.460201] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1301client # [ 0.432308] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1302lb # [ 0.460224] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1303client # [ 0.432329] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1304lb # [ 0.460248] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1305client # [ 0.432351] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1306lb # [ 0.460272] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1307client # [ 0.432371] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1308lb # [ 0.460295] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1309client # [ 0.432393] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1310lb # [ 0.460317] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1311client # [ 0.432418] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1312lb # [ 0.460339] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1313client # [ 0.432428] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1314worker2 # [ 0.493457] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1315lb # [ 0.460361] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1316client # [ 0.432432] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1317lb # [ 0.460383] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1318client # [ 0.433278] pci 0000:00:07.0: enabling device (0000 -> 0002)1319lb # [ 0.460405] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1320lb # [ 0.460434] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1321lb # [ 0.460444] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1322lb # [ 0.460450] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1323lb # [ 0.461308] pci 0000:00:07.0: enabling device (0000 -> 0002)1324worker1 # [ 0.502151] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1325worker2 # [ 0.499697] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1326worker1 # [ 0.504310] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1327worker2 # [ 0.501674] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1328worker2 # [ 0.512001] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1329worker1 # [ 0.514471] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1330worker2 # [ 0.513947] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1331worker1 # [ 0.516475] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1332client # [ 0.496833] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1333client # [ 0.498971] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1334lb # [ 0.502493] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1335worker1 # [ 0.526145] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1336worker2 # [ 0.524480] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1337worker1 # [ 0.528220] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1338worker2 # [ 0.526279] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1339worker2 # [ 0.529760] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1340client # [ 0.509283] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1341client # [ 0.511256] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1342worker1 # [ 0.537989] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1343worker1 # [ 0.539835] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1344lb # [ 0.512614] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1345worker2 # [ 0.536337] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1346lb # [ 0.515693] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1347worker2 # [ 0.538263] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1348lb # [ 0.518023] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1349client # [ 0.521342] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1350client # [ 0.523340] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1351worker2 # [ 0.549106] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1352worker1 # [ 0.549935] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1353lb # [ 0.528220] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1354worker1 # [ 0.551960] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1355lb # [ 0.530689] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1356client # [ 0.533284] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1357worker1 # [ 0.556081] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1358client # [ 0.535147] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1359client # [ 0.538645] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1360lb # [ 0.540646] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1361lb # [ 0.542621] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1362worker2 # [ 0.563472] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1363worker2 # [ 0.566137] msm_serial: driver initialized1364worker2 # [ 0.566292] SuperH (H)SCI(F) driver initialized1365worker2 # [ 0.566347] STM32 USART driver initialized1366worker1 # [ 0.569877] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1367client # [ 0.544559] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1368worker1 # [ 0.572448] msm_serial: driver initialized1369client # [ 0.547052] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1370worker1 # [ 0.572620] SuperH (H)SCI(F) driver initialized1371lb # [ 0.552519] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1372lb # [ 0.554624] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1373lb # [ 0.556825] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1374client # [ 0.565282] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1375worker1 # [ 0.580783] STM32 USART driver initialized1376lb # [ 0.569902] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1377client # [ 0.568027] msm_serial: driver initialized1378client # [ 0.568173] SuperH (H)SCI(F) driver initialized1379client # [ 0.568228] STM32 USART driver initialized1380lb # [ 0.572518] msm_serial: driver initialized1381lb # [ 0.572672] SuperH (H)SCI(F) driver initialized1382lb # [ 0.572729] STM32 USART driver initialized1383worker2 # [ 0.599868] loop: module loaded1384worker2 # [ 0.600064] virtio_blk virtio2: 1/0/0 default/read/poll queues1385worker2 # [ 0.601077] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1386worker1 # [ 0.609568] loop: module loaded1387worker1 # [ 0.609755] virtio_blk virtio2: 1/0/0 default/read/poll queues1388worker1 # [ 0.610481] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1389worker2 # [ 0.607198] megasas: 07.734.00.00-rc11390worker2 # [ 0.607946] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1391client # [ 0.600247] loop: module loaded1392client # [ 0.600427] virtio_blk virtio2: 1/0/0 default/read/poll queues1393client # [ 0.601275] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1394worker2 # [ 0.618655] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001395worker2 # [ 0.618687] Intel/Sharp Extended Query Table at 0x00311396worker1 # [ 0.621218] megasas: 07.734.00.00-rc11397worker2 # [ 0.620890] Using buffer write method1398worker1 # [ 0.621932] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1399lb # [ 0.606815] loop: module loaded1400worker2 # [ 0.620965] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1401lb # [ 0.606995] virtio_blk virtio2: 1/0/0 default/read/poll queues1402worker1 # [ 0.623906] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001403worker2 # [ 0.622670] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001404worker1 # [ 0.623930] Intel/Sharp Extended Query Table at 0x00311405lb # [ 0.607780] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1406worker2 # [ 0.622689] Intel/Sharp Extended Query Table at 0x00311407worker1 # [ 0.625574] Using buffer write method1408worker2 # [ 0.624448] Using buffer write method1409worker2 # [ 0.624472] Concatenating MTD devices:1410worker2 # [ 0.624476] (0): "0.flash"1411worker1 # [ 0.625641] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1412worker2 # [ 0.624480] (1): "0.flash"1413worker2 # [ 0.624492] into device "0.flash"1414worker1 # [ 0.627453] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001415worker1 # [ 0.627474] Intel/Sharp Extended Query Table at 0x00311416client # [ 0.608571] megasas: 07.734.00.00-rc11417client # [ 0.609279] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1418client # [ 0.611173] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001419client # [ 0.611197] Intel/Sharp Extended Query Table at 0x00311420lb # [ 0.612547] megasas: 07.734.00.00-rc11421lb # [ 0.613249] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1422client # [ 0.612841] Using buffer write method1423client # [ 0.612908] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1424client # [ 0.614807] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001425client # [ 0.614830] Intel/Sharp Extended Query Table at 0x00311426worker1 # [ 0.646146] Using buffer write method1427worker1 # [ 0.646185] Concatenating MTD devices:1428worker1 # [ 0.646190] (0): "0.flash"1429worker1 # [ 0.646194] (1): "0.flash"1430worker1 # [ 0.646203] into device "0.flash"1431client # [ 0.628595] Using buffer write method1432client # [ 0.628628] Concatenating MTD devices:1433client # [ 0.628633] (0): "0.flash"1434client # [ 0.628637] (1): "0.flash"1435client # [ 0.628646] into device "0.flash"1436lb # [ 0.633110] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001437lb # [ 0.633144] Intel/Sharp Extended Query Table at 0x00311438lb # [ 0.634944] Using buffer write method1439lb # [ 0.635016] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1440lb # [ 0.640201] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001441lb # [ 0.640221] Intel/Sharp Extended Query Table at 0x00311442lb # [ 0.642370] Using buffer write method1443lb # [ 0.642396] Concatenating MTD devices:1444lb # [ 0.642400] (0): "0.flash"1445lb # [ 0.642405] (1): "0.flash"1446lb # [ 0.642416] into device "0.flash"1447worker2 # [ 0.891325] Freeing initrd memory: 26944K1448worker2 # [ 0.897629] tun: Universal TUN/TAP device driver, 1.61449worker1 # [ 0.900734] Freeing initrd memory: 26976K1450worker2 # [ 0.901393] thunder_xcv, ver 1.01451worker2 # [ 0.901437] thunder_bgx, ver 1.01452worker2 # [ 0.901461] nicpf, ver 1.01453worker2 # [ 0.902015] e1000: Intel(R) PRO/1000 Network Driver1454worker1 # [ 0.906909] tun: Universal TUN/TAP device driver, 1.61455worker2 # [ 0.902023] e1000: Copyright (c) 1999-2006 Intel Corporation.1456worker2 # [ 0.902047] e1000e: Intel(R) PRO/1000 Network Driver1457worker2 # [ 0.902056] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1458worker2 # [ 0.902085] igb: Intel(R) Gigabit Ethernet Network Driver1459worker2 # [ 0.902091] igb: Copyright (c) 2007-2014 Intel Corporation.1460worker2 # [ 0.902131] igbvf: Intel(R) Gigabit Virtual Function Network Driver1461worker2 # [ 0.902138] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1462worker2 # [ 0.902271] sky2: driver version 1.301463worker1 # [ 0.910812] thunder_xcv, ver 1.01464worker1 # [ 0.910855] thunder_bgx, ver 1.01465worker1 # [ 0.910878] nicpf, ver 1.01466worker1 # [ 0.911430] e1000: Intel(R) PRO/1000 Network Driver1467worker1 # [ 0.911438] e1000: Copyright (c) 1999-2006 Intel Corporation.1468client # [ 0.892144] Freeing initrd memory: 26972K1469worker1 # [ 0.911462] e1000e: Intel(R) PRO/1000 Network Driver1470worker1 # [ 0.911470] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1471worker2 # [ 0.911387] usbcore: registered new interface driver usb-storage1472lb # [ 0.892991] Freeing initrd memory: 26948K1473worker1 # [ 0.911500] igb: Intel(R) Gigabit Ethernet Network Driver1474worker2 # [ 0.911443] usbcore: registered new interface driver usbserial_generic1475worker1 # [ 0.911506] igb: Copyright (c) 2007-2014 Intel Corporation.1476worker2 # [ 0.911458] usbserial: USB Serial support registered for generic1477worker1 # [ 0.911545] igbvf: Intel(R) Gigabit Virtual Function Network Driver1478worker2 # [ 0.912068] hv_vmbus: registering driver hyperv_keyboard1479worker1 # [ 0.911551] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1480worker2 # [ 0.913323] ehci-pci 0000:00:07.0: EHCI Host Controller1481client # [ 0.898299] tun: Universal TUN/TAP device driver, 1.61482worker1 # [ 0.911756] sky2: driver version 1.301483worker2 # [ 0.913350] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11484worker2 # [ 0.913556] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001485lb # [ 0.899107] tun: Universal TUN/TAP device driver, 1.61486worker1 # [ 0.921274] ehci-pci 0000:00:07.0: EHCI Host Controller1487worker1 # [ 0.921309] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11488worker2 # [ 0.919345] rtc-pl031 9010000.pl031: registered as rtc01489worker1 # [ 0.921525] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001490worker2 # [ 0.919374] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T16:49:46 UTC (1790354986)1491worker2 # [ 0.919703] i2c_dev: i2c /dev entries driver1492worker1 # [ 0.924333] usbcore: registered new interface driver usb-storage1493client # [ 0.902136] thunder_xcv, ver 1.01494client # [ 0.902178] thunder_bgx, ver 1.01495lb # [ 0.902863] thunder_xcv, ver 1.01496client # [ 0.902202] nicpf, ver 1.01497worker1 # [ 0.924496] usbcore: registered new interface driver usbserial_generic1498lb # [ 0.902907] thunder_bgx, ver 1.01499client # [ 0.902871] e1000: Intel(R) PRO/1000 Network Driver1500worker2 # [ 0.922894] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001501lb # [ 0.902931] nicpf, ver 1.01502worker1 # [ 0.924511] usbserial: USB Serial support registered for generic1503worker2 # [ 0.923214] hub 1-0:1.0: USB hub found1504lb # [ 0.903479] e1000: Intel(R) PRO/1000 Network Driver1505client # [ 0.902879] e1000: Copyright (c) 1999-2006 Intel Corporation.1506worker2 # [ 0.923243] hub 1-0:1.0: 6 ports detected1507client # [ 0.902908] e1000e: Intel(R) PRO/1000 Network Driver1508lb # [ 0.903487] e1000: Copyright (c) 1999-2006 Intel Corporation.1509client # [ 0.902917] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1510lb # [ 0.903512] e1000e: Intel(R) PRO/1000 Network Driver1511worker1 # [ 0.927480] hv_vmbus: registering driver hyperv_keyboard1512client # [ 0.902946] igb: Intel(R) Gigabit Ethernet Network Driver1513lb # [ 0.903520] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1514client # [ 0.902952] igb: Copyright (c) 2007-2014 Intel Corporation.1515lb # [ 0.903549] igb: Intel(R) Gigabit Ethernet Network Driver1516worker1 # [ 0.929189] rtc-pl031 9010000.pl031: registered as rtc01517lb # [ 0.903555] igb: Copyright (c) 2007-2014 Intel Corporation.1518client # [ 0.902976] igbvf: Intel(R) Gigabit Virtual Function Network Driver1519client # [ 0.902982] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1520lb # [ 0.903578] igbvf: Intel(R) Gigabit Virtual Function Network Driver1521worker1 # [ 0.929216] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T16:49:46 UTC (1790354986)1522client # [ 0.903115] sky2: driver version 1.301523lb # [ 0.903584] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1524worker1 # [ 0.929524] i2c_dev: i2c /dev entries driver1525lb # [ 0.903717] sky2: driver version 1.301526worker1 # [ 0.932744] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001527worker2 # [ 0.927819] sdhci: Secure Digital Host Controller Interface driver1528worker1 # [ 0.933818] hub 1-0:1.0: USB hub found1529worker2 # [ 0.927831] sdhci: Copyright(c) Pierre Ossman1530client # [ 0.912604] ehci-pci 0000:00:07.0: EHCI Host Controller1531worker1 # [ 0.934374] hub 1-0:1.0: 6 ports detected1532worker2 # [ 0.928098] Synopsys Designware Multimedia Card Interface Driver1533client # [ 0.912633] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11534worker2 # [ 0.928473] sdhci-pltfm: SDHCI platform and OF driver helper1535client # [ 0.912837] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001536worker2 # [ 0.929998] hid: raw HID events driver (C) Jiri Kosina1537worker2 # [ 0.930250] usbcore: registered new interface driver usbhid1538lb # [ 0.912845] usbcore: registered new interface driver usb-storage1539client # [ 0.915452] usbcore: registered new interface driver usb-storage1540worker2 # [ 0.930257] usbhid: USB HID core driver1541lb # [ 0.912944] usbcore: registered new interface driver usbserial_generic1542client # [ 0.915509] usbcore: registered new interface driver usbserial_generic1543lb # [ 0.912959] usbserial: USB Serial support registered for generic1544client # [ 0.915524] usbserial: USB Serial support registered for generic1545lb # [ 0.913553] hv_vmbus: registering driver hyperv_keyboard1546lb # [ 0.914918] ehci-pci 0000:00:07.0: EHCI Host Controller1547client # [ 0.918492] hv_vmbus: registering driver hyperv_keyboard1548worker1 # [ 0.937736] sdhci: Secure Digital Host Controller Interface driver1549worker2 # [ 0.936054] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1550lb # [ 0.914966] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11551worker1 # [ 0.937750] sdhci: Copyright(c) Pierre Ossman1552lb # [ 0.915161] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001553worker2 # [ 0.937602] drop_monitor: Initializing network drop monitor service1554client # [ 0.920094] rtc-pl031 9010000.pl031: registered as rtc01555worker1 # [ 0.938008] Synopsys Designware Multimedia Card Interface Driver1556worker2 # [ 0.937735] NET: Registered PF_INET6 protocol family1557worker1 # [ 0.938387] sdhci-pltfm: SDHCI platform and OF driver helper1558client # [ 0.920119] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T16:49:46 UTC (1790354986)1559worker2 # [ 0.940917] Segment Routing with IPv61560worker1 # [ 0.939894] hid: raw HID events driver (C) Jiri Kosina1561lb # [ 0.920847] rtc-pl031 9010000.pl031: registered as rtc01562client # [ 0.920418] i2c_dev: i2c /dev entries driver1563worker2 # [ 0.940935] In-situ OAM (IOAM) with IPv61564worker1 # [ 0.940139] usbcore: registered new interface driver usbhid1565worker2 # [ 0.940979] NET: Registered PF_PACKET protocol family1566worker1 # [ 0.940146] usbhid: USB HID core driver1567lb # [ 0.920880] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T16:49:46 UTC (1790354986)1568lb # [ 0.921282] i2c_dev: i2c /dev entries driver1569lb # [ 0.924460] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001570lb # [ 0.924748] hub 1-0:1.0: USB hub found1571lb # [ 0.924768] hub 1-0:1.0: 6 ports detected1572worker1 # [ 0.946177] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1573client # [ 0.925806] sdhci: Secure Digital Host Controller Interface driver1574worker2 # [ 0.942729] 9pnet: Installing 9P2000 support1575worker1 # [ 0.947742] drop_monitor: Initializing network drop monitor service1576client # [ 0.925818] sdhci: Copyright(c) Pierre Ossman1577worker2 # [ 0.942773] Key type dns_resolver registered1578worker1 # [ 0.947885] NET: Registered PF_INET6 protocol family1579client # [ 0.926081] Synopsys Designware Multimedia Card Interface Driver1580worker1 # [ 0.951073] Segment Routing with IPv61581client # [ 0.926450] sdhci-pltfm: SDHCI platform and OF driver helper1582worker1 # [ 0.951094] In-situ OAM (IOAM) with IPv61583worker1 # [ 0.951121] NET: Registered PF_PACKET protocol family1584client # [ 0.930793] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001585client # [ 0.931133] hub 1-0:1.0: USB hub found1586worker2 # [ 0.949465] registered taskstats version 11587client # [ 0.931154] hub 1-0:1.0: 6 ports detected1588worker2 # [ 0.949650] Loading compiled-in X.509 certificates1589lb # [ 0.929326] sdhci: Secure Digital Host Controller Interface driver1590lb # [ 0.929339] sdhci: Copyright(c) Pierre Ossman1591lb # [ 0.929603] Synopsys Designware Multimedia Card Interface Driver1592client # [ 0.933411] hid: raw HID events driver (C) Jiri Kosina1593lb # [ 0.929991] sdhci-pltfm: SDHCI platform and OF driver helper1594client # [ 0.933665] usbcore: registered new interface driver usbhid1595lb # [ 0.931527] hid: raw HID events driver (C) Jiri Kosina1596worker1 # [ 0.952910] 9pnet: Installing 9P2000 support1597client # [ 0.933671] usbhid: USB HID core driver1598worker1 # [ 0.952958] Key type dns_resolver registered1599lb # [ 0.931786] usbcore: registered new interface driver usbhid1600lb # [ 0.931795] usbhid: USB HID core driver1601client # [ 0.936763] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1602worker1 # [ 0.960110] registered taskstats version 11603client # [ 0.938294] drop_monitor: Initializing network drop monitor service1604worker2 # [ 0.958489] Demotion targets for Node 0: null1605worker1 # [ 0.960304] Loading compiled-in X.509 certificates1606client # [ 0.938437] NET: Registered PF_INET6 protocol family1607worker2 # [ 0.958601] Key type .fscrypt registered1608lb # [ 0.937812] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1609worker2 # [ 0.958607] Key type fscrypt-provisioning registered1610client # [ 0.941535] Segment Routing with IPv61611lb # [ 0.939427] drop_monitor: Initializing network drop monitor service1612client # [ 0.941565] In-situ OAM (IOAM) with IPv61613worker2 # [ 0.960488] ima: No TPM chip found, activating TPM-bypass!1614lb # [ 0.939559] NET: Registered PF_INET6 protocol family1615client # [ 0.941593] NET: Registered PF_PACKET protocol family1616worker2 # [ 0.960512] ima: Allocated hash algorithm: sha11617worker2 # [ 0.960534] ima: No architecture policies found1618lb # [ 0.942939] Segment Routing with IPv61619client # [ 0.943396] 9pnet: Installing 9P2000 support1620lb # [ 0.942959] In-situ OAM (IOAM) with IPv61621client # [ 0.943447] Key type dns_resolver registered1622lb # [ 0.942988] NET: Registered PF_PACKET protocol family1623worker2 # [ 0.963069] input: gpio-keys as /devices/platform/gpio-keys/input/input01624worker1 # [ 0.969031] Demotion targets for Node 0: null1625lb # [ 0.944716] 9pnet: Installing 9P2000 support1626worker1 # [ 0.969142] Key type .fscrypt registered1627lb # [ 0.944765] Key type dns_resolver registered1628worker1 # [ 0.969148] Key type fscrypt-provisioning registered1629worker1 # [ 0.969245] ima: No TPM chip found, activating TPM-bypass!1630worker1 # [ 0.969265] ima: Allocated hash algorithm: sha11631worker1 # [ 0.969288] ima: No architecture policies found1632client # [ 0.950277] registered taskstats version 11633client # [ 0.950429] Loading compiled-in X.509 certificates1634lb # [ 0.951678] registered taskstats version 11635worker1 # [ 0.973681] input: gpio-keys as /devices/platform/gpio-keys/input/input01636lb # [ 0.951844] Loading compiled-in X.509 certificates1637client # [ 0.959543] Demotion targets for Node 0: null1638client # [ 0.959656] Key type .fscrypt registered1639client # [ 0.959664] Key type fscrypt-provisioning registered1640client # [ 0.959757] ima: No TPM chip found, activating TPM-bypass!1641lb # [ 0.960610] Demotion targets for Node 0: null1642lb # [ 0.960714] Key type .fscrypt registered1643client # [ 0.959777] ima: Allocated hash algorithm: sha11644lb # [ 0.960721] Key type fscrypt-provisioning registered1645client # [ 0.959798] ima: No architecture policies found1646lb # [ 0.960820] ima: No TPM chip found, activating TPM-bypass!1647worker2 # [ 0.980972] clk: Disabling unused clocks1648lb # [ 0.960840] ima: Allocated hash algorithm: sha11649client # [ 0.964016] input: gpio-keys as /devices/platform/gpio-keys/input/input01650worker2 # [ 0.981001] PM: genpd: Disabling unused power domains1651lb # [ 0.960863] ima: No architecture policies found1652lb # [ 0.965286] input: gpio-keys as /devices/platform/gpio-keys/input/input01653worker2 # [ 0.985290] Freeing unused kernel memory: 4736K1654worker2 # [ 0.985493] Run /init as init process1655worker1 # [ 0.992317] clk: Disabling unused clocks1656worker1 # [ 0.992357] PM: genpd: Disabling unused power domains1657worker1 # [ 0.996703] Freeing unused kernel memory: 4736K1658worker1 # [ 0.996945] Run /init as init process1659worker2 # [ 1.000908] systemd[1]: Successfully made /usr/ read-only.1660client # [ 0.982386] clk: Disabling unused clocks1661client # [ 0.982415] PM: genpd: Disabling unused power domains1662lb # [ 0.983546] clk: Disabling unused clocks1663lb # [ 0.983572] PM: genpd: Disabling unused power domains1664client # [ 0.986700] Freeing unused kernel memory: 4736K1665client # [ 0.986909] Run /init as init process1666lb # [ 0.987757] Freeing unused kernel memory: 4736K1667lb # [ 0.988584] Run /init as init process1668worker1 # [ 1.014930] systemd[1]: Successfully made /usr/ read-only.1669client # [ 1.002217] systemd[1]: Successfully made /usr/ read-only.1670lb # [ 1.002228] systemd[1]: Successfully made /usr/ read-only.1671worker2 # [ 1.170656] usb 1-1: new high-speed USB device number 2 using ehci-pci1672worker1 # [ 1.180675] usb 1-1: new high-speed USB device number 2 using ehci-pci1673lb # [ 1.172003] usb 1-1: new high-speed USB device number 2 using ehci-pci1674client # [ 1.180040] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker2 # [ 1.320345] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11676worker1 # [ 1.330275] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677worker2 # [ 1.335659] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1678worker2 # [ 1.342096] systemd[1]: Detected virtualization qemu.1679worker2 # [ 1.343198] systemd[1]: Detected architecture arm64.1680worker2 # [ 1.344210] systemd[1]: Running in initrd.1681lb # [ 1.323647] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11682worker2 # [ 1.345649] systemd[1]: Initializing machine ID from random generator.1683worker2 # [ 1.347214] systemd[1]: Hostname set to <worker2>.1684client # [ 1.329758] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11685worker1 # [ 1.349744] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1686worker1 # [ 1.356325] systemd[1]: Detected virtualization qemu.1687worker1 # [ 1.357467] systemd[1]: Detected architecture arm64.1688worker1 # [ 1.358477] systemd[1]: Running in initrd.1689worker1 # [ 1.359876] systemd[1]: Initializing machine ID from random generator.1690worker1 # [ 1.361441] systemd[1]: Hostname set to <worker1>.1691client # [ 1.336957] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1692lb # [ 1.337072] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1693client # [ 1.343475] systemd[1]: Detected virtualization qemu.1694lb # [ 1.343739] systemd[1]: Detected virtualization qemu.1695client # [ 1.344661] systemd[1]: Detected architecture arm64.1696client # [ 1.345664] systemd[1]: Running in initrd.1697lb # [ 1.344873] systemd[1]: Detected architecture arm64.1698lb # [ 1.345886] systemd[1]: Running in initrd.1699client # [ 1.347033] systemd[1]: Initializing machine ID from random generator.1700lb # [ 1.347387] systemd[1]: Initializing machine ID from random generator.1701client # [ 1.348620] systemd[1]: Hostname set to <client>.1702lb # [ 1.348950] systemd[1]: Hostname set to <lb>.1703worker2 # [ 1.398851] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01704worker1 # [ 1.408843] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01705lb # [ 1.400180] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706client # [ 1.412208] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker2 # [ 1.518669] usb 1-2: new high-speed USB device number 3 using ehci-pci1708worker1 # [ 1.528671] usb 1-2: new high-speed USB device number 3 using ehci-pci1709lb # [ 1.520009] usb 1-2: new high-speed USB device number 3 using ehci-pci1710client # [ 1.532070] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker2 # [ 1.595187] systemd[1]: bpf-restrict-fs: LSM BPF program attached1712worker1 # [ 1.624235] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713lb # [ 1.603783] systemd[1]: bpf-restrict-fs: LSM BPF program attached1714client # [ 1.604553] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715worker2 # [ 1.651446] systemd[1]: Queued start job for default target Initrd Default Target.1716worker2 # [ 1.660640] systemd[1]: Created slice Slice /system/modprobe.1717worker2 # [ 1.661724] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1718worker2 # [ 1.662826] systemd[1]: Expecting device /dev/disk/by-label/nixos...1719worker2 # [ 1.663685] systemd[1]: Reached target Path Units.1720worker2 # [ 1.664381] systemd[1]: Reached target Slice Units.1721worker2 # [ 1.665052] systemd[1]: Reached target Swaps.1722worker2 # [ 1.665653] systemd[1]: Reached target Timer Units.1723worker2 # [ 1.666450] systemd[1]: Listening on D-Bus System Message Bus Socket.1724worker2 # [ 1.667691] systemd[1]: Listening on Journal Socket (/dev/log).1725worker2 # [ 1.668712] systemd[1]: Listening on Journal Sockets.1726worker2 # [ 1.669521] systemd[1]: Listening on udev Control Socket.1727worker2 # [ 1.670508] systemd[1]: Listening on udev Kernel Socket.1728worker2 # [ 1.670533] systemd[1]: Reached target Socket Units.1729worker2 # [ 1.674146] systemd[1]: Starting Create List of Static Device Nodes...1730worker2 # [ 1.675185] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1731worker2 # [ 1.679030] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21732worker2 # [ 1.679215] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01733worker1 # [ 1.683442] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21734worker1 # [ 1.683640] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01735worker2 # [ 1.686804] systemd[1]: Mounting Kernel Configuration File System...1736client # [ 1.675285] systemd[1]: Queued start job for default target Initrd Default Target.1737worker2 # [ 1.694876] systemd[1]: Starting Journal Service...1738lb # [ 1.678203] systemd[1]: Queued start job for default target Initrd Default Target.1739lb # [ 1.679876] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21740lb # [ 1.680059] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01741client # [ 1.683662] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21742client # [ 1.687416] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01743worker1 # [ 1.709173] systemd[1]: Queued start job for default target Initrd Default Target.1744client # [ 1.691118] systemd[1]: Created slice Slice /system/modprobe.1745client # [ 1.692376] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1746client # [ 1.693785] systemd[1]: Expecting device /dev/disk/by-label/nixos...1747client # [ 1.695112] systemd[1]: Reached target Path Units.1748lb # [ 1.694554] systemd[1]: Created slice Slice /system/modprobe.1749client # [ 1.695975] systemd[1]: Reached target Slice Units.1750client # [ 1.697016] systemd[1]: Reached target Swaps.1751lb # [ 1.695803] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1752client # [ 1.697810] systemd[1]: Reached target Timer Units.1753lb # [ 1.697300] systemd[1]: Expecting device /dev/disk/by-label/nixos...1754worker1 # [ 1.719383] systemd[1]: Created slice Slice /system/modprobe.1755worker2 # [ 1.717519] systemd[1]: Starting Load Kernel Modules...1756lb # [ 1.698426] systemd[1]: Reached target Path Units.1757client # [ 1.698924] systemd[1]: Listening on D-Bus System Message Bus Socket.1758lb # [ 1.699268] systemd[1]: Reached target Slice Units.1759worker2 # [ 1.718320] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1760worker1 # [ 1.720791] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1761client # [ 1.700233] systemd[1]: Listening on Journal Socket (/dev/log).1762lb # [ 1.700249] systemd[1]: Reached target Swaps.1763client # [ 1.701458] systemd[1]: Listening on Journal Sockets.1764worker1 # [ 1.722171] systemd[1]: Expecting device /dev/disk/by-label/nixos...1765lb # [ 1.701041] systemd[1]: Reached target Timer Units.1766worker1 # [ 1.723332] systemd[1]: Reached target Path Units.1767client # [ 1.702460] systemd[1]: Listening on udev Control Socket.1768lb # [ 1.702117] systemd[1]: Listening on D-Bus System Message Bus Socket.1769worker1 # [ 1.724199] systemd[1]: Reached target Slice Units.1770client # [ 1.703546] systemd[1]: Listening on udev Kernel Socket.1771worker1 # [ 1.725131] systemd[1]: Reached target Swaps.1772lb # [ 1.703400] systemd[1]: Listening on Journal Socket (/dev/log).1773worker1 # [ 1.725915] systemd[1]: Reached target Timer Units.1774client # [ 1.704502] systemd[1]: Reached target Socket Units.1775worker1 # [ 1.726955] systemd[1]: Listening on D-Bus System Message Bus Socket.1776lb # [ 1.704640] systemd[1]: Listening on Journal Sockets.1777client # [ 1.707016] systemd[1]: Starting Create List of Static Device Nodes...1778worker1 # [ 1.728315] systemd[1]: Listening on Journal Socket (/dev/log).1779lb # [ 1.704787] systemd[1]: Listening on udev Control Socket.1780lb # [ 1.704904] systemd[1]: Listening on udev Kernel Socket.1781lb # [ 1.704928] systemd[1]: Reached target Socket Units.1782client # [ 1.708166] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1783worker1 # [ 1.729647] systemd[1]: Listening on Journal Sockets.1784worker1 # [ 1.729796] systemd[1]: Listening on udev Control Socket.1785lb # [ 1.710274] systemd[1]: Starting Create List of Static Device Nodes...1786worker1 # [ 1.729912] systemd[1]: Listening on udev Kernel Socket.1787worker1 # [ 1.729936] systemd[1]: Reached target Socket Units.1788lb # [ 1.711472] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1789worker1 # [ 1.735094] systemd[1]: Starting Create List of Static Device Nodes...1790client # [ 1.716179] systemd[1]: Mounting Kernel Configuration File System...1791worker1 # [ 1.736424] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1792worker2 # [ 1.734871] systemd[1]: Starting Coldplug All udev Devices...1793worker2 # [ 1.738494] systemd-journald[72]: Collecting audit messages is disabled.1794lb # [ 1.720107] systemd[1]: Mounting Kernel Configuration File System...1795client # [ 1.724302] systemd[1]: Starting Journal Service...1796worker1 # [ 1.747866] systemd[1]: Mounting Kernel Configuration File System...1797lb # [ 1.728228] systemd[1]: Starting Journal Service...1798worker2 # [ 1.750805] systemd[1]: Finished Create List of Static Device Nodes.1799worker2 # [ 1.751527] systemd[1]: Mounted Kernel Configuration File System.1800client # [ 1.748343] systemd[1]: Starting Load Kernel Modules...1801client # [ 1.749241] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1802worker2 # [ 1.767061] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1803lb # [ 1.752255] systemd[1]: Starting Load Kernel Modules...1804worker1 # [ 1.772949] systemd[1]: Starting Journal Service...1805lb # [ 1.756189] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1806worker1 # [ 1.782778] systemd[1]: Starting Load Kernel Modules...1807worker1 # [ 1.782908] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1808lb # [ 1.772136] systemd[1]: Starting Coldplug All udev Devices...1809client # [ 1.772231] systemd[1]: Starting Coldplug All udev Devices...1810lb # [ 1.775020] systemd-journald[72]: Collecting audit messages is disabled.1811worker1 # [ 1.797017] systemd[1]: Starting Coldplug All udev Devices...1812client # [ 1.778297] systemd-journald[72]: Collecting audit messages is disabled.1813worker2 # [ 1.807227] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1814lb # [ 1.789487] systemd[1]: Finished Create List of Static Device Nodes.1815worker2 # [ 1.809885] systemd[1]: Starting Create Static Device Nodes in /dev...1816lb # [ 1.790170] systemd[1]: Mounted Kernel Configuration File System.1817client # [ 1.792114] systemd[1]: Finished Create List of Static Device Nodes.1818client # [ 1.792824] systemd[1]: Mounted Kernel Configuration File System.1819worker1 # [ 1.816870] systemd[1]: Finished Create List of Static Device Nodes.1820worker1 # [ 1.817654] systemd[1]: Mounted Kernel Configuration File System.1821lb # [ 1.796323] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1822worker2 # [ 1.818332] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1823client # [ 1.801693] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1824worker1 # [ 1.832984] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1825client # [ 1.816586] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1826worker1 # [ 1.842998] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1827worker2 # [ 1.834790] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1828worker2 # [ 1.840787] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01829worker1 # [ 1.846263] systemd-journald[72]: Collecting audit messages is disabled.1830worker2 # [ 1.841028] [drm] features: -virgl +edid -resource_blob -host_visible1831worker2 # [ 1.841038] [drm] features: -context_init1832worker2 # [ 1.841828] [drm] number of scanouts: 11833worker2 # [ 1.841846] [drm] number of cap sets: 01834client # [ 1.832118] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1835client # [ 1.838302] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01836client # [ 1.838553] [drm] features: -virgl +edid -resource_blob -host_visible1837client # [ 1.838563] [drm] features: -context_init1838worker2 # [ 1.859295] systemd[1]: Finished Create Static Device Nodes in /dev.1839client # [ 1.839347] [drm] number of scanouts: 11840client # [ 1.839366] [drm] number of cap sets: 01841worker2 # [ 1.859645] systemd[1]: Reached target Preparation for Local File Systems.1842worker2 # [ 1.859675] systemd[1]: Reached target Local File Systems.1843lb # [ 1.840658] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1844worker2 # [ 1.863715] systemd[1]: Starting Rule-based Manager for Device Events and Files...1845lb # [ 1.845336] systemd[1]: Starting Create Static Device Nodes in /dev...1846worker1 # [ 1.865014] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1847worker1 # [ 1.871292] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01848worker1 # [ 1.871531] [drm] features: -virgl +edid -resource_blob -host_visible1849worker1 # [ 1.871542] [drm] features: -context_init1850worker1 # [ 1.872289] [drm] number of scanouts: 11851worker1 # [ 1.872308] [drm] number of cap sets: 01852lb # [ 1.857865] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1853client # [ 1.860624] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1854worker2 # [ 1.874941] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1855worker2 # [ 1.874962] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01856client # [ 1.863219] systemd[1]: Starting Create Static Device Nodes in /dev...1857lb # [ 1.864013] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1858lb # [ 1.870303] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01859worker1 # [ 1.893337] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1860lb # [ 1.870549] [drm] features: -virgl +edid -resource_blob -host_visible1861lb # [ 1.870559] [drm] features: -context_init1862worker1 # [ 1.895885] systemd[1]: Starting Create Static Device Nodes in /dev...1863lb # [ 1.871276] [drm] number of scanouts: 11864lb # [ 1.871294] [drm] number of cap sets: 01865client # [ 1.872374] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1866client # [ 1.872390] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01867worker2 # [ 1.894996] Console: switching to colour frame buffer device 160x501868worker1 # [ 1.905113] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1869worker1 # [ 1.905129] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01870worker2 # [ 1.919253] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1871lb # [ 1.900515] systemd[1]: Finished Create Static Device Nodes in /dev.1872lb # [ 1.900846] systemd[1]: Reached target Preparation for Local File Systems.1873lb # [ 1.900877] systemd[1]: Reached target Local File Systems.1874worker2 # [ 1.924432] systemd[1]: Started Journal Service.1875lb # [ 1.904905] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1876lb # [ 1.904921] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01877client # [ 1.908670] systemd[1]: Finished Create Static Device Nodes in /dev.1878client # [ 1.908986] systemd[1]: Reached target Preparation for Local File Systems.1879client # [ 1.909012] systemd[1]: Reached target Local File Systems.1880lb # [ 1.910105] systemd[1]: Starting Rule-based Manager for Device Events and Files...1881client # [ 1.912966] systemd[1]: Starting Rule-based Manager for Device Events and Files...1882worker2 # [ 1.917615] systemd-modules-load[74]: Inserted module 'dm_mod'1883worker2 # [ 1.918919] systemd-modules-load[74]: Module 'virtio_balloon' is built in1884worker1 # [ 1.933075] Console: switching to colour frame buffer device 160x501885worker1 # [ 1.937244] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1886client # [ 1.920170] Console: switching to colour frame buffer device 160x501887client # [ 1.920864] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1888worker2 # [ 1.928125] systemd-modules-load[74]: Module 'virtio_console' is built in1889worker2 # [ 1.932430] systemd-modules-load[74]: Inserted module 'virtio_gpu'1890worker2 # [ 1.933491] systemd-modules-load[74]: Module 'virtio_rng' is built in1891worker1 # [ 1.950141] systemd[1]: Finished Create Static Device Nodes in /dev.1892worker2 # [ 1.936265] systemd[1]: Starting Create System Files and Directories...1893worker1 # [ 1.950504] systemd[1]: Reached target Preparation for Local File Systems.1894worker1 # [ 1.950540] systemd[1]: Reached target Local File Systems.1895worker1 # [ 1.955391] systemd[1]: Starting Rule-based Manager for Device Events and Files...1896worker2 # [ 1.944299] systemd[1]: Finished Load Kernel Modules.1897lb # [ 1.936286] Console: switching to colour frame buffer device 160x501898lb # [ 1.940510] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1899client # [ 1.944272] systemd[1]: Finished Load Kernel Modules.1900worker2 # [ 1.950720] systemd-udevd[79]: Using default interface naming scheme 'v261'.1901client # [ 1.947441] systemd[1]: Starting Apply Kernel Variables...1902worker1 # [ 1.972743] systemd[1]: Finished Load Kernel Modules.1903worker1 # [ 1.977026] systemd[1]: Starting Apply Kernel Variables...1904lb # [ 1.956643] systemd[1]: Finished Load Kernel Modules.1905worker2 # [ 1.964098] systemd[1]: Starting Apply Kernel Variables...1906lb # [ 1.946560] systemd-modules-load[73]: Inserted module 'dm_mod'1907client # [ 1.968198] systemd[1]: Started Journal Service.1908client # [ 1.953587] systemd-modules-load[73]: Inserted module 'dm_mod'1909lb # [ 1.972372] systemd[1]: Starting Apply Kernel Variables...1910lb # [ 1.972602] systemd[1]: Started Journal Service.1911client # [ 1.960301] systemd-modules-load[73]: Module 'virtio_balloon' is built in1912lb # [ 1.964155] systemd-modules-load[73]: Module 'virtio_balloon' is built in1913lb # [ 1.965329] systemd-modules-load[73]: Module 'virtio_console' is built in1914lb # [ 1.966432] systemd-modules-load[73]: Inserted module 'virtio_gpu'1915client # [ 1.968079] systemd-modules-load[73]: Module 'virtio_console' is built in1916lb # [ 1.967452] systemd-modules-load[73]: Module 'virtio_rng' is built in1917worker2 # [ 1.989776] systemd[1]: Finished Create System Files and Directories.1918client # [ 1.972294] systemd-modules-load[73]: Inserted module 'virtio_gpu'1919client # [ 1.977307] systemd-modules-load[73]: Module 'virtio_rng' is built in1920worker1 # [ 2.012812] systemd[1]: Started Journal Service.1921client # [ 1.984302] systemd[1]: Starting Create System Files and Directories...1922worker2 # [ 2.005698] systemd[1]: Finished Apply Kernel Variables.1923worker2 # [ 2.009417] systemd[1]: Started Rule-based Manager for Device Events and Files.1924client # [ 1.989904] systemd[1]: Finished Apply Kernel Variables.1925worker1 # [ 2.008306] systemd-modules-load[73]: Inserted module 'dm_mod'1926worker1 # [ 2.009603] systemd-modules-load[73]: Module 'virtio_balloon' is built in1927lb # [ 1.993653] systemd[1]: Starting Create System Files and Directories...1928worker1 # [ 2.010701] systemd-modules-load[73]: Module 'virtio_console' is built in1929worker1 # [ 2.011889] systemd-modules-load[73]: Inserted module 'virtio_gpu'1930lb # [ 2.002435] systemd-udevd[79]: Using default interface naming scheme 'v261'.1931worker1 # [ 2.024475] systemd-modules-load[73]: Module 'virtio_rng' is built in1932worker1 # [ 2.025523] systemd[1]: Starting Create System Files and Directories...1933client # [ 2.009571] systemd[1]: Finished Create System Files and Directories.1934client # [ 2.011706] systemd-udevd[78]: Using default interface naming scheme 'v261'.1935worker1 # [ 2.029638] systemd[1]: Finished Apply Kernel Variables.1936lb # [ 2.016122] systemd[1]: Finished Create System Files and Directories.1937lb # [ 2.021768] systemd[1]: Finished Apply Kernel Variables.1938worker1 # [ 2.053059] systemd[1]: Finished Create System Files and Directories.1939worker1 # [ 2.058188] systemd-udevd[78]: Using default interface naming scheme 'v261'.1940client # [ 2.042422] systemd[1]: Started Rule-based Manager for Device Events and Files.1941lb # [ 2.045534] systemd[1]: Started Rule-based Manager for Device Events and Files.1942worker2 # [ 2.076098] systemd[1]: Starting Virtual Console Setup...1943worker1 # [ 2.089163] systemd[1]: Started Rule-based Manager for Device Events and Files.1944client # [ 2.092581] systemd[1]: Starting Virtual Console Setup...1945lb # [ 2.102862] systemd[1]: Starting Virtual Console Setup...1946worker2 # [ 2.148467] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1947worker1 # [ 2.145881] systemd[1]: Starting Virtual Console Setup...1948worker2 # [ 2.151990] systemd[1]: Finished Virtual Console Setup.1949client # [ 2.156543] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1950lb # [ 2.156510] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1951client # [ 2.160179] systemd[1]: Finished Virtual Console Setup.1952lb # [ 2.159984] systemd[1]: Finished Virtual Console Setup.1953worker1 # [ 2.196502] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1954worker1 # [ 2.200099] systemd[1]: Finished Virtual Console Setup.1955worker2 # [ 2.780944] systemd[1]: Finished Coldplug All udev Devices.1956lb # [ 2.760558] systemd[1]: Finished Coldplug All udev Devices.1957worker2 # [ 2.781893] systemd[1]: Reached target System Initialization.1958worker2 # [ 2.782738] systemd[1]: Reached target Basic System.1959lb # [ 2.764837] systemd[1]: Reached target System Initialization.1960lb # [ 2.765742] systemd[1]: Reached target Basic System.1961client # [ 2.767643] systemd[1]: Finished Coldplug All udev Devices.1962client # [ 2.768672] systemd[1]: Reached target System Initialization.1963client # [ 2.772109] systemd[1]: Reached target Basic System.1964worker1 # [ 2.798921] systemd[1]: Finished Coldplug All udev Devices.1965worker1 # [ 2.800418] systemd[1]: Reached target System Initialization.1966worker1 # [ 2.804460] systemd[1]: Reached target Basic System.1967lb # [ 2.908157] (udev-worker)[99]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1968client # [ 2.911806] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969worker1 # [ 2.952103] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1970worker2 # [ 2.955054] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1971worker2 # [ 2.960263] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1972lb # [ 2.947911] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1973worker2 # [ 2.970012] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1974lb # [ 2.951441] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1975client # [ 2.954438] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1976client # [ 2.965582] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1977worker1 # [ 2.987261] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1978worker1 # [ 2.992237] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1979worker2 # [ 3.048513] systemd[1]: Found device /dev/disk/by-label/nixos.1980lb # [ 3.029756] systemd[1]: Found device /dev/disk/by-label/nixos.1981worker2 # [ 3.053864] systemd[1]: Reached target Initrd Root Device.1982lb # [ 3.035158] systemd[1]: Reached target Initrd Root Device.1983client # [ 3.037570] systemd[1]: Found device /dev/disk/by-label/nixos.1984worker2 # [ 3.057896] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1985lb # [ 3.040849] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1986client # [ 3.044086] systemd[1]: Reached target Initrd Root Device.1987client # [ 3.048089] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1988worker1 # [ 3.064825] systemd[1]: Found device /dev/disk/by-label/nixos.1989worker1 # [ 3.071036] systemd[1]: Reached target Initrd Root Device.1990worker1 # [ 3.073076] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1991worker2 # [ 3.100757] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1992lb # [ 3.085243] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1993worker2 # [ 3.108207] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1994worker2 # [ 3.112131] systemd[1]: Mounting /sysroot...1995lb # [ 3.092118] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1996client # [ 3.094524] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1997lb # [ 3.096134] systemd[1]: Mounting /sysroot...1998worker1 # [ 3.116856] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1999client # [ 3.102492] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.2000client # [ 3.104198] systemd[1]: Mounting /sysroot...
loading 2,091 hidden lines…
4092worker1 # [ 68.778542] nix-grpc-daemon[487]: ts=2026-09-25T16:50:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:382304093client # don't know how to build these paths:4094client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4095worker1 # [ 68.785770] nix-grpc-daemon[487]: ts=2026-09-25T16:50:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382304096worker1 # [ 68.800734] nix-grpc-daemon[487]: ts=2026-09-25T16:50:54Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04097worker1 # [ 68.803707] nix-grpc-daemon[487]: ts=2026-09-25T16:50:54Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=14098worker2 # [ 68.808701] nix-grpc-daemon[771]: ts=2026-09-25T16:50:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044099worker2 # [ 68.815169] nix-grpc-daemon[771]: ts=2026-09-25T16:50:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:511044100client # copying 1 paths to 'lb:50051'...4101client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4102worker2 # [ 68.824241] nix-grpc-daemon[771]: ts=2026-09-25T16:50:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:511044103worker2 # [ 68.833543] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4104worker2 # [ 68.851679] nix-grpc-daemon[771]: ts=2026-09-25T16:50:54Z level=info event=niks3_push_started pid=8764105worker2 # [ 69.001010] nix-grpc-daemon[876]: time=2026-09-25T16:50:54.594Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""4106worker2 # [ 69.103460] nix-daemon[842]: accepted connection from pid 888, user nix-grpc-daemon (trusted)4107worker2 # [ 69.122648] nix-daemon[842]: reaped child process 891, status = succeeded4108lb # [ 69.108767] niks3-server[985]: 2026/09/25 16:50:54 INFO Received push request method=POST path=/api/pushes4109worker2 # [ 69.136105] nix-grpc-daemon[876]: time=2026-09-25T16:50:54.730Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4110worker2 # [ 69.137639] nix-grpc-daemon[876]: time=2026-09-25T16:50:54.730Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4111lb # [ 69.136612] niks3-server[985]: 2026/09/25 16:50:54 INFO Received sign narinfos request method=POST path=/api/pushes/9/sign4112worker2 # [ 69.161881] nix-grpc-daemon[876]: time=2026-09-25T16:50:54.756Z level=INFO msg="Uploading 1 narinfos"4113lb # [ 69.143911] niks3-server[985]: 2026/09/25 16:50:54 INFO Signed narinfos id=9 count=14114lb # [ 69.148686] niks3-server[985]: 2026/09/25 16:50:54 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4115lb # [ 69.151706] niks3-server[985]: 2026/09/25 16:50:54 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4116lb # [ 69.160915] niks3-server[985]: 2026/09/25 16:50:54 INFO Received complete push request method=POST path=/api/pushes/9/complete4117worker2 # [ 69.186483] nix-grpc-daemon[876]: time=2026-09-25T16:50:54.781Z level=INFO msg="Upload complete. (182ms)"4118lb # [ 69.166532] niks3-server[985]: 2026/09/25 16:50:54 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4119worker2 # [ 69.192788] nix-grpc-daemon[771]: ts=2026-09-25T16:50:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 paths=1 nar_bytes_in=5124120worker2 # [ 69.195777] nix-grpc-daemon[771]: ts=2026-09-25T16:50:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044121worker2 # [ 69.207300] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4122worker2 # [ 69.210862] nix-daemon[842]: reaped child process 874, status = succeeded4123client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4124worker2 # [ 69.218985] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4125client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4126client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4127client # farm-a> LOG-farm-a-sig4128worker2 # [ 70.872403] nix-daemon[842]: reaped child process 895, status = succeeded4129worker2 # [ 70.970713] nix-daemon[842]: accepted connection from pid 911, user nix-grpc-daemon (trusted)4130worker2 # [ 70.991720] nix-daemon[842]: reaped child process 914, status = succeeded4131lb # [ 70.972937] niks3-server[985]: 2026/09/25 16:50:56 INFO Received push request method=POST path=/api/pushes4132worker2 # [ 71.000468] nix-grpc-daemon[876]: time=2026-09-25T16:50:56.594Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4133worker2 # [ 71.002094] nix-grpc-daemon[876]: time=2026-09-25T16:50:56.594Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4134lb # [ 71.001319] niks3-server[985]: 2026/09/25 16:50:56 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign4135worker2 # [ 71.029147] nix-grpc-daemon[876]: time=2026-09-25T16:50:56.623Z level=INFO msg="Uploading 1 narinfos"4136lb # [ 71.009619] niks3-server[985]: 2026/09/25 16:50:56 INFO Signed narinfos id=10 count=14137lb # [ 71.015562] niks3-server[985]: 2026/09/25 16:50:56 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4138lb # [ 71.021084] niks3-server[985]: 2026/09/25 16:50:56 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4139lb # [ 71.023780] niks3-server[985]: 2026/09/25 16:50:56 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4140lb # [ 71.034091] niks3-server[985]: 2026/09/25 16:50:56 INFO Received complete push request method=POST path=/api/pushes/10/complete4141lb # [ 71.038581] niks3-server[985]: 2026/09/25 16:50:56 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4142worker2 # [ 71.062260] nix-grpc-daemon[876]: time=2026-09-25T16:50:56.657Z level=INFO msg="Upload complete. (233ms)"4143worker2 # [ 71.068743] nix-grpc-daemon[771]: ts=2026-09-25T16:50:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04144worker1 # [ 71.068470] nix-grpc-daemon[487]: ts=2026-09-25T16:50:56Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04145worker1 # [ 71.072094] nix-grpc-daemon[487]: ts=2026-09-25T16:50:56Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=14146worker2 # [ 71.083488] nix-grpc-daemon[771]: ts=2026-09-25T16:50:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044147worker2 # [ 71.088305] nix-daemon[842]: reaped child process 893, status = succeeded4148worker2 # [ 71.093511] nix-grpc-daemon[771]: ts=2026-09-25T16:50:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:511044149worker2 # [ 71.101198] nix-grpc-daemon[876]: time=2026-09-25T16:50:56.696Z level=INFO msg="All 1 paths already cached"4150client # copying 1 paths to 'lb:50051'...4151client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4152worker2 # [ 71.107359] nix-grpc-daemon[771]: ts=2026-09-25T16:50:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:511044153worker2 # [ 71.115132] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4154worker2 # [ 71.233724] nix-daemon[842]: accepted connection from pid 918, user nix-grpc-daemon (trusted)4155worker2 # [ 71.250406] nix-daemon[842]: reaped child process 921, status = succeeded4156lb # [ 71.235476] niks3-server[985]: 2026/09/25 16:50:56 INFO Received push request method=POST path=/api/pushes4157worker2 # [ 71.261197] nix-grpc-daemon[876]: time=2026-09-25T16:50:56.855Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4158worker2 # [ 71.262862] nix-grpc-daemon[876]: time=2026-09-25T16:50:56.856Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4159lb # [ 71.249908] niks3-server[985]: 2026/09/25 16:50:56 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign4160worker2 # [ 71.273568] nix-grpc-daemon[876]: time=2026-09-25T16:50:56.868Z level=INFO msg="Uploading 1 narinfos"4161lb # [ 71.256637] niks3-server[985]: 2026/09/25 16:50:56 INFO Signed narinfos id=11 count=14162lb # [ 71.258406] niks3-server[985]: 2026/09/25 16:50:56 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4163lb # [ 71.266481] niks3-server[985]: 2026/09/25 16:50:56 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4164lb # [ 71.276135] niks3-server[985]: 2026/09/25 16:50:56 INFO Received complete push request method=POST path=/api/pushes/11/complete4165worker2 # [ 71.300442] nix-grpc-daemon[876]: time=2026-09-25T16:50:56.895Z level=INFO msg="Upload complete. (174ms)"4166lb # [ 71.281113] niks3-server[985]: 2026/09/25 16:50:56 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4167worker2 # [ 71.307484] nix-grpc-daemon[771]: ts=2026-09-25T16:50:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 paths=1 nar_bytes_in=6244168worker2 # [ 71.314018] nix-grpc-daemon[771]: ts=2026-09-25T16:50:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044169worker2 # [ 71.320176] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4170worker2 # [ 71.322353] nix-daemon[842]: reaped child process 916, status = succeeded4171client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4172client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4173worker2 # [ 71.331514] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4174client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4175client # farm-b> LOG-farm-b-sig4176worker2 # [ 72.184216] nix-daemon[842]: reaped child process 925, status = succeeded4177worker2 # [ 72.245311] nix-daemon[842]: accepted connection from pid 941, user nix-grpc-daemon (trusted)4178worker2 # [ 72.262023] nix-daemon[842]: reaped child process 944, status = succeeded4179lb # [ 72.247222] niks3-server[985]: 2026/09/25 16:50:57 INFO Received push request method=POST path=/api/pushes4180worker2 # [ 72.274187] nix-grpc-daemon[876]: time=2026-09-25T16:50:57.868Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4181worker2 # [ 72.275677] nix-grpc-daemon[876]: time=2026-09-25T16:50:57.868Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4182lb # [ 72.272433] niks3-server[985]: 2026/09/25 16:50:57 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign4183worker2 # [ 72.298794] nix-grpc-daemon[876]: time=2026-09-25T16:50:57.893Z level=INFO msg="Uploading 1 narinfos"4184lb # [ 72.281557] niks3-server[985]: 2026/09/25 16:50:57 INFO Signed narinfos id=12 count=14185lb # [ 72.286350] niks3-server[985]: 2026/09/25 16:50:57 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4186lb # [ 72.293579] niks3-server[985]: 2026/09/25 16:50:57 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4187lb # [ 72.302084] niks3-server[985]: 2026/09/25 16:50:57 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4188lb # [ 72.307012] niks3-server[985]: 2026/09/25 16:50:57 INFO Received complete push request method=POST path=/api/pushes/12/complete4189worker2 # [ 72.331649] nix-grpc-daemon[876]: time=2026-09-25T16:50:57.926Z level=INFO msg="Upload complete. (193ms)"4190lb # [ 72.311744] niks3-server[985]: 2026/09/25 16:50:57 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4191worker2 # [ 72.340648] nix-grpc-daemon[771]: ts=2026-09-25T16:50:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04192worker1 # [ 72.339671] nix-grpc-daemon[487]: ts=2026-09-25T16:50:57Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04193worker1 # [ 72.343034] nix-grpc-daemon[487]: ts=2026-09-25T16:50:57Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=14194worker2 # [ 72.350938] nix-grpc-daemon[771]: ts=2026-09-25T16:50:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044195worker2 # [ 72.354939] nix-daemon[842]: reaped child process 923, status = succeeded4196worker2 # [ 72.359980] nix-grpc-daemon[771]: ts=2026-09-25T16:50:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:511044197worker2 # [ 72.367836] nix-grpc-daemon[876]: time=2026-09-25T16:50:57.962Z level=INFO msg="All 2 paths already cached"4198client # copying 1 paths to 'lb:50051'...4199client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4200worker2 # [ 72.374150] nix-grpc-daemon[771]: ts=2026-09-25T16:50:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:511044201worker2 # [ 72.381589] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4202worker2 # [ 72.495844] nix-daemon[842]: accepted connection from pid 948, user nix-grpc-daemon (trusted)4203worker2 # [ 72.513738] nix-daemon[842]: reaped child process 951, status = succeeded4204lb # [ 72.495732] niks3-server[985]: 2026/09/25 16:50:58 INFO Received push request method=POST path=/api/pushes4205worker2 # [ 72.521451] nix-grpc-daemon[876]: time=2026-09-25T16:50:58.116Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4206worker2 # [ 72.522991] nix-grpc-daemon[876]: time=2026-09-25T16:50:58.116Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4207lb # [ 72.510756] niks3-server[985]: 2026/09/25 16:50:58 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign4208worker2 # [ 72.536030] nix-grpc-daemon[876]: time=2026-09-25T16:50:58.130Z level=INFO msg="Uploading 1 narinfos"4209lb # [ 72.519381] niks3-server[985]: 2026/09/25 16:50:58 INFO Signed narinfos id=13 count=14210lb # [ 72.523893] niks3-server[985]: 2026/09/25 16:50:58 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4211lb # [ 72.532164] niks3-server[985]: 2026/09/25 16:50:58 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4212lb # [ 72.534698] niks3-server[985]: 2026/09/25 16:50:58 INFO Received complete push request method=POST path=/api/pushes/13/complete4213worker2 # [ 72.560551] nix-grpc-daemon[876]: time=2026-09-25T16:50:58.155Z level=INFO msg="Upload complete. (170ms)"4214lb # [ 72.541551] niks3-server[985]: 2026/09/25 16:50:58 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4215worker2 # [ 72.568303] nix-grpc-daemon[771]: ts=2026-09-25T16:50:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 paths=1 nar_bytes_in=7524216worker2 # [ 72.571359] nix-grpc-daemon[771]: ts=2026-09-25T16:50:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044217worker2 # [ 72.580657] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4218worker2 # [ 72.582750] nix-daemon[842]: reaped child process 946, status = succeeded4219client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4220worker2 # [ 72.591166] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4221client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4222client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4223client # farm-top> LOG-farm-top-sig4224worker2 # [ 73.640623] nix-daemon[842]: reaped child process 955, status = succeeded4225worker2 # [ 73.736939] nix-daemon[842]: accepted connection from pid 971, user nix-grpc-daemon (trusted)4226worker2 # [ 73.752513] nix-daemon[842]: reaped child process 974, status = succeeded4227lb # [ 73.738089] niks3-server[985]: 2026/09/25 16:50:59 INFO Received push request method=POST path=/api/pushes4228worker2 # [ 73.764370] nix-grpc-daemon[876]: time=2026-09-25T16:50:59.359Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4229worker2 # [ 73.767682] nix-grpc-daemon[876]: time=2026-09-25T16:50:59.361Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4230lb # [ 73.763282] niks3-server[985]: 2026/09/25 16:50:59 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4231worker2 # [ 73.792964] nix-grpc-daemon[876]: time=2026-09-25T16:50:59.387Z level=INFO msg="Uploading 1 narinfos"4232lb # [ 73.771956] niks3-server[985]: 2026/09/25 16:50:59 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4233lb # [ 73.775361] niks3-server[985]: 2026/09/25 16:50:59 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign4234lb # [ 73.788074] niks3-server[985]: 2026/09/25 16:50:59 INFO Signed narinfos id=14 count=14235lb # [ 73.789894] niks3-server[985]: 2026/09/25 16:50:59 INFO Received complete push request method=POST path=/api/pushes/14/complete4236worker2 # [ 73.815758] nix-grpc-daemon[876]: time=2026-09-25T16:50:59.410Z level=INFO msg="Upload complete. (214ms)"4237lb # [ 73.796535] niks3-server[985]: 2026/09/25 16:50:59 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4238lb # [ 73.800577] niks3-server[985]: 2026/09/25 16:50:59 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4239worker2 # [ 73.822982] nix-grpc-daemon[771]: ts=2026-09-25T16:50:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=04240worker2 # [ 73.833778] nix-daemon[842]: reaped child process 953, status = succeeded4241client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.24 seconds)4242client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4243client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.14 seconds)4244client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4245worker1 # [ 74.106885] nix-grpc-daemon[487]: ts=2026-09-25T16:50:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:340524246client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4247client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4248client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4249client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4250worker1 # [ 74.125194] nix-grpc-daemon[487]: ts=2026-09-25T16:50:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:340524251worker1 # [ 74.130285] nix-grpc-daemon[487]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4252worker1 # [ 74.140497] nix-grpc-daemon[487]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4253worker1 # [ 74.148121] nix-grpc-daemon[487]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4254client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.19 seconds)4255client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4256worker2 # [ 74.308410] nix-grpc-daemon[771]: ts=2026-09-25T16:50:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:509884257worker2 # [ 74.312671] nix-grpc-daemon[771]: ts=2026-09-25T16:50:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:509884258client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.15 seconds)4259client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4260client # finding garbage collector roots...4261client # removing stale temporary roots file "/nix/var/nix/temproots/847"4262client # removing stale temporary roots file "/nix/var/nix/temproots/855"4263client # removing stale temporary roots file "/nix/var/nix/temproots/799"4264client # removing stale temporary roots file "/nix/var/nix/temproots/814"4265client # removing stale temporary roots file "/nix/var/nix/temproots/891"4266client # removing stale temporary roots file "/nix/var/nix/temproots/835"4267client # removing stale temporary roots file "/nix/var/nix/temproots/876"4268client # removing stale temporary roots file "/nix/var/nix/temproots/868"4269client # deleting specified paths...4270client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4271client # deleting unused links...4272client # note: hard linking is currently saving 0.0 KiB4273worker1 # [ 74.589128] nix-grpc-daemon[487]: ts=2026-09-25T16:51:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:382304274worker1 # [ 74.594796] nix-grpc-daemon[487]: ts=2026-09-25T16:51:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382304275worker1 # [ 74.601025] nix-grpc-daemon[870]: time=2026-09-25T16:51:00.191Z level=INFO msg="All 1 paths already cached"4276worker1 # [ 74.605569] nix-grpc-daemon[487]: ts=2026-09-25T16:51:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:382304277worker1 # [ 74.610452] nix-grpc-daemon[487]: ts=2026-09-25T16:51:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:382304278worker2 # [ 74.619325] nix-grpc-daemon[771]: ts=2026-09-25T16:51:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:511044279client # copying 3 paths...4280worker2 # [ 74.628261] nix-grpc-daemon[771]: ts=2026-09-25T16:51:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:511044281client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4282worker1 # [ 74.640365] nix-grpc-daemon[487]: ts=2026-09-25T16:51:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:382304283worker1 # [ 74.644378] nix-grpc-daemon[487]: ts=2026-09-25T16:51:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=0 paths=1 nar_bytes_out=2324284worker2 # [ 74.653669] nix-grpc-daemon[771]: ts=2026-09-25T16:51:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:511044285worker2 # [ 74.657734] nix-grpc-daemon[771]: ts=2026-09-25T16:51:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 paths=1 nar_bytes_out=1764286worker2 # [ 74.666594] nix-grpc-daemon[771]: ts=2026-09-25T16:51:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:511044287worker2 # [ 74.670218] nix-grpc-daemon[771]: ts=2026-09-25T16:51:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 paths=1 nar_bytes_out=1284288client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4289client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4290client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.37 seconds)4291(finished: subtest: outputs read back through any worker carry the cache signature, in 6.09 seconds)4292subtest: two clients wanting the same drv build it once4293client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4294client # [ 74.714204] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4295client # Running as unit: dedup1.service; invocation ID: f65925fd93cb4b9b8ed1d27f04bbecfd4296client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.04 seconds)4297client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4298worker2 # [ 74.903657] nix-grpc-daemon[771]: ts=2026-09-25T16:51:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:511044299client # [ 74.888207] nix[944]: don't know how to build these paths:4300client # [ 74.889242] nix[944]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4301worker1 # [ 74.913013] nix-grpc-daemon[487]: ts=2026-09-25T16:51:00Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04302worker1 # [ 74.918272] nix-grpc-daemon[487]: ts=2026-09-25T16:51:00Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14303worker1 # [ 74.923125] nix-grpc-daemon[487]: ts=2026-09-25T16:51:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382304304worker1 # [ 74.927605] nix-grpc-daemon[487]: ts=2026-09-25T16:51:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382304305client # [ 74.917361] nix[944]: copying 1 paths to 'lb:50051'...4306client # [ 74.918483] nix[944]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4307worker1 # [ 74.933775] nix-grpc-daemon[487]: ts=2026-09-25T16:51:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:382304308worker1 # [ 74.943322] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4309worker1 # [ 75.069780] nix-daemon[856]: accepted connection from pid 1056, user nix-grpc-daemon (trusted)4310worker1 # [ 75.089012] nix-daemon[856]: reaped child process 1059, status = succeeded4311lb # [ 75.074641] niks3-server[985]: 2026/09/25 16:51:00 INFO Received push request method=POST path=/api/pushes4312worker1 # [ 75.097103] nix-grpc-daemon[870]: time=2026-09-25T16:51:00.688Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4313worker1 # [ 75.098695] nix-grpc-daemon[870]: time=2026-09-25T16:51:00.688Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4314lb # [ 75.092078] niks3-server[985]: 2026/09/25 16:51:00 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign4315worker1 # [ 75.113505] nix-grpc-daemon[870]: time=2026-09-25T16:51:00.704Z level=INFO msg="Uploading 1 narinfos"4316lb # [ 75.100319] niks3-server[985]: 2026/09/25 16:51:00 INFO Signed narinfos id=15 count=14317lb # [ 75.102110] niks3-server[985]: 2026/09/25 16:51:00 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4318lb # [ 75.113561] niks3-server[985]: 2026/09/25 16:51:00 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4319worker1 # [ 75.134676] nix-grpc-daemon[870]: time=2026-09-25T16:51:00.725Z level=INFO msg="Upload complete. (181ms)"4320lb # [ 75.118468] niks3-server[985]: 2026/09/25 16:51:00 INFO Received complete push request method=POST path=/api/pushes/15/complete4321lb # [ 75.123073] niks3-server[985]: 2026/09/25 16:51:00 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4322worker1 # [ 75.140586] nix-grpc-daemon[487]: ts=2026-09-25T16:51:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=0 paths=1 nar_bytes_in=4324323worker1 # [ 75.150254] nix-grpc-daemon[487]: ts=2026-09-25T16:51:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382304324worker1 # [ 75.158229] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4325worker1 # [ 75.160084] nix-daemon[856]: reaped child process 1054, status = succeeded4326client # [ 75.151451] nix[944]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4327worker1 # [ 75.170372] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4328client # [ 75.778259] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4329client # Running as unit: dedup2.service; invocation ID: 9797ea7b7c564d93ae4cb7b4718594834330client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.07 seconds)4331worker2 # [ 75.990305] nix-grpc-daemon[771]: ts=2026-09-25T16:51:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:511044332client # [ 75.975762] nix[960]: don't know how to build these paths:4333client # [ 75.977040] nix[960]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4334worker1 # [ 75.996851] nix-grpc-daemon[487]: ts=2026-09-25T16:51:01Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04335worker1 # [ 76.000860] nix-grpc-daemon[487]: ts=2026-09-25T16:51:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382304336client # [ 75.988511] nix[960]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4337client: must succeed: sleep 24338lb # [ 77.043550] envoy[886]: conn peer=[2001:db8:1::1]:37404 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4168 tx=1896 ms=19474339lb # [ 77.046396] envoy[886]: conn peer=[2001:db8:1::1]:37406 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9511 tx=4331 ms=50534340lb # [ 77.049126] envoy[886]: conn peer=[2001:db8:1::1]:48674 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1091 tx=2908 ms=974341lb # [ 77.051550] envoy[886]: conn peer=[2001:db8:1::1]:48708 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=284342lb # [ 77.054090] envoy[886]: conn peer=[2001:db8:1::1]:48700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=375 ms=374343lb # [ 77.056515] envoy[886]: conn peer=[2001:db8:1::1]:48690 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=409 ms=464344client: (finished: must succeed: sleep 2, in 2.03 seconds)4345client: waiting for success: ! systemctl is-active dedup1 dedup24346worker1 # [ 79.199703] nix-daemon[856]: reaped child process 1063, status = succeeded4347worker1 # [ 79.312790] nix-daemon[856]: accepted connection from pid 1095, user nix-grpc-daemon (trusted)4348worker1 # [ 79.331883] nix-daemon[856]: reaped child process 1098, status = succeeded4349lb # [ 79.319093] niks3-server[985]: 2026/09/25 16:51:04 INFO Received push request method=POST path=/api/pushes4350worker1 # [ 79.344371] nix-grpc-daemon[870]: time=2026-09-25T16:51:04.933Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4351worker1 # [ 79.345883] nix-grpc-daemon[870]: time=2026-09-25T16:51:04.933Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4352lb # [ 79.355934] niks3-server[985]: 2026/09/25 16:51:04 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign4353worker1 # [ 79.379767] nix-grpc-daemon[870]: time=2026-09-25T16:51:04.970Z level=INFO msg="Uploading 1 narinfos"4354lb # [ 79.364129] niks3-server[985]: 2026/09/25 16:51:04 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4355lb # [ 79.366862] niks3-server[985]: 2026/09/25 16:51:04 INFO Signed narinfos id=16 count=14356lb # [ 79.373627] niks3-server[985]: 2026/09/25 16:51:04 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4357lb # [ 79.384576] niks3-server[985]: 2026/09/25 16:51:04 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4358worker1 # [ 79.406802] nix-grpc-daemon[870]: time=2026-09-25T16:51:04.998Z level=INFO msg="Upload complete. (222ms)"4359lb # [ 79.389741] niks3-server[985]: 2026/09/25 16:51:04 INFO Received complete push request method=POST path=/api/pushes/16/complete4360lb # [ 79.392682] niks3-server[985]: 2026/09/25 16:51:04 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4361worker1 # [ 79.415520] nix-grpc-daemon[487]: ts=2026-09-25T16:51:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04362worker1 # [ 79.427713] nix-grpc-daemon[487]: ts=2026-09-25T16:51:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04363worker1 # [ 79.433769] nix-daemon[856]: reaped child process 1061, status = succeeded4364client # [ 79.445684] systemd[1]: dedup2.service: Deactivated successfully.4365client # [ 79.446751] systemd[1]: dedup2.service: Consumed 177ms CPU time over 3.668s wall clock time, 35.2M memory peak, 6.4K incoming IP traffic, 5.9K outgoing IP traffic.4366client # [ 79.453642] systemd[1]: dedup1.service: Deactivated successfully.4367client # [ 79.456528] systemd[1]: dedup1.service: Consumed 152ms CPU time over 4.741s wall clock time, 41.5M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic.4368client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.07 seconds)4369client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4370client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.05 seconds)4371worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4372worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4373worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4374worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4375worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4376worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4377worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4378worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4379(finished: subtest: two clients wanting the same drv build it once, in 5.69 seconds)4380subtest: --timeout and --max-silent-time reach the builder4381worker1 # [ 80.560294] nix-grpc-daemon[487]: ts=2026-09-25T16:51:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:382304382worker1 # [ 80.569823] nix-grpc-daemon[487]: ts=2026-09-25T16:51:06Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04383worker2 # [ 80.576491] nix-grpc-daemon[771]: ts=2026-09-25T16:51:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044384worker1 # [ 80.573194] nix-grpc-daemon[487]: ts=2026-09-25T16:51:06Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=14385worker2 # [ 80.585471] nix-grpc-daemon[771]: ts=2026-09-25T16:51:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:511044386worker2 # [ 80.593683] nix-grpc-daemon[771]: ts=2026-09-25T16:51:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:511044387worker2 # [ 80.602349] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4388worker2 # [ 80.730856] nix-daemon[842]: accepted connection from pid 999, user nix-grpc-daemon (trusted)4389worker2 # [ 80.749772] nix-daemon[842]: reaped child process 1002, status = succeeded4390lb # [ 80.727642] niks3-server[985]: 2026/09/25 16:51:06 INFO Received push request method=POST path=/api/pushes4391worker2 # [ 80.754162] nix-grpc-daemon[876]: time=2026-09-25T16:51:06.348Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4392worker2 # [ 80.755668] nix-grpc-daemon[876]: time=2026-09-25T16:51:06.348Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4393lb # [ 80.746499] niks3-server[985]: 2026/09/25 16:51:06 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4394worker2 # [ 80.774596] nix-grpc-daemon[876]: time=2026-09-25T16:51:06.369Z level=INFO msg="Uploading 1 narinfos"4395lb # [ 80.755508] niks3-server[985]: 2026/09/25 16:51:06 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign4396lb # [ 80.761422] niks3-server[985]: 2026/09/25 16:51:06 INFO Signed narinfos id=17 count=14397lb # [ 80.767892] niks3-server[985]: 2026/09/25 16:51:06 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4398worker2 # [ 80.793909] nix-grpc-daemon[876]: time=2026-09-25T16:51:06.389Z level=INFO msg="Upload complete. (182ms)"4399lb # [ 80.775087] niks3-server[985]: 2026/09/25 16:51:06 INFO Received complete push request method=POST path=/api/pushes/17/complete4400lb # [ 80.778660] niks3-server[985]: 2026/09/25 16:51:06 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4401worker2 # [ 80.800189] nix-grpc-daemon[771]: ts=2026-09-25T16:51:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 paths=1 nar_bytes_in=4244402worker2 # [ 80.809166] nix-grpc-daemon[771]: ts=2026-09-25T16:51:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044403worker2 # [ 80.815801] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4404worker2 # [ 80.818002] nix-daemon[842]: reaped child process 997, status = succeeded4405worker2 # [ 80.826735] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4406worker2 # [ 87.041644] nix-grpc-daemon[771]: ts=2026-09-25T16:51:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04407lb # [ 87.044284] envoy[886]: conn peer=[2001:db8:1::1]:48710 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=1519 ms=45384408worker2 # [ 87.067965] nix-daemon[842]: reaped child process 1004, status = succeeded4409lb # [ 87.047184] envoy[886]: conn peer=[2001:db8:1::1]:48716 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1473 tx=1399 ms=34574410worker2 # [ 87.069911] nix-daemon[842]: reaped child process 1006, status = succeeded4411lb # [ 87.050145] envoy[886]: conn peer=[2001:db8:1::1]:48720 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2971 tx=958 ms=64964412worker1 # [ 87.236593] nix-grpc-daemon[487]: ts=2026-09-25T16:51:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:382304413worker1 # [ 87.247588] nix-grpc-daemon[487]: ts=2026-09-25T16:51:12Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04414worker1 # [ 87.251906] nix-grpc-daemon[487]: ts=2026-09-25T16:51:12Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=14415worker1 # [ 87.257267] nix-grpc-daemon[487]: ts=2026-09-25T16:51:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382304416worker1 # [ 87.263806] nix-grpc-daemon[487]: ts=2026-09-25T16:51:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382304417worker1 # [ 87.272544] nix-grpc-daemon[487]: ts=2026-09-25T16:51:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:382304418worker1 # [ 87.281471] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4419worker1 # [ 87.411462] nix-daemon[856]: accepted connection from pid 1115, user nix-grpc-daemon (trusted)4420worker1 # [ 87.430584] nix-daemon[856]: reaped child process 1118, status = succeeded4421lb # [ 87.420244] niks3-server[985]: 2026/09/25 16:51:13 INFO Received push request method=POST path=/api/pushes4422worker1 # [ 87.442996] nix-grpc-daemon[870]: time=2026-09-25T16:51:13.033Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4423worker1 # [ 87.445207] nix-grpc-daemon[870]: time=2026-09-25T16:51:13.033Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4424lb # [ 87.441815] niks3-server[985]: 2026/09/25 16:51:13 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign4425worker1 # [ 87.462499] nix-grpc-daemon[870]: time=2026-09-25T16:51:13.053Z level=INFO msg="Uploading 1 narinfos"4426lb # [ 87.448544] niks3-server[985]: 2026/09/25 16:51:13 INFO Signed narinfos id=18 count=14427lb # [ 87.454373] niks3-server[985]: 2026/09/25 16:51:13 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4428lb # [ 87.461312] niks3-server[985]: 2026/09/25 16:51:13 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4429lb # [ 87.469724] niks3-server[985]: 2026/09/25 16:51:13 INFO Received complete push request method=POST path=/api/pushes/18/complete4430worker1 # [ 87.490837] nix-grpc-daemon[870]: time=2026-09-25T16:51:13.082Z level=INFO msg="Upload complete. (199ms)"4431lb # [ 87.475486] niks3-server[985]: 2026/09/25 16:51:13 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4432worker1 # [ 87.496196] nix-grpc-daemon[487]: ts=2026-09-25T16:51:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=0 paths=1 nar_bytes_in=4244433worker1 # [ 87.504476] nix-grpc-daemon[487]: ts=2026-09-25T16:51:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382304434worker1 # [ 87.514282] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4435worker1 # [ 87.518167] nix-daemon[856]: reaped child process 1113, status = succeeded4436worker1 # [ 87.527294] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4437worker1 # [ 93.541211] nix-grpc-daemon[487]: ts=2026-09-25T16:51:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04438worker1 # [ 93.559713] nix-daemon[856]: reaped child process 1122, status = succeeded4439worker1 # [ 93.561053] nix-daemon[856]: reaped child process 1120, status = succeeded4440worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4441worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4442worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4443worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4444(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.33 seconds)4445subtest: a failing build is counted with its own reason4446worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4447worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4448worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4449worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4450worker1 # [ 93.957288] nix-grpc-daemon[487]: ts=2026-09-25T16:51:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:382304451worker1 # [ 93.966569] nix-grpc-daemon[487]: ts=2026-09-25T16:51:19Z level=debug event=want client=9 drv=j4rjl229w5xzbr9614xcipiv7652fdka-fail-boom.drv cached=04452worker1 # [ 93.969932] nix-grpc-daemon[487]: ts=2026-09-25T16:51:19Z level=debug event=assigned drv=j4rjl229w5xzbr9614xcipiv7652fdka-fail-boom.drv worker=192.168.1.4:50051 clients=14453worker2 # [ 93.976868] nix-grpc-daemon[771]: ts=2026-09-25T16:51:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044454worker2 # [ 93.986342] nix-grpc-daemon[771]: ts=2026-09-25T16:51:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:511044455worker2 # [ 93.994774] nix-grpc-daemon[771]: ts=2026-09-25T16:51:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:511044456worker2 # [ 94.003840] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4457worker2 # [ 94.132871] nix-daemon[842]: accepted connection from pid 1035, user nix-grpc-daemon (trusted)4458worker2 # [ 94.153061] nix-daemon[842]: reaped child process 1038, status = succeeded4459lb # [ 94.138885] niks3-server[985]: 2026/09/25 16:51:19 INFO Received push request method=POST path=/api/pushes4460worker2 # [ 94.165557] nix-grpc-daemon[876]: time=2026-09-25T16:51:19.759Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4461worker2 # [ 94.167174] nix-grpc-daemon[876]: time=2026-09-25T16:51:19.760Z level=INFO msg="Uploading j4rjl229w5xzbr9614xcipiv7652fdka-fail-boom.drv (400B)"4462lb # [ 94.157683] niks3-server[985]: 2026/09/25 16:51:19 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign4463worker2 # [ 94.183262] nix-grpc-daemon[876]: time=2026-09-25T16:51:19.778Z level=INFO msg="Uploading 1 narinfos"4464lb # [ 94.164604] niks3-server[985]: 2026/09/25 16:51:19 INFO Signed narinfos id=19 count=14465lb # [ 94.166352] niks3-server[985]: 2026/09/25 16:51:19 INFO Registered completed upload object_key=j4rjl229w5xzbr9614xcipiv7652fdka.ls4466lb # [ 94.174734] niks3-server[985]: 2026/09/25 16:51:19 INFO Registered completed upload object_key=nar/0d79vqzwc4s1agk431w3csq2p4brahnzmanwj14fjf383ld797i6.nar.zst4467lb # [ 94.185197] niks3-server[985]: 2026/09/25 16:51:19 INFO Received complete push request method=POST path=/api/pushes/19/complete4468worker2 # [ 94.210751] nix-grpc-daemon[876]: time=2026-09-25T16:51:19.805Z level=INFO msg="Upload complete. (196ms)"4469lb # [ 94.191683] niks3-server[985]: 2026/09/25 16:51:19 INFO Registered completed upload object_key=j4rjl229w5xzbr9614xcipiv7652fdka.narinfo4470worker2 # [ 94.216673] nix-grpc-daemon[771]: ts=2026-09-25T16:51:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 paths=1 nar_bytes_in=4004471worker2 # [ 94.224462] nix-grpc-daemon[771]: ts=2026-09-25T16:51:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044472worker2 # [ 94.232885] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4473worker2 # [ 94.236790] nix-daemon[842]: reaped child process 1033, status = succeeded4474worker2 # [ 94.248626] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4475worker2 # [ 95.058205] nix-grpc-daemon[771]: ts=2026-09-25T16:51:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 drv=j4rjl229w5xzbr9614xcipiv7652fdka-fail-boom.drv assign_id=11 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04476worker2 # [ 95.081557] nix-daemon[842]: reaped child process 1040, status = succeeded4477worker2 # [ 95.083354] nix-daemon[842]: reaped child process 1042, status = succeeded4478worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4479worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4480worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4481worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4482worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4483worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4484worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4485worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4486worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4487worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4488worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4489worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4490worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4491worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4492worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4493worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4494(finished: subtest: a failing build is counted with its own reason, in 1.65 seconds)4495subtest: the first client leaving does not fail the second4496client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4497client # [ 95.384944] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4498client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.05 seconds)4499client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4500worker2 # [ 95.584990] nix-grpc-daemon[771]: ts=2026-09-25T16:51:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:511044501client # Running as unit: share1.service; invocation ID: b1e67ce0896745b19a452478e04989934502client # [ 95.569814] nix[1048]: don't know how to build these paths:4503client # [ 95.572659] nix[1048]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4504worker1 # [ 95.596225] nix-grpc-daemon[487]: ts=2026-09-25T16:51:21Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04505worker1 # [ 95.601053] nix-grpc-daemon[487]: ts=2026-09-25T16:51:21Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14506worker1 # [ 95.605574] nix-grpc-daemon[487]: ts=2026-09-25T16:51:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382304507worker1 # [ 95.609598] nix-grpc-daemon[487]: ts=2026-09-25T16:51:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382304508client # [ 95.599076] nix[1048]: copying 1 paths to 'lb:50051'...4509client # [ 95.600206] nix[1048]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4510worker1 # [ 95.615445] nix-grpc-daemon[487]: ts=2026-09-25T16:51:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:382304511worker1 # [ 95.624363] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4512worker1 # [ 95.751556] nix-daemon[856]: accepted connection from pid 1167, user nix-grpc-daemon (trusted)4513worker1 # [ 95.767910] nix-daemon[856]: reaped child process 1170, status = succeeded4514lb # [ 95.757617] niks3-server[985]: 2026/09/25 16:51:21 INFO Received push request method=POST path=/api/pushes4515worker1 # [ 95.781455] nix-grpc-daemon[870]: time=2026-09-25T16:51:21.372Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4516worker1 # [ 95.783029] nix-grpc-daemon[870]: time=2026-09-25T16:51:21.372Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4517lb # [ 95.775296] niks3-server[985]: 2026/09/25 16:51:21 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign4518worker1 # [ 95.797544] nix-grpc-daemon[870]: time=2026-09-25T16:51:21.388Z level=INFO msg="Uploading 1 narinfos"4519lb # [ 95.785781] niks3-server[985]: 2026/09/25 16:51:21 INFO Signed narinfos id=20 count=14520lb # [ 95.790349] niks3-server[985]: 2026/09/25 16:51:21 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4521lb # [ 95.795302] niks3-server[985]: 2026/09/25 16:51:21 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4522lb # [ 95.805432] niks3-server[985]: 2026/09/25 16:51:21 INFO Received complete push request method=POST path=/api/pushes/20/complete4523worker1 # [ 95.827050] nix-grpc-daemon[870]: time=2026-09-25T16:51:21.418Z level=INFO msg="Upload complete. (193ms)"4524lb # [ 95.811843] niks3-server[985]: 2026/09/25 16:51:21 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4525worker1 # [ 95.834554] nix-grpc-daemon[487]: ts=2026-09-25T16:51:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=0 paths=1 nar_bytes_in=4324526worker1 # [ 95.843468] nix-grpc-daemon[487]: ts=2026-09-25T16:51:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382304527worker1 # [ 95.849472] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4528worker1 # [ 95.853662] nix-daemon[856]: reaped child process 1165, status = succeeded4529client # [ 95.845509] nix[1048]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4530worker1 # [ 95.864796] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4531client # [ 96.459829] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4532client # Running as unit: share2.service; invocation ID: 3cc5beacf4e74ff2806b8e2a1b18d31b4533client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.08 seconds)4534worker2 # [ 96.665834] nix-grpc-daemon[771]: ts=2026-09-25T16:51:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:511044535client # [ 96.650663] nix[1064]: don't know how to build these paths:4536client # [ 96.652418] nix[1064]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4537worker1 # [ 96.674955] nix-grpc-daemon[487]: ts=2026-09-25T16:51:22Z level=debug event=want client=11 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04538client # [ 96.663448] nix[1064]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4539worker1 # [ 96.679408] nix-grpc-daemon[487]: ts=2026-09-25T16:51:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382304540lb # [ 97.038568] envoy[886]: conn peer=[2001:db8:1::1]:45498 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2985 tx=932 ms=63214541lb # [ 97.041361] envoy[886]: conn peer=[2001:db8:1::1]:38030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2868 tx=1247 ms=11144542client: must succeed: sleep 2; systemctl kill -s INT share14543client # [ 99.712604] systemd[1]: share1.service: Sent signal SIGINT to main process 1048 (nix) on client request.4544client # [ 99.719239] nix[1048]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4545client # [ 99.727202] nix[1048]: error: interrupted by the user4546client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.05 seconds)4547client: waiting for success: ! systemctl is-active share14548client # [ 99.774214] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4549client # [ 99.777590] systemd[1]: share1.service: Failed with result 'exit-code'.4550client # [ 99.779259] systemd[1]: share1.service: Consumed 155ms CPU time over 4.392s wall clock time, 41.4M memory peak, 6.1K incoming IP traffic, 8.1K outgoing IP traffic.4551client: (finished: waiting for success: ! systemctl is-active share1, in 1.07 seconds)4552client: waiting for success: ! systemctl is-active share24553worker1 # [ 101.094806] nix-daemon[856]: reaped child process 1174, status = succeeded4554worker1 # [ 101.209010] nix-daemon[856]: accepted connection from pid 1210, user nix-grpc-daemon (trusted)4555lb # [ 101.211055] niks3-server[985]: 2026/09/25 16:51:26 INFO Received push request method=POST path=/api/pushes4556worker1 # [ 101.230937] nix-daemon[856]: reaped child process 1213, status = succeeded4557worker1 # [ 101.235372] nix-grpc-daemon[870]: time=2026-09-25T16:51:26.825Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4558worker1 # [ 101.237573] nix-grpc-daemon[870]: time=2026-09-25T16:51:26.825Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4559lb # [ 101.235644] niks3-server[985]: 2026/09/25 16:51:26 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign4560worker1 # [ 101.257900] nix-grpc-daemon[870]: time=2026-09-25T16:51:26.849Z level=INFO msg="Uploading 1 narinfos"4561lb # [ 101.242093] niks3-server[985]: 2026/09/25 16:51:26 INFO Signed narinfos id=21 count=14562lb # [ 101.247603] niks3-server[985]: 2026/09/25 16:51:26 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4563lb # [ 101.257830] niks3-server[985]: 2026/09/25 16:51:26 INFO Received complete push request method=POST path=/api/pushes/21/complete4564worker1 # [ 101.279112] nix-grpc-daemon[870]: time=2026-09-25T16:51:26.870Z level=INFO msg="Upload complete. (198ms)"4565lb # [ 101.264646] niks3-server[985]: 2026/09/25 16:51:26 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4566worker1 # [ 101.286706] nix-grpc-daemon[487]: ts=2026-09-25T16:51:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=12 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04567lb # [ 101.272767] niks3-server[985]: 2026/09/25 16:51:26 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4568lb # [ 101.275755] niks3-server[985]: 2026/09/25 16:51:26 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4569worker1 # [ 101.299969] nix-grpc-daemon[487]: ts=2026-09-25T16:51:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=12 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04570worker1 # [ 101.305280] nix-daemon[856]: reaped child process 1172, status = succeeded4571client # [ 101.306404] systemd[1]: share2.service: Deactivated successfully.4572client # [ 101.309327] systemd[1]: share2.service: Consumed 167ms CPU time over 4.847s wall clock time, 33.3M memory peak, 6.5K incoming IP traffic, 6K outgoing IP traffic.4573client: (finished: waiting for success: ! systemctl is-active share2, in 1.07 seconds)4574client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4575client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.04 seconds)4576(finished: subtest: the first client leaving does not fail the second, in 6.76 seconds)4577subtest: build hook: nix-daemon with builders = grpc://lb4578client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4579client # [ 102.230237] systemd[1]: Started Nix Daemon.4580client # [ 102.348576] nix-daemon[1116]: accepted connection from pid 1113, user root (trusted)4581worker1 # [ 102.508879] nix-grpc-daemon[487]: ts=2026-09-25T16:51:28Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:382304582worker2 # [ 102.515832] nix-grpc-daemon[771]: ts=2026-09-25T16:51:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:511044583worker1 # [ 102.520118] nix-grpc-daemon[487]: ts=2026-09-25T16:51:28Z level=debug event=want client=12 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04584worker1 # [ 102.523058] nix-grpc-daemon[487]: ts=2026-09-25T16:51:28Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14585worker2 # [ 102.532702] nix-grpc-daemon[771]: ts=2026-09-25T16:51:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044586worker2 # [ 102.542501] nix-grpc-daemon[771]: ts=2026-09-25T16:51:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:511044587worker2 # [ 102.551563] nix-grpc-daemon[771]: ts=2026-09-25T16:51:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:511044588worker2 # [ 102.561302] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4589worker2 # [ 102.700589] nix-daemon[842]: accepted connection from pid 1088, user nix-grpc-daemon (trusted)4590worker2 # [ 102.723663] nix-daemon[842]: reaped child process 1091, status = succeeded4591lb # [ 102.707109] niks3-server[985]: 2026/09/25 16:51:28 INFO Received push request method=POST path=/api/pushes4592worker2 # [ 102.734111] nix-grpc-daemon[876]: time=2026-09-25T16:51:28.328Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4593worker2 # [ 102.735734] nix-grpc-daemon[876]: time=2026-09-25T16:51:28.328Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4594lb # [ 102.724347] niks3-server[985]: 2026/09/25 16:51:28 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign4595worker2 # [ 102.751000] nix-grpc-daemon[876]: time=2026-09-25T16:51:28.346Z level=INFO msg="Uploading 1 narinfos"4596lb # [ 102.732961] niks3-server[985]: 2026/09/25 16:51:28 INFO Signed narinfos id=22 count=14597lb # [ 102.737730] niks3-server[985]: 2026/09/25 16:51:28 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4598lb # [ 102.744672] niks3-server[985]: 2026/09/25 16:51:28 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4599lb # [ 102.753408] niks3-server[985]: 2026/09/25 16:51:28 INFO Received complete push request method=POST path=/api/pushes/22/complete4600worker2 # [ 102.778973] nix-grpc-daemon[876]: time=2026-09-25T16:51:28.374Z level=INFO msg="Upload complete. (206ms)"4601lb # [ 102.759361] niks3-server[985]: 2026/09/25 16:51:28 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4602worker2 # [ 102.786281] nix-grpc-daemon[771]: ts=2026-09-25T16:51:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 paths=1 nar_bytes_in=5124603worker2 # [ 102.792233] nix-grpc-daemon[771]: ts=2026-09-25T16:51:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044604worker2 # [ 102.804713] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4605worker2 # [ 102.807046] nix-daemon[842]: reaped child process 1086, status = succeeded4606worker2 # [ 102.816965] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4607worker2 # [ 103.677005] nix-daemon[842]: reaped child process 1095, status = succeeded4608worker2 # [ 103.765195] nix-daemon[842]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)4609worker2 # [ 103.787196] nix-daemon[842]: reaped child process 1114, status = succeeded4610lb # [ 103.767869] niks3-server[985]: 2026/09/25 16:51:29 INFO Received push request method=POST path=/api/pushes4611worker2 # [ 103.796525] nix-grpc-daemon[876]: time=2026-09-25T16:51:29.389Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4612worker2 # [ 103.798159] nix-grpc-daemon[876]: time=2026-09-25T16:51:29.389Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4613lb # [ 103.799056] niks3-server[985]: 2026/09/25 16:51:29 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4614worker2 # [ 103.826096] nix-grpc-daemon[876]: time=2026-09-25T16:51:29.421Z level=INFO msg="Uploading 1 narinfos"4615lb # [ 103.807447] niks3-server[985]: 2026/09/25 16:51:29 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign4616lb # [ 103.813583] niks3-server[985]: 2026/09/25 16:51:29 INFO Signed narinfos id=23 count=14617lb # [ 103.815355] niks3-server[985]: 2026/09/25 16:51:29 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4618lb # [ 103.828554] niks3-server[985]: 2026/09/25 16:51:29 INFO Received complete push request method=POST path=/api/pushes/23/complete4619worker2 # [ 103.852786] nix-grpc-daemon[876]: time=2026-09-25T16:51:29.447Z level=INFO msg="Upload complete. (225ms)"4620lb # [ 103.833573] niks3-server[985]: 2026/09/25 16:51:29 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4621lb # [ 103.839398] niks3-server[985]: 2026/09/25 16:51:29 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4622worker2 # [ 103.860729] nix-grpc-daemon[771]: ts=2026-09-25T16:51:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=13 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04623worker2 # [ 103.876390] nix-grpc-daemon[771]: ts=2026-09-25T16:51:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:511044624worker2 # [ 103.879455] nix-daemon[842]: reaped child process 1093, status = succeeded4625worker1 # [ 103.880509] nix-grpc-daemon[487]: ts=2026-09-25T16:51:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:382304626worker1 # [ 103.888490] nix-grpc-daemon[487]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4627worker1 # [ 103.898903] nix-grpc-daemon[487]: ts=2026-09-25T16:51:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=0 paths=1 nar_bytes_out=1284628worker1 # [ 104.104297] nix-grpc-daemon[487]: ts=2026-09-25T16:51:29Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:382304629worker1 # [ 104.108797] nix-grpc-daemon[487]: ts=2026-09-25T16:51:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382304630worker1 # [ 104.113163] nix-grpc-daemon[870]: time=2026-09-25T16:51:29.703Z level=INFO msg="All 1 paths already cached"4631worker1 # [ 104.122371] nix-grpc-daemon[487]: ts=2026-09-25T16:51:29Z level=debug event=want client=13 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04632worker1 # [ 104.126058] nix-grpc-daemon[487]: ts=2026-09-25T16:51:29Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14633worker2 # [ 104.131987] nix-grpc-daemon[771]: ts=2026-09-25T16:51:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044634worker2 # [ 104.140706] nix-grpc-daemon[771]: ts=2026-09-25T16:51:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:511044635worker2 # [ 104.150095] nix-grpc-daemon[876]: time=2026-09-25T16:51:29.745Z level=INFO msg="All 1 paths already cached"4636worker2 # [ 104.156336] nix-grpc-daemon[771]: ts=2026-09-25T16:51:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:511044637worker2 # [ 104.165490] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4638worker2 # [ 104.307045] nix-daemon[842]: accepted connection from pid 1118, user nix-grpc-daemon (trusted)4639lb # [ 104.307032] niks3-server[985]: 2026/09/25 16:51:29 INFO Received push request method=POST path=/api/pushes4640worker2 # [ 104.333168] nix-grpc-daemon[876]: time=2026-09-25T16:51:29.927Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4641worker2 # [ 104.334905] nix-grpc-daemon[876]: time=2026-09-25T16:51:29.927Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4642worker2 # [ 104.337113] nix-daemon[842]: reaped child process 1121, status = succeeded4643lb # [ 104.324276] niks3-server[985]: 2026/09/25 16:51:29 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign4644worker2 # [ 104.349071] nix-grpc-daemon[876]: time=2026-09-25T16:51:29.944Z level=INFO msg="Uploading 1 narinfos"4645lb # [ 104.332450] niks3-server[985]: 2026/09/25 16:51:29 INFO Signed narinfos id=24 count=14646lb # [ 104.337440] niks3-server[985]: 2026/09/25 16:51:29 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4647lb # [ 104.348335] niks3-server[985]: 2026/09/25 16:51:29 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4648lb # [ 104.351528] niks3-server[985]: 2026/09/25 16:51:29 INFO Received complete push request method=POST path=/api/pushes/24/complete4649worker2 # [ 104.375794] nix-grpc-daemon[876]: time=2026-09-25T16:51:29.970Z level=INFO msg="Upload complete. (197ms)"4650lb # [ 104.356176] niks3-server[985]: 2026/09/25 16:51:29 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4651worker2 # [ 104.384563] nix-grpc-daemon[771]: ts=2026-09-25T16:51:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 paths=1 nar_bytes_in=6244652worker2 # [ 104.387874] nix-grpc-daemon[771]: ts=2026-09-25T16:51:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044653worker2 # [ 104.400442] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4654worker2 # [ 104.405124] nix-daemon[842]: reaped child process 1116, status = succeeded4655worker2 # [ 104.415724] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4656worker2 # [ 105.860686] nix-daemon[842]: reaped child process 1125, status = succeeded4657worker2 # [ 105.971850] nix-daemon[842]: accepted connection from pid 1141, user nix-grpc-daemon (trusted)4658lb # [ 105.971690] niks3-server[985]: 2026/09/25 16:51:31 INFO Received push request method=POST path=/api/pushes4659worker2 # [ 105.996222] nix-daemon[842]: reaped child process 1144, status = succeeded4660worker2 # [ 106.000549] nix-grpc-daemon[876]: time=2026-09-25T16:51:31.594Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4661worker2 # [ 106.002160] nix-grpc-daemon[876]: time=2026-09-25T16:51:31.594Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4662lb # [ 105.991986] niks3-server[985]: 2026/09/25 16:51:31 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4663lb # [ 106.003588] niks3-server[985]: 2026/09/25 16:51:31 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4664lb # [ 106.009824] niks3-server[985]: 2026/09/25 16:51:31 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign4665worker2 # [ 106.035428] nix-grpc-daemon[876]: time=2026-09-25T16:51:31.630Z level=INFO msg="Uploading 1 narinfos"4666lb # [ 106.016782] niks3-server[985]: 2026/09/25 16:51:31 INFO Signed narinfos id=25 count=14667lb # [ 106.018613] niks3-server[985]: 2026/09/25 16:51:31 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4668lb # [ 106.030066] niks3-server[985]: 2026/09/25 16:51:31 INFO Received complete push request method=POST path=/api/pushes/25/complete4669worker2 # [ 106.054902] nix-grpc-daemon[876]: time=2026-09-25T16:51:31.650Z level=INFO msg="Upload complete. (226ms)"4670lb # [ 106.036825] niks3-server[985]: 2026/09/25 16:51:31 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4671worker2 # [ 106.064754] nix-grpc-daemon[771]: ts=2026-09-25T16:51:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=14 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04672worker2 # [ 106.075945] nix-grpc-daemon[771]: ts=2026-09-25T16:51:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:511044673worker2 # [ 106.079674] nix-daemon[842]: reaped child process 1123, status = succeeded4674worker1 # [ 106.095368] nix-grpc-daemon[487]: ts=2026-09-25T16:51:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:382304675worker1 # [ 106.103611] nix-grpc-daemon[487]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4676worker1 # [ 106.113952] nix-grpc-daemon[487]: ts=2026-09-25T16:51:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=0 paths=1 nar_bytes_out=1764677worker2 # [ 106.322087] nix-grpc-daemon[771]: ts=2026-09-25T16:51:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:511044678worker2 # [ 106.327105] nix-grpc-daemon[771]: ts=2026-09-25T16:51:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:511044679worker2 # [ 106.332526] nix-grpc-daemon[876]: time=2026-09-25T16:51:31.925Z level=INFO msg="All 2 paths already cached"4680worker1 # [ 106.338529] nix-grpc-daemon[487]: ts=2026-09-25T16:51:31Z level=debug event=want client=14 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04681worker1 # [ 106.342316] nix-grpc-daemon[487]: ts=2026-09-25T16:51:31Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14682worker2 # [ 106.348278] nix-grpc-daemon[771]: ts=2026-09-25T16:51:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044683worker2 # [ 106.359488] nix-grpc-daemon[771]: ts=2026-09-25T16:51:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:511044684worker2 # [ 106.369161] nix-grpc-daemon[876]: time=2026-09-25T16:51:31.964Z level=INFO msg="All 2 paths already cached"4685worker2 # [ 106.376684] nix-grpc-daemon[771]: ts=2026-09-25T16:51:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:511044686worker2 # [ 106.387345] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4687worker2 # [ 106.519839] nix-daemon[842]: accepted connection from pid 1148, user nix-grpc-daemon (trusted)4688lb # [ 106.519423] niks3-server[985]: 2026/09/25 16:51:32 INFO Received push request method=POST path=/api/pushes4689worker2 # [ 106.544979] nix-daemon[842]: reaped child process 1151, status = succeeded4690worker2 # [ 106.547112] nix-grpc-daemon[876]: time=2026-09-25T16:51:32.141Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4691worker2 # [ 106.548740] nix-grpc-daemon[876]: time=2026-09-25T16:51:32.141Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4692lb # [ 106.540342] niks3-server[985]: 2026/09/25 16:51:32 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign4693worker2 # [ 106.564892] nix-grpc-daemon[876]: time=2026-09-25T16:51:32.159Z level=INFO msg="Uploading 1 narinfos"4694lb # [ 106.548402] niks3-server[985]: 2026/09/25 16:51:32 INFO Signed narinfos id=26 count=14695lb # [ 106.550294] niks3-server[985]: 2026/09/25 16:51:32 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4696lb # [ 106.560585] niks3-server[985]: 2026/09/25 16:51:32 INFO Received complete push request method=POST path=/api/pushes/26/complete4697lb # [ 106.568350] niks3-server[985]: 2026/09/25 16:51:32 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4698worker2 # [ 106.593222] nix-grpc-daemon[876]: time=2026-09-25T16:51:32.188Z level=INFO msg="Upload complete. (190ms)"4699lb # [ 106.573733] niks3-server[985]: 2026/09/25 16:51:32 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4700worker2 # [ 106.600392] nix-grpc-daemon[771]: ts=2026-09-25T16:51:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 paths=1 nar_bytes_in=7524701worker2 # [ 106.603477] nix-grpc-daemon[771]: ts=2026-09-25T16:51:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044702worker2 # [ 106.614811] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4703worker2 # [ 106.621008] nix-daemon[842]: reaped child process 1146, status = succeeded4704worker2 # [ 106.629113] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4705lb # [ 107.043933] envoy[886]: conn peer=[2001:db8:1::1]:38038 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2973 tx=792 ms=41664706lb # [ 107.050113] envoy[886]: conn peer=[2001:db8:1::1]:38046 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1499 tx=1399 ms=46484707lb # [ 107.055750] envoy[886]: conn peer=[2001:db8:1::1]:46158 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3397 tx=1943 ms=14054708lb # [ 107.059616] envoy[886]: conn peer=[2001:db8:1::1]:46172 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=334 ms=354709lb # [ 107.062493] envoy[886]: conn peer=[2001:db8:1::1]:46174 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3871 tx=2138 ms=20234710lb # [ 107.066507] envoy[886]: conn peer=[2001:db8:1::1]:46176 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=376 ms=374711worker2 # [ 107.491252] nix-daemon[842]: reaped child process 1155, status = succeeded4712worker2 # [ 107.571510] nix-daemon[842]: accepted connection from pid 1171, user nix-grpc-daemon (trusted)4713worker2 # [ 107.588515] nix-daemon[842]: reaped child process 1174, status = succeeded4714lb # [ 107.574541] niks3-server[985]: 2026/09/25 16:51:33 INFO Received push request method=POST path=/api/pushes4715worker2 # [ 107.602179] nix-grpc-daemon[876]: time=2026-09-25T16:51:33.196Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4716worker2 # [ 107.603701] nix-grpc-daemon[876]: time=2026-09-25T16:51:33.196Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4717worker2 # [ 107.625865] nix-grpc-daemon[876]: time=2026-09-25T16:51:33.220Z level=INFO msg="Uploading 1 narinfos"4718lb # [ 107.598888] niks3-server[985]: 2026/09/25 16:51:33 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4719lb # [ 107.607656] niks3-server[985]: 2026/09/25 16:51:33 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign4720lb # [ 107.613550] niks3-server[985]: 2026/09/25 16:51:33 INFO Signed narinfos id=27 count=14721lb # [ 107.621434] niks3-server[985]: 2026/09/25 16:51:33 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4722lb # [ 107.626202] niks3-server[985]: 2026/09/25 16:51:33 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4723worker2 # [ 107.652063] nix-grpc-daemon[876]: time=2026-09-25T16:51:33.247Z level=INFO msg="Upload complete. (213ms)"4724lb # [ 107.631704] niks3-server[985]: 2026/09/25 16:51:33 INFO Received complete push request method=POST path=/api/pushes/27/complete4725lb # [ 107.636576] niks3-server[985]: 2026/09/25 16:51:33 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4726worker2 # [ 107.660550] nix-grpc-daemon[771]: ts=2026-09-25T16:51:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=15 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=04727worker2 # [ 107.670729] nix-grpc-daemon[771]: ts=2026-09-25T16:51:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:511044728worker2 # [ 107.674259] nix-daemon[842]: reaped child process 1153, status = succeeded4729worker2 # [ 107.687955] nix-grpc-daemon[771]: ts=2026-09-25T16:51:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:511044730worker2 # [ 107.691729] nix-grpc-daemon[771]: ts=2026-09-25T16:51:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 paths=1 nar_bytes_out=2324731client # [ 107.734951] nix-daemon[1116]: reaped child process 1118, status = succeeded4732client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.64 seconds)4733client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4734client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.02 seconds)4735client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4736client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)4737client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4738client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4739(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.73 seconds)4740subtest: an input only one worker has still reaches the builder4741client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4742client # this derivation will be built:4743client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4744client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4745client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.28 seconds)4746client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4747client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.11 seconds)4748worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4749worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.15 seconds)4750worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4751worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.02 seconds)4752client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24753client # [ 108.516896] nix-daemon[1116]: accepted connection from pid 1199, user root (trusted)4754client # this derivation will be built:4755client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4756worker1 # [ 108.669186] nix-grpc-daemon[487]: ts=2026-09-25T16:51:34Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:382304757client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4758worker1 # [ 108.673132] nix-grpc-daemon[487]: ts=2026-09-25T16:51:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382304759worker1 # [ 108.799909] nix-daemon[856]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)4760worker1 # [ 108.817156] nix-daemon[856]: reaped child process 1231, status = succeeded4761lb # [ 108.805664] niks3-server[985]: 2026/09/25 16:51:34 INFO Received push request method=POST path=/api/pushes4762worker1 # [ 108.829369] nix-grpc-daemon[870]: time=2026-09-25T16:51:34.420Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4763worker1 # [ 108.830952] nix-grpc-daemon[870]: time=2026-09-25T16:51:34.420Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4764lb # [ 108.824933] niks3-server[985]: 2026/09/25 16:51:34 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4765worker1 # [ 108.852115] nix-grpc-daemon[870]: time=2026-09-25T16:51:34.443Z level=INFO msg="Uploading 1 narinfos"4766lb # [ 108.831227] niks3-server[985]: 2026/09/25 16:51:34 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign4767lb # [ 108.838270] niks3-server[985]: 2026/09/25 16:51:34 INFO Signed narinfos id=28 count=14768lb # [ 108.842482] niks3-server[985]: 2026/09/25 16:51:34 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4769lb # [ 108.850878] niks3-server[985]: 2026/09/25 16:51:34 INFO Received complete push request method=POST path=/api/pushes/28/complete4770worker1 # [ 108.876026] nix-grpc-daemon[870]: time=2026-09-25T16:51:34.467Z level=INFO msg="Upload complete. (203ms)"4771lb # [ 108.859795] niks3-server[985]: 2026/09/25 16:51:34 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4772worker1 # [ 108.890163] nix-grpc-daemon[487]: ts=2026-09-25T16:51:34Z level=debug event=want client=15 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04773worker1 # [ 108.895740] nix-grpc-daemon[487]: ts=2026-09-25T16:51:34Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14774worker1 # [ 108.900397] nix-grpc-daemon[487]: ts=2026-09-25T16:51:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382304775worker1 # [ 108.906092] nix-grpc-daemon[487]: ts=2026-09-25T16:51:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382304776client # copying 2 paths to 'lb:50051'...4777client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4778worker1 # [ 108.916376] nix-grpc-daemon[487]: ts=2026-09-25T16:51:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:382304779client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4780worker1 # [ 108.924094] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4781worker1 # [ 109.043293] nix-daemon[856]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)4782worker1 # [ 109.061382] nix-daemon[856]: reaped child process 1240, status = succeeded4783lb # [ 109.047595] niks3-server[985]: 2026/09/25 16:51:34 INFO Received push request method=POST path=/api/pushes4784worker1 # [ 109.071135] nix-grpc-daemon[870]: time=2026-09-25T16:51:34.660Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4785worker1 # [ 109.073657] nix-grpc-daemon[870]: time=2026-09-25T16:51:34.660Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4786worker1 # [ 109.075928] nix-grpc-daemon[870]: time=2026-09-25T16:51:34.660Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4787lb # [ 109.075372] niks3-server[985]: 2026/09/25 16:51:34 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4788worker1 # [ 109.101382] nix-grpc-daemon[870]: time=2026-09-25T16:51:34.692Z level=INFO msg="Uploading 2 narinfos"4789lb # [ 109.084563] niks3-server[985]: 2026/09/25 16:51:34 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4790lb # [ 109.091196] niks3-server[985]: 2026/09/25 16:51:34 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4791lb # [ 109.097236] niks3-server[985]: 2026/09/25 16:51:34 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign4792lb # [ 109.104399] niks3-server[985]: 2026/09/25 16:51:34 INFO Signed narinfos id=29 count=24793lb # [ 109.106128] niks3-server[985]: 2026/09/25 16:51:34 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4794lb # [ 109.117452] niks3-server[985]: 2026/09/25 16:51:34 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4795lb # [ 109.119953] niks3-server[985]: 2026/09/25 16:51:34 INFO Received complete push request method=POST path=/api/pushes/29/complete4796lb # [ 109.126127] niks3-server[985]: 2026/09/25 16:51:34 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4797worker1 # [ 109.142023] nix-grpc-daemon[870]: time=2026-09-25T16:51:34.733Z level=INFO msg="Upload complete. (208ms)"4798worker1 # [ 109.148516] nix-grpc-daemon[487]: ts=2026-09-25T16:51:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=0 paths=2 nar_bytes_in=9844799worker1 # [ 109.151784] nix-grpc-daemon[487]: ts=2026-09-25T16:51:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382304800worker1 # [ 109.163487] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4801worker1 # [ 109.167497] nix-daemon[856]: reaped child process 1235, status = succeeded4802client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4803client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4804worker1 # [ 109.178994] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4805client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4806worker1 # [ 110.214912] nix-daemon[856]: reaped child process 1244, status = succeeded4807worker1 # [ 110.321258] nix-daemon[856]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)4808worker1 # [ 110.338187] nix-daemon[856]: reaped child process 1263, status = succeeded4809lb # [ 110.325913] niks3-server[985]: 2026/09/25 16:51:35 INFO Received push request method=POST path=/api/pushes4810worker1 # [ 110.350903] nix-grpc-daemon[870]: time=2026-09-25T16:51:35.939Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4811lb # [ 110.340565] niks3-server[985]: 2026/09/25 16:51:35 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4812worker1 # [ 110.366395] nix-grpc-daemon[870]: time=2026-09-25T16:51:35.957Z level=INFO msg="Uploading 1 narinfos"4813lb # [ 110.349251] niks3-server[985]: 2026/09/25 16:51:35 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign4814lb # [ 110.351807] niks3-server[985]: 2026/09/25 16:51:35 INFO Signed narinfos id=30 count=14815lb # [ 110.359336] niks3-server[985]: 2026/09/25 16:51:35 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4816lb # [ 110.367251] niks3-server[985]: 2026/09/25 16:51:35 INFO Received complete push request method=POST path=/api/pushes/30/complete4817worker1 # [ 110.389701] nix-grpc-daemon[870]: time=2026-09-25T16:51:35.981Z level=INFO msg="Upload complete. (202ms)"4818lb # [ 110.375378] niks3-server[985]: 2026/09/25 16:51:35 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4819worker1 # [ 110.397050] nix-grpc-daemon[487]: ts=2026-09-25T16:51:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=16 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04820client # copying 1 paths...4821worker1 # [ 110.408990] nix-daemon[856]: reaped child process 1242, status = succeeded4822worker2 # [ 110.413123] nix-grpc-daemon[771]: ts=2026-09-25T16:51:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:511044823worker2 # [ 110.420883] nix-grpc-daemon[771]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4824client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4825worker2 # [ 110.447177] nix-grpc-daemon[771]: ts=2026-09-25T16:51:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:511044826worker2 # [ 110.451602] nix-grpc-daemon[771]: ts=2026-09-25T16:51:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 paths=1 nar_bytes_out=1204827client # [ 110.492769] nix-daemon[1116]: reaped child process 1202, status = succeeded4828client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.12 seconds)4829client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&24830client # [ 110.625846] nix-daemon[1116]: accepted connection from pid 1221, user root (trusted)4831client # this derivation will be built:4832client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4833worker2 # [ 110.782876] nix-grpc-daemon[771]: ts=2026-09-25T16:51:36Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:511044834client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4835worker2 # [ 110.788259] nix-grpc-daemon[771]: ts=2026-09-25T16:51:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:511044836worker1 # [ 110.797537] nix-grpc-daemon[487]: ts=2026-09-25T16:51:36Z level=debug event=want client=16 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04837worker2 # [ 110.803457] nix-grpc-daemon[771]: ts=2026-09-25T16:51:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044838worker1 # [ 110.801010] nix-grpc-daemon[487]: ts=2026-09-25T16:51:36Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14839worker2 # [ 110.812752] nix-grpc-daemon[771]: ts=2026-09-25T16:51:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:511044840client # copying 1 paths to 'lb:50051'...4841client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4842worker2 # [ 110.826470] nix-grpc-daemon[771]: ts=2026-09-25T16:51:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:511044843worker2 # [ 110.835457] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4844worker2 # [ 110.849215] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 88 ms (attempt 1/5)4845worker2 # [ 110.942199] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 2/5)4846worker2 # [ 110.952307] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 61 ms (attempt 3/5)4847worker2 # [ 111.018226] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 238 ms (attempt 4/5)4848worker2 # [ 111.257863] nix-grpc-daemon[771]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4849worker2 # [ 111.259814] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4850worker2 # [ 111.265142] nix-grpc-daemon[771]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4851worker2 # [ 111.407189] nix-daemon[842]: accepted connection from pid 1187, user nix-grpc-daemon (trusted)4852worker2 # [ 111.425190] nix-daemon[842]: reaped child process 1190, status = succeeded4853lb # [ 111.411194] niks3-server[985]: 2026/09/25 16:51:37 INFO Received push request method=POST path=/api/pushes4854worker2 # [ 111.438008] nix-grpc-daemon[876]: time=2026-09-25T16:51:37.032Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4855worker2 # [ 111.439527] nix-grpc-daemon[876]: time=2026-09-25T16:51:37.032Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4856lb # [ 111.428546] niks3-server[985]: 2026/09/25 16:51:37 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign4857worker2 # [ 111.452786] nix-grpc-daemon[876]: time=2026-09-25T16:51:37.047Z level=INFO msg="Uploading 1 narinfos"4858lb # [ 111.434181] niks3-server[985]: 2026/09/25 16:51:37 INFO Signed narinfos id=31 count=14859lb # [ 111.440407] niks3-server[985]: 2026/09/25 16:51:37 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4860lb # [ 111.442861] niks3-server[985]: 2026/09/25 16:51:37 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4861lb # [ 111.454062] niks3-server[985]: 2026/09/25 16:51:37 INFO Received complete push request method=POST path=/api/pushes/31/complete4862worker2 # [ 111.477985] nix-grpc-daemon[876]: time=2026-09-25T16:51:37.073Z level=INFO msg="Upload complete. (201ms)"4863lb # [ 111.459197] niks3-server[985]: 2026/09/25 16:51:37 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4864worker2 # [ 111.484625] nix-grpc-daemon[771]: ts=2026-09-25T16:51:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 paths=1 nar_bytes_in=5684865worker2 # [ 111.491828] nix-grpc-daemon[771]: ts=2026-09-25T16:51:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:511044866worker2 # [ 111.500570] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4867worker2 # [ 111.505150] nix-daemon[842]: reaped child process 1180, status = succeeded4868worker2 # [ 111.511993] nix-grpc-daemon[771]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4869client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4870worker2 # [ 111.523092] nix-daemon[842]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4871client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4872client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4873worker2 # [ 112.360862] nix-daemon[842]: reaped child process 1195, status = succeeded4874worker2 # [ 112.470925] nix-daemon[842]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)4875worker2 # [ 112.487951] nix-daemon[842]: reaped child process 1214, status = succeeded4876lb # [ 112.471637] niks3-server[985]: 2026/09/25 16:51:38 INFO Received push request method=POST path=/api/pushes4877worker2 # [ 112.500379] nix-grpc-daemon[876]: time=2026-09-25T16:51:38.092Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4878lb # [ 112.489093] niks3-server[985]: 2026/09/25 16:51:38 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4879worker2 # [ 112.515346] nix-grpc-daemon[876]: time=2026-09-25T16:51:38.110Z level=INFO msg="Uploading 1 narinfos"4880lb # [ 112.495598] niks3-server[985]: 2026/09/25 16:51:38 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign4881lb # [ 112.501076] niks3-server[985]: 2026/09/25 16:51:38 INFO Signed narinfos id=32 count=14882lb # [ 112.506779] niks3-server[985]: 2026/09/25 16:51:38 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4883lb # [ 112.515606] niks3-server[985]: 2026/09/25 16:51:38 INFO Received complete push request method=POST path=/api/pushes/32/complete4884worker2 # [ 112.541407] nix-grpc-daemon[876]: time=2026-09-25T16:51:38.136Z level=INFO msg="Upload complete. (208ms)"4885lb # [ 112.521764] niks3-server[985]: 2026/09/25 16:51:38 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4886client # copying 1 paths...4887worker2 # [ 112.548559] nix-grpc-daemon[771]: ts=2026-09-25T16:51:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=17 outputs=1 inputs=1 inputs_fetched=1 input_bytes=120 input_s=04888worker2 # [ 112.560717] nix-grpc-daemon[771]: ts=2026-09-25T16:51:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:511044889worker2 # [ 112.564176] nix-daemon[842]: reaped child process 1192, status = succeeded4890client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4891worker1 # [ 112.575559] nix-grpc-daemon[487]: ts=2026-09-25T16:51:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:382304892worker1 # [ 112.582867] nix-grpc-daemon[487]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4893worker1 # [ 112.592626] nix-grpc-daemon[487]: ts=2026-09-25T16:51:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=0 paths=1 nar_bytes_out=1204894client # [ 112.636941] nix-daemon[1116]: reaped child process 1224, status = succeeded4895client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.14 seconds)4896client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&24897client # [ 112.770981] nix-daemon[1116]: accepted connection from pid 1243, user root (trusted)4898client # this derivation will be built:4899client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4900worker1 # [ 112.921702] nix-grpc-daemon[487]: ts=2026-09-25T16:51:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:382304901client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4902worker1 # [ 112.928088] nix-grpc-daemon[487]: ts=2026-09-25T16:51:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382304903worker1 # [ 112.934489] nix-grpc-daemon[870]: time=2026-09-25T16:51:38.525Z level=INFO msg="All 1 paths already cached"4904worker1 # [ 112.945747] nix-grpc-daemon[487]: ts=2026-09-25T16:51:38Z level=debug event=want client=17 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04905worker1 # [ 112.950878] nix-grpc-daemon[487]: ts=2026-09-25T16:51:38Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14906worker1 # [ 112.955733] nix-grpc-daemon[487]: ts=2026-09-25T16:51:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382304907worker1 # [ 112.960068] nix-grpc-daemon[487]: ts=2026-09-25T16:51:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:382304908worker1 # [ 112.965591] nix-grpc-daemon[870]: time=2026-09-25T16:51:38.556Z level=INFO msg="All 1 paths already cached"4909client # copying 1 paths to 'lb:50051'...4910client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4911worker1 # [ 112.971700] nix-grpc-daemon[487]: ts=2026-09-25T16:51:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:382304912worker1 # [ 112.980580] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4913worker1 # [ 113.101254] nix-daemon[856]: accepted connection from pid 1272, user nix-grpc-daemon (trusted)4914worker1 # [ 113.119198] nix-daemon[856]: reaped child process 1275, status = succeeded4915lb # [ 113.105486] niks3-server[985]: 2026/09/25 16:51:38 INFO Received push request method=POST path=/api/pushes4916worker1 # [ 113.128000] nix-grpc-daemon[870]: time=2026-09-25T16:51:38.719Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4917worker1 # [ 113.129599] nix-grpc-daemon[870]: time=2026-09-25T16:51:38.719Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4918lb # [ 113.123648] niks3-server[985]: 2026/09/25 16:51:38 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign4919worker1 # [ 113.144677] nix-grpc-daemon[870]: time=2026-09-25T16:51:38.735Z level=INFO msg="Uploading 1 narinfos"4920lb # [ 113.131142] niks3-server[985]: 2026/09/25 16:51:38 INFO Signed narinfos id=33 count=14921lb # [ 113.135711] niks3-server[985]: 2026/09/25 16:51:38 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4922lb # [ 113.145045] niks3-server[985]: 2026/09/25 16:51:38 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4923lb # [ 113.147981] niks3-server[985]: 2026/09/25 16:51:38 INFO Received complete push request method=POST path=/api/pushes/33/complete4924lb # [ 113.156093] niks3-server[985]: 2026/09/25 16:51:38 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4925worker1 # [ 113.176318] nix-grpc-daemon[870]: time=2026-09-25T16:51:38.767Z level=INFO msg="Upload complete. (186ms)"4926worker1 # [ 113.182139] nix-grpc-daemon[487]: ts=2026-09-25T16:51:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=0 paths=1 nar_bytes_in=5684927worker1 # [ 113.193276] nix-grpc-daemon[487]: ts=2026-09-25T16:51:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:382304928worker1 # [ 113.197742] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4929worker1 # [ 113.201933] nix-daemon[856]: reaped child process 1270, status = succeeded4930client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4931client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4932worker1 # [ 113.212154] nix-daemon[856]: accepted connection from pid 487, user nix-grpc-daemon (trusted)4933client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4934worker1 # [ 114.090701] nix-daemon[856]: reaped child process 1279, status = succeeded4935worker1 # [ 114.147471] nix-daemon[856]: accepted connection from pid 1295, user nix-grpc-daemon (trusted)4936worker1 # [ 114.162773] nix-daemon[856]: reaped child process 1298, status = succeeded4937lb # [ 114.152136] niks3-server[985]: 2026/09/25 16:51:39 INFO Received push request method=POST path=/api/pushes4938worker1 # [ 114.175440] nix-grpc-daemon[870]: time=2026-09-25T16:51:39.765Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4939lb # [ 114.171227] niks3-server[985]: 2026/09/25 16:51:39 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4940lb # [ 114.175831] niks3-server[985]: 2026/09/25 16:51:39 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign4941worker1 # [ 114.196845] nix-grpc-daemon[870]: time=2026-09-25T16:51:39.787Z level=INFO msg="Uploading 1 narinfos"4942lb # [ 114.181921] niks3-server[985]: 2026/09/25 16:51:39 INFO Signed narinfos id=34 count=14943lb # [ 114.186797] niks3-server[985]: 2026/09/25 16:51:39 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4944lb # [ 114.196293] niks3-server[985]: 2026/09/25 16:51:39 INFO Received complete push request method=POST path=/api/pushes/34/complete4945worker1 # [ 114.218236] nix-grpc-daemon[870]: time=2026-09-25T16:51:39.809Z level=INFO msg="Upload complete. (198ms)"4946lb # [ 114.203030] niks3-server[985]: 2026/09/25 16:51:39 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4947client # copying 1 paths...4948worker1 # [ 114.226063] nix-grpc-daemon[487]: ts=2026-09-25T16:51:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38230 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=18 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04949worker1 # [ 114.237740] nix-grpc-daemon[487]: ts=2026-09-25T16:51:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:382304950worker1 # [ 114.241347] nix-daemon[856]: reaped child process 1277, status = succeeded4951client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4952worker2 # [ 114.262720] nix-grpc-daemon[771]: ts=2026-09-25T16:51:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:511044953worker2 # [ 114.267278] nix-grpc-daemon[771]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4954worker2 # [ 114.278686] nix-grpc-daemon[771]: ts=2026-09-25T16:51:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:51104 duration_s=0 paths=1 nar_bytes_out=1204955client # [ 114.319576] nix-daemon[1116]: reaped child process 1246, status = succeeded4956client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.68 seconds)4957(finished: subtest: an input only one worker has still reaches the builder, in 6.50 seconds)4958subtest: upload whose reference the worker lacks is completed from the cache4959client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache4960client # this derivation will be built:4961client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4962client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4963client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.29 seconds)4964client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache4965client # this derivation will be built:4966client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4967client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4968client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.27 seconds)4969client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4970client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.11 seconds)4971worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4972worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.15 seconds)4973worker2 # [ 115.193481] systemd[1]: Stopping Nix worker-protocol over gRPC...4974worker2 # [ 116.139886] nix-grpc-daemon[771]: ts=2026-09-25T16:51:41Z level=info event=signal_exit4975lb # [ 117.043880] envoy[886]: conn peer=[2001:db8:1::1]:46180 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4322 tx=2372 ms=13804976lb # [ 117.046703] envoy[886]: conn peer=[2001:db8:1::1]:46192 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=154977lb # [ 117.049261] envoy[886]: conn peer=[2001:db8:1::1]:46194 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3862 tx=2065 ms=17904978lb # [ 117.051721] envoy[886]: conn peer=[2001:db8:1::1]:46208 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=154979lb # [ 117.054258] envoy[886]: conn peer=[2001:db8:1::1]:46214 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3740 tx=2101 ms=18244980lb # [ 117.056764] envoy[886]: conn peer=[2001:db8:1::1]:48948 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=329 ms=294981lb # [ 117.059200] envoy[886]: conn peer=[2001:db8:1::1]:48956 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3724 tx=2081 ms=13634982lb # [ 117.061758] envoy[886]: conn peer=[2001:db8:1::1]:48964 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=329 ms=304983worker1 # [ 118.048709] nix-grpc-daemon[487]: ts=2026-09-25T16:51:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:468064984worker2 # [ 118.055049] nix-grpc-daemon[771]: ts=2026-09-25T16:51:43Z level=info event=scheduler_restarting4985worker1 # [ 118.053132] nix-grpc-daemon[487]: ts=2026-09-25T16:51:43Z level=info event=worker_gone addr=192.168.1.4:500514986worker2 # [ 118.069607] nix-grpc-daemon[771]: ts=2026-09-25T16:51:43Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4987worker2 # [ 118.085458] nix-daemon[842]: reaped child process 844, status = succeeded4988worker2 # [ 119.413271] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4989worker2 # [ 119.417805] systemd[1]: Stopped Nix worker-protocol over gRPC.4990worker2 # [ 119.419285] systemd[1]: nix-grpc-daemon.service: Consumed 2.551s CPU time over 1min 41.448s wall clock time, 59.2M memory peak, 174.3K incoming IP traffic, 315.3K outgoing IP traffic.4991lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux4992lb: must succeed: curl -sf localhost:9901/clusters4993lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4994lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.04 seconds)4995client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24996worker1 # [ 119.639024] nix-grpc-daemon[487]: ts=2026-09-25T16:51:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:382304997worker1 # [ 119.642677] nix-grpc-daemon[487]: ts=2026-09-25T16:51:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:382304998client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4999client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.19 seconds)5000worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5001worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.02 seconds)5002worker2 # [ 119.729677] systemd[1]: Starting scheduler-token.service...5003worker2 # [ 119.784855] systemd[1]: scheduler-token.service: Deactivated successfully.5004worker2 # [ 119.791164] systemd[1]: Finished scheduler-token.service.5005worker2 # [ 119.792706] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 58ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5006worker2 # [ 119.797681] systemd[1]: Starting Nix worker-protocol over gRPC...5007worker2 # [ 119.909242] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:45Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5008worker2 # [ 119.911756] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:45Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5009worker2 # [ 119.916677] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:45Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15010worker2 # [ 119.922467] nix-grpc-daemon[1239]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5011worker2 # [ 119.925772] nix-grpc-daemon[1239]: E0000 00:00:1790355105.516927 1239 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5012worker2 # [ 119.948203] nix-grpc-daemon[1239]: E0000 00:00:1790355105.520780 1239 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5013worker2 # [ 119.950673] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:45Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5014worker2 # [ 119.976437] nix-grpc-daemon[1239]: E0000 00:00:1790355105.530674 1280 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5015worker2 # [ 119.982299] systemd[1]: Started Nix worker-protocol over gRPC.5016worker2 # [ 119.985214] nix-grpc-daemon[1239]: E0000 00:00:1790355105.554103 1280 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5017worker1 # [ 119.983769] nix-grpc-daemon[487]: ts=2026-09-25T16:51:45Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:468065018worker2 # [ 119.987810] nix-grpc-daemon[1239]: E0000 00:00:1790355105.554273 1280 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5019worker2 # [ 119.994521] nix-grpc-daemon[1239]: E0000 00:00:1790355105.570224 1280 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5020worker1 # [ 119.991733] nix-grpc-daemon[487]: ts=2026-09-25T16:51:45Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05021worker2 # [ 119.998268] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:45Z level=info event=scheduler_connected addr=lb:500515022worker1 # [ 120.008388] systemd[1]: Stopping Nix worker-protocol over gRPC...5023worker1 # [ 120.037037] nix-grpc-daemon[487]: ts=2026-09-25T16:51:45Z level=info event=signal_exit5024lb # [ 120.024773] niks3-server[985]: 2026/09/25 16:51:45 INFO lead: released remote=192.168.1.3:542425025worker1 # [ 121.041085] nix-grpc-daemon[487]: ts=2026-09-25T16:51:46Z level=info event=scheduler_restarting5026worker2 # [ 121.046841] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:46Z level=info event=scheduler_restarting addr=lb:500515027lb # [ 121.024270] envoy[886]: [2026-09-25 16:51:46.618][886][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5028worker1 # [ 121.044645] nix-grpc-daemon[487]: ts=2026-09-25T16:51:46Z level=info event=scheduler_restarting addr=lb:500515029worker1 # [ 121.051050] nix-grpc-daemon[487]: ts=2026-09-25T16:51:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:468065030worker1 # [ 121.053871] nix-grpc-daemon[487]: ts=2026-09-25T16:51:46Z level=info event=worker_gone addr=192.168.1.4:500515031worker1 # [ 121.057137] nix-grpc-daemon[487]: ts=2026-09-25T16:51:46Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:468065032worker1 # [ 121.060597] nix-grpc-daemon[487]: ts=2026-09-25T16:51:46Z level=info event=worker_gone addr=192.168.1.3:500515033worker1 # [ 121.074670] nix-daemon[856]: reaped child process 858, status = succeeded5034worker2 # [ 121.246952] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:46Z level=info event=scheduler_connected addr=lb:500515035worker2 # [ 121.250073] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5036worker1 # [ 121.259739] nix-grpc-daemon[487]: ts=2026-09-25T16:51:46Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5037worker1 # [ 121.370582] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5038worker1 # [ 121.374719] systemd[1]: Stopped Nix worker-protocol over gRPC.5039worker1 # [ 121.377862] systemd[1]: nix-grpc-daemon.service: Consumed 2.914s CPU time over 1min 51.668s wall clock time, 74M memory peak, 450.9K incoming IP traffic, 449.5K outgoing IP traffic.5040lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux5041lb: must succeed: curl -sf localhost:9901/clusters5042lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5043lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.04 seconds)5044client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25045worker2 # [ 121.450715] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:47Z level=info event=scheduler_connected addr=lb:500515046worker2 # [ 121.453682] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5047worker2 # [ 121.603253] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409185048worker2 # [ 121.612836] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5049worker2 # [ 121.628270] nix-grpc-daemon[1239]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 1/5)5050worker2 # [ 121.654287] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:47Z level=info event=scheduler_connected addr=lb:500515051worker2 # [ 121.657103] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5052worker2 # [ 121.708621] nix-grpc-daemon[1239]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 162 ms (attempt 2/5)5053worker2 # [ 121.858086] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:47Z level=info event=scheduler_connected addr=lb:500515054worker2 # [ 121.861062] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5055worker2 # [ 121.876240] nix-grpc-daemon[1239]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 228 ms (attempt 3/5)5056worker2 # [ 122.063132] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:47Z level=info event=scheduler_connected addr=lb:500515057worker2 # [ 122.065540] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5058worker2 # [ 122.105825] nix-grpc-daemon[1239]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 4/5)5059worker2 # [ 122.201386] nix-grpc-daemon[1239]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5060worker2 # [ 122.203467] nix-grpc-daemon[1239]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5061client # copying 2 paths...5062client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5063worker2 # [ 122.218935] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409185064client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5065worker2 # [ 122.228978] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5066worker2 # [ 122.253484] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:47Z level=info event=niks3_push_started pid=12995067worker2 # [ 122.270031] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5068worker2 # [ 122.389030] nix-grpc-daemon[1299]: time=2026-09-25T16:51:47.983Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5069worker2 # [ 122.505332] nix-daemon[842]: accepted connection from pid 1311, user nix-grpc-daemon (trusted)5070worker2 # [ 122.529230] nix-daemon[842]: reaped child process 1314, status = succeeded5071lb # [ 122.509106] niks3-server[985]: 2026/09/25 16:51:48 INFO Received push request method=POST path=/api/pushes5072worker2 # [ 122.536122] nix-grpc-daemon[1299]: time=2026-09-25T16:51:48.130Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5073worker2 # [ 122.537771] nix-grpc-daemon[1299]: time=2026-09-25T16:51:48.130Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5074worker2 # [ 122.539567] nix-grpc-daemon[1299]: time=2026-09-25T16:51:48.130Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5075lb # [ 122.550939] niks3-server[985]: 2026/09/25 16:51:48 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5076lb # [ 122.564379] niks3-server[985]: 2026/09/25 16:51:48 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign5077lb # [ 122.566859] niks3-server[985]: 2026/09/25 16:51:48 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5078lb # [ 122.580636] niks3-server[985]: 2026/09/25 16:51:48 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5079lb # [ 122.583088] niks3-server[985]: 2026/09/25 16:51:48 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5080lb # [ 122.589900] niks3-server[985]: 2026/09/25 16:51:48 INFO Signed narinfos id=35 count=25081worker2 # [ 122.612104] nix-grpc-daemon[1299]: time=2026-09-25T16:51:48.205Z level=INFO msg="Uploading 2 narinfos"5082lb # [ 122.611550] niks3-server[985]: 2026/09/25 16:51:48 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5083lb # [ 122.616237] niks3-server[985]: 2026/09/25 16:51:48 INFO Received complete push request method=POST path=/api/pushes/35/complete5084lb # [ 122.626151] niks3-server[985]: 2026/09/25 16:51:48 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5085worker2 # [ 122.646639] nix-grpc-daemon[1299]: time=2026-09-25T16:51:48.241Z level=INFO msg="Upload complete. (253ms)"5086worker2 # [ 122.655179] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=0 paths=2 nar_bytes_in=3125087worker2 # [ 122.666040] nix-daemon[842]: reaped child process 1297, status = succeeded5088worker2 # [ 122.668637] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:48Z level=info event=scheduler_connected addr=lb:500515089worker2 # [ 122.671701] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5090client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.25 seconds)5091worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5092worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.02 seconds)5093worker1 # [ 122.733326] systemd[1]: Starting Nix worker-protocol over gRPC...5094worker1 # [ 122.854650] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5095worker1 # [ 122.857040] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5096worker1 # [ 122.860938] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15097worker1 # [ 122.866355] nix-grpc-daemon[1321]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5098worker1 # [ 122.867885] nix-grpc-daemon[1321]: E0000 00:00:1790355108.456807 1321 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5099worker2 # [ 122.872638] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:48Z level=info event=scheduler_connected addr=lb:500515100worker1 # [ 122.872479] nix-grpc-daemon[1321]: E0000 00:00:1790355108.463602 1321 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5101worker2 # [ 122.875728] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5102worker1 # [ 122.875877] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5103lb # [ 122.865634] niks3-server[985]: 2026/09/25 16:51:48 INFO lead: acquired remote=192.168.1.3:603985104worker1 # [ 122.901997] systemd[1]: Started Nix worker-protocol over gRPC.5105worker1 # [ 122.902926] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=scheduler_take_over5106lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux5107lb: must succeed: curl -sf localhost:9901/clusters5108worker1 # [ 122.920737] nix-grpc-daemon[1321]: E0000 00:00:1790355108.474445 1364 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5109worker1 # [ 122.923696] nix-grpc-daemon[1321]: E0000 00:00:1790355108.493086 1364 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5110worker1 # [ 122.926524] nix-grpc-daemon[1321]: E0000 00:00:1790355108.495586 1364 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5111worker1 # [ 122.929084] nix-grpc-daemon[1321]: E0000 00:00:1790355108.511188 1364 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5112worker1 # [ 122.936266] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=scheduler_connected addr=lb:500515113worker1 # [ 122.938361] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5114lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5115worker2 # [ 123.078828] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:48Z level=info event=scheduler_connected addr=lb:500515116worker2 # [ 123.081710] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5117worker1 # [ 123.139171] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=scheduler_connected addr=lb:500515118worker1 # [ 123.142099] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5119worker2 # [ 123.278982] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:48Z level=info event=scheduler_connected addr=lb:500515120worker1 # [ 123.289197] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=oidc_jwks provider=mock keys=15121worker1 # [ 123.291447] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:576905122worker1 # [ 123.295897] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05123worker1 # [ 123.342700] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=scheduler_connected addr=lb:500515124worker1 # [ 123.345968] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:576905125worker1 # [ 123.348580] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:48Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05126lb: must succeed: curl -sf localhost:9901/clusters5127lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5128lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 1.09 seconds)5129(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.66 seconds)5130subtest: a niks3 restart does not cost the leader its role5131lb: must succeed: curl -sf localhost:9901/clusters5132lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5133lb: must succeed: curl -sf localhost:9901/clusters5134lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5135worker1 # [ 124.096527] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:49Z level=info event=scheduler_lock_lost status=2005136worker2 # [ 124.100432] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:49Z level=info event=scheduler_lock_lost status=2005137lb # [ 124.079040] niks3-server[985]: 2026/09/25 16:51:49 INFO Shutdown signal received, draining in-flight requests timeout=10s5138lb # [ 124.079219] niks3-server[985]: 2026/09/25 16:51:49 INFO lead: released remote=192.168.1.3:603985139lb # [ 124.085060] systemd[1]: Stopping niks3 server...5140worker1 # [ 126.824844] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:52Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:577085141lb # [ 127.044260] envoy[886]: conn peer=[2001:db8:1::4]:39258 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2863 tx=1021 ms=836855142lb # [ 127.047204] envoy[886]: conn peer=[2001:db8:1::1]:48980 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=185143lb # [ 127.049774] envoy[886]: conn peer=192.168.1.3:46996 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2387 tx=1090 ms=862575144lb # [ 127.052267] envoy[886]: conn peer=[2001:db8:1::1]:36484 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=282 ms=10615145lb # [ 127.750310] systemd[1]: niks3.service: Deactivated successfully.5146lb # [ 127.754220] systemd[1]: Stopped niks3 server.5147lb # [ 127.755354] systemd[1]: niks3.service: Consumed 434ms CPU time over 1min 42.966s wall clock time, 28M memory peak, 97.6K incoming IP traffic, 140.9K outgoing IP traffic.5148lb # [ 127.769402] systemd[1]: Starting niks3 server...5149lb # [ 127.848273] niks3-server[1196]: 2026/09/25 16:51:53 goose: no migrations to run. current version: 202609231200005150lb # [ 127.853827] niks3-server[1196]: 2026/09/25 16:51:53 OK 1_commit_pending_closure.sql (7.07ms)5151lb # [ 127.857194] niks3-server[1196]: 2026/09/25 16:51:53 OK 2_object_stats_trigger.sql (3.31ms)5152lb # [ 127.860257] niks3-server[1196]: 2026/09/25 16:51:53 OK 3_commit_push.sql (2.69ms)5153lb # [ 127.861498] niks3-server[1196]: 2026/09/25 16:51:53 goose: up to current file version: 35154lb # [ 127.863216] niks3-server[1196]: 2026/09/25 16:51:53 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5155lb # [ 127.874204] niks3-server[1196]: 2026/09/25 16:51:53 INFO Read proxy enabled — serving cache objects from S35156lb # [ 127.875679] niks3-server[1196]: 2026/09/25 16:51:53 INFO Using socket-activated listener address=0.0.0.0:57515157lb # [ 127.882737] niks3-server[1196]: 2026/09/25 16:51:53 INFO systemd watchdog enabled interval=15s5158lb # [ 127.884271] niks3-server[1196]: 2026/09/25 16:51:53 INFO Starting HTTP server address=0.0.0.0:57515159lb # [ 127.887270] niks3-server[1196]: 2026/09/25 16:51:53 INFO lead: acquired remote=192.168.1.3:604005160lb # [ 127.889263] systemd[1]: Started niks3 server.5161lb: waiting for unit niks3.service5162lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5163lb: must succeed: curl -sf localhost:9901/clusters5164lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5165worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5166worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5167lb: must succeed: curl -sf localhost:9901/clusters5168lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5169lb: must succeed: curl -sf localhost:9901/clusters5170lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5171worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5172worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.03 seconds)5173client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25174worker2 # [ 128.261281] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409185175client # don't know how to build these paths:5176client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5177worker2 # [ 128.268911] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409185178worker1 # [ 128.291560] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:53Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05179worker1 # [ 128.295031] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:53Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15180worker2 # [ 128.300502] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409185181worker2 # [ 128.310080] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409185182client # copying 1 paths to 'lb:50051'...5183client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5184worker2 # [ 128.319059] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409185185worker2 # [ 128.328720] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5186worker2 # [ 128.461352] nix-daemon[842]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5187worker2 # [ 128.480126] nix-daemon[842]: reaped child process 1326, status = succeeded5188lb # [ 128.462116] niks3-server[1196]: 2026/09/25 16:51:54 INFO Received push request method=POST path=/api/pushes5189worker2 # [ 128.492381] nix-grpc-daemon[1299]: time=2026-09-25T16:51:54.086Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5190worker2 # [ 128.493974] nix-grpc-daemon[1299]: time=2026-09-25T16:51:54.086Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5191lb # [ 128.488567] niks3-server[1196]: 2026/09/25 16:51:54 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign5192lb # [ 128.514283] niks3-server[1196]: 2026/09/25 16:51:54 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5193lb # [ 128.525446] niks3-server[1196]: 2026/09/25 16:51:54 INFO Signed narinfos id=36 count=15194worker2 # [ 128.548990] nix-grpc-daemon[1299]: time=2026-09-25T16:51:54.143Z level=INFO msg="Uploading 1 narinfos"5195lb # [ 128.534541] niks3-server[1196]: 2026/09/25 16:51:54 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5196lb # [ 128.544210] niks3-server[1196]: 2026/09/25 16:51:54 INFO Received complete push request method=POST path=/api/pushes/36/complete5197worker2 # [ 128.572262] nix-grpc-daemon[1299]: time=2026-09-25T16:51:54.167Z level=INFO msg="Upload complete. (232ms)"5198lb # [ 128.554381] niks3-server[1196]: 2026/09/25 16:51:54 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5199worker2 # [ 128.579498] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=0 paths=1 nar_bytes_in=5445200worker2 # [ 128.588983] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409185201worker2 # [ 128.593800] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5202worker2 # [ 128.598082] nix-daemon[842]: reaped child process 1321, status = succeeded5203client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5204client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5205worker2 # [ 128.607677] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5206client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5207client # farm-a> LOG-farm-a-after-niks3-restart5208worker2 # [ 129.441859] nix-daemon[842]: reaped child process 1330, status = succeeded5209worker2 # [ 129.540164] nix-daemon[842]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5210lb # [ 129.537112] niks3-server[1196]: 2026/09/25 16:51:55 INFO Received push request method=POST path=/api/pushes5211worker2 # [ 129.560158] nix-daemon[842]: reaped child process 1349, status = succeeded5212worker2 # [ 129.567053] nix-grpc-daemon[1299]: time=2026-09-25T16:51:55.160Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5213worker2 # [ 129.568812] nix-grpc-daemon[1299]: time=2026-09-25T16:51:55.160Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5214lb # [ 129.563757] niks3-server[1196]: 2026/09/25 16:51:55 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign5215worker2 # [ 129.590521] nix-grpc-daemon[1299]: time=2026-09-25T16:51:55.185Z level=INFO msg="Uploading 1 narinfos"5216lb # [ 129.572255] niks3-server[1196]: 2026/09/25 16:51:55 INFO Signed narinfos id=37 count=15217lb # [ 129.574080] niks3-server[1196]: 2026/09/25 16:51:55 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5218lb # [ 129.583829] niks3-server[1196]: 2026/09/25 16:51:55 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5219lb # [ 129.594772] niks3-server[1196]: 2026/09/25 16:51:55 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5220lb # [ 129.601687] niks3-server[1196]: 2026/09/25 16:51:55 INFO Received complete push request method=POST path=/api/pushes/37/complete5221lb # [ 129.603935] niks3-server[1196]: 2026/09/25 16:51:55 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5222worker2 # [ 129.627408] nix-grpc-daemon[1299]: time=2026-09-25T16:51:55.222Z level=INFO msg="Upload complete. (209ms)"5223worker1 # [ 129.634650] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:55Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05224worker2 # [ 129.636627] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05225worker1 # [ 129.634874] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:55Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15226worker2 # [ 129.647135] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409185227worker2 # [ 129.650279] nix-daemon[842]: reaped child process 1328, status = succeeded5228worker2 # [ 129.657726] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409185229worker2 # [ 129.666171] nix-grpc-daemon[1299]: time=2026-09-25T16:51:55.261Z level=INFO msg="All 1 paths already cached"5230client # copying 1 paths to 'lb:50051'...5231client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5232worker2 # [ 129.672305] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409185233worker2 # [ 129.679576] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5234worker2 # [ 129.792696] nix-daemon[842]: accepted connection from pid 1353, user nix-grpc-daemon (trusted)5235lb # [ 129.791000] niks3-server[1196]: 2026/09/25 16:51:55 INFO Received push request method=POST path=/api/pushes5236worker2 # [ 129.813653] nix-daemon[842]: reaped child process 1356, status = succeeded5237worker2 # [ 129.816733] nix-grpc-daemon[1299]: time=2026-09-25T16:51:55.411Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5238worker2 # [ 129.818314] nix-grpc-daemon[1299]: time=2026-09-25T16:51:55.411Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5239lb # [ 129.808887] niks3-server[1196]: 2026/09/25 16:51:55 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign5240worker2 # [ 129.833249] nix-grpc-daemon[1299]: time=2026-09-25T16:51:55.428Z level=INFO msg="Uploading 1 narinfos"5241lb # [ 129.815341] niks3-server[1196]: 2026/09/25 16:51:55 INFO Signed narinfos id=38 count=15242lb # [ 129.820733] niks3-server[1196]: 2026/09/25 16:51:55 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5243lb # [ 129.823848] niks3-server[1196]: 2026/09/25 16:51:55 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5244lb # [ 129.833044] niks3-server[1196]: 2026/09/25 16:51:55 INFO Received complete push request method=POST path=/api/pushes/38/complete5245worker2 # [ 129.861594] nix-grpc-daemon[1299]: time=2026-09-25T16:51:55.456Z level=INFO msg="Upload complete. (172ms)"5246lb # [ 129.840791] niks3-server[1196]: 2026/09/25 16:51:55 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5247worker2 # [ 129.867748] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=0 paths=1 nar_bytes_in=6565248worker2 # [ 129.876186] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409185249worker2 # [ 129.880342] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5250worker2 # [ 129.883941] nix-daemon[842]: reaped child process 1351, status = succeeded5251client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5252worker2 # [ 129.892236] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5253client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5254client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5255client # farm-b> LOG-farm-b-after-niks3-restart5256worker2 # [ 131.130156] nix-daemon[842]: reaped child process 1360, status = succeeded5257worker2 # [ 131.237264] nix-daemon[842]: accepted connection from pid 1376, user nix-grpc-daemon (trusted)5258worker2 # [ 131.256097] nix-daemon[842]: reaped child process 1379, status = succeeded5259lb # [ 131.239814] niks3-server[1196]: 2026/09/25 16:51:56 INFO Received push request method=POST path=/api/pushes5260worker2 # [ 131.268322] nix-grpc-daemon[1299]: time=2026-09-25T16:51:56.862Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5261worker2 # [ 131.270003] nix-grpc-daemon[1299]: time=2026-09-25T16:51:56.862Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5262lb # [ 131.264761] niks3-server[1196]: 2026/09/25 16:51:56 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5263worker2 # [ 131.291076] nix-grpc-daemon[1299]: time=2026-09-25T16:51:56.886Z level=INFO msg="Uploading 1 narinfos"5264lb # [ 131.271433] niks3-server[1196]: 2026/09/25 16:51:56 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign5265lb # [ 131.279810] niks3-server[1196]: 2026/09/25 16:51:56 INFO Signed narinfos id=39 count=15266lb # [ 131.284708] niks3-server[1196]: 2026/09/25 16:51:56 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5267lb # [ 131.293500] niks3-server[1196]: 2026/09/25 16:51:56 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5268lb # [ 131.300200] niks3-server[1196]: 2026/09/25 16:51:56 INFO Received complete push request method=POST path=/api/pushes/39/complete5269worker2 # [ 131.323836] nix-grpc-daemon[1299]: time=2026-09-25T16:51:56.918Z level=INFO msg="Upload complete. (222ms)"5270lb # [ 131.303964] niks3-server[1196]: 2026/09/25 16:51:56 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5271worker1 # [ 131.330670] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:56Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05272worker2 # [ 131.332685] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05273worker1 # [ 131.330917] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:56Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15274worker2 # [ 131.343686] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409185275worker2 # [ 131.347911] nix-daemon[842]: reaped child process 1358, status = succeeded5276worker2 # [ 131.353134] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409185277worker2 # [ 131.360799] nix-grpc-daemon[1299]: time=2026-09-25T16:51:56.955Z level=INFO msg="All 2 paths already cached"5278client # copying 1 paths to 'lb:50051'...5279client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5280worker2 # [ 131.367035] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409185281worker2 # [ 131.374547] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5282worker2 # [ 131.490637] nix-daemon[842]: accepted connection from pid 1383, user nix-grpc-daemon (trusted)5283worker2 # [ 131.509764] nix-daemon[842]: reaped child process 1386, status = succeeded5284lb # [ 131.489796] niks3-server[1196]: 2026/09/25 16:51:57 INFO Received push request method=POST path=/api/pushes5285worker2 # [ 131.515614] nix-grpc-daemon[1299]: time=2026-09-25T16:51:57.110Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5286worker2 # [ 131.517305] nix-grpc-daemon[1299]: time=2026-09-25T16:51:57.110Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5287lb # [ 131.506798] niks3-server[1196]: 2026/09/25 16:51:57 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5288worker2 # [ 131.533597] nix-grpc-daemon[1299]: time=2026-09-25T16:51:57.128Z level=INFO msg="Uploading 1 narinfos"5289lb # [ 131.514680] niks3-server[1196]: 2026/09/25 16:51:57 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign5290lb # [ 131.520121] niks3-server[1196]: 2026/09/25 16:51:57 INFO Signed narinfos id=40 count=15291lb # [ 131.521852] niks3-server[1196]: 2026/09/25 16:51:57 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5292lb # [ 131.533695] niks3-server[1196]: 2026/09/25 16:51:57 INFO Received complete push request method=POST path=/api/pushes/40/complete5293worker2 # [ 131.557878] nix-grpc-daemon[1299]: time=2026-09-25T16:51:57.153Z level=INFO msg="Upload complete. (174ms)"5294lb # [ 131.537316] niks3-server[1196]: 2026/09/25 16:51:57 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5295worker2 # [ 131.569326] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=0 paths=1 nar_bytes_in=7845296worker2 # [ 131.572581] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409185297worker2 # [ 131.577802] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5298worker2 # [ 131.581609] nix-daemon[842]: reaped child process 1381, status = succeeded5299client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5300client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5301worker2 # [ 131.591891] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5302client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5303client # farm-top> LOG-farm-top-after-niks3-restart5304worker2 # [ 133.229272] nix-daemon[842]: reaped child process 1390, status = succeeded5305worker2 # [ 133.335372] nix-daemon[842]: accepted connection from pid 1406, user nix-grpc-daemon (trusted)5306worker2 # [ 133.351237] nix-daemon[842]: reaped child process 1409, status = succeeded5307lb # [ 133.337062] niks3-server[1196]: 2026/09/25 16:51:58 INFO Received push request method=POST path=/api/pushes5308worker2 # [ 133.363861] nix-grpc-daemon[1299]: time=2026-09-25T16:51:58.958Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5309worker2 # [ 133.365647] nix-grpc-daemon[1299]: time=2026-09-25T16:51:58.958Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5310lb # [ 133.359232] niks3-server[1196]: 2026/09/25 16:51:58 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5311worker2 # [ 133.387346] nix-grpc-daemon[1299]: time=2026-09-25T16:51:58.982Z level=INFO msg="Uploading 1 narinfos"5312lb # [ 133.369200] niks3-server[1196]: 2026/09/25 16:51:58 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign5313lb # [ 133.374718] niks3-server[1196]: 2026/09/25 16:51:58 INFO Signed narinfos id=41 count=15314lb # [ 133.378725] niks3-server[1196]: 2026/09/25 16:51:58 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5315lb # [ 133.384447] niks3-server[1196]: 2026/09/25 16:51:58 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5316lb # [ 133.391966] niks3-server[1196]: 2026/09/25 16:51:58 INFO Received complete push request method=POST path=/api/pushes/41/complete5317worker2 # [ 133.416387] nix-grpc-daemon[1299]: time=2026-09-25T16:51:59.011Z level=INFO msg="Upload complete. (216ms)"5318lb # [ 133.396558] niks3-server[1196]: 2026/09/25 16:51:58 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5319worker2 # [ 133.424246] nix-grpc-daemon[1239]: ts=2026-09-25T16:51:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=05320worker2 # [ 133.434583] nix-daemon[842]: reaped child process 1388, status = succeeded5321client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 5.36 seconds)5322client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5323client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.14 seconds)5324(finished: subtest: a niks3 restart does not cost the leader its role, in 9.59 seconds)5325subtest: scheduler down: the other node takes the lock and keeps it5326lb: must succeed: curl -sf localhost:9901/clusters5327lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5328lb: must succeed: curl -sf localhost:9901/clusters5329lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5330lb: must succeed: curl -sf localhost:9901/clusters5331lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5332lb: must succeed: curl -sf localhost:9901/clusters5333lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5334worker1 # [ 133.749707] systemd[1]: Stopping Nix worker-protocol over gRPC...5335worker1 # [ 133.882766] nix-grpc-daemon[1321]: ts=2026-09-25T16:51:59Z level=info event=signal_exit5336lb # [ 133.886137] niks3-server[1196]: 2026/09/25 16:51:59 INFO lead: released remote=192.168.1.3:604005337worker1 # [ 134.900654] nix-grpc-daemon[1321]: ts=2026-09-25T16:52:00Z level=info event=scheduler_restarting5338worker2 # [ 134.906496] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:00Z level=info event=scheduler_restarting addr=lb:500515339lb # [ 134.883930] envoy[886]: [2026-09-25 16:52:00.477][886][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5340worker1 # [ 134.904789] nix-grpc-daemon[1321]: ts=2026-09-25T16:52:00Z level=info event=scheduler_restarting addr=lb:500515341worker1 # [ 134.908999] nix-grpc-daemon[1321]: ts=2026-09-25T16:52:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:576905342worker1 # [ 134.911436] nix-grpc-daemon[1321]: ts=2026-09-25T16:52:00Z level=info event=worker_gone addr=192.168.1.4:500515343worker1 # [ 134.913673] nix-grpc-daemon[1321]: ts=2026-09-25T16:52:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:576905344worker1 # [ 134.915964] nix-grpc-daemon[1321]: ts=2026-09-25T16:52:00Z level=info event=worker_gone addr=192.168.1.3:500515345worker2 # [ 135.106833] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:00Z level=info event=scheduler_connected addr=lb:500515346worker2 # [ 135.109893] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5347worker2 # [ 135.310471] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:00Z level=info event=scheduler_connected addr=lb:500515348worker2 # [ 135.313257] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5349worker2 # [ 135.513978] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:01Z level=info event=scheduler_connected addr=lb:500515350worker2 # [ 135.516744] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5351worker2 # [ 135.717527] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:01Z level=info event=scheduler_connected addr=lb:500515352worker2 # [ 135.720291] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5353worker2 # [ 135.922019] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:01Z level=info event=scheduler_connected addr=lb:500515354worker2 # [ 135.924230] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5355worker1 # [ 136.015186] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5356worker1 # [ 136.018188] systemd[1]: Stopped Nix worker-protocol over gRPC.5357worker1 # [ 136.022214] systemd[1]: nix-grpc-daemon.service: Consumed 230ms CPU time over 13.283s wall clock time, 15.2M memory peak, 147.7K incoming IP traffic, 34.7K outgoing IP traffic.5358worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5359worker2 # [ 136.122776] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:01Z level=info event=scheduler_connected addr=lb:500515360worker2 # [ 136.125785] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5361worker2 # [ 136.326326] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:01Z level=info event=scheduler_connected addr=lb:500515362worker2 # [ 136.329141] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5363worker2 # [ 136.529785] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:02Z level=info event=scheduler_connected addr=lb:500515364worker2 # [ 136.532589] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5365worker2 # [ 136.733135] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:02Z level=info event=scheduler_connected addr=lb:500515366worker2 # [ 136.735938] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5367worker2 # [ 136.937813] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:02Z level=info event=scheduler_connected addr=lb:500515368worker2 # [ 136.940354] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5369lb # [ 137.048590] envoy[886]: conn peer=[2001:db8:1::1]:36492 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9719 tx=4406 ms=51725370lb # [ 137.051446] envoy[886]: conn peer=[2001:db8:1::3]:43684 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=801 tx=310 ms=121815371worker2 # [ 137.141150] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:02Z level=info event=scheduler_connected addr=lb:500515372worker2 # [ 137.143942] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5373worker2 # [ 137.344671] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:02Z level=info event=scheduler_connected addr=lb:500515374worker2 # [ 137.347583] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5375worker2 # [ 137.548214] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:03Z level=info event=scheduler_connected addr=lb:500515376worker2 # [ 137.551226] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5377worker2 # [ 137.751724] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:03Z level=info event=scheduler_connected addr=lb:500515378worker2 # [ 137.754712] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5379worker2 # [ 137.901410] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:03Z level=info event=scheduler_take_over5380lb # [ 137.881360] niks3-server[1196]: 2026/09/25 16:52:03 INFO lead: acquired remote=192.168.1.4:592165381worker2 # [ 137.956586] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:03Z level=info event=scheduler_connected addr=lb:500515382worker2 # [ 137.958826] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5383worker2 # [ 138.160102] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:03Z level=info event=scheduler_connected addr=lb:500515384worker2 # [ 138.162417] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5385worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 2.14 seconds)5386worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5387worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5388worker2 # [ 138.358303] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:03Z level=info event=scheduler_connected addr=lb:500515389worker2 # [ 138.372085] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:03Z level=info event=oidc_jwks provider=mock keys=15390worker2 # [ 138.375567] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:03Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:600165391worker2 # [ 138.379148] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:03Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05392worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5393worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5394client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25395worker2 # [ 139.430245] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:409185396client # don't know how to build these paths:5397client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5398worker2 # [ 139.437927] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409185399worker2 # [ 139.454845] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:05Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05400worker2 # [ 139.460428] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:05Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15401worker2 # [ 139.465026] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409185402worker2 # [ 139.469266] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409185403client # copying 1 paths to 'lb:50051'...5404client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5405worker2 # [ 139.476079] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409185406worker2 # [ 139.485529] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5407worker2 # [ 139.617554] nix-daemon[842]: accepted connection from pid 1439, user nix-grpc-daemon (trusted)5408worker2 # [ 139.638804] nix-daemon[842]: reaped child process 1442, status = succeeded5409lb # [ 139.622593] niks3-server[1196]: 2026/09/25 16:52:05 INFO Received push request method=POST path=/api/pushes5410worker2 # [ 139.650201] nix-grpc-daemon[1299]: time=2026-09-25T16:52:05.244Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5411worker2 # [ 139.651846] nix-grpc-daemon[1299]: time=2026-09-25T16:52:05.244Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5412lb # [ 139.640964] niks3-server[1196]: 2026/09/25 16:52:05 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5413lb # [ 139.648672] niks3-server[1196]: 2026/09/25 16:52:05 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign5414worker2 # [ 139.673642] nix-grpc-daemon[1299]: time=2026-09-25T16:52:05.268Z level=INFO msg="Uploading 1 narinfos"5415lb # [ 139.655396] niks3-server[1196]: 2026/09/25 16:52:05 INFO Signed narinfos id=42 count=15416lb # [ 139.662655] niks3-server[1196]: 2026/09/25 16:52:05 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5417lb # [ 139.672265] niks3-server[1196]: 2026/09/25 16:52:05 INFO Received complete push request method=POST path=/api/pushes/42/complete5418lb # [ 139.679605] niks3-server[1196]: 2026/09/25 16:52:05 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5419worker2 # [ 139.705609] nix-grpc-daemon[1299]: time=2026-09-25T16:52:05.300Z level=INFO msg="Upload complete. (209ms)"5420worker2 # [ 139.711637] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=0 paths=1 nar_bytes_in=5205421worker2 # [ 139.720584] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409185422worker2 # [ 139.737254] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5423worker2 # [ 139.740396] nix-daemon[842]: reaped child process 1437, status = succeeded5424client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5425client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5426worker2 # [ 139.758800] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5427client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5428client # farm-a> LOG-farm-a-on-standby5429worker2 # [ 140.800256] nix-daemon[842]: reaped child process 1446, status = succeeded5430worker2 # [ 140.909373] nix-daemon[842]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)5431worker2 # [ 140.930422] nix-daemon[842]: reaped child process 1465, status = succeeded5432lb # [ 140.908799] niks3-server[1196]: 2026/09/25 16:52:06 INFO Received push request method=POST path=/api/pushes5433worker2 # [ 140.936453] nix-grpc-daemon[1299]: time=2026-09-25T16:52:06.530Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5434worker2 # [ 140.939707] nix-grpc-daemon[1299]: time=2026-09-25T16:52:06.530Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5435lb # [ 140.935303] niks3-server[1196]: 2026/09/25 16:52:06 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign5436worker2 # [ 140.963265] nix-grpc-daemon[1299]: time=2026-09-25T16:52:06.558Z level=INFO msg="Uploading 1 narinfos"5437lb # [ 140.944410] niks3-server[1196]: 2026/09/25 16:52:06 INFO Signed narinfos id=43 count=15438lb # [ 140.946305] niks3-server[1196]: 2026/09/25 16:52:06 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5439lb # [ 140.959895] niks3-server[1196]: 2026/09/25 16:52:06 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5440worker2 # [ 140.998050] nix-grpc-daemon[1299]: time=2026-09-25T16:52:06.593Z level=INFO msg="Upload complete. (227ms)"5441lb # [ 140.971853] niks3-server[1196]: 2026/09/25 16:52:06 INFO Received complete push request method=POST path=/api/pushes/43/complete5442lb # [ 140.982127] niks3-server[1196]: 2026/09/25 16:52:06 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5443lb # [ 140.991981] niks3-server[1196]: 2026/09/25 16:52:06 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5444worker2 # [ 141.009031] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05445worker2 # [ 141.025282] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:06Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05446worker2 # [ 141.029583] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:06Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15447worker2 # [ 141.033755] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409185448worker2 # [ 141.037421] nix-daemon[842]: reaped child process 1444, status = succeeded5449worker2 # [ 141.040428] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409185450worker2 # [ 141.046975] nix-grpc-daemon[1299]: time=2026-09-25T16:52:06.642Z level=INFO msg="All 1 paths already cached"5451client # copying 1 paths to 'lb:50051'...5452client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5453worker2 # [ 141.053659] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409185454worker2 # [ 141.061859] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5455worker2 # [ 141.182574] nix-daemon[842]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5456lb # [ 141.179550] niks3-server[1196]: 2026/09/25 16:52:06 INFO Received push request method=POST path=/api/pushes5457worker2 # [ 141.203480] nix-daemon[842]: reaped child process 1474, status = succeeded5458worker2 # [ 141.207639] nix-grpc-daemon[1299]: time=2026-09-25T16:52:06.802Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5459worker2 # [ 141.209517] nix-grpc-daemon[1299]: time=2026-09-25T16:52:06.802Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5460lb # [ 141.200166] niks3-server[1196]: 2026/09/25 16:52:06 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5461lb # [ 141.208905] niks3-server[1196]: 2026/09/25 16:52:06 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign5462worker2 # [ 141.232671] nix-grpc-daemon[1299]: time=2026-09-25T16:52:06.827Z level=INFO msg="Uploading 1 narinfos"5463lb # [ 141.215222] niks3-server[1196]: 2026/09/25 16:52:06 INFO Signed narinfos id=44 count=15464lb # [ 141.219979] niks3-server[1196]: 2026/09/25 16:52:06 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5465lb # [ 141.228252] niks3-server[1196]: 2026/09/25 16:52:06 INFO Received complete push request method=POST path=/api/pushes/44/complete5466lb # [ 141.236544] niks3-server[1196]: 2026/09/25 16:52:06 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5467worker2 # [ 141.262400] nix-grpc-daemon[1299]: time=2026-09-25T16:52:06.857Z level=INFO msg="Upload complete. (191ms)"5468worker2 # [ 141.268589] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=0 paths=1 nar_bytes_in=6405469worker2 # [ 141.271914] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409185470worker2 # [ 141.285017] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5471worker2 # [ 141.289591] nix-daemon[842]: reaped child process 1469, status = succeeded5472client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5473client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5474worker2 # [ 141.300252] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5475client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5476client # farm-b> LOG-farm-b-on-standby5477worker2 # [ 142.004235] nix-daemon[842]: reaped child process 1478, status = succeeded5478worker2 # [ 142.089604] nix-daemon[842]: accepted connection from pid 1494, user nix-grpc-daemon (trusted)5479worker2 # [ 142.112582] nix-daemon[842]: reaped child process 1497, status = succeeded5480lb # [ 142.095755] niks3-server[1196]: 2026/09/25 16:52:07 INFO Received push request method=POST path=/api/pushes5481worker2 # [ 142.124532] nix-grpc-daemon[1299]: time=2026-09-25T16:52:07.717Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5482worker2 # [ 142.126166] nix-grpc-daemon[1299]: time=2026-09-25T16:52:07.717Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5483lb # [ 142.131226] niks3-server[1196]: 2026/09/25 16:52:07 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign5484worker2 # [ 142.155735] nix-grpc-daemon[1299]: time=2026-09-25T16:52:07.750Z level=INFO msg="Uploading 1 narinfos"5485lb # [ 142.138366] niks3-server[1196]: 2026/09/25 16:52:07 INFO Signed narinfos id=45 count=15486lb # [ 142.143139] niks3-server[1196]: 2026/09/25 16:52:07 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5487lb # [ 142.151410] niks3-server[1196]: 2026/09/25 16:52:07 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5488lb # [ 142.156454] niks3-server[1196]: 2026/09/25 16:52:07 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5489worker2 # [ 142.180632] nix-grpc-daemon[1299]: time=2026-09-25T16:52:07.775Z level=INFO msg="Upload complete. (271ms)"5490lb # [ 142.161344] niks3-server[1196]: 2026/09/25 16:52:07 INFO Received complete push request method=POST path=/api/pushes/45/complete5491lb # [ 142.163724] niks3-server[1196]: 2026/09/25 16:52:07 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5492worker2 # [ 142.188834] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=0 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05493worker2 # [ 142.204875] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:07Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05494worker2 # [ 142.207641] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:07Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15495worker2 # [ 142.213953] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409185496worker2 # [ 142.217877] nix-daemon[842]: reaped child process 1476, status = succeeded5497worker2 # [ 142.220200] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409185498worker2 # [ 142.227657] nix-grpc-daemon[1299]: time=2026-09-25T16:52:07.822Z level=INFO msg="All 2 paths already cached"5499client # copying 1 paths to 'lb:50051'...5500client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5501worker2 # [ 142.233897] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409185502worker2 # [ 142.242116] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5503worker2 # [ 142.361178] nix-daemon[842]: accepted connection from pid 1503, user nix-grpc-daemon (trusted)5504lb # [ 142.358003] niks3-server[1196]: 2026/09/25 16:52:07 INFO Received push request method=POST path=/api/pushes5505worker2 # [ 142.382249] nix-daemon[842]: reaped child process 1506, status = succeeded5506worker2 # [ 142.384133] nix-grpc-daemon[1299]: time=2026-09-25T16:52:07.979Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5507worker2 # [ 142.386215] nix-grpc-daemon[1299]: time=2026-09-25T16:52:07.980Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5508lb # [ 142.376689] niks3-server[1196]: 2026/09/25 16:52:07 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5509worker2 # [ 142.403637] nix-grpc-daemon[1299]: time=2026-09-25T16:52:07.998Z level=INFO msg="Uploading 1 narinfos"5510lb # [ 142.382785] niks3-server[1196]: 2026/09/25 16:52:07 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign5511lb # [ 142.389066] niks3-server[1196]: 2026/09/25 16:52:07 INFO Signed narinfos id=46 count=15512lb # [ 142.393287] niks3-server[1196]: 2026/09/25 16:52:07 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5513lb # [ 142.400867] niks3-server[1196]: 2026/09/25 16:52:07 INFO Received complete push request method=POST path=/api/pushes/46/complete5514worker2 # [ 142.430533] nix-grpc-daemon[1299]: time=2026-09-25T16:52:08.025Z level=INFO msg="Upload complete. (178ms)"5515lb # [ 142.410913] niks3-server[1196]: 2026/09/25 16:52:08 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5516worker2 # [ 142.436396] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=0 paths=1 nar_bytes_in=7685517worker2 # [ 142.446920] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409185518worker2 # [ 142.454051] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5519worker2 # [ 142.455733] nix-daemon[842]: reaped child process 1501, status = succeeded5520client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5521worker2 # [ 142.464789] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5522client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5523worker2 # [ 143.037697] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:08Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:408505524client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5525client # farm-top> LOG-farm-top-on-standby5526worker2 # [ 143.312846] nix-daemon[842]: reaped child process 1510, status = succeeded5527worker2 # [ 143.396723] nix-daemon[842]: accepted connection from pid 1526, user nix-grpc-daemon (trusted)5528lb # [ 143.394208] niks3-server[1196]: 2026/09/25 16:52:08 INFO Received push request method=POST path=/api/pushes5529worker2 # [ 143.416503] nix-daemon[842]: reaped child process 1529, status = succeeded5530worker2 # [ 143.424478] nix-grpc-daemon[1299]: time=2026-09-25T16:52:09.016Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5531worker2 # [ 143.426069] nix-grpc-daemon[1299]: time=2026-09-25T16:52:09.016Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5532lb # [ 143.424295] niks3-server[1196]: 2026/09/25 16:52:09 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5533worker2 # [ 143.451202] nix-grpc-daemon[1299]: time=2026-09-25T16:52:09.046Z level=INFO msg="Uploading 1 narinfos"5534lb # [ 143.431736] niks3-server[1196]: 2026/09/25 16:52:09 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5535lb # [ 143.439209] niks3-server[1196]: 2026/09/25 16:52:09 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign5536lb # [ 143.444406] niks3-server[1196]: 2026/09/25 16:52:09 INFO Signed narinfos id=47 count=15537lb # [ 143.450905] niks3-server[1196]: 2026/09/25 16:52:09 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5538lb # [ 143.455365] niks3-server[1196]: 2026/09/25 16:52:09 INFO Received complete push request method=POST path=/api/pushes/47/complete5539worker2 # [ 143.479921] nix-grpc-daemon[1299]: time=2026-09-25T16:52:09.075Z level=INFO msg="Upload complete. (206ms)"5540lb # [ 143.460560] niks3-server[1196]: 2026/09/25 16:52:09 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5541worker2 # [ 143.487184] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=05542worker2 # [ 143.499331] nix-daemon[842]: reaped child process 1508, status = succeeded5543client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 4.25 seconds)5544client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5545client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.14 seconds)5546worker1 # [ 143.692810] systemd[1]: Starting Nix worker-protocol over gRPC...5547worker1 # [ 143.814431] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5548worker1 # [ 143.816816] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5549worker1 # [ 143.820698] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15550worker1 # [ 143.826008] nix-grpc-daemon[1404]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5551worker1 # [ 143.827512] nix-grpc-daemon[1404]: E0000 00:00:1790355129.416539 1404 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5552worker1 # [ 143.831751] nix-grpc-daemon[1404]: E0000 00:00:1790355129.422852 1404 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5553lb: must succeed: curl -sf localhost:9901/clusters5554worker1 # [ 143.875152] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:09Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5555worker1 # [ 143.877515] nix-grpc-daemon[1404]: E0000 00:00:1790355129.431546 1444 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5556worker1 # [ 143.883181] nix-grpc-daemon[1404]: E0000 00:00:1790355129.448680 1444 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5557worker1 # [ 143.887420] nix-grpc-daemon[1404]: E0000 00:00:1790355129.448950 1444 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5558worker2 # [ 143.891300] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:600165559worker1 # [ 143.890477] nix-grpc-daemon[1404]: E0000 00:00:1790355129.465520 1444 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5560worker2 # [ 143.895303] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05561worker1 # [ 143.895735] systemd[1]: Started Nix worker-protocol over gRPC.5562worker1 # [ 143.897894] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:09Z level=info event=scheduler_connected addr=lb:500515563lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5564worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5565worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5566lb: must succeed: curl -sf localhost:9901/clusters5567lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5568lb: must succeed: curl -sf localhost:9901/clusters5569lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5570worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5571worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5572(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 10.47 seconds)5573subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5574client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5575client # [ 144.082451] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5576client # Running as unit: rs.service; invocation ID: b0928b459fa143769ca9523f38136fce5577client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.05 seconds)5578worker1 # [ 144.287421] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334125579worker1 # [ 144.296322] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)5580client # [ 144.289696] nix[1370]: don't know how to build these paths:5581client # [ 144.291579] nix[1370]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5582worker2 # [ 144.316740] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:09Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05583worker2 # [ 144.322454] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:09Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=15584worker2 # [ 144.327389] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409185585worker2 # [ 144.331820] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:409185586client # [ 144.318488] nix[1370]: copying 1 paths to 'lb:50051'...5587worker2 # [ 144.338237] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:409185588client # [ 144.319863] nix[1370]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5589worker2 # [ 144.347630] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5590worker2 # [ 144.479298] nix-daemon[842]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5591lb # [ 144.477625] niks3-server[1196]: 2026/09/25 16:52:10 INFO Received push request method=POST path=/api/pushes5592worker2 # [ 144.499991] nix-daemon[842]: reaped child process 1545, status = succeeded5593worker2 # [ 144.504434] nix-grpc-daemon[1299]: time=2026-09-25T16:52:10.099Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5594worker2 # [ 144.505991] nix-grpc-daemon[1299]: time=2026-09-25T16:52:10.099Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5595worker2 # [ 144.526451] nix-grpc-daemon[1299]: time=2026-09-25T16:52:10.121Z level=INFO msg="Uploading 1 narinfos"5596lb # [ 144.499471] niks3-server[1196]: 2026/09/25 16:52:10 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign5597lb # [ 144.508415] niks3-server[1196]: 2026/09/25 16:52:10 INFO Signed narinfos id=48 count=15598lb # [ 144.510168] niks3-server[1196]: 2026/09/25 16:52:10 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5599lb # [ 144.520938] niks3-server[1196]: 2026/09/25 16:52:10 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5600lb # [ 144.523958] niks3-server[1196]: 2026/09/25 16:52:10 INFO Received complete push request method=POST path=/api/pushes/48/complete5601worker2 # [ 144.549846] nix-grpc-daemon[1299]: time=2026-09-25T16:52:10.145Z level=INFO msg="Upload complete. (190ms)"5602lb # [ 144.530696] niks3-server[1196]: 2026/09/25 16:52:10 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5603worker2 # [ 144.557176] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=0 paths=1 nar_bytes_in=4165604worker2 # [ 144.566146] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:409185605worker2 # [ 144.572293] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5606worker2 # [ 144.575590] nix-daemon[842]: reaped child process 1540, status = succeeded5607client # [ 144.563463] nix[1370]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5608worker2 # [ 144.586270] nix-daemon[842]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5609lb: must succeed: curl -sf localhost:9901/clusters5610lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5611lb: must succeed: curl -sf localhost:9901/clusters5612lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5613lb: must succeed: curl -sf localhost:9901/clusters5614lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5615lb: must succeed: curl -sf localhost:9901/clusters5616lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5617worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5618worker1 # [ 146.647492] systemd-journald[272]: Received client request to rotate journal, rotating.5619worker1 # [ 146.659290] systemd-journald[272]: Vacuuming done, freed 0B of archived journals from /var/log/journal/a6739303e1814a628d8ca3a15873844a.5620worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.04 seconds)5621worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5622worker2 # [ 146.685913] systemd[1]: Stopping Nix worker-protocol over gRPC...5623worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5624client: must succeed: sleep 5.05625worker2 # [ 146.942420] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:12Z level=info event=draining5626lb # [ 147.043126] envoy[886]: conn peer=[2001:db8:1::1]:33286 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9585 tx=4333 ms=40705627client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5628worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5629worker2 # [ 152.064414] nix-daemon[842]: reaped child process 1549, status = succeeded5630worker2 # [ 152.132411] nix-daemon[842]: accepted connection from pid 1590, user nix-grpc-daemon (trusted)5631worker2 # [ 152.153280] nix-daemon[842]: reaped child process 1593, status = succeeded5632lb # [ 152.136864] niks3-server[1196]: 2026/09/25 16:52:17 INFO Received push request method=POST path=/api/pushes5633worker2 # [ 152.164371] nix-grpc-daemon[1299]: time=2026-09-25T16:52:17.758Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5634worker2 # [ 152.165952] nix-grpc-daemon[1299]: time=2026-09-25T16:52:17.758Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5635lb # [ 152.168842] niks3-server[1196]: 2026/09/25 16:52:17 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5636worker2 # [ 152.197403] nix-grpc-daemon[1299]: time=2026-09-25T16:52:17.792Z level=INFO msg="Uploading 1 narinfos"5637lb # [ 152.176650] niks3-server[1196]: 2026/09/25 16:52:17 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5638lb # [ 152.179804] niks3-server[1196]: 2026/09/25 16:52:17 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign5639lb # [ 152.192441] niks3-server[1196]: 2026/09/25 16:52:17 INFO Signed narinfos id=49 count=15640lb # [ 152.194187] niks3-server[1196]: 2026/09/25 16:52:17 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5641worker2 # [ 152.219605] nix-grpc-daemon[1299]: time=2026-09-25T16:52:17.814Z level=INFO msg="Upload complete. (221ms)"5642lb # [ 152.200540] niks3-server[1196]: 2026/09/25 16:52:17 INFO Received complete push request method=POST path=/api/pushes/49/complete5643lb # [ 152.204454] niks3-server[1196]: 2026/09/25 16:52:17 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5644worker2 # [ 152.229320] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40918 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05645worker2 # [ 152.240379] nix-daemon[842]: reaped child process 1547, status = succeeded5646client # [ 152.232932] systemd[1]: rs.service: Deactivated successfully.5647client # [ 152.235772] systemd[1]: rs.service: Consumed 177ms CPU time over 8.151s wall clock time, 39.5M memory peak, 7K incoming IP traffic, 8.2K outgoing IP traffic.5648worker2 # [ 152.943267] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:18Z level=info event=signal_exit5649lb # [ 153.886883] niks3-server[1196]: 2026/09/25 16:52:19 INFO lead: released remote=192.168.1.4:592165650worker2 # [ 154.903553] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:20Z level=info event=scheduler_restarting5651worker1 # [ 154.902340] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:20Z level=info event=scheduler_restarting addr=lb:500515652lb # [ 154.883227] envoy[886]: [2026-09-25 16:52:20.477][886][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5653worker2 # [ 154.907453] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:20Z level=info event=scheduler_restarting addr=lb:500515654worker2 # [ 154.912646] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:600165655worker2 # [ 154.915205] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:20Z level=info event=worker_gone addr=192.168.1.3:500515656worker2 # [ 154.918226] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:600165657worker2 # [ 154.921443] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:20Z level=info event=worker_gone addr=192.168.1.4:500515658worker2 # [ 154.937055] nix-daemon[842]: reaped child process 1291, status = succeeded5659worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.14 seconds)5660client: waiting for success: ! systemctl is-active rs5661client: (finished: waiting for success: ! systemctl is-active rs, in 0.03 seconds)5662client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5663worker1 # [ 155.102426] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:20Z level=info event=scheduler_connected addr=lb:500515664worker1 # [ 155.105419] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5665client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5666client: must fail: journalctl -u rs | grep -q 'reconnecting'5667worker2 # [ 155.121606] nix-grpc-daemon[1239]: ts=2026-09-25T16:52:20Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5668client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5669lb: must succeed: curl -sf localhost:9901/clusters5670lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5671worker1 # [ 155.305796] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:20Z level=info event=scheduler_connected addr=lb:500515672worker1 # [ 155.308564] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5673worker1 # [ 155.508958] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:21Z level=info event=scheduler_connected addr=lb:500515674worker1 # [ 155.511696] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5675worker1 # [ 155.712660] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:21Z level=info event=scheduler_connected addr=lb:500515676worker1 # [ 155.715431] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5677worker1 # [ 155.916801] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:21Z level=info event=scheduler_connected addr=lb:500515678worker1 # [ 155.918995] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5679worker2 # [ 155.989566] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5680worker2 # [ 155.992517] systemd[1]: Stopped Nix worker-protocol over gRPC.5681worker2 # [ 155.993954] systemd[1]: nix-grpc-daemon.service: Consumed 2.244s CPU time over 36.191s wall clock time, 47.5M memory peak, 379.4K incoming IP traffic, 249.1K outgoing IP traffic.5682worker2 # [ 156.006002] systemd[1]: Starting scheduler-token.service...5683worker2 # [ 156.057093] systemd[1]: scheduler-token.service: Deactivated successfully.5684worker2 # [ 156.062470] systemd[1]: Finished scheduler-token.service.5685worker2 # [ 156.063743] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 50ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5686worker2 # [ 156.068778] systemd[1]: Starting Nix worker-protocol over gRPC...5687worker1 # [ 156.117043] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:21Z level=info event=scheduler_connected addr=lb:500515688worker1 # [ 156.119846] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5689worker2 # [ 156.179761] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:21Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5690worker2 # [ 156.182258] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:21Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5691lb: must succeed: curl -sf localhost:9901/clusters5692worker2 # [ 156.186230] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:21Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15693worker2 # [ 156.192060] nix-grpc-daemon[1604]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5694worker2 # [ 156.193647] nix-grpc-daemon[1604]: E0000 00:00:1790355141.785633 1604 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5695worker2 # [ 156.197204] nix-grpc-daemon[1604]: E0000 00:00:1790355141.792137 1604 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5696worker2 # [ 156.200607] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:21Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5697worker2 # [ 156.202490] systemd[1]: Started Nix worker-protocol over gRPC.5698lb # [ 156.189517] niks3-server[1196]: 2026/09/25 16:52:21 INFO lead: acquired remote=192.168.1.4:339325699lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5700worker2 # [ 156.232052] nix-grpc-daemon[1604]: E0000 00:00:1790355141.803719 1647 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5701worker2 # [ 156.234567] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:21Z level=info event=scheduler_take_over5702worker2 # [ 156.235701] nix-grpc-daemon[1604]: E0000 00:00:1790355141.819221 1647 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5703worker2 # [ 156.246151] nix-grpc-daemon[1604]: E0000 00:00:1790355141.819391 1647 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5704worker2 # [ 156.255555] nix-grpc-daemon[1604]: E0000 00:00:1790355141.840305 1647 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5705worker2 # [ 156.260513] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:21Z level=info event=scheduler_connected addr=lb:500515706worker2 # [ 156.265889] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:21Z level=info event=oidc_jwks provider=mock keys=15707worker2 # [ 156.267227] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:21Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:603105708worker2 # [ 156.269270] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:21Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05709worker1 # [ 156.320383] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:21Z level=info event=scheduler_connected addr=lb:500515710worker2 # [ 156.324811] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:21Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:603105711worker2 # [ 156.327447] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:21Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05712lb # [ 157.045137] envoy[886]: conn peer=[2001:db8:1::1]:59396 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2949 tx=1499 ms=79505713lb # [ 157.047815] envoy[886]: conn peer=[2001:db8:1::4]:50640 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=23077 tx=1826 ms=351325714worker2 # [ 157.215116] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:22Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:603325715lb: must succeed: curl -sf localhost:9901/clusters5716lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5717worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5718worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5719(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.23 seconds)5720subtest: scheduler restart mid-build: build finishes, no double build5721client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5722client # [ 157.318714] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5723client # Running as unit: mid.service; invocation ID: 7a29c31402e14da9abd816d06467f9605724client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.05 seconds)5725worker2 # [ 157.520178] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:603425726worker2 # [ 157.528629] nix-daemon[842]: accepted connection from pid 1604, user nix-grpc-daemon (trusted)5727client # [ 157.520905] nix[1406]: don't know how to build these paths:5728client # [ 157.522273] nix[1406]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5729worker2 # [ 157.547184] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:23Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05730worker2 # [ 157.551164] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:23Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15731worker2 # [ 157.558822] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603425732worker2 # [ 157.564115] nix-grpc-daemon[1604]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 2 ms (attempt 1/5)5733worker2 # [ 157.569104] nix-grpc-daemon[1604]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 176 ms (attempt 2/5)5734worker2 # [ 157.750525] nix-grpc-daemon[1604]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 380 ms (attempt 3/5)5735worker2 # [ 158.132535] nix-grpc-daemon[1604]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 208 ms (attempt 4/5)5736worker2 # [ 158.343829] nix-grpc-daemon[1604]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5737worker2 # [ 158.348822] nix-grpc-daemon[1604]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5738worker2 # [ 158.360809] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:603425739client # [ 158.349156] nix[1406]: copying 1 paths to 'lb:50051'...5740worker2 # [ 158.370577] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:603425741client # [ 158.351491] nix[1406]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5742worker2 # [ 158.378389] nix-daemon[842]: accepted connection from pid 1604, user nix-grpc-daemon (trusted)5743worker2 # [ 158.399740] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:23Z level=info event=niks3_push_started pid=16735744worker2 # [ 158.551405] nix-grpc-daemon[1673]: time=2026-09-25T16:52:24.145Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5745worker2 # [ 158.692755] nix-daemon[842]: accepted connection from pid 1690, user nix-grpc-daemon (trusted)5746worker2 # [ 158.726364] nix-daemon[842]: reaped child process 1693, status = succeeded5747lb # [ 158.705067] niks3-server[1196]: 2026/09/25 16:52:24 INFO Received push request method=POST path=/api/pushes5748worker2 # [ 158.735384] nix-grpc-daemon[1673]: time=2026-09-25T16:52:24.329Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5749worker2 # [ 158.740074] nix-grpc-daemon[1673]: time=2026-09-25T16:52:24.329Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5750lb # [ 158.737724] niks3-server[1196]: 2026/09/25 16:52:24 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign5751lb # [ 158.748145] niks3-server[1196]: 2026/09/25 16:52:24 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5752lb # [ 158.750764] niks3-server[1196]: 2026/09/25 16:52:24 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5753lb # [ 158.763626] niks3-server[1196]: 2026/09/25 16:52:24 INFO Signed narinfos id=50 count=15754worker2 # [ 158.785585] nix-grpc-daemon[1673]: time=2026-09-25T16:52:24.379Z level=INFO msg="Uploading 1 narinfos"5755lb # [ 158.771670] niks3-server[1196]: 2026/09/25 16:52:24 INFO Received complete push request method=POST path=/api/pushes/50/complete5756lb # [ 158.780242] niks3-server[1196]: 2026/09/25 16:52:24 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5757worker2 # [ 158.815093] nix-grpc-daemon[1673]: time=2026-09-25T16:52:24.410Z level=INFO msg="Upload complete. (260ms)"5758worker2 # [ 158.822117] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60342 duration_s=0 paths=1 nar_bytes_in=4245759worker2 # [ 158.834335] nix-grpc-daemon[1604]: ts=2026-09-25T16:52:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:603425760worker2 # [ 158.839985] nix-daemon[842]: accepted connection from pid 1604, user nix-grpc-daemon (trusted)5761worker2 # [ 158.844819] nix-daemon[842]: reaped child process 1671, status = succeeded5762client # [ 158.834897] nix[1406]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5763worker2 # [ 158.857630] nix-daemon[842]: accepted connection from pid 1604, user nix-grpc-daemon (trusted)5764lb: must succeed: curl -sf localhost:9901/clusters5765lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5766lb: must succeed: curl -sf localhost:9901/clusters5767lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5768worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5769worker2 # [ 160.026998] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1604 (nix-grpc-daemon) on client request.5770worker2 # [ 160.029077] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5771lb # [ 160.010312] niks3-server[1196]: 2026/09/25 16:52:25 INFO lead: released remote=192.168.1.4:339325772client # [ 160.015831] nix[1406]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5773worker1 # [ 160.032118] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:25Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5774lb # [ 160.014451] envoy[886]: [2026-09-25 16:52:25.607][886][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}5775client # [ 160.019387] nix[1406]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5776worker2 # [ 160.046285] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5777worker2 # [ 160.047879] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5778worker2 # [ 160.067731] systemd[1]: nix-grpc-daemon.service: Consumed 438ms CPU time over 3.973s wall clock time, 34.7M memory peak, 148.7K incoming IP traffic, 33.2K outgoing IP traffic.5779worker2 # [ 160.070054] systemd[1]: Starting scheduler-token.service...5780worker2 # [ 160.102562] nix-daemon[842]: reaped child process 1695, status = succeeded5781worker2 # [ 160.106032] nix-daemon[842]: reaped child process 1663, status = succeeded5782worker2 # [ 160.112895] nix-daemon[1697]: unexpected Nix daemon error: error: interrupted by the user5783worker2 # [ 160.126204] nix-daemon[842]: reaped child process 1697, status = failed with exit code 15784worker2 # [ 160.135949] systemd[1]: scheduler-token.service: Deactivated successfully.5785worker2 # [ 160.139030] systemd[1]: Finished scheduler-token.service.5786worker2 # [ 160.139862] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 79ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5787worker2 # [ 160.145333] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5788worker2 # [ 160.146867] systemd[1]: Starting scheduler-token.service...5789worker2 # [ 160.185192] systemd[1]: scheduler-token.service: Deactivated successfully.5790worker2 # [ 160.188079] systemd[1]: Finished scheduler-token.service.5791worker2 # [ 160.188901] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 41ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5792worker2 # [ 160.193256] systemd[1]: Starting Nix worker-protocol over gRPC...5793worker1 # [ 160.232237] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:25Z level=info event=scheduler_connected addr=lb:500515794worker1 # [ 160.283054] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:25Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. retried and the latest reset reason: remote connection failure"5795worker2 # [ 160.290830] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:25Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5796worker2 # [ 160.293220] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:25Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5797worker2 # [ 160.297023] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:25Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15798worker2 # [ 160.302370] nix-grpc-daemon[1736]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5799worker2 # [ 160.303882] nix-grpc-daemon[1736]: E0000 00:00:1790355145.896780 1736 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5800lb # [ 160.293288] niks3-server[1196]: 2026/09/25 16:52:25 INFO lead: acquired remote=192.168.1.4:339865801worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.36 seconds)5802client: must succeed: sleep 5.05803worker2 # [ 160.308139] nix-grpc-daemon[1736]: E0000 00:00:1790355145.902978 1736 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5804worker2 # [ 160.351213] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:25Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5805worker2 # [ 160.353249] nix-grpc-daemon[1736]: E0000 00:00:1790355145.909085 1779 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5806worker2 # [ 160.358770] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:25Z level=info event=scheduler_take_over5807worker2 # [ 160.363985] nix-grpc-daemon[1736]: E0000 00:00:1790355145.926995 1779 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5808worker2 # [ 160.367516] nix-grpc-daemon[1736]: E0000 00:00:1790355145.927141 1779 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5809worker2 # [ 160.371239] nix-grpc-daemon[1736]: E0000 00:00:1790355145.942779 1779 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5810worker2 # [ 160.374390] systemd[1]: Started Nix worker-protocol over gRPC.5811worker2 # [ 160.376986] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:25Z level=info event=scheduler_connected addr=lb:500515812worker2 # [ 160.378345] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:25Z level=info event=oidc_jwks provider=mock keys=15813worker2 # [ 160.379616] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:25Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:603905814worker2 # [ 160.381456] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:25Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05815worker1 # [ 160.483601] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:26Z level=info event=scheduler_connected addr=lb:500515816worker2 # [ 160.488449] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:26Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:603905817worker2 # [ 160.491472] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05818worker2 # [ 160.541918] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:26Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05819worker2 # [ 160.548164] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:26Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15820worker2 # [ 160.554161] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604005821worker2 # [ 160.561607] nix-daemon[842]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)5822worker2 # [ 160.574144] nix-daemon[842]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)5823client # [ 160.563026] nix[1406]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5824worker2 # [ 160.586284] nix-daemon[842]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)5825worker2 # [ 162.194678] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:27Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:423245826client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5827client: waiting for success: ! systemctl is-active mid5828worker2 # [ 165.607666] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:31Z level=info event=niks3_push_started pid=18195829worker2 # [ 165.674254] nix-daemon[842]: reaped child process 1794, status = succeeded5830worker2 # [ 165.742272] nix-grpc-daemon[1819]: time=2026-09-25T16:52:31.336Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5831worker2 # [ 165.852927] nix-daemon[842]: accepted connection from pid 1831, user nix-grpc-daemon (trusted)5832lb # [ 165.852181] niks3-server[1196]: 2026/09/25 16:52:31 INFO Received push request method=POST path=/api/pushes5833worker2 # [ 165.875198] nix-daemon[842]: reaped child process 1834, status = succeeded5834worker2 # [ 165.880111] nix-grpc-daemon[1819]: time=2026-09-25T16:52:31.474Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5835worker2 # [ 165.881759] nix-grpc-daemon[1819]: time=2026-09-25T16:52:31.474Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5836lb # [ 165.892481] niks3-server[1196]: 2026/09/25 16:52:31 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign5837lb # [ 165.903637] niks3-server[1196]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5838lb # [ 165.911637] niks3-server[1196]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5839lb # [ 165.923607] niks3-server[1196]: 2026/09/25 16:52:31 INFO Signed narinfos id=51 count=15840worker2 # [ 165.944956] nix-grpc-daemon[1819]: time=2026-09-25T16:52:31.538Z level=INFO msg="Uploading 1 narinfos"5841lb # [ 165.928181] niks3-server[1196]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5842lb # [ 165.938274] niks3-server[1196]: 2026/09/25 16:52:31 INFO Received complete push request method=POST path=/api/pushes/51/complete5843lb # [ 165.953822] niks3-server[1196]: 2026/09/25 16:52:31 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5844worker2 # [ 165.970912] nix-grpc-daemon[1819]: time=2026-09-25T16:52:31.566Z level=INFO msg="Upload complete. (224ms)"5845worker2 # [ 165.984162] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60400 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05846worker2 # [ 165.994137] nix-daemon[842]: reaped child process 1792, status = succeeded5847client # [ 165.984158] systemd[1]: mid.service: Deactivated successfully.5848client # [ 165.986355] systemd[1]: mid.service: Consumed 173ms CPU time over 8.667s wall clock time, 41.5M memory peak, 8.6K incoming IP traffic, 10.4K outgoing IP traffic.5849client: (finished: waiting for success: ! systemctl is-active mid, in 1.06 seconds)5850client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5851client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5852(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.38 seconds)5853subtest: interrupting the client stops the build on the worker5854client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr5855client # [ 166.693663] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.5856client # Running as unit: intr.service; invocation ID: 8918c6077f294f7f8370d84a1c520c9a5857client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.04 seconds)5858worker1 # [ 166.869724] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334125859client # [ 166.858390] nix[1442]: don't know how to build these paths:5860client # [ 166.859471] nix[1442]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5861worker2 # [ 166.888918] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:32Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05862worker2 # [ 166.892393] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:32Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15863worker1 # [ 166.891005] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334125864worker1 # [ 166.899673] nix-grpc-daemon[1404]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 78 ms (attempt 1/5)5865worker1 # [ 166.982877] nix-grpc-daemon[1404]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 184 ms (attempt 2/5)5866lb # [ 167.047845] envoy[886]: conn peer=[2001:db8:1::4]:36686 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1101 tx=239 ms=37845867lb # [ 167.050593] envoy[886]: conn peer=[2001:db8:1::1]:42844 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3927 tx=1964 ms=84745868worker1 # [ 167.167987] nix-grpc-daemon[1404]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 3/5)5869worker1 # [ 167.262831] nix-grpc-daemon[1404]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 4/5)5870worker1 # [ 167.320417] nix-grpc-daemon[1404]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5871worker1 # [ 167.322411] nix-grpc-daemon[1404]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5872worker1 # [ 167.333269] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334125873client # [ 167.323679] nix[1442]: copying 1 paths to 'lb:50051'...5874worker1 # [ 167.343244] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334125875client # [ 167.327349] nix[1442]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5876worker1 # [ 167.352417] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)5877worker1 # [ 167.371673] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:32Z level=info event=niks3_push_started pid=15485878worker1 # [ 167.491457] nix-grpc-daemon[1548]: time=2026-09-25T16:52:33.082Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5879worker1 # [ 167.591956] nix-daemon[856]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)5880worker1 # [ 167.608433] nix-daemon[856]: reaped child process 1563, status = succeeded5881lb # [ 167.597388] niks3-server[1196]: 2026/09/25 16:52:33 INFO Received push request method=POST path=/api/pushes5882worker1 # [ 167.622556] nix-grpc-daemon[1548]: time=2026-09-25T16:52:33.212Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5883worker1 # [ 167.626910] nix-grpc-daemon[1548]: time=2026-09-25T16:52:33.212Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5884lb # [ 167.626685] niks3-server[1196]: 2026/09/25 16:52:33 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5885worker1 # [ 167.649460] nix-grpc-daemon[1548]: time=2026-09-25T16:52:33.239Z level=INFO msg="Uploading 1 narinfos"5886lb # [ 167.633569] niks3-server[1196]: 2026/09/25 16:52:33 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign5887lb # [ 167.635882] niks3-server[1196]: 2026/09/25 16:52:33 INFO Signed narinfos id=52 count=15888lb # [ 167.643367] niks3-server[1196]: 2026/09/25 16:52:33 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5889lb # [ 167.654523] niks3-server[1196]: 2026/09/25 16:52:33 INFO Received complete push request method=POST path=/api/pushes/52/complete5890worker1 # [ 167.677179] nix-grpc-daemon[1548]: time=2026-09-25T16:52:33.268Z level=INFO msg="Upload complete. (182ms)"5891lb # [ 167.660700] niks3-server[1196]: 2026/09/25 16:52:33 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5892worker1 # [ 167.683486] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=4245893worker1 # [ 167.692789] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334125894worker1 # [ 167.697556] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)5895worker1 # [ 167.701640] nix-daemon[856]: reaped child process 1546, status = succeeded5896client # [ 167.692726] nix[1442]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5897worker1 # [ 167.712074] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)5898client: must succeed: systemctl kill -s INT intr5899client # [ 169.018294] systemd[1]: intr.service: Sent signal SIGINT to main process 1442 (nix) on client request.5900worker1 # [ 169.037684] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:34Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5901client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)5902client # [ 169.023668] nix[1442]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5903client # [ 169.024599] nix[1442]: error: interrupted by the user5904client # [ 169.043766] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5905client # [ 169.052191] systemd[1]: intr.service: Failed with result 'exit-code'.5906client # [ 169.054185] systemd[1]: intr.service: Consumed 151ms CPU time over 2.356s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.5907worker1 # [ 169.131730] nix-daemon[1567]: unexpected Nix daemon error: error: interrupted by the user5908worker1 # [ 169.138942] nix-daemon[856]: reaped child process 1565, status = succeeded5909worker1 # [ 169.143794] nix-daemon[856]: reaped child process 1567, status = failed with exit code 15910(finished: subtest: interrupting the client stops the build on the worker, in 3.55 seconds)5911subtest: a deploy mid-build drains: the build finishes, the new generation takes over5912client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw5913client # [ 170.252518] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.5914client # Running as unit: sw.service; invocation ID: ce106fc96f784ed390ce8a6b8d04061e5915client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.05 seconds)5916worker2 # [ 170.441748] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:604005917client # [ 170.427394] nix[1462]: don't know how to build these paths:5918client # [ 170.428560] nix[1462]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5919worker2 # [ 170.452622] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:36Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05920worker2 # [ 170.456775] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:36Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15921worker2 # [ 170.464286] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604005922worker2 # [ 170.470161] nix-grpc-daemon[1736]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 3 ms (attempt 1/5)5923worker2 # [ 170.478333] nix-grpc-daemon[1736]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 134 ms (attempt 2/5)5924worker2 # [ 170.613713] nix-grpc-daemon[1736]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 3/5)5925worker2 # [ 170.638960] nix-grpc-daemon[1736]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 768 ms (attempt 4/5)5926worker2 # [ 171.412611] nix-grpc-daemon[1736]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5927worker2 # [ 171.415067] nix-grpc-daemon[1736]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5928worker2 # [ 171.429066] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:604005929client # [ 171.418456] nix[1462]: copying 1 paths to 'lb:50051'...5930worker2 # [ 171.437948] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:604005931client # [ 171.418706] nix[1462]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5932worker2 # [ 171.451719] nix-daemon[842]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)5933worker2 # [ 171.601102] nix-daemon[842]: accepted connection from pid 1874, user nix-grpc-daemon (trusted)5934lb # [ 171.598798] niks3-server[1196]: 2026/09/25 16:52:37 INFO Received push request method=POST path=/api/pushes5935worker2 # [ 171.622878] nix-daemon[842]: reaped child process 1877, status = succeeded5936worker2 # [ 171.626604] nix-grpc-daemon[1819]: time=2026-09-25T16:52:37.220Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5937worker2 # [ 171.628272] nix-grpc-daemon[1819]: time=2026-09-25T16:52:37.220Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5938lb # [ 171.618712] niks3-server[1196]: 2026/09/25 16:52:37 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5939worker2 # [ 171.646550] nix-grpc-daemon[1819]: time=2026-09-25T16:52:37.241Z level=INFO msg="Uploading 1 narinfos"5940lb # [ 171.627387] niks3-server[1196]: 2026/09/25 16:52:37 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign5941lb # [ 171.633148] niks3-server[1196]: 2026/09/25 16:52:37 INFO Signed narinfos id=53 count=15942lb # [ 171.634880] niks3-server[1196]: 2026/09/25 16:52:37 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5943lb # [ 171.645527] niks3-server[1196]: 2026/09/25 16:52:37 INFO Received complete push request method=POST path=/api/pushes/53/complete5944worker2 # [ 171.669932] nix-grpc-daemon[1819]: time=2026-09-25T16:52:37.265Z level=INFO msg="Upload complete. (196ms)"5945lb # [ 171.650354] niks3-server[1196]: 2026/09/25 16:52:37 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5946worker2 # [ 171.676611] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60400 duration_s=0 paths=1 nar_bytes_in=4165947worker2 # [ 171.679946] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:604005948worker2 # [ 171.691799] nix-daemon[842]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)5949worker2 # [ 171.695877] nix-daemon[842]: reaped child process 1871, status = succeeded5950client # [ 171.684366] nix[1462]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5951worker2 # [ 171.704737] nix-daemon[842]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)5952worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25953worker2 # Checking switch inhibitors... done5954worker2 # activating the configuration...5955worker2 # [ 173.172376] nixos[1909]: switching to system configuration /nix/store/a6ydxzg4iqvbdapmq69lgr1p3bfizbid-nixos-system-worker2-test5956worker2 # [ 173.174271] systemd[1]: Stopped target Local File Systems.5957worker2 # [ 173.175097] systemd[1]: Stopped target Remote File Systems.5958worker2 # setting up /etc...5959worker2 # [ 173.713345] systemd[1]: Reload requested from client PID 1909 ('.switch-to-conf') (unit backdoor.service)...5960worker2 # [ 173.720121] systemd[1]: Reloading...5961lb # [ 177.050670] envoy[886]: conn peer=[2001:db8:1::1]:43932 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3013 tx=799 ms=21815962worker2 # [ 178.327606] systemd[1]: Reloading finished in 4608 ms.5963worker2 # restarting sysinit-reactivation.target5964worker2 # [ 178.455463] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5965worker2 # [ 178.606429] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5966worker2 # [ 178.607697] systemd[1]: Reached target Reactivate sysinit units.5967worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5968worker2 # [ 178.614874] systemd[1]: Reloading D-Bus System Message Bus...5969worker2 # [ 178.617573] dbus-broker-launch[484]: Caught SIGHUP, trigger reload.5970worker2 # [ 178.628593] systemd[1]: Reloading Nix worker-protocol over gRPC...5971worker2 # [ 178.640629] dbus-broker-launch[484]: Looking up NSS user entry for 'systemd-timesync'...5972worker2 # [ 178.641851] dbus-broker-launch[484]: NSS returned no entry for 'systemd-timesync'5973worker2 # [ 178.642885] dbus-broker-launch[484]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5974worker2 # [ 178.672341] systemd[1]: Reloaded Nix worker-protocol over gRPC.5975worker2 # [ 178.685469] systemd[1]: Reloaded D-Bus System Message Bus.5976worker2 # [ 178.692361] systemd[1]: Reached target Local File Systems.5977worker2 # [ 178.700271] systemd[1]: Reached target Remote File Systems.5978worker2 # [ 178.709824] systemd[1]: Starting linger-users.service...5979worker2 # [ 178.710731] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5980worker2 # [ 178.717732] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5981worker2 # [ 178.719123] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5982worker2 # [ 178.735536] systemd[1]: Starting Create SUID/SGID Wrappers...5983worker2 # [ 178.740143] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5984worker2 # [ 178.741451] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675985worker2 # [ 178.753631] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5986worker2 # [ 178.755358] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5987worker2 # [ 178.765516] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5988worker2 # [ 178.794721] systemd[1]: linger-users.service: Deactivated successfully.5989worker2 # [ 178.795783] systemd[1]: Finished linger-users.service.5990worker2 # [ 179.129719] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5991worker2 # [ 179.134668] systemd[1]: Finished Create SUID/SGID Wrappers.5992worker2 # [ 179.316166] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:44Z level=info event=draining5993worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5994worker2 # [ 179.394251] nixos[1909]: finished switching to system configuration /nix/store/a6ydxzg4iqvbdapmq69lgr1p3bfizbid-nixos-system-worker2-test5995worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.64 seconds)5996worker2: must succeed: pgrep -f 'read -t [9]0 x'5997worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5998client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&25999client # don't know how to build these paths:6000client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6001worker1 # [ 179.635329] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334126002worker1 # [ 179.640768] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126003worker2 # [ 179.659990] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:45Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06004worker2 # [ 179.662355] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:45Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=16005worker1 # [ 179.658322] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126006worker1 # [ 179.667231] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126007client # copying 1 paths to 'lb:50051'...6008worker1 # [ 179.676298] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334126009client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6010worker1 # [ 179.693273] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6011worker1 # [ 179.816121] nix-daemon[856]: accepted connection from pid 1618, user nix-grpc-daemon (trusted)6012lb # [ 179.817509] niks3-server[1196]: 2026/09/25 16:52:45 INFO Received push request method=POST path=/api/pushes6013worker1 # [ 179.836827] nix-daemon[856]: reaped child process 1621, status = succeeded6014worker1 # [ 179.841720] nix-grpc-daemon[1548]: time=2026-09-25T16:52:45.432Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6015worker1 # [ 179.843339] nix-grpc-daemon[1548]: time=2026-09-25T16:52:45.432Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6016lb # [ 179.835879] niks3-server[1196]: 2026/09/25 16:52:45 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign6017worker1 # [ 179.858488] nix-grpc-daemon[1548]: time=2026-09-25T16:52:45.449Z level=INFO msg="Uploading 1 narinfos"6018lb # [ 179.844136] niks3-server[1196]: 2026/09/25 16:52:45 INFO Signed narinfos id=54 count=16019lb # [ 179.848161] niks3-server[1196]: 2026/09/25 16:52:45 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6020lb # [ 179.857449] niks3-server[1196]: 2026/09/25 16:52:45 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6021worker1 # [ 179.881786] nix-grpc-daemon[1548]: time=2026-09-25T16:52:45.473Z level=INFO msg="Upload complete. (178ms)"6022lb # [ 179.864730] niks3-server[1196]: 2026/09/25 16:52:45 INFO Received complete push request method=POST path=/api/pushes/54/complete6023lb # [ 179.869515] niks3-server[1196]: 2026/09/25 16:52:45 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6024worker1 # [ 179.888164] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=5286025worker1 # [ 179.898824] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126026worker1 # [ 179.904483] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6027worker1 # [ 179.908522] nix-daemon[856]: reaped child process 1616, status = succeeded6028client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6029client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6030worker1 # [ 179.919371] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6031client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6032client # farm-a> LOG-farm-a-during-drain6033worker1 # [ 181.194975] nix-daemon[856]: reaped child process 1625, status = succeeded6034worker1 # [ 181.261913] nix-daemon[856]: accepted connection from pid 1641, user nix-grpc-daemon (trusted)6035worker1 # [ 181.281567] nix-daemon[856]: reaped child process 1644, status = succeeded6036lb # [ 181.267666] niks3-server[1196]: 2026/09/25 16:52:46 INFO Received push request method=POST path=/api/pushes6037worker1 # [ 181.292599] nix-grpc-daemon[1548]: time=2026-09-25T16:52:46.882Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6038worker1 # [ 181.294374] nix-grpc-daemon[1548]: time=2026-09-25T16:52:46.882Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6039lb # [ 181.296069] niks3-server[1196]: 2026/09/25 16:52:46 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6040worker1 # [ 181.320201] nix-grpc-daemon[1548]: time=2026-09-25T16:52:46.911Z level=INFO msg="Uploading 1 narinfos"6041lb # [ 181.302777] niks3-server[1196]: 2026/09/25 16:52:46 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign6042lb # [ 181.308481] niks3-server[1196]: 2026/09/25 16:52:46 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6043lb # [ 181.311164] niks3-server[1196]: 2026/09/25 16:52:46 INFO Signed narinfos id=55 count=16044lb # [ 181.321150] niks3-server[1196]: 2026/09/25 16:52:46 INFO Received complete push request method=POST path=/api/pushes/55/complete6045lb # [ 181.323825] niks3-server[1196]: 2026/09/25 16:52:46 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6046worker1 # [ 181.350600] nix-grpc-daemon[1548]: time=2026-09-25T16:52:46.941Z level=INFO msg="Upload complete. (223ms)"6047lb # [ 181.334600] niks3-server[1196]: 2026/09/25 16:52:46 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6048worker2 # [ 181.363675] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:46Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06049worker1 # [ 181.357967] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06050worker2 # [ 181.365759] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:46Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=16051worker1 # [ 181.371396] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126052worker1 # [ 181.375845] nix-daemon[856]: reaped child process 1623, status = succeeded6053worker1 # [ 181.381158] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126054worker1 # [ 181.389568] nix-grpc-daemon[1548]: time=2026-09-25T16:52:46.980Z level=INFO msg="All 1 paths already cached"6055client # copying 1 paths to 'lb:50051'...6056worker1 # [ 181.395741] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334126057client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6058worker1 # [ 181.408621] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6059worker1 # [ 181.526836] nix-daemon[856]: accepted connection from pid 1648, user nix-grpc-daemon (trusted)6060worker1 # [ 181.545206] nix-daemon[856]: reaped child process 1651, status = succeeded6061lb # [ 181.533641] niks3-server[1196]: 2026/09/25 16:52:47 INFO Received push request method=POST path=/api/pushes6062worker1 # [ 181.555798] nix-grpc-daemon[1548]: time=2026-09-25T16:52:47.146Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6063worker1 # [ 181.557649] nix-grpc-daemon[1548]: time=2026-09-25T16:52:47.146Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6064lb # [ 181.552633] niks3-server[1196]: 2026/09/25 16:52:47 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign6065lb # [ 181.555059] niks3-server[1196]: 2026/09/25 16:52:47 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6066worker1 # [ 181.576618] nix-grpc-daemon[1548]: time=2026-09-25T16:52:47.167Z level=INFO msg="Uploading 1 narinfos"6067lb # [ 181.568302] niks3-server[1196]: 2026/09/25 16:52:47 INFO Signed narinfos id=56 count=16068worker1 # [ 181.588696] nix-grpc-daemon[1548]: time=2026-09-25T16:52:47.180Z level=INFO msg="Upload complete. (170ms)"6069lb # [ 181.572540] niks3-server[1196]: 2026/09/25 16:52:47 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6070lb # [ 181.578228] niks3-server[1196]: 2026/09/25 16:52:47 INFO Received complete push request method=POST path=/api/pushes/56/complete6071worker1 # [ 181.594622] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=6406072lb # [ 181.581043] niks3-server[1196]: 2026/09/25 16:52:47 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6073worker1 # [ 181.605635] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126074worker1 # [ 181.609802] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6075worker1 # [ 181.613651] nix-daemon[856]: reaped child process 1646, status = succeeded6076client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6077client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6078worker1 # [ 181.623312] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6079client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6080client # farm-b> LOG-farm-b-during-drain6081worker1 # [ 182.904733] nix-daemon[856]: reaped child process 1655, status = succeeded6082worker1 # [ 182.962088] nix-daemon[856]: accepted connection from pid 1671, user nix-grpc-daemon (trusted)6083lb # [ 182.961675] niks3-server[1196]: 2026/09/25 16:52:48 INFO Received push request method=POST path=/api/pushes6084worker1 # [ 182.981469] nix-daemon[856]: reaped child process 1674, status = succeeded6085worker1 # [ 182.985400] nix-grpc-daemon[1548]: time=2026-09-25T16:52:48.575Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6086worker1 # [ 182.986915] nix-grpc-daemon[1548]: time=2026-09-25T16:52:48.575Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6087lb # [ 182.983971] niks3-server[1196]: 2026/09/25 16:52:48 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign6088worker1 # [ 183.006477] nix-grpc-daemon[1548]: time=2026-09-25T16:52:48.597Z level=INFO msg="Uploading 1 narinfos"6089lb # [ 182.992669] niks3-server[1196]: 2026/09/25 16:52:48 INFO Signed narinfos id=57 count=16090lb # [ 182.998128] niks3-server[1196]: 2026/09/25 16:52:48 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6091lb # [ 183.004533] niks3-server[1196]: 2026/09/25 16:52:48 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6092lb # [ 183.014191] niks3-server[1196]: 2026/09/25 16:52:48 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6093worker1 # [ 183.037107] nix-grpc-daemon[1548]: time=2026-09-25T16:52:48.628Z level=INFO msg="Upload complete. (206ms)"6094lb # [ 183.020593] niks3-server[1196]: 2026/09/25 16:52:48 INFO Received complete push request method=POST path=/api/pushes/57/complete6095lb # [ 183.024213] niks3-server[1196]: 2026/09/25 16:52:48 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6096worker1 # [ 183.044555] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06097worker2 # [ 183.053510] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:48Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06098worker2 # [ 183.056160] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:48Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=16099worker1 # [ 183.057704] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126100worker1 # [ 183.061821] nix-daemon[856]: reaped child process 1653, status = succeeded6101worker1 # [ 183.067207] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126102worker1 # [ 183.075502] nix-grpc-daemon[1548]: time=2026-09-25T16:52:48.666Z level=INFO msg="All 2 paths already cached"6103client # copying 1 paths to 'lb:50051'...6104worker1 # [ 183.082228] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334126105client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6106worker1 # [ 183.094961] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6107worker1 # [ 183.215456] nix-daemon[856]: accepted connection from pid 1678, user nix-grpc-daemon (trusted)6108worker1 # [ 183.233736] nix-daemon[856]: reaped child process 1681, status = succeeded6109lb # [ 183.219733] niks3-server[1196]: 2026/09/25 16:52:48 INFO Received push request method=POST path=/api/pushes6110worker1 # [ 183.242556] nix-grpc-daemon[1548]: time=2026-09-25T16:52:48.833Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6111worker1 # [ 183.244100] nix-grpc-daemon[1548]: time=2026-09-25T16:52:48.833Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6112lb # [ 183.236173] niks3-server[1196]: 2026/09/25 16:52:48 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign6113worker1 # [ 183.256361] nix-grpc-daemon[1548]: time=2026-09-25T16:52:48.847Z level=INFO msg="Uploading 1 narinfos"6114lb # [ 183.242908] niks3-server[1196]: 2026/09/25 16:52:48 INFO Signed narinfos id=58 count=16115lb # [ 183.246950] niks3-server[1196]: 2026/09/25 16:52:48 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6116lb # [ 183.257130] niks3-server[1196]: 2026/09/25 16:52:48 INFO Received complete push request method=POST path=/api/pushes/58/complete6117worker1 # [ 183.277651] nix-grpc-daemon[1548]: time=2026-09-25T16:52:48.868Z level=INFO msg="Upload complete. (173ms)"6118lb # [ 183.262529] niks3-server[1196]: 2026/09/25 16:52:48 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6119worker1 # [ 183.283530] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=7686120lb # [ 183.269574] niks3-server[1196]: 2026/09/25 16:52:48 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6121worker1 # [ 183.293223] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126122worker1 # [ 183.297132] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6123worker1 # [ 183.300793] nix-daemon[856]: reaped child process 1676, status = succeeded6124client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6125client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6126worker1 # [ 183.310518] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6127client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6128client # farm-top> LOG-farm-top-during-drain6129worker1 # [ 184.544090] nix-daemon[856]: reaped child process 1685, status = succeeded6130worker1 # [ 184.649286] nix-daemon[856]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6131worker1 # [ 184.670896] nix-daemon[856]: reaped child process 1704, status = succeeded6132lb # [ 184.655080] niks3-server[1196]: 2026/09/25 16:52:50 INFO Received push request method=POST path=/api/pushes6133worker1 # [ 184.679869] nix-grpc-daemon[1548]: time=2026-09-25T16:52:50.269Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6134worker1 # [ 184.681933] nix-grpc-daemon[1548]: time=2026-09-25T16:52:50.269Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6135lb # [ 184.683277] niks3-server[1196]: 2026/09/25 16:52:50 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6136worker1 # [ 184.706784] nix-grpc-daemon[1548]: time=2026-09-25T16:52:50.297Z level=INFO msg="Uploading 1 narinfos"6137lb # [ 184.690916] niks3-server[1196]: 2026/09/25 16:52:50 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign6138lb # [ 184.696591] niks3-server[1196]: 2026/09/25 16:52:50 INFO Signed narinfos id=59 count=16139lb # [ 184.698319] niks3-server[1196]: 2026/09/25 16:52:50 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6140lb # [ 184.709053] niks3-server[1196]: 2026/09/25 16:52:50 INFO Received complete push request method=POST path=/api/pushes/59/complete6141worker1 # [ 184.730240] nix-grpc-daemon[1548]: time=2026-09-25T16:52:50.321Z level=INFO msg="Upload complete. (212ms)"6142lb # [ 184.713167] niks3-server[1196]: 2026/09/25 16:52:50 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6143lb # [ 184.717943] niks3-server[1196]: 2026/09/25 16:52:50 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6144worker1 # [ 184.736630] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06145worker1 # [ 184.749485] nix-daemon[856]: reaped child process 1683, status = succeeded6146client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.31 seconds)6147client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6148client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.14 seconds)6149worker2: must succeed: pkill -f 'read -t [9]0 x'6150worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)6151client: waiting for success: ! systemctl is-active sw6152worker2 # [ 185.196866] nix-daemon[842]: reaped child process 1881, status = succeeded6153worker2 # [ 185.276975] nix-daemon[842]: accepted connection from pid 2055, user nix-grpc-daemon (trusted)6154lb # [ 185.276320] niks3-server[1196]: 2026/09/25 16:52:50 INFO Received push request method=POST path=/api/pushes6155worker2 # [ 185.299355] nix-daemon[842]: reaped child process 2058, status = succeeded6156worker2 # [ 185.303302] nix-grpc-daemon[1819]: time=2026-09-25T16:52:50.897Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6157worker2 # [ 185.305155] nix-grpc-daemon[1819]: time=2026-09-25T16:52:50.897Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6158lb # [ 185.297606] niks3-server[1196]: 2026/09/25 16:52:50 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign6159worker2 # [ 185.323063] nix-grpc-daemon[1819]: time=2026-09-25T16:52:50.918Z level=INFO msg="Uploading 1 narinfos"6160lb # [ 185.305269] niks3-server[1196]: 2026/09/25 16:52:50 INFO Signed narinfos id=60 count=16161lb # [ 185.309391] niks3-server[1196]: 2026/09/25 16:52:50 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6162lb # [ 185.316319] niks3-server[1196]: 2026/09/25 16:52:50 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6163lb # [ 185.319390] niks3-server[1196]: 2026/09/25 16:52:50 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6164lb # [ 185.329051] niks3-server[1196]: 2026/09/25 16:52:50 INFO Received complete push request method=POST path=/api/pushes/60/complete6165worker2 # [ 185.354007] nix-grpc-daemon[1819]: time=2026-09-25T16:52:50.949Z level=INFO msg="Upload complete. (222ms)"6166lb # [ 185.334734] niks3-server[1196]: 2026/09/25 16:52:50 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6167worker2 # [ 185.364798] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60400 duration_s=13 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06168worker2 # [ 185.375930] nix-daemon[842]: reaped child process 1879, status = succeeded6169client # [ 185.364182] systemd[1]: sw.service: Deactivated successfully.6170client # [ 185.367155] systemd[1]: sw.service: Consumed 165ms CPU time over 15.112s wall clock time, 39.4M memory peak, 7.5K incoming IP traffic, 8.5K outgoing IP traffic.6171client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6172client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6173client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6174worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6175worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.04 seconds)6176lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux6177lb: must succeed: curl -sf localhost:9901/clusters6178lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6179worker2 # [ 186.317848] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:51Z level=info event=signal_exit6180lb # [ 187.048989] envoy[886]: conn peer=[2001:db8:1::1]:47136 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9617 tx=4380 ms=51176181lb # [ 187.051658] envoy[886]: conn peer=[2001:db8:1::1]:43934 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3008 tx=1560 ms=149306182lb: must succeed: curl -sf localhost:9901/clusters6183lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6184lb # [ 187.297860] niks3-server[1196]: 2026/09/25 16:52:52 INFO lead: released remote=192.168.1.4:339866185lb: must succeed: curl -sf localhost:9901/clusters6186lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6187worker2 # [ 188.318246] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:53Z level=info event=scheduler_restarting6188worker1 # [ 188.316647] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:53Z level=info event=scheduler_restarting addr=lb:500516189lb # [ 188.297593] envoy[886]: [2026-09-25 16:52:53.891][886][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}6190worker2 # [ 188.322189] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:53Z level=info event=scheduler_restarting addr=lb:500516191worker2 # [ 188.325018] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:603906192worker2 # [ 188.332379] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:53Z level=info event=worker_gone addr=192.168.1.3:500516193worker2 # [ 188.334722] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:53Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:603906194worker2 # [ 188.337518] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:53Z level=info event=worker_gone addr=192.168.1.4:500516195worker2 # [ 188.356716] nix-daemon[842]: reaped child process 1790, status = succeeded6196worker1 # [ 188.516421] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:54Z level=info event=scheduler_connected addr=lb:500516197worker1 # [ 188.519359] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6198worker2 # [ 188.535540] nix-grpc-daemon[1736]: ts=2026-09-25T16:52:54Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6199worker1 # [ 188.719869] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:54Z level=info event=scheduler_connected addr=lb:500516200worker1 # [ 188.722923] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6201lb # [ 188.823659] niks3-server[1196]: 2026/09/25 16:52:54 INFO lead: acquired remote=192.168.1.3:440686202worker1 # [ 188.843264] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:54Z level=info event=scheduler_take_over6203worker1 # [ 188.923476] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:54Z level=info event=scheduler_connected addr=lb:500516204worker1 # [ 188.926467] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6205worker1 # [ 189.127000] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:54Z level=info event=scheduler_connected addr=lb:500516206worker1 # [ 189.129770] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6207lb: must succeed: curl -sf localhost:9901/clusters6208lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6209worker1 # [ 189.334549] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:54Z level=info event=scheduler_connected addr=lb:500516210worker1 # [ 189.339283] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:54Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:570086211worker1 # [ 189.342048] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:54Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06212worker1 # [ 189.450182] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:55Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:570126213lb: must succeed: curl -sf localhost:9901/clusters6214lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6215worker2 # [ 190.343357] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6216worker2 # [ 190.346497] systemd[1]: nix-grpc-daemon.service: Consumed 751ms CPU time over 30.299s wall clock time, 38.2M memory peak, 234.1K incoming IP traffic, 68.7K outgoing IP traffic.6217worker2 # [ 190.671710] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.6218worker2 # [ 190.679809] systemd[1]: Starting scheduler-token.service...6219worker2 # [ 190.739427] systemd[1]: scheduler-token.service: Deactivated successfully.6220worker2 # [ 190.744222] systemd[1]: Finished scheduler-token.service.6221worker2 # [ 190.745630] systemd[1]: scheduler-token.service: Consumed 37ms CPU time over 61ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6222worker2 # [ 190.751985] systemd[1]: Starting Nix worker-protocol over gRPC...6223worker2 # [ 190.866476] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:56Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6224worker2 # [ 190.874693] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:56Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6225worker2 # [ 190.890423] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:56Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16226worker2 # [ 190.916123] nix-grpc-daemon[2074]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6227worker2 # [ 190.917660] nix-grpc-daemon[2074]: E0000 00:00:1790355176.466989 2074 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6228worker2 # [ 190.922312] nix-grpc-daemon[2074]: E0000 00:00:1790355176.469580 2074 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6229worker2 # [ 190.928082] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:56Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6230worker2 # [ 190.931848] nix-grpc-daemon[2074]: E0000 00:00:1790355176.478352 2117 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6231worker2 # [ 190.935078] nix-grpc-daemon[2074]: E0000 00:00:1790355176.496596 2117 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6232worker1 # [ 190.933051] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:56Z level=info event=oidc_jwks provider=mock keys=16233worker2 # [ 190.937622] nix-grpc-daemon[2074]: E0000 00:00:1790355176.496737 2117 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6234worker2 # [ 190.940137] nix-grpc-daemon[2074]: E0000 00:00:1790355176.515918 2117 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6235worker1 # [ 190.936603] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:56Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:570086236worker2 # [ 190.942910] systemd[1]: Started Nix worker-protocol over gRPC.6237worker1 # [ 190.941189] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:56Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06238worker2 # [ 190.947505] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:56Z level=info event=scheduler_connected addr=lb:500516239lb: must succeed: curl -sf localhost:9901/clusters6240lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6241lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 5.23 seconds)6242lb: must succeed: curl -sf localhost:9901/clusters6243lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6244worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6245worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6246(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 21.17 seconds)6247subtest: queue gauges show a burst while every build blocks6248lb: must succeed: curl -sf localhost:9901/clusters6249lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6250lb: must succeed: curl -sf localhost:9901/clusters6251lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6252worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6253worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6254client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6255client # [ 191.525215] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6256client # Running as unit: burst.service; invocation ID: 2a5ef62b41594961b59718c4f7e3a44e6257client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.06 seconds)6258lb: must succeed: curl -sf localhost:9901/clusters6259lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6260lb: must succeed: curl -sf localhost:9901/clusters6261lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6262worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6263worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6264worker2 # [ 191.730857] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355386265worker2 # [ 191.740107] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6266client # [ 191.731253] nix[1518]: don't know how to build these paths:6267client # [ 191.733976] nix[1518]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6268client # [ 191.735489] nix[1518]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6269client # [ 191.737010] nix[1518]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6270client # [ 191.738341] nix[1518]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6271worker1 # [ 191.755879] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:57Z level=debug event=want client=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06272worker2 # [ 191.761801] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355386273worker1 # [ 191.760576] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:57Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.3:50051 clients=16274worker1 # [ 191.763710] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:57Z level=debug event=want client=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06275worker1 # [ 191.767405] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:57Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.4:50051 clients=16276worker2 # [ 191.770812] nix-grpc-daemon[2074]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 1/5)6277worker1 # [ 191.772132] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:57Z level=debug event=want client=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06278worker1 # [ 191.776149] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:57Z level=debug event=want client=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06279worker1 # [ 191.779815] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126280client # [ 191.765866] nix[1518]: copying 1 paths to 'lb:50051'...6281client # [ 191.767203] nix[1518]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6282worker1 # [ 191.783909] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126283worker1 # [ 191.791474] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334126284worker1 # [ 191.798925] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6285worker2 # [ 191.823154] nix-grpc-daemon[2074]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 2/5)6286worker2 # [ 191.851376] nix-grpc-daemon[2074]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 3/5)6287worker2 # [ 191.912166] nix-grpc-daemon[2074]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 378 ms (attempt 4/5)6288worker1 # [ 191.916622] nix-daemon[856]: accepted connection from pid 1719, user nix-grpc-daemon (trusted)6289worker1 # [ 191.938501] nix-daemon[856]: reaped child process 1722, status = succeeded6290lb # [ 191.922584] niks3-server[1196]: 2026/09/25 16:52:57 INFO Received push request method=POST path=/api/pushes6291worker1 # [ 191.945989] nix-grpc-daemon[1548]: time=2026-09-25T16:52:57.536Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6292worker1 # [ 191.947479] nix-grpc-daemon[1548]: time=2026-09-25T16:52:57.536Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6293worker1 # [ 191.963971] nix-grpc-daemon[1548]: time=2026-09-25T16:52:57.555Z level=INFO msg="Uploading 1 narinfos"6294lb # [ 191.942029] niks3-server[1196]: 2026/09/25 16:52:57 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign6295lb # [ 191.950648] niks3-server[1196]: 2026/09/25 16:52:57 INFO Signed narinfos id=61 count=16296lb # [ 191.955740] niks3-server[1196]: 2026/09/25 16:52:57 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6297lb # [ 191.964356] niks3-server[1196]: 2026/09/25 16:52:57 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6298lb # [ 191.968729] niks3-server[1196]: 2026/09/25 16:52:57 INFO Received complete push request method=POST path=/api/pushes/61/complete6299worker1 # [ 191.990077] nix-grpc-daemon[1548]: time=2026-09-25T16:52:57.581Z level=INFO msg="Upload complete. (186ms)"6300lb # [ 191.973957] niks3-server[1196]: 2026/09/25 16:52:57 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6301worker1 # [ 191.997238] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=4166302worker1 # [ 192.004438] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126303worker1 # [ 192.011373] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6304worker1 # [ 192.015104] nix-daemon[856]: reaped child process 1717, status = succeeded6305client # [ 192.006194] nix[1518]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6306worker1 # [ 192.025003] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6307worker2 # [ 192.291066] nix-grpc-daemon[2074]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6308worker2 # [ 192.295129] nix-grpc-daemon[2074]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6309worker2 # [ 192.305332] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355386310client # [ 192.293790] nix[1518]: copying 1 paths to 'lb:50051'...6311worker2 # [ 192.313298] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355386312client # [ 192.295136] nix[1518]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6313worker2 # [ 192.321998] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6314worker2 # [ 192.348123] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:57Z level=info event=niks3_push_started pid=21346315worker2 # [ 192.469287] nix-grpc-daemon[2134]: time=2026-09-25T16:52:58.063Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6316worker2 # [ 192.575008] nix-daemon[842]: accepted connection from pid 2146, user nix-grpc-daemon (trusted)6317worker2 # [ 192.594207] nix-daemon[842]: reaped child process 2149, status = succeeded6318lb # [ 192.577615] niks3-server[1196]: 2026/09/25 16:52:58 INFO Received push request method=POST path=/api/pushes6319worker2 # [ 192.604206] nix-grpc-daemon[2134]: time=2026-09-25T16:52:58.198Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6320worker2 # [ 192.605797] nix-grpc-daemon[2134]: time=2026-09-25T16:52:58.198Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6321lb # [ 192.604867] niks3-server[1196]: 2026/09/25 16:52:58 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign6322worker2 # [ 192.631770] nix-grpc-daemon[2134]: time=2026-09-25T16:52:58.226Z level=INFO msg="Uploading 1 narinfos"6323lb # [ 192.612546] niks3-server[1196]: 2026/09/25 16:52:58 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6324lb # [ 192.615550] niks3-server[1196]: 2026/09/25 16:52:58 INFO Signed narinfos id=62 count=16325lb # [ 192.631754] niks3-server[1196]: 2026/09/25 16:52:58 INFO Received complete push request method=POST path=/api/pushes/62/complete6326worker2 # [ 192.658027] nix-grpc-daemon[2134]: time=2026-09-25T16:52:58.253Z level=INFO msg="Upload complete. (185ms)"6327lb # [ 192.640064] niks3-server[1196]: 2026/09/25 16:52:58 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6328worker2 # [ 192.664110] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35538 duration_s=0 paths=1 nar_bytes_in=4166329lb: must succeed: curl -sf localhost:9901/clusters6330worker2 # [ 192.674056] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355386331worker2 # [ 192.678448] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6332worker2 # [ 192.682052] nix-daemon[842]: reaped child process 2132, status = succeeded6333client # [ 192.671269] nix[1518]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6334worker2 # [ 192.692982] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6335lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6336lb: must succeed: curl -sf localhost:9901/clusters6337lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6338worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6339worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6340worker1 # [ 194.068200] nix-daemon[856]: reaped child process 1726, status = succeeded6341worker2 # [ 194.144286] nix-daemon[842]: reaped child process 2153, status = succeeded6342worker1 # [ 194.149026] nix-daemon[856]: accepted connection from pid 1754, user nix-grpc-daemon (trusted)6343worker1 # [ 194.170302] nix-daemon[856]: reaped child process 1757, status = succeeded6344lb # [ 194.157541] niks3-server[1196]: 2026/09/25 16:52:59 INFO Received push request method=POST path=/api/pushes6345worker1 # [ 194.184415] nix-grpc-daemon[1548]: time=2026-09-25T16:52:59.773Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6346worker1 # [ 194.186004] nix-grpc-daemon[1548]: time=2026-09-25T16:52:59.773Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6347lb # [ 194.192348] niks3-server[1196]: 2026/09/25 16:52:59 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign6348worker1 # [ 194.214489] nix-grpc-daemon[1548]: time=2026-09-25T16:52:59.805Z level=INFO msg="Uploading 1 narinfos"6349worker2 # [ 194.218591] nix-daemon[842]: accepted connection from pid 2178, user nix-grpc-daemon (trusted)6350lb # [ 194.202364] niks3-server[1196]: 2026/09/25 16:52:59 INFO Signed narinfos id=63 count=16351lb # [ 194.206442] niks3-server[1196]: 2026/09/25 16:52:59 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6352lb # [ 194.213588] niks3-server[1196]: 2026/09/25 16:52:59 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6353worker2 # [ 194.239133] nix-daemon[842]: reaped child process 2181, status = succeeded6354lb # [ 194.223524] niks3-server[1196]: 2026/09/25 16:52:59 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6355worker1 # [ 194.245362] nix-grpc-daemon[1548]: time=2026-09-25T16:52:59.836Z level=INFO msg="Upload complete. (211ms)"6356lb # [ 194.228517] niks3-server[1196]: 2026/09/25 16:52:59 INFO Received complete push request method=POST path=/api/pushes/63/complete6357lb # [ 194.230782] niks3-server[1196]: 2026/09/25 16:52:59 INFO Received push request method=POST path=/api/pushes6358lb # [ 194.234360] niks3-server[1196]: 2026/09/25 16:52:59 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6359worker2 # [ 194.256612] nix-grpc-daemon[2134]: time=2026-09-25T16:52:59.849Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6360worker2 # [ 194.260299] nix-grpc-daemon[2134]: time=2026-09-25T16:52:59.849Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6361worker1 # [ 194.253731] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=2 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06362lb # [ 194.249565] niks3-server[1196]: 2026/09/25 16:52:59 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6363worker1 # [ 194.268499] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:59Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.3:50051 clients=16364worker1 # [ 194.273831] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126365worker2 # [ 194.279877] nix-grpc-daemon[2134]: time=2026-09-25T16:52:59.874Z level=INFO msg="Uploading 1 narinfos"6366lb # [ 194.258848] niks3-server[1196]: 2026/09/25 16:52:59 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign6367worker1 # [ 194.276861] nix-daemon[856]: reaped child process 1724, status = succeeded6368lb # [ 194.263888] niks3-server[1196]: 2026/09/25 16:52:59 INFO Signed narinfos id=64 count=16369lb # [ 194.269011] niks3-server[1196]: 2026/09/25 16:52:59 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6370worker1 # [ 194.291302] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126371lb # [ 194.275471] niks3-server[1196]: 2026/09/25 16:52:59 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6372lb # [ 194.283394] niks3-server[1196]: 2026/09/25 16:52:59 INFO Received complete push request method=POST path=/api/pushes/64/complete6373worker2 # [ 194.307597] nix-grpc-daemon[2134]: time=2026-09-25T16:52:59.902Z level=INFO msg="Upload complete. (206ms)"6374lb # [ 194.288446] niks3-server[1196]: 2026/09/25 16:52:59 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6375client # [ 194.291518] nix[1518]: copying 1 paths to 'lb:50051'...6376client # [ 194.291681] nix[1518]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6377worker1 # [ 194.309450] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334126378worker1 # [ 194.313323] nix-grpc-daemon[1404]: ts=2026-09-25T16:52:59Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.4:50051 clients=16379worker2 # [ 194.317431] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35538 duration_s=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=2 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06380worker1 # [ 194.321088] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6381worker2 # [ 194.331011] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355386382worker2 # [ 194.334734] nix-daemon[842]: reaped child process 2151, status = succeeded6383worker2 # [ 194.341719] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355386384client # [ 194.328714] nix[1518]: copying 1 paths to 'lb:50051'...6385client # [ 194.331379] nix[1518]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6386worker2 # [ 194.351856] nix-grpc-daemon[2074]: ts=2026-09-25T16:52:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355386387worker2 # [ 194.360448] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6388worker1 # [ 194.441206] nix-daemon[856]: accepted connection from pid 1761, user nix-grpc-daemon (trusted)6389worker1 # [ 194.461793] nix-daemon[856]: reaped child process 1764, status = succeeded6390lb # [ 194.446843] niks3-server[1196]: 2026/09/25 16:53:00 INFO Received push request method=POST path=/api/pushes6391worker1 # [ 194.468830] nix-grpc-daemon[1548]: time=2026-09-25T16:53:00.059Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6392worker1 # [ 194.470431] nix-grpc-daemon[1548]: time=2026-09-25T16:53:00.059Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6393worker2 # [ 194.483251] nix-daemon[842]: accepted connection from pid 2185, user nix-grpc-daemon (trusted)6394lb # [ 194.464416] niks3-server[1196]: 2026/09/25 16:53:00 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign6395worker1 # [ 194.486533] nix-grpc-daemon[1548]: time=2026-09-25T16:53:00.077Z level=INFO msg="Uploading 1 narinfos"6396lb # [ 194.471522] niks3-server[1196]: 2026/09/25 16:53:00 INFO Signed narinfos id=65 count=16397lb # [ 194.476622] niks3-server[1196]: 2026/09/25 16:53:00 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6398worker2 # [ 194.501991] nix-daemon[842]: reaped child process 2188, status = succeeded6399lb # [ 194.485752] niks3-server[1196]: 2026/09/25 16:53:00 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6400worker1 # [ 194.508344] nix-grpc-daemon[1548]: time=2026-09-25T16:53:00.099Z level=INFO msg="Upload complete. (177ms)"6401lb # [ 194.493825] niks3-server[1196]: 2026/09/25 16:53:00 INFO Received complete push request method=POST path=/api/pushes/65/complete6402worker2 # [ 194.518731] nix-grpc-daemon[2134]: time=2026-09-25T16:53:00.113Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6403worker2 # [ 194.520440] nix-grpc-daemon[2134]: time=2026-09-25T16:53:00.113Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6404worker1 # [ 194.516203] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=4166405lb # [ 194.497929] niks3-server[1196]: 2026/09/25 16:53:00 INFO Received push request method=POST path=/api/pushes6406worker1 # [ 194.520177] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126407lb # [ 194.503300] niks3-server[1196]: 2026/09/25 16:53:00 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6408worker1 # [ 194.529521] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6409worker1 # [ 194.533556] nix-daemon[856]: reaped child process 1759, status = succeeded6410worker2 # [ 194.538106] nix-grpc-daemon[2134]: time=2026-09-25T16:53:00.132Z level=INFO msg="Uploading 1 narinfos"6411lb # [ 194.517106] niks3-server[1196]: 2026/09/25 16:53:00 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign6412lb # [ 194.519644] niks3-server[1196]: 2026/09/25 16:53:00 INFO Signed narinfos id=66 count=16413client # [ 194.525725] nix[1518]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6414worker1 # [ 194.544930] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6415lb # [ 194.527472] niks3-server[1196]: 2026/09/25 16:53:00 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6416lb # [ 194.533022] niks3-server[1196]: 2026/09/25 16:53:00 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6417lb # [ 194.538864] niks3-server[1196]: 2026/09/25 16:53:00 INFO Received complete push request method=POST path=/api/pushes/66/complete6418lb # [ 194.545351] niks3-server[1196]: 2026/09/25 16:53:00 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6419worker2 # [ 194.573745] nix-grpc-daemon[2134]: time=2026-09-25T16:53:00.168Z level=INFO msg="Upload complete. (203ms)"6420worker2 # [ 194.579804] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35538 duration_s=0 paths=1 nar_bytes_in=4166421worker2 # [ 194.588394] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355386422worker2 # [ 194.597465] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6423worker2 # [ 194.601939] nix-daemon[842]: reaped child process 2183, status = succeeded6424client # [ 194.590949] nix[1518]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6425worker2 # [ 194.612217] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6426worker1 # [ 196.381993] nix-daemon[856]: reaped child process 1768, status = succeeded6427worker2 # [ 196.468763] nix-daemon[842]: reaped child process 2192, status = succeeded6428worker1 # [ 196.482463] nix-daemon[856]: accepted connection from pid 1793, user nix-grpc-daemon (trusted)6429worker1 # [ 196.501473] nix-daemon[856]: reaped child process 1796, status = succeeded6430lb # [ 196.488463] niks3-server[1196]: 2026/09/25 16:53:02 INFO Received push request method=POST path=/api/pushes6431worker1 # [ 196.512478] nix-grpc-daemon[1548]: time=2026-09-25T16:53:02.102Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6432worker1 # [ 196.515981] nix-grpc-daemon[1548]: time=2026-09-25T16:53:02.102Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6433lb # [ 196.512670] niks3-server[1196]: 2026/09/25 16:53:02 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6434lb # [ 196.521831] niks3-server[1196]: 2026/09/25 16:53:02 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6435worker1 # [ 196.546756] nix-grpc-daemon[1548]: time=2026-09-25T16:53:02.137Z level=INFO msg="Uploading 1 narinfos"6436lb # [ 196.529420] niks3-server[1196]: 2026/09/25 16:53:02 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign6437lb # [ 196.531881] niks3-server[1196]: 2026/09/25 16:53:02 INFO Signed narinfos id=67 count=16438lb # [ 196.538713] niks3-server[1196]: 2026/09/25 16:53:02 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6439worker2 # [ 196.563300] nix-daemon[842]: accepted connection from pid 2217, user nix-grpc-daemon (trusted)6440lb # [ 196.547090] niks3-server[1196]: 2026/09/25 16:53:02 INFO Received complete push request method=POST path=/api/pushes/67/complete6441worker1 # [ 196.568963] nix-grpc-daemon[1548]: time=2026-09-25T16:53:02.160Z level=INFO msg="Upload complete. (215ms)"6442lb # [ 196.552478] niks3-server[1196]: 2026/09/25 16:53:02 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6443worker1 # [ 196.576630] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=2 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=3 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06444lb # [ 196.564430] niks3-server[1196]: 2026/09/25 16:53:02 INFO Received push request method=POST path=/api/pushes6445worker2 # [ 196.588695] nix-daemon[842]: reaped child process 2220, status = succeeded6446worker2 # [ 196.591444] nix-grpc-daemon[2134]: time=2026-09-25T16:53:02.185Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6447worker1 # [ 196.588327] nix-daemon[856]: reaped child process 1766, status = succeeded6448worker2 # [ 196.593200] nix-grpc-daemon[2134]: time=2026-09-25T16:53:02.185Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6449lb # [ 196.580422] niks3-server[1196]: 2026/09/25 16:53:02 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6450lb # [ 196.589775] niks3-server[1196]: 2026/09/25 16:53:02 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign6451worker2 # [ 196.613415] nix-grpc-daemon[2134]: time=2026-09-25T16:53:02.208Z level=INFO msg="Uploading 1 narinfos"6452lb # [ 196.594884] niks3-server[1196]: 2026/09/25 16:53:02 INFO Signed narinfos id=68 count=16453lb # [ 196.599271] niks3-server[1196]: 2026/09/25 16:53:02 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6454lb # [ 196.607954] niks3-server[1196]: 2026/09/25 16:53:02 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6455worker2 # [ 196.632915] nix-grpc-daemon[2134]: time=2026-09-25T16:53:02.228Z level=INFO msg="Upload complete. (208ms)"6456lb # [ 196.613041] niks3-server[1196]: 2026/09/25 16:53:02 INFO Received complete push request method=POST path=/api/pushes/68/complete6457lb # [ 196.615086] niks3-server[1196]: 2026/09/25 16:53:02 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6458worker2 # [ 196.640622] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35538 duration_s=2 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=4 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06459worker2 # [ 196.649956] nix-daemon[842]: reaped child process 2190, status = succeeded6460client # [ 196.643574] systemd[1]: burst.service: Deactivated successfully.6461client # [ 196.646062] systemd[1]: burst.service: Consumed 190ms CPU time over 5.120s wall clock time, 55.6M memory peak, 13.8K incoming IP traffic, 18.5K outgoing IP traffic.6462lb # [ 197.049636] envoy[886]: conn peer=[2001:db8:1::4]:36702 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1554 tx=378 ms=281716463lb # [ 197.052370] envoy[886]: conn peer=[2001:db8:1::1]:38822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9671 tx=5178 ms=49236464client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6465client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds)6466lb: must succeed: curl -sf localhost:9901/clusters6467lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6468lb: must succeed: curl -sf localhost:9901/clusters6469lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6470worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6471worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6472lb: must succeed: curl -sf localhost:9901/clusters6473lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6474lb: must succeed: curl -sf localhost:9901/clusters6475lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6476worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6477worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)6478(finished: subtest: queue gauges show a burst while every build blocks, in 6.35 seconds)6479subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6480client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16481worker1 # [ 197.926222] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334126482worker2 # [ 197.937720] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355386483worker1 # [ 197.961035] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:03Z level=debug event=want client=2 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06484worker1 # [ 197.966290] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:03Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16485worker1 # [ 197.971046] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:03Z level=debug event=want client=2 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06486worker1 # [ 197.975177] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:03Z level=debug event=want client=2 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06487worker1 # [ 197.979363] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126488worker1 # [ 197.984131] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126489worker1 # [ 197.986715] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334126490worker1 # [ 197.993748] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6491worker1 # [ 198.113124] nix-daemon[856]: accepted connection from pid 1812, user nix-grpc-daemon (trusted)6492lb # [ 198.113296] niks3-server[1196]: 2026/09/25 16:53:03 INFO Received push request method=POST path=/api/pushes6493worker1 # [ 198.132152] nix-daemon[856]: reaped child process 1815, status = succeeded6494worker1 # [ 198.136149] nix-grpc-daemon[1548]: time=2026-09-25T16:53:03.726Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6495worker1 # [ 198.137663] nix-grpc-daemon[1548]: time=2026-09-25T16:53:03.726Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6496lb # [ 198.130323] niks3-server[1196]: 2026/09/25 16:53:03 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign6497worker1 # [ 198.153027] nix-grpc-daemon[1548]: time=2026-09-25T16:53:03.744Z level=INFO msg="Uploading 1 narinfos"6498lb # [ 198.138171] niks3-server[1196]: 2026/09/25 16:53:03 INFO Signed narinfos id=69 count=16499lb # [ 198.142344] niks3-server[1196]: 2026/09/25 16:53:03 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6500lb # [ 198.147870] niks3-server[1196]: 2026/09/25 16:53:03 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6501lb # [ 198.153350] niks3-server[1196]: 2026/09/25 16:53:03 INFO Received complete push request method=POST path=/api/pushes/69/complete6502worker1 # [ 198.178849] nix-grpc-daemon[1548]: time=2026-09-25T16:53:03.770Z level=INFO msg="Upload complete. (176ms)"6503lb # [ 198.162253] niks3-server[1196]: 2026/09/25 16:53:03 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6504worker1 # [ 198.185124] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=5126505worker1 # [ 198.194755] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126506worker1 # [ 198.197633] nix-daemon[856]: reaped child process 1810, status = succeeded6507worker1 # [ 198.202554] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6508worker1 # [ 198.211313] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6509worker1 # [ 199.445750] nix-daemon[856]: reaped child process 1819, status = succeeded6510worker1 # [ 199.548998] nix-daemon[856]: accepted connection from pid 1835, user nix-grpc-daemon (trusted)6511worker1 # [ 199.566540] nix-daemon[856]: reaped child process 1838, status = succeeded6512lb # [ 199.554637] niks3-server[1196]: 2026/09/25 16:53:05 INFO Received push request method=POST path=/api/pushes6513worker1 # [ 199.578261] nix-grpc-daemon[1548]: time=2026-09-25T16:53:05.168Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6514worker1 # [ 199.579747] nix-grpc-daemon[1548]: time=2026-09-25T16:53:05.168Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6515lb # [ 199.576133] niks3-server[1196]: 2026/09/25 16:53:05 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign6516worker1 # [ 199.597110] nix-grpc-daemon[1548]: time=2026-09-25T16:53:05.188Z level=INFO msg="Uploading 1 narinfos"6517lb # [ 199.582878] niks3-server[1196]: 2026/09/25 16:53:05 INFO Signed narinfos id=70 count=16518lb # [ 199.589159] niks3-server[1196]: 2026/09/25 16:53:05 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6519lb # [ 199.591751] niks3-server[1196]: 2026/09/25 16:53:05 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6520worker1 # [ 199.620395] nix-grpc-daemon[1548]: time=2026-09-25T16:53:05.211Z level=INFO msg="Upload complete. (199ms)"6521lb # [ 199.603869] niks3-server[1196]: 2026/09/25 16:53:05 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6522lb # [ 199.609009] niks3-server[1196]: 2026/09/25 16:53:05 INFO Received complete push request method=POST path=/api/pushes/70/complete6523lb # [ 199.615711] niks3-server[1196]: 2026/09/25 16:53:05 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6524worker1 # [ 199.631822] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=5 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06525worker1 # [ 199.642694] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:05Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16526worker1 # [ 199.646851] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126527worker1 # [ 199.651191] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:05Z level=debug event=want client=2 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06528worker1 # [ 199.655103] nix-daemon[856]: reaped child process 1817, status = succeeded6529worker1 # [ 199.657531] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126530worker1 # [ 199.669183] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334126531worker1 # [ 199.671540] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6532worker1 # [ 199.772410] nix-daemon[856]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6533worker1 # [ 199.787783] nix-daemon[856]: reaped child process 1847, status = succeeded6534lb # [ 199.775352] niks3-server[1196]: 2026/09/25 16:53:05 INFO Received push request method=POST path=/api/pushes6535worker1 # [ 199.797361] nix-grpc-daemon[1548]: time=2026-09-25T16:53:05.388Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6536worker1 # [ 199.798852] nix-grpc-daemon[1548]: time=2026-09-25T16:53:05.388Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6537lb # [ 199.791597] niks3-server[1196]: 2026/09/25 16:53:05 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6538worker1 # [ 199.814933] nix-grpc-daemon[1548]: time=2026-09-25T16:53:05.406Z level=INFO msg="Uploading 1 narinfos"6539lb # [ 199.799325] niks3-server[1196]: 2026/09/25 16:53:05 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign6540lb # [ 199.804844] niks3-server[1196]: 2026/09/25 16:53:05 INFO Signed narinfos id=71 count=16541lb # [ 199.806480] niks3-server[1196]: 2026/09/25 16:53:05 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6542lb # [ 199.816902] niks3-server[1196]: 2026/09/25 16:53:05 INFO Received complete push request method=POST path=/api/pushes/71/complete6543worker1 # [ 199.838285] nix-grpc-daemon[1548]: time=2026-09-25T16:53:05.429Z level=INFO msg="Upload complete. (163ms)"6544lb # [ 199.821087] niks3-server[1196]: 2026/09/25 16:53:05 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6545worker1 # [ 199.844704] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=5126546worker1 # [ 199.847779] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126547worker1 # [ 199.857802] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6548worker1 # [ 199.861489] nix-daemon[856]: reaped child process 1842, status = succeeded6549worker1 # [ 199.871285] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6550worker1 # [ 201.350031] nix-daemon[856]: reaped child process 1851, status = succeeded6551worker1 # [ 201.410496] nix-daemon[856]: accepted connection from pid 1867, user nix-grpc-daemon (trusted)6552worker1 # [ 201.425683] nix-daemon[856]: reaped child process 1870, status = succeeded6553lb # [ 201.414250] niks3-server[1196]: 2026/09/25 16:53:07 INFO Received push request method=POST path=/api/pushes6554worker1 # [ 201.437367] nix-grpc-daemon[1548]: time=2026-09-25T16:53:07.027Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6555worker1 # [ 201.438851] nix-grpc-daemon[1548]: time=2026-09-25T16:53:07.027Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6556lb # [ 201.443493] niks3-server[1196]: 2026/09/25 16:53:07 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6557worker1 # [ 201.465984] nix-grpc-daemon[1548]: time=2026-09-25T16:53:07.057Z level=INFO msg="Uploading 1 narinfos"6558lb # [ 201.451073] niks3-server[1196]: 2026/09/25 16:53:07 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign6559lb # [ 201.458016] niks3-server[1196]: 2026/09/25 16:53:07 INFO Signed narinfos id=72 count=16560lb # [ 201.459765] niks3-server[1196]: 2026/09/25 16:53:07 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6561lb # [ 201.467112] niks3-server[1196]: 2026/09/25 16:53:07 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6562lb # [ 201.474854] niks3-server[1196]: 2026/09/25 16:53:07 INFO Received complete push request method=POST path=/api/pushes/72/complete6563worker1 # [ 201.495174] nix-grpc-daemon[1548]: time=2026-09-25T16:53:07.086Z level=INFO msg="Upload complete. (216ms)"6564lb # [ 201.479300] niks3-server[1196]: 2026/09/25 16:53:07 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6565worker1 # [ 201.505443] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=6 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06566worker1 # [ 201.518736] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:07Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16567worker1 # [ 201.522645] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126568worker1 # [ 201.526985] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:07Z level=debug event=want client=2 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06569worker1 # [ 201.530773] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126570worker1 # [ 201.544153] nix-daemon[856]: reaped child process 1849, status = succeeded6571worker1 # [ 201.545654] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6572worker1 # [ 201.547319] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334126573worker1 # [ 201.656380] nix-daemon[856]: accepted connection from pid 1876, user nix-grpc-daemon (trusted)6574worker1 # [ 201.671137] nix-daemon[856]: reaped child process 1879, status = succeeded6575lb # [ 201.659919] niks3-server[1196]: 2026/09/25 16:53:07 INFO Received push request method=POST path=/api/pushes6576worker1 # [ 201.681967] nix-grpc-daemon[1548]: time=2026-09-25T16:53:07.272Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6577worker1 # [ 201.683460] nix-grpc-daemon[1548]: time=2026-09-25T16:53:07.273Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6578lb # [ 201.675395] niks3-server[1196]: 2026/09/25 16:53:07 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign6579worker1 # [ 201.695704] nix-grpc-daemon[1548]: time=2026-09-25T16:53:07.286Z level=INFO msg="Uploading 1 narinfos"6580lb # [ 201.682665] niks3-server[1196]: 2026/09/25 16:53:07 INFO Signed narinfos id=73 count=16581lb # [ 201.687436] niks3-server[1196]: 2026/09/25 16:53:07 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6582lb # [ 201.697063] niks3-server[1196]: 2026/09/25 16:53:07 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6583lb # [ 201.699527] niks3-server[1196]: 2026/09/25 16:53:07 INFO Received complete push request method=POST path=/api/pushes/73/complete6584worker1 # [ 201.722324] nix-grpc-daemon[1548]: time=2026-09-25T16:53:07.313Z level=INFO msg="Upload complete. (171ms)"6585lb # [ 201.706308] niks3-server[1196]: 2026/09/25 16:53:07 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6586worker1 # [ 201.728744] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=5126587worker1 # [ 201.731867] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126588worker1 # [ 201.742735] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6589worker1 # [ 201.746345] nix-daemon[856]: reaped child process 1874, status = succeeded6590worker1 # [ 201.756080] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6591worker1 # [ 203.004903] nix-daemon[856]: reaped child process 1883, status = succeeded6592worker1 # [ 203.094860] nix-daemon[856]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6593worker1 # [ 203.115469] nix-daemon[856]: reaped child process 1902, status = succeeded6594lb # [ 203.099570] niks3-server[1196]: 2026/09/25 16:53:08 INFO Received push request method=POST path=/api/pushes6595worker1 # [ 203.123626] nix-grpc-daemon[1548]: time=2026-09-25T16:53:08.713Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6596worker1 # [ 203.125744] nix-grpc-daemon[1548]: time=2026-09-25T16:53:08.713Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6597lb # [ 203.123415] niks3-server[1196]: 2026/09/25 16:53:08 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6598worker1 # [ 203.148478] nix-grpc-daemon[1548]: time=2026-09-25T16:53:08.739Z level=INFO msg="Uploading 1 narinfos"6599lb # [ 203.131945] niks3-server[1196]: 2026/09/25 16:53:08 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6600lb # [ 203.138865] niks3-server[1196]: 2026/09/25 16:53:08 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign6601lb # [ 203.143768] niks3-server[1196]: 2026/09/25 16:53:08 INFO Signed narinfos id=74 count=16602lb # [ 203.150708] niks3-server[1196]: 2026/09/25 16:53:08 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6603worker1 # [ 203.172520] nix-grpc-daemon[1548]: time=2026-09-25T16:53:08.763Z level=INFO msg="Upload complete. (209ms)"6604lb # [ 203.156792] niks3-server[1196]: 2026/09/25 16:53:08 INFO Received complete push request method=POST path=/api/pushes/74/complete6605lb # [ 203.160426] niks3-server[1196]: 2026/09/25 16:53:08 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6606worker1 # [ 203.183730] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=7 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06607worker1 # [ 203.195688] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:08Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16608worker1 # [ 203.200339] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126609worker1 # [ 203.204472] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:08Z level=debug event=want client=2 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06610worker1 # [ 203.208616] nix-daemon[856]: reaped child process 1881, status = succeeded6611worker1 # [ 203.211180] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126612worker1 # [ 203.215024] nix-grpc-daemon[1548]: time=2026-09-25T16:53:08.805Z level=INFO msg="All 1 paths already cached"6613worker1 # [ 203.226805] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334126614worker1 # [ 203.231255] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6615worker1 # [ 203.336896] nix-daemon[856]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)6616worker1 # [ 203.353279] nix-daemon[856]: reaped child process 1911, status = succeeded6617lb # [ 203.341746] niks3-server[1196]: 2026/09/25 16:53:08 INFO Received push request method=POST path=/api/pushes6618worker1 # [ 203.363707] nix-grpc-daemon[1548]: time=2026-09-25T16:53:08.954Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6619worker1 # [ 203.365777] nix-grpc-daemon[1548]: time=2026-09-25T16:53:08.954Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6620lb # [ 203.361038] niks3-server[1196]: 2026/09/25 16:53:08 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6621worker1 # [ 203.383423] nix-grpc-daemon[1548]: time=2026-09-25T16:53:08.974Z level=INFO msg="Uploading 1 narinfos"6622lb # [ 203.368339] niks3-server[1196]: 2026/09/25 16:53:08 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign6623lb # [ 203.370669] niks3-server[1196]: 2026/09/25 16:53:08 INFO Signed narinfos id=75 count=16624lb # [ 203.378132] niks3-server[1196]: 2026/09/25 16:53:08 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6625lb # [ 203.386348] niks3-server[1196]: 2026/09/25 16:53:08 INFO Received complete push request method=POST path=/api/pushes/75/complete6626lb # [ 203.390891] niks3-server[1196]: 2026/09/25 16:53:08 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6627worker1 # [ 203.411429] nix-grpc-daemon[1548]: time=2026-09-25T16:53:09.002Z level=INFO msg="Upload complete. (177ms)"6628worker1 # [ 203.417300] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=6246629worker1 # [ 203.426955] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126630worker1 # [ 203.432645] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6631worker1 # [ 203.436702] nix-daemon[856]: reaped child process 1906, status = succeeded6632worker1 # [ 203.445522] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6633worker1 # [ 204.674279] nix-daemon[856]: reaped child process 1915, status = succeeded6634worker1 # [ 204.762764] nix-daemon[856]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)6635worker1 # [ 204.782004] nix-daemon[856]: reaped child process 1934, status = succeeded6636lb # [ 204.769034] niks3-server[1196]: 2026/09/25 16:53:10 INFO Received push request method=POST path=/api/pushes6637worker1 # [ 204.792405] nix-grpc-daemon[1548]: time=2026-09-25T16:53:10.382Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6638worker1 # [ 204.793949] nix-grpc-daemon[1548]: time=2026-09-25T16:53:10.382Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6639lb # [ 204.792981] niks3-server[1196]: 2026/09/25 16:53:10 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6640worker1 # [ 204.815659] nix-grpc-daemon[1548]: time=2026-09-25T16:53:10.406Z level=INFO msg="Uploading 1 narinfos"6641lb # [ 204.800958] niks3-server[1196]: 2026/09/25 16:53:10 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign6642lb # [ 204.803267] niks3-server[1196]: 2026/09/25 16:53:10 INFO Signed narinfos id=76 count=16643lb # [ 204.810211] niks3-server[1196]: 2026/09/25 16:53:10 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6644lb # [ 204.818933] niks3-server[1196]: 2026/09/25 16:53:10 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6645lb # [ 204.823880] niks3-server[1196]: 2026/09/25 16:53:10 INFO Received complete push request method=POST path=/api/pushes/76/complete6646worker1 # [ 204.844100] nix-grpc-daemon[1548]: time=2026-09-25T16:53:10.435Z level=INFO msg="Upload complete. (188ms)"6647lb # [ 204.828373] niks3-server[1196]: 2026/09/25 16:53:10 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6648worker1 # [ 204.855408] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=8 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06649worker1 # [ 204.866797] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:10Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16650worker1 # [ 204.874835] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126651worker1 # [ 204.878593] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:10Z level=debug event=want client=2 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06652worker1 # [ 204.882477] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126653worker1 # [ 204.886203] nix-daemon[856]: reaped child process 1913, status = succeeded6654worker1 # [ 204.887734] nix-grpc-daemon[1548]: time=2026-09-25T16:53:10.476Z level=INFO msg="All 1 paths already cached"6655worker1 # [ 204.900712] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334126656worker1 # [ 204.903120] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6657worker1 # [ 205.008692] nix-daemon[856]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)6658worker1 # [ 205.027308] nix-daemon[856]: reaped child process 1943, status = succeeded6659lb # [ 205.012474] niks3-server[1196]: 2026/09/25 16:53:10 INFO Received push request method=POST path=/api/pushes6660worker1 # [ 205.034498] nix-grpc-daemon[1548]: time=2026-09-25T16:53:10.625Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6661worker1 # [ 205.035988] nix-grpc-daemon[1548]: time=2026-09-25T16:53:10.625Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6662lb # [ 205.027704] niks3-server[1196]: 2026/09/25 16:53:10 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6663worker1 # [ 205.052116] nix-grpc-daemon[1548]: time=2026-09-25T16:53:10.643Z level=INFO msg="Uploading 1 narinfos"6664lb # [ 205.036548] niks3-server[1196]: 2026/09/25 16:53:10 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign6665lb # [ 205.038973] niks3-server[1196]: 2026/09/25 16:53:10 INFO Signed narinfos id=77 count=16666lb # [ 205.047843] niks3-server[1196]: 2026/09/25 16:53:10 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6667worker1 # [ 205.071500] nix-grpc-daemon[1548]: time=2026-09-25T16:53:10.662Z level=INFO msg="Upload complete. (165ms)"6668lb # [ 205.055710] niks3-server[1196]: 2026/09/25 16:53:10 INFO Received complete push request method=POST path=/api/pushes/77/complete6669worker1 # [ 205.076155] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=6246670lb # [ 205.058999] niks3-server[1196]: 2026/09/25 16:53:10 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6671worker1 # [ 205.084980] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126672worker1 # [ 205.089535] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6673worker1 # [ 205.093048] nix-daemon[856]: reaped child process 1938, status = succeeded6674worker1 # [ 205.100628] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6675worker1 # [ 206.362716] nix-daemon[856]: reaped child process 1947, status = succeeded6676worker1 # [ 206.432595] nix-daemon[856]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)6677worker1 # [ 206.450525] nix-daemon[856]: reaped child process 1966, status = succeeded6678lb # [ 206.436955] niks3-server[1196]: 2026/09/25 16:53:12 INFO Received push request method=POST path=/api/pushes6679worker1 # [ 206.460547] nix-grpc-daemon[1548]: time=2026-09-25T16:53:12.050Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6680worker1 # [ 206.462086] nix-grpc-daemon[1548]: time=2026-09-25T16:53:12.050Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6681lb # [ 206.459849] niks3-server[1196]: 2026/09/25 16:53:12 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6682worker1 # [ 206.482407] nix-grpc-daemon[1548]: time=2026-09-25T16:53:12.073Z level=INFO msg="Uploading 1 narinfos"6683lb # [ 206.467074] niks3-server[1196]: 2026/09/25 16:53:12 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign6684lb # [ 206.473564] niks3-server[1196]: 2026/09/25 16:53:12 INFO Signed narinfos id=78 count=16685lb # [ 206.475322] niks3-server[1196]: 2026/09/25 16:53:12 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6686lb # [ 206.483136] niks3-server[1196]: 2026/09/25 16:53:12 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6687lb # [ 206.490988] niks3-server[1196]: 2026/09/25 16:53:12 INFO Received complete push request method=POST path=/api/pushes/78/complete6688worker1 # [ 206.511515] nix-grpc-daemon[1548]: time=2026-09-25T16:53:12.102Z level=INFO msg="Upload complete. (202ms)"6689lb # [ 206.495276] niks3-server[1196]: 2026/09/25 16:53:12 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6690worker1 # [ 206.522466] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=9 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06691worker1 # [ 206.536393] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:12Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16692worker1 # [ 206.542054] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126693worker1 # [ 206.546393] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:12Z level=debug event=want client=2 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06694worker1 # [ 206.550423] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126695worker1 # [ 206.554248] nix-daemon[856]: reaped child process 1945, status = succeeded6696worker1 # [ 206.555828] nix-grpc-daemon[1548]: time=2026-09-25T16:53:12.144Z level=INFO msg="All 1 paths already cached"6697worker1 # [ 206.564181] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334126698worker1 # [ 206.566769] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6699worker1 # [ 206.673851] nix-daemon[856]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)6700worker1 # [ 206.691300] nix-daemon[856]: reaped child process 1975, status = succeeded6701lb # [ 206.679857] niks3-server[1196]: 2026/09/25 16:53:12 INFO Received push request method=POST path=/api/pushes6702worker1 # [ 206.701970] nix-grpc-daemon[1548]: time=2026-09-25T16:53:12.292Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6703worker1 # [ 206.703466] nix-grpc-daemon[1548]: time=2026-09-25T16:53:12.293Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6704lb # [ 206.696229] niks3-server[1196]: 2026/09/25 16:53:12 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6705worker1 # [ 206.719290] nix-grpc-daemon[1548]: time=2026-09-25T16:53:12.310Z level=INFO msg="Uploading 1 narinfos"6706lb # [ 206.704939] niks3-server[1196]: 2026/09/25 16:53:12 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign6707lb # [ 206.710248] niks3-server[1196]: 2026/09/25 16:53:12 INFO Signed narinfos id=79 count=16708lb # [ 206.711957] niks3-server[1196]: 2026/09/25 16:53:12 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6709lb # [ 206.720784] niks3-server[1196]: 2026/09/25 16:53:12 INFO Received complete push request method=POST path=/api/pushes/79/complete6710worker1 # [ 206.742381] nix-grpc-daemon[1548]: time=2026-09-25T16:53:12.333Z level=INFO msg="Upload complete. (168ms)"6711lb # [ 206.725559] niks3-server[1196]: 2026/09/25 16:53:12 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6712worker1 # [ 206.748584] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=6246713worker1 # [ 206.751793] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126714worker1 # [ 206.763542] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6715worker1 # [ 206.767360] nix-daemon[856]: reaped child process 1970, status = succeeded6716worker1 # [ 206.776184] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6717worker1 # [ 208.017580] nix-daemon[856]: reaped child process 1979, status = succeeded6718worker1 # [ 208.124118] nix-daemon[856]: accepted connection from pid 1995, user nix-grpc-daemon (trusted)6719lb # [ 208.127146] niks3-server[1196]: 2026/09/25 16:53:13 INFO Received push request method=POST path=/api/pushes6720worker1 # [ 208.148109] nix-daemon[856]: reaped child process 1998, status = succeeded6721worker1 # [ 208.152476] nix-grpc-daemon[1548]: time=2026-09-25T16:53:13.741Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6722worker1 # [ 208.154034] nix-grpc-daemon[1548]: time=2026-09-25T16:53:13.741Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6723lb # [ 208.158961] niks3-server[1196]: 2026/09/25 16:53:13 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6724worker1 # [ 208.182186] nix-grpc-daemon[1548]: time=2026-09-25T16:53:13.773Z level=INFO msg="Uploading 1 narinfos"6725lb # [ 208.166168] niks3-server[1196]: 2026/09/25 16:53:13 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6726lb # [ 208.174379] niks3-server[1196]: 2026/09/25 16:53:13 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign6727lb # [ 208.179346] niks3-server[1196]: 2026/09/25 16:53:13 INFO Signed narinfos id=80 count=16728lb # [ 208.187466] niks3-server[1196]: 2026/09/25 16:53:13 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6729worker1 # [ 208.209598] nix-grpc-daemon[1548]: time=2026-09-25T16:53:13.800Z level=INFO msg="Upload complete. (223ms)"6730lb # [ 208.193061] niks3-server[1196]: 2026/09/25 16:53:13 INFO Received complete push request method=POST path=/api/pushes/80/complete6731lb # [ 208.197115] niks3-server[1196]: 2026/09/25 16:53:13 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6732worker1 # [ 208.216617] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=10 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06733worker1 # [ 208.235002] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:13Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16734worker1 # [ 208.238342] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126735worker1 # [ 208.241838] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:13Z level=debug event=want client=2 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06736worker1 # [ 208.246046] nix-daemon[856]: reaped child process 1977, status = succeeded6737worker1 # [ 208.248600] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126738worker1 # [ 208.256593] nix-grpc-daemon[1548]: time=2026-09-25T16:53:13.847Z level=INFO msg="All 2 paths already cached"6739worker1 # [ 208.262800] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334126740worker1 # [ 208.271080] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6741worker1 # [ 208.390171] nix-daemon[856]: accepted connection from pid 2004, user nix-grpc-daemon (trusted)6742worker1 # [ 208.411753] nix-daemon[856]: reaped child process 2007, status = succeeded6743lb # [ 208.399081] niks3-server[1196]: 2026/09/25 16:53:13 INFO Received push request method=POST path=/api/pushes6744worker1 # [ 208.421298] nix-grpc-daemon[1548]: time=2026-09-25T16:53:14.012Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6745worker1 # [ 208.422825] nix-grpc-daemon[1548]: time=2026-09-25T16:53:14.012Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6746lb # [ 208.417657] niks3-server[1196]: 2026/09/25 16:53:14 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign6747worker1 # [ 208.438204] nix-grpc-daemon[1548]: time=2026-09-25T16:53:14.029Z level=INFO msg="Uploading 1 narinfos"6748lb # [ 208.424603] niks3-server[1196]: 2026/09/25 16:53:14 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6749lb # [ 208.427130] niks3-server[1196]: 2026/09/25 16:53:14 INFO Signed narinfos id=81 count=16750lb # [ 208.437027] niks3-server[1196]: 2026/09/25 16:53:14 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6751lb # [ 208.439986] niks3-server[1196]: 2026/09/25 16:53:14 INFO Received complete push request method=POST path=/api/pushes/81/complete6752worker1 # [ 208.463212] nix-grpc-daemon[1548]: time=2026-09-25T16:53:14.054Z level=INFO msg="Upload complete. (182ms)"6753lb # [ 208.446468] niks3-server[1196]: 2026/09/25 16:53:14 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6754worker1 # [ 208.468189] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=7526755worker1 # [ 208.479167] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126756worker1 # [ 208.484195] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6757worker1 # [ 208.488247] nix-daemon[856]: reaped child process 2002, status = succeeded6758worker1 # [ 208.499047] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6759worker1 # [ 209.544900] nix-daemon[856]: reaped child process 2011, status = succeeded6760worker1 # [ 209.643903] nix-daemon[856]: accepted connection from pid 2027, user nix-grpc-daemon (trusted)6761worker1 # [ 209.665310] nix-daemon[856]: reaped child process 2030, status = succeeded6762lb # [ 209.655084] niks3-server[1196]: 2026/09/25 16:53:15 INFO Received push request method=POST path=/api/pushes6763worker1 # [ 209.679709] nix-grpc-daemon[1548]: time=2026-09-25T16:53:15.270Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6764worker1 # [ 209.682191] nix-grpc-daemon[1548]: time=2026-09-25T16:53:15.270Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6765lb # [ 209.681897] niks3-server[1196]: 2026/09/25 16:53:15 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6766worker1 # [ 209.711818] nix-grpc-daemon[1548]: time=2026-09-25T16:53:15.302Z level=INFO msg="Uploading 1 narinfos"6767lb # [ 209.690126] niks3-server[1196]: 2026/09/25 16:53:15 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign6768lb # [ 209.699944] niks3-server[1196]: 2026/09/25 16:53:15 INFO Signed narinfos id=82 count=16769lb # [ 209.704234] niks3-server[1196]: 2026/09/25 16:53:15 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6770lb # [ 209.706702] niks3-server[1196]: 2026/09/25 16:53:15 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6771lb # [ 209.717908] niks3-server[1196]: 2026/09/25 16:53:15 INFO Received complete push request method=POST path=/api/pushes/82/complete6772worker1 # [ 209.739680] nix-grpc-daemon[1548]: time=2026-09-25T16:53:15.330Z level=INFO msg="Upload complete. (232ms)"6773lb # [ 209.723612] niks3-server[1196]: 2026/09/25 16:53:15 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6774worker1 # [ 209.747892] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=11 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06775worker1 # [ 209.760276] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:15Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16776worker1 # [ 209.766072] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126777worker1 # [ 209.770579] nix-daemon[856]: reaped child process 2009, status = succeeded6778worker1 # [ 209.773882] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126779worker1 # [ 209.780245] nix-grpc-daemon[1548]: time=2026-09-25T16:53:15.371Z level=INFO msg="All 2 paths already cached"6780worker1 # [ 209.786748] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334126781worker1 # [ 209.795002] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6782worker1 # [ 209.917004] nix-daemon[856]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)6783lb # [ 209.917411] niks3-server[1196]: 2026/09/25 16:53:15 INFO Received push request method=POST path=/api/pushes6784worker1 # [ 209.938895] nix-daemon[856]: reaped child process 2037, status = succeeded6785worker1 # [ 209.940890] nix-grpc-daemon[1548]: time=2026-09-25T16:53:15.531Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6786worker1 # [ 209.942601] nix-grpc-daemon[1548]: time=2026-09-25T16:53:15.531Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6787lb # [ 209.936036] niks3-server[1196]: 2026/09/25 16:53:15 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6788lb # [ 209.942398] niks3-server[1196]: 2026/09/25 16:53:15 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign6789worker1 # [ 209.962453] nix-grpc-daemon[1548]: time=2026-09-25T16:53:15.553Z level=INFO msg="Uploading 1 narinfos"6790lb # [ 209.948515] niks3-server[1196]: 2026/09/25 16:53:15 INFO Signed narinfos id=83 count=16791lb # [ 209.953214] niks3-server[1196]: 2026/09/25 16:53:15 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6792lb # [ 209.963425] niks3-server[1196]: 2026/09/25 16:53:15 INFO Received complete push request method=POST path=/api/pushes/83/complete6793worker1 # [ 209.985587] nix-grpc-daemon[1548]: time=2026-09-25T16:53:15.576Z level=INFO msg="Upload complete. (180ms)"6794lb # [ 209.969470] niks3-server[1196]: 2026/09/25 16:53:15 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6795worker1 # [ 209.991893] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=7526796worker1 # [ 210.003458] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126797worker1 # [ 210.008693] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6798worker1 # [ 210.012891] nix-daemon[856]: reaped child process 2032, status = succeeded6799worker1 # [ 210.023841] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6800worker1 # [ 211.308767] nix-daemon[856]: reaped child process 2041, status = succeeded6801worker1 # [ 211.370432] nix-daemon[856]: accepted connection from pid 2057, user nix-grpc-daemon (trusted)6802lb # [ 211.372714] niks3-server[1196]: 2026/09/25 16:53:16 INFO Received push request method=POST path=/api/pushes6803worker1 # [ 211.392736] nix-daemon[856]: reaped child process 2060, status = succeeded6804worker1 # [ 211.396454] nix-grpc-daemon[1548]: time=2026-09-25T16:53:16.986Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6805worker1 # [ 211.398069] nix-grpc-daemon[1548]: time=2026-09-25T16:53:16.986Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6806lb # [ 211.403880] niks3-server[1196]: 2026/09/25 16:53:16 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6807worker1 # [ 211.428248] nix-grpc-daemon[1548]: time=2026-09-25T16:53:17.019Z level=INFO msg="Uploading 1 narinfos"6808lb # [ 211.412380] niks3-server[1196]: 2026/09/25 16:53:17 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6809lb # [ 211.419722] niks3-server[1196]: 2026/09/25 16:53:17 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign6810lb # [ 211.422784] niks3-server[1196]: 2026/09/25 16:53:17 INFO Signed narinfos id=84 count=16811lb # [ 211.432298] niks3-server[1196]: 2026/09/25 16:53:17 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6812lb # [ 211.435004] niks3-server[1196]: 2026/09/25 16:53:17 INFO Received complete push request method=POST path=/api/pushes/84/complete6813worker1 # [ 211.456279] nix-grpc-daemon[1548]: time=2026-09-25T16:53:17.047Z level=INFO msg="Upload complete. (223ms)"6814lb # [ 211.439939] niks3-server[1196]: 2026/09/25 16:53:17 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6815worker1 # [ 211.468354] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=12 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06816worker1 # [ 211.476251] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:17Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16817worker1 # [ 211.479475] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126818worker1 # [ 211.487652] nix-daemon[856]: reaped child process 2039, status = succeeded6819worker1 # [ 211.490739] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:334126820worker1 # [ 211.496920] nix-grpc-daemon[1548]: time=2026-09-25T16:53:17.088Z level=INFO msg="All 2 paths already cached"6821worker1 # [ 211.503126] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:334126822worker1 # [ 211.511376] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6823worker1 # [ 211.631882] nix-daemon[856]: accepted connection from pid 2064, user nix-grpc-daemon (trusted)6824lb # [ 211.633562] niks3-server[1196]: 2026/09/25 16:53:17 INFO Received push request method=POST path=/api/pushes6825worker1 # [ 211.654537] nix-daemon[856]: reaped child process 2067, status = succeeded6826worker1 # [ 211.656578] nix-grpc-daemon[1548]: time=2026-09-25T16:53:17.247Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6827worker1 # [ 211.658103] nix-grpc-daemon[1548]: time=2026-09-25T16:53:17.247Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6828lb # [ 211.649015] niks3-server[1196]: 2026/09/25 16:53:17 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign6829worker1 # [ 211.671557] nix-grpc-daemon[1548]: time=2026-09-25T16:53:17.262Z level=INFO msg="Uploading 1 narinfos"6830lb # [ 211.659072] niks3-server[1196]: 2026/09/25 16:53:17 INFO Signed narinfos id=85 count=16831lb # [ 211.664274] niks3-server[1196]: 2026/09/25 16:53:17 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6832lb # [ 211.666713] niks3-server[1196]: 2026/09/25 16:53:17 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6833lb # [ 211.677695] niks3-server[1196]: 2026/09/25 16:53:17 INFO Received complete push request method=POST path=/api/pushes/85/complete6834worker1 # [ 211.700620] nix-grpc-daemon[1548]: time=2026-09-25T16:53:17.291Z level=INFO msg="Upload complete. (179ms)"6835lb # [ 211.683816] niks3-server[1196]: 2026/09/25 16:53:17 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6836worker1 # [ 211.707120] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=0 paths=1 nar_bytes_in=7526837worker1 # [ 211.718322] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:334126838worker1 # [ 211.722094] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6839worker1 # [ 211.727545] nix-daemon[856]: reaped child process 2062, status = succeeded6840worker1 # [ 211.734884] nix-daemon[856]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)6841worker1 # [ 213.172408] nix-daemon[856]: reaped child process 2071, status = succeeded6842worker1 # [ 213.283801] nix-daemon[856]: accepted connection from pid 2087, user nix-grpc-daemon (trusted)6843worker1 # [ 213.299926] nix-daemon[856]: reaped child process 2090, status = succeeded6844lb # [ 213.289572] niks3-server[1196]: 2026/09/25 16:53:18 INFO Received push request method=POST path=/api/pushes6845worker1 # [ 213.313831] nix-grpc-daemon[1548]: time=2026-09-25T16:53:18.903Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6846worker1 # [ 213.315353] nix-grpc-daemon[1548]: time=2026-09-25T16:53:18.903Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6847lb # [ 213.314048] niks3-server[1196]: 2026/09/25 16:53:18 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign6848worker1 # [ 213.335786] nix-grpc-daemon[1548]: time=2026-09-25T16:53:18.926Z level=INFO msg="Uploading 1 narinfos"6849lb # [ 213.323129] niks3-server[1196]: 2026/09/25 16:53:18 INFO Signed narinfos id=86 count=16850lb # [ 213.328334] niks3-server[1196]: 2026/09/25 16:53:18 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6851lb # [ 213.336935] niks3-server[1196]: 2026/09/25 16:53:18 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6852worker1 # [ 213.360405] nix-grpc-daemon[1548]: time=2026-09-25T16:53:18.951Z level=INFO msg="Upload complete. (216ms)"6853lb # [ 213.344214] niks3-server[1196]: 2026/09/25 16:53:18 INFO Received complete push request method=POST path=/api/pushes/86/complete6854lb # [ 213.351191] niks3-server[1196]: 2026/09/25 16:53:18 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6855lb # [ 213.354906] niks3-server[1196]: 2026/09/25 16:53:18 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6856worker1 # [ 213.371800] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33412 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=13 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06857worker1 # [ 213.381552] nix-daemon[856]: reaped child process 2069, status = succeeded6858client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.65 seconds)6859worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6860worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6861client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6862client # [ 213.460696] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6863client # Running as unit: gpu.service; invocation ID: 55425169e0804b87b6758c98fc99a6226864client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)6865lb: must succeed: curl -sf localhost:9901/clusters6866lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6867lb: must succeed: curl -sf localhost:9901/clusters6868lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6869worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6870worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)6871worker1 # [ 213.648469] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:334126872client # [ 213.636504] nix[1592]: don't know how to build these paths:6873client # [ 213.637486] nix[1592]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6874worker2 # [ 213.661826] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355386875worker1 # [ 213.673788] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:19Z level=debug event=want client=3 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06876client # [ 213.658036] nix[1592]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker6877lb: must succeed: curl -sf localhost:9901/clusters6878lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6879lb: must succeed: curl -sf localhost:9901/clusters6880lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6881worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6882worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6883client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6884lb # [ 217.055237] envoy[886]: conn peer=[2001:db8:1::1]:38828 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27166 tx=12313 ms=154576885client # [ 243.658868] nix[1592]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}6886client # [ 243.668068] nix[1592]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6887client # [ 243.692697] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6888client # [ 243.697162] systemd[1]: gpu.service: Failed with result 'exit-code'.6889client # [ 243.699170] systemd[1]: gpu.service: Consumed 167ms CPU time over 30.235s wall clock time, 35.4M memory peak, 4K written to disk, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.6890client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.22 seconds)6891client: must succeed: journalctl -u gpu -o cat6892client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)6893lb: must succeed: curl -sf localhost:9901/clusters6894lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6895lb: must succeed: curl -sf localhost:9901/clusters6896lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6897worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6898worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6899(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.33 seconds)6900subtest: low disk drains a worker and builds go to the other6901worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6902worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.18 seconds)6903worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6904worker1 # [ 245.878831] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:51Z level=info event=unhealthy reason=min_free6905worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.07 seconds)6906client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26907client # don't know how to build these paths:6908client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6909worker2 # [ 246.501414] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:355386910worker2 # [ 246.506722] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355386911worker1 # [ 246.515033] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:52Z level=debug event=want client=4 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06912worker1 # [ 246.518523] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:52Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16913worker2 # [ 246.524053] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355386914worker2 # [ 246.533832] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355386915client # copying 1 paths to 'lb:50051'...6916client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6917worker2 # [ 246.543799] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355386918worker2 # [ 246.553412] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6919worker2 # [ 246.692998] nix-daemon[842]: accepted connection from pid 2228, user nix-grpc-daemon (trusted)6920worker2 # [ 246.717669] nix-daemon[842]: reaped child process 2231, status = succeeded6921lb # [ 246.696831] niks3-server[1196]: 2026/09/25 16:53:52 INFO Received push request method=POST path=/api/pushes6922worker2 # [ 246.726194] nix-grpc-daemon[2134]: time=2026-09-25T16:53:52.320Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6923worker2 # [ 246.727881] nix-grpc-daemon[2134]: time=2026-09-25T16:53:52.320Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6924lb # [ 246.723102] niks3-server[1196]: 2026/09/25 16:53:52 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign6925lb # [ 246.732258] niks3-server[1196]: 2026/09/25 16:53:52 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6926worker2 # [ 246.756846] nix-grpc-daemon[2134]: time=2026-09-25T16:53:52.351Z level=INFO msg="Uploading 1 narinfos"6927lb # [ 246.740036] niks3-server[1196]: 2026/09/25 16:53:52 INFO Signed narinfos id=87 count=16928lb # [ 246.746173] niks3-server[1196]: 2026/09/25 16:53:52 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6929lb # [ 246.756126] niks3-server[1196]: 2026/09/25 16:53:52 INFO Received complete push request method=POST path=/api/pushes/87/complete6930worker2 # [ 246.781349] nix-grpc-daemon[2134]: time=2026-09-25T16:53:52.376Z level=INFO msg="Upload complete. (216ms)"6931lb # [ 246.764975] niks3-server[1196]: 2026/09/25 16:53:52 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6932worker2 # [ 246.788180] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35538 duration_s=0 paths=1 nar_bytes_in=5126933worker2 # [ 246.796370] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355386934worker2 # [ 246.806562] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6935worker2 # [ 246.810895] nix-daemon[842]: reaped child process 2226, status = succeeded6936client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6937client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6938worker2 # [ 246.823761] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6939lb # [ 247.064678] envoy[886]: conn peer=[2001:db8:1::1]:50532 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=300426940client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6941client # farm-a> LOG-farm-a-drain6942worker2 # [ 248.465402] nix-daemon[842]: reaped child process 2235, status = succeeded6943worker2 # [ 248.571667] nix-daemon[842]: accepted connection from pid 2251, user nix-grpc-daemon (trusted)6944lb # [ 248.574229] niks3-server[1196]: 2026/09/25 16:53:54 INFO Received push request method=POST path=/api/pushes6945worker2 # [ 248.597351] nix-daemon[842]: reaped child process 2254, status = succeeded6946worker2 # [ 248.600767] nix-grpc-daemon[2134]: time=2026-09-25T16:53:54.195Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6947worker2 # [ 248.604449] nix-grpc-daemon[2134]: time=2026-09-25T16:53:54.197Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6948lb # [ 248.594673] niks3-server[1196]: 2026/09/25 16:53:54 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6949lb # [ 248.604986] niks3-server[1196]: 2026/09/25 16:53:54 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6950worker2 # [ 248.631916] nix-grpc-daemon[2134]: time=2026-09-25T16:53:54.226Z level=INFO msg="Uploading 1 narinfos"6951lb # [ 248.612323] niks3-server[1196]: 2026/09/25 16:53:54 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign6952lb # [ 248.618760] niks3-server[1196]: 2026/09/25 16:53:54 INFO Signed narinfos id=88 count=16953lb # [ 248.622216] niks3-server[1196]: 2026/09/25 16:53:54 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6954lb # [ 248.632497] niks3-server[1196]: 2026/09/25 16:53:54 INFO Received complete push request method=POST path=/api/pushes/88/complete6955worker2 # [ 248.656623] nix-grpc-daemon[2134]: time=2026-09-25T16:53:54.251Z level=INFO msg="Upload complete. (223ms)"6956lb # [ 248.635937] niks3-server[1196]: 2026/09/25 16:53:54 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6957worker1 # [ 248.663744] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:54Z level=debug event=want client=4 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06958worker2 # [ 248.664783] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35538 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=14 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06959worker1 # [ 248.667080] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:54Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16960worker2 # [ 248.677447] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355386961worker2 # [ 248.685167] nix-daemon[842]: reaped child process 2233, status = succeeded6962worker2 # [ 248.688412] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355386963worker2 # [ 248.694440] nix-grpc-daemon[2134]: time=2026-09-25T16:53:54.289Z level=INFO msg="All 1 paths already cached"6964client # copying 1 paths to 'lb:50051'...6965client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6966worker2 # [ 248.700858] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355386967worker2 # [ 248.710527] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6968worker2 # [ 248.830850] nix-daemon[842]: accepted connection from pid 2258, user nix-grpc-daemon (trusted)6969lb # [ 248.830587] niks3-server[1196]: 2026/09/25 16:53:54 INFO Received push request method=POST path=/api/pushes6970worker2 # [ 248.855879] nix-grpc-daemon[2134]: time=2026-09-25T16:53:54.451Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6971worker2 # [ 248.858483] nix-grpc-daemon[2134]: time=2026-09-25T16:53:54.452Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6972worker2 # [ 248.860382] nix-daemon[842]: reaped child process 2261, status = succeeded6973lb # [ 248.848967] niks3-server[1196]: 2026/09/25 16:53:54 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6974worker2 # [ 248.875400] nix-grpc-daemon[2134]: time=2026-09-25T16:53:54.470Z level=INFO msg="Uploading 1 narinfos"6975lb # [ 248.856563] niks3-server[1196]: 2026/09/25 16:53:54 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign6976lb # [ 248.858950] niks3-server[1196]: 2026/09/25 16:53:54 INFO Signed narinfos id=89 count=16977lb # [ 248.866078] niks3-server[1196]: 2026/09/25 16:53:54 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6978worker2 # [ 248.895348] nix-grpc-daemon[2134]: time=2026-09-25T16:53:54.490Z level=INFO msg="Upload complete. (175ms)"6979lb # [ 248.874761] niks3-server[1196]: 2026/09/25 16:53:54 INFO Received complete push request method=POST path=/api/pushes/89/complete6980worker2 # [ 248.900127] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35538 duration_s=0 paths=1 nar_bytes_in=6326981lb # [ 248.879377] niks3-server[1196]: 2026/09/25 16:53:54 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6982worker2 # [ 248.909905] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355386983worker2 # [ 248.914939] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6984worker2 # [ 248.918595] nix-daemon[842]: reaped child process 2256, status = succeeded6985client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6986worker2 # [ 248.927912] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)6987client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6988client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6989client # farm-b> LOG-farm-b-drain6990worker2 # [ 250.361964] nix-daemon[842]: reaped child process 2265, status = succeeded6991worker2 # [ 250.482229] nix-daemon[842]: accepted connection from pid 2281, user nix-grpc-daemon (trusted)6992worker2 # [ 250.502732] nix-daemon[842]: reaped child process 2284, status = succeeded6993lb # [ 250.486903] niks3-server[1196]: 2026/09/25 16:53:56 INFO Received push request method=POST path=/api/pushes6994worker2 # [ 250.515574] nix-grpc-daemon[2134]: time=2026-09-25T16:53:56.107Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6995worker2 # [ 250.517424] nix-grpc-daemon[2134]: time=2026-09-25T16:53:56.107Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6996lb # [ 250.506046] niks3-server[1196]: 2026/09/25 16:53:56 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6997worker2 # [ 250.537012] nix-grpc-daemon[2134]: time=2026-09-25T16:53:56.132Z level=INFO msg="Uploading 1 narinfos"6998lb # [ 250.516232] niks3-server[1196]: 2026/09/25 16:53:56 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign6999lb # [ 250.518659] niks3-server[1196]: 2026/09/25 16:53:56 INFO Signed narinfos id=90 count=17000lb # [ 250.525992] niks3-server[1196]: 2026/09/25 16:53:56 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7001lb # [ 250.533075] niks3-server[1196]: 2026/09/25 16:53:56 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7002lb # [ 250.539474] niks3-server[1196]: 2026/09/25 16:53:56 INFO Received complete push request method=POST path=/api/pushes/90/complete7003worker2 # [ 250.571774] nix-grpc-daemon[2134]: time=2026-09-25T16:53:56.166Z level=INFO msg="Upload complete. (233ms)"7004lb # [ 250.551077] niks3-server[1196]: 2026/09/25 16:53:56 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7005worker1 # [ 250.579256] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:56Z level=debug event=want client=4 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07006worker2 # [ 250.580834] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35538 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=15 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=07007worker1 # [ 250.582648] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:56Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17008worker2 # [ 250.593259] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355387009worker2 # [ 250.597541] nix-daemon[842]: reaped child process 2263, status = succeeded7010worker2 # [ 250.603211] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:355387011worker2 # [ 250.612154] nix-grpc-daemon[2134]: time=2026-09-25T16:53:56.207Z level=INFO msg="All 2 paths already cached"7012client # copying 1 paths to 'lb:50051'...7013client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7014worker2 # [ 250.618836] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:355387015worker2 # [ 250.627631] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)7016worker2 # [ 250.753749] nix-daemon[842]: accepted connection from pid 2288, user nix-grpc-daemon (trusted)7017lb # [ 250.752079] niks3-server[1196]: 2026/09/25 16:53:56 INFO Received push request method=POST path=/api/pushes7018worker2 # [ 250.775317] nix-daemon[842]: reaped child process 2291, status = succeeded7019worker2 # [ 250.778201] nix-grpc-daemon[2134]: time=2026-09-25T16:53:56.372Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7020worker2 # [ 250.779762] nix-grpc-daemon[2134]: time=2026-09-25T16:53:56.372Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7021lb # [ 250.768310] niks3-server[1196]: 2026/09/25 16:53:56 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7022worker2 # [ 250.795420] nix-grpc-daemon[2134]: time=2026-09-25T16:53:56.390Z level=INFO msg="Uploading 1 narinfos"7023lb # [ 250.775650] niks3-server[1196]: 2026/09/25 16:53:56 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign7024lb # [ 250.781159] niks3-server[1196]: 2026/09/25 16:53:56 INFO Signed narinfos id=91 count=17025lb # [ 250.785187] niks3-server[1196]: 2026/09/25 16:53:56 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7026worker2 # [ 250.814785] nix-grpc-daemon[2134]: time=2026-09-25T16:53:56.409Z level=INFO msg="Upload complete. (175ms)"7027lb # [ 250.794537] niks3-server[1196]: 2026/09/25 16:53:56 INFO Received complete push request method=POST path=/api/pushes/91/complete7028lb # [ 250.798081] niks3-server[1196]: 2026/09/25 16:53:56 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7029worker2 # [ 250.820210] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35538 duration_s=0 paths=1 nar_bytes_in=7607030worker2 # [ 250.829388] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:355387031worker2 # [ 250.834620] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)7032worker2 # [ 250.838147] nix-daemon[842]: reaped child process 2286, status = succeeded7033client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7034client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7035worker2 # [ 250.849024] nix-daemon[842]: accepted connection from pid 2074, user nix-grpc-daemon (trusted)7036client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7037client # farm-top> LOG-farm-top-drain7038worker2 # [ 251.693606] nix-daemon[842]: reaped child process 2295, status = succeeded7039worker2 # [ 251.786318] nix-daemon[842]: accepted connection from pid 2311, user nix-grpc-daemon (trusted)7040lb # [ 251.784702] niks3-server[1196]: 2026/09/25 16:53:57 INFO Received push request method=POST path=/api/pushes7041worker2 # [ 251.809128] nix-daemon[842]: reaped child process 2314, status = succeeded7042worker2 # [ 251.812699] nix-grpc-daemon[2134]: time=2026-09-25T16:53:57.405Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7043worker2 # [ 251.814263] nix-grpc-daemon[2134]: time=2026-09-25T16:53:57.405Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7044lb # [ 251.807016] niks3-server[1196]: 2026/09/25 16:53:57 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7045worker2 # [ 251.834963] nix-grpc-daemon[2134]: time=2026-09-25T16:53:57.429Z level=INFO msg="Uploading 1 narinfos"7046lb # [ 251.814471] niks3-server[1196]: 2026/09/25 16:53:57 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7047lb # [ 251.821523] niks3-server[1196]: 2026/09/25 16:53:57 INFO Received sign narinfos request method=POST path=/api/pushes/92/sign7048lb # [ 251.823856] niks3-server[1196]: 2026/09/25 16:53:57 INFO Signed narinfos id=92 count=17049lb # [ 251.831765] niks3-server[1196]: 2026/09/25 16:53:57 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7050worker2 # [ 251.857148] nix-grpc-daemon[2134]: time=2026-09-25T16:53:57.452Z level=INFO msg="Upload complete. (201ms)"7051lb # [ 251.838424] niks3-server[1196]: 2026/09/25 16:53:57 INFO Received complete push request method=POST path=/api/pushes/92/complete7052lb # [ 251.843059] niks3-server[1196]: 2026/09/25 16:53:57 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7053worker2 # [ 251.863703] nix-grpc-daemon[2074]: ts=2026-09-25T16:53:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35538 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=16 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=07054worker2 # [ 251.878095] nix-daemon[842]: reaped child process 2293, status = succeeded7055client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.57 seconds)7056client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7057client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.14 seconds)7058worker1: must succeed: rm /nix/.rw-store/fill7059worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7060worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7061worker1 # [ 252.881168] nix-grpc-daemon[1404]: ts=2026-09-25T16:53:58Z level=info event=healthy reason=min_free7062worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.06 seconds)7063(finished: subtest: low disk drains a worker and builds go to the other, in 9.17 seconds)7064(finished: run the VM test script, in 254.09 seconds)7065test script finished in 254.23s7066cleanup7067kill QemuMachine (pid 49)7068client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7069client # [2026-09-25T16:53:58Z INFO virtiofsd] Client disconnected, shutting down7070client # [2026-09-25T16:53:58Z INFO virtiofsd] Client disconnected, shutting down7071client # [2026-09-25T16:53:58Z INFO virtiofsd] Client disconnected, shutting down7072kill QemuMachine (pid 50)7073lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7074lb # [2026-09-25T16:53:58Z INFO virtiofsd] Client disconnected, shutting down7075lb # [2026-09-25T16:53:58Z INFO virtiofsd] Client disconnected, shutting down7076lb # [2026-09-25T16:53:58Z INFO virtiofsd] Client disconnected, shutting down7077kill QemuMachine (pid 52)7078worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7079worker1 # [2026-09-25T16:53:59Z INFO virtiofsd] Client disconnected, shutting down7080worker1 # [2026-09-25T16:53:59Z INFO virtiofsd] Client disconnected, shutting down7081worker1 # [2026-09-25T16:53:59Z INFO virtiofsd] Client disconnected, shutting down7082kill QemuMachine (pid 51)7083worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7084worker2 # [2026-09-25T16:53:59Z INFO virtiofsd] Client disconnected, shutting down7085worker2 # [2026-09-25T16:53:59Z INFO virtiofsd] Client disconnected, shutting down7086worker2 # [2026-09-25T16:53:59Z INFO virtiofsd] Client disconnected, shutting down7087(finished: cleanup, in 1.00 seconds)7088additionally exposed symbols:7089 client, lb, worker1, worker2,7090 vlan1,7091 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh