niks3-go-unit-tests
checks.aarch64-darwin.go-unit-tests
· build #260
· raw
1Running client tests...2=== RUN TestDoServerRequestAttachesToken3=== PAUSE TestDoServerRequestAttachesToken4=== RUN TestRegisterUploadedObjectReusesConnections5=== PAUSE TestRegisterUploadedObjectReusesConnections6=== RUN TestCaseHackSuffix7=== PAUSE TestCaseHackSuffix8=== RUN TestFilterOversizedClosures9=== PAUSE TestFilterOversizedClosures10=== RUN TestUploadMultipart_PartsInParallel11=== PAUSE TestUploadMultipart_PartsInParallel12=== RUN TestPartSizeForNAR13=== PAUSE TestPartSizeForNAR14=== RUN TestUploadMultipart_SupersededByPeer15=== PAUSE TestUploadMultipart_SupersededByPeer16=== RUN TestDumpPathCaseHackMatchesNix17--- PASS: TestDumpPathCaseHackMatchesNix (0.05s)18=== RUN TestDumpPathCaseHackCollision19--- PASS: TestDumpPathCaseHackCollision (0.00s)20=== RUN TestDumpPathMatchesNix21=== PAUSE TestDumpPathMatchesNix22=== RUN TestDumpPathSingleFile23=== PAUSE TestDumpPathSingleFile24=== RUN TestDumpPathWriterError25=== PAUSE TestDumpPathWriterError26=== RUN TestEncodeNixBase3227=== PAUSE TestEncodeNixBase3228=== RUN TestEncodeNixBase32WithRealHash29=== PAUSE TestEncodeNixBase32WithRealHash30=== RUN TestConvertHashToNix3231=== PAUSE TestConvertHashToNix3232=== RUN TestGetStorePathHash33=== PAUSE TestGetStorePathHash34=== RUN TestPathInfoHashCompatibility35=== PAUSE TestPathInfoHashCompatibility36=== RUN TestParsePathInfoJSON37=== PAUSE TestParsePathInfoJSON38=== RUN TestParsePathInfoJSONMultiplePaths39=== PAUSE TestParsePathInfoJSONMultiplePaths40=== RUN TestPathInfoCACompatibility41=== PAUSE TestPathInfoCACompatibility42=== RUN TestRateLimiterFeedback43=== PAUSE TestRateLimiterFeedback44=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess45=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess46=== RUN TestResolveStorePath47=== PAUSE TestResolveStorePath48=== RUN TestDoWithRetry_BodyReplayedViaGetBody49=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody50=== RUN TestShellSplit51=== PAUSE TestShellSplit52=== RUN TestShellSplitErrors53=== PAUSE TestShellSplitErrors54=== RUN TestStreamPushReportsEveryPath55=== PAUSE TestStreamPushReportsEveryPath56=== RUN TestStreamPushBatchesUnderLoad57=== PAUSE TestStreamPushBatchesUnderLoad58=== RUN TestStreamPushIsolatesFailures59=== PAUSE TestStreamPushIsolatesFailures60=== RUN TestStreamPushGivesUpOnDeadServer61=== PAUSE TestStreamPushGivesUpOnDeadServer62=== RUN TestStreamPushRequestLine63=== PAUSE TestStreamPushRequestLine64=== RUN TestStreamPushReportsSignatures65=== PAUSE TestStreamPushReportsSignatures66=== RUN TestClientSignaturesByStorePath67=== PAUSE TestClientSignaturesByStorePath68=== RUN TestSetClientTLS69=== PAUSE TestSetClientTLS70=== RUN TestSetClientTLSDoesNotMutateDefaultTransport71=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport72=== RUN TestSetClientTLSErrors73=== PAUSE TestSetClientTLSErrors74=== RUN TestStaticToken75=== PAUSE TestStaticToken76=== RUN TestFileTokenReadsAndCaches77=== PAUSE TestFileTokenReadsAndCaches78=== RUN TestFileTokenMissing79=== PAUSE TestFileTokenMissing80=== RUN TestFileTokenEmpty81=== PAUSE TestFileTokenEmpty82=== RUN TestScriptTokenNoExpiryRerunsEveryCall83=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall84=== RUN TestScriptTokenCachesUntilRefresh85=== PAUSE TestScriptTokenCachesUntilRefresh86=== RUN TestScriptTokenEmptyToken87=== PAUSE TestScriptTokenEmptyToken88=== RUN TestScriptTokenBadJSON89=== PAUSE TestScriptTokenBadJSON90=== RUN TestScriptTokenScriptFails91=== PAUSE TestScriptTokenScriptFails92=== RUN TestScriptTokenEmptyCommand93=== PAUSE TestScriptTokenEmptyCommand94=== CONT TestDoServerRequestAttachesToken95=== CONT TestStreamPushRequestLine96=== CONT TestStreamPushReportsEveryPath97=== CONT TestDumpPathMatchesNix98=== CONT TestScriptTokenEmptyCommand99--- PASS: TestScriptTokenEmptyCommand (0.00s)100=== CONT TestScriptTokenScriptFails101=== CONT TestEncodeNixBase32WithRealHash102--- PASS: TestEncodeNixBase32WithRealHash (0.00s)103=== CONT TestShellSplitErrors104=== CONT TestPathInfoHashCompatibility105=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)106=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)107=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon108=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon109=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI110=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI111=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512112=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512113=== CONT TestStreamPushGivesUpOnDeadServer114=== CONT TestDoWithRetry_BodyReplayedViaGetBody115=== CONT TestStreamPushIsolatesFailures116=== CONT TestStreamPushBatchesUnderLoad117--- PASS: TestShellSplitErrors (0.00s)118=== CONT TestShellSplit119--- PASS: TestShellSplit (0.00s)120=== CONT TestResolveStorePath121--- PASS: TestStreamPushReportsEveryPath (0.00s)122=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess1232026/09/23 12:21:43 ERROR Upload failed error="connection refused" count=201242026/09/23 12:21:43 ERROR Server seems unavailable, giving up on batch untried=171252026/09/23 12:21:43 ERROR Upload failed error=boom count=1126--- PASS: TestStreamPushGivesUpOnDeadServer (0.00s)127=== CONT TestRateLimiterFeedback128=== RUN TestRateLimiterFeedback/429_enables_limiter129=== PAUSE TestRateLimiterFeedback/429_enables_limiter130=== RUN TestRateLimiterFeedback/503_enables_limiter131=== PAUSE TestRateLimiterFeedback/503_enables_limiter132=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter133=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter134=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter135=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter1362026/09/23 12:21:43 WARN Rate limiter enabled after throttle name=server-test rate=5137=== CONT TestPathInfoCACompatibility138=== RUN TestPathInfoCACompatibility/null_ca_field139=== PAUSE TestPathInfoCACompatibility/null_ca_field140=== RUN TestPathInfoCACompatibility/old_string_format_-_text141=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text142=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive143=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive144=== RUN TestPathInfoCACompatibility/new_structured_format_-_text145=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text146=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method147=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method148=== CONT TestParsePathInfoJSONMultiplePaths149=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths1502026/09/23 12:21:43 ERROR Upload failed error="bad path" count=3151=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths152--- PASS: TestStreamPushIsolatesFailures (0.00s)153=== CONT TestParsePathInfoJSON154=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths1552026/09/23 12:21:43 WARN Rate limiter enabled after throttle name=server-test rate=5156=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths1572026/09/23 12:21:43 WARN Request returned retryable status, retrying attempt=1 max_attempts=6 backoff=0s status=503 url=http://127.0.0.1:56583158=== RUN TestParsePathInfoJSON/Nix_format159=== CONT TestUploadMultipart_PartsInParallel160=== PAUSE TestParsePathInfoJSON/Nix_format161=== RUN TestParsePathInfoJSON/Lix_format162=== PAUSE TestParsePathInfoJSON/Lix_format163=== RUN TestParsePathInfoJSON/empty_input164=== PAUSE TestParsePathInfoJSON/empty_input165=== RUN TestParsePathInfoJSON/whitespace_only166=== PAUSE TestParsePathInfoJSON/whitespace_only167=== RUN TestParsePathInfoJSON/invalid_JSON168=== PAUSE TestParsePathInfoJSON/invalid_JSON169--- PASS: TestDoServerRequestAttachesToken (0.00s)170=== CONT TestPartSizeForNAR171=== RUN TestPartSizeForNAR/zero_stays_at_minimum1722026/09/23 12:21:43 WARN Rate limiter backed off name=server-test rate=51732026/09/23 12:21:43 WARN Request returned retryable status, retrying attempt=2 max_attempts=6 backoff=0s status=503 url=http://127.0.0.1:56583174--- PASS: TestResolveStorePath (0.00s)175=== CONT TestUploadMultipart_SupersededByPeer176=== RUN TestUploadMultipart_SupersededByPeer/exists177=== CONT TestDumpPathWriterError178--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.01s)179=== CONT TestEncodeNixBase32180=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum181=== PAUSE TestUploadMultipart_SupersededByPeer/exists182=== RUN TestPartSizeForNAR/small_stays_at_minimum183=== PAUSE TestPartSizeForNAR/small_stays_at_minimum184=== RUN TestEncodeNixBase32/test_string_hash185=== RUN TestUploadMultipart_SupersededByPeer/missing186=== PAUSE TestEncodeNixBase32/test_string_hash187=== RUN TestEncodeNixBase32/empty_input188=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum189=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum190=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts191=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts192=== PAUSE TestUploadMultipart_SupersededByPeer/missing193=== PAUSE TestEncodeNixBase32/empty_input194--- PASS: TestScriptTokenScriptFails (0.01s)195=== CONT TestStaticToken196=== RUN TestPartSizeForNAR/1_TiB197=== PAUSE TestPartSizeForNAR/1_TiB198=== RUN TestPartSizeForNAR/5_TiB_S3_max_object199=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object200=== RUN TestPartSizeForNAR/capped_at_5_GiB201=== PAUSE TestPartSizeForNAR/capped_at_5_GiB202=== CONT TestDumpPathSingleFile203--- PASS: TestStaticToken (0.00s)204=== CONT TestScriptTokenBadJSON205=== CONT TestScriptTokenEmptyToken206=== CONT TestScriptTokenCachesUntilRefresh207--- PASS: TestStreamPushRequestLine (0.02s)208=== CONT TestScriptTokenNoExpiryRerunsEveryCall209=== CONT TestFileTokenEmpty210--- PASS: TestScriptTokenBadJSON (0.01s)211--- PASS: TestScriptTokenEmptyToken (0.01s)212=== CONT TestFileTokenMissing213--- PASS: TestFileTokenMissing (0.00s)214=== CONT TestFileTokenReadsAndCaches215--- PASS: TestFileTokenEmpty (0.00s)216=== CONT TestClientSignaturesByStorePath217--- PASS: TestClientSignaturesByStorePath (0.00s)218=== CONT TestSetClientTLSErrors219--- PASS: TestFileTokenReadsAndCaches (0.00s)220=== CONT TestSetClientTLSDoesNotMutateDefaultTransport221=== RUN TestSetClientTLSErrors/missing_cert_file222=== PAUSE TestSetClientTLSErrors/missing_cert_file223=== RUN TestSetClientTLSErrors/missing_key_file224=== PAUSE TestSetClientTLSErrors/missing_key_file225=== RUN TestSetClientTLSErrors/missing_ca_file226=== PAUSE TestSetClientTLSErrors/missing_ca_file227=== RUN TestSetClientTLSErrors/invalid_ca_file228=== PAUSE TestSetClientTLSErrors/invalid_ca_file229=== CONT TestSetClientTLS230--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.00s)231=== CONT TestConvertHashToNix32232=== RUN TestConvertHashToNix32/SRI_format_to_Nix32233=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32234=== RUN TestConvertHashToNix32/already_Nix32_format235=== PAUSE TestConvertHashToNix32/already_Nix32_format236=== RUN TestConvertHashToNix32/invalid_format237=== PAUSE TestConvertHashToNix32/invalid_format238=== CONT TestGetStorePathHash239=== RUN TestGetStorePathHash/valid_store_path240=== PAUSE TestGetStorePathHash/valid_store_path241=== RUN TestGetStorePathHash/basename_without_hyphen_should_error242=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error243=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error244=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error245=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error246=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error247=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)248=== CONT TestStreamPushReportsSignatures2492026/09/23 12:21:43 ERROR Upload failed error=boom count=1250--- PASS: TestStreamPushReportsSignatures (0.00s)251=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI252=== CONT TestCaseHackSuffix253=== RUN TestSetClientTLS/rejects_connection_without_client_cert254=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert255=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA256=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA257=== RUN TestSetClientTLS/preserves_debug_logging_transport258=== PAUSE TestSetClientTLS/preserves_debug_logging_transport259=== CONT TestFilterOversizedClosures260=== RUN TestFilterOversizedClosures/no_limit_keeps_everything261=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything262=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped263=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped264=== RUN TestFilterOversizedClosures/all_closures_skipped265=== PAUSE TestFilterOversizedClosures/all_closures_skipped266=== CONT TestRegisterUploadedObjectReusesConnections267--- PASS: TestScriptTokenCachesUntilRefresh (0.04s)268=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon269=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512270--- PASS: TestPathInfoHashCompatibility (0.00s)271 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)272 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)273 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)274 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)275=== CONT TestRateLimiterFeedback/429_enables_limiter2762026/09/23 12:21:43 WARN Rate limiter enabled after throttle name=server-test rate=52772026/09/23 12:21:43 WARN Request returned retryable status, retrying attempt=1 max_attempts=2 backoff=0s status=429 url=http://127.0.0.1:56659278--- PASS: TestDumpPathWriterError (0.04s)279=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter2802026/09/23 12:21:43 WARN Rate limiter backed off name=server-test rate=5281=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter282=== CONT TestPathInfoCACompatibility/null_ca_field283=== CONT TestRateLimiterFeedback/503_enables_limiter284=== CONT TestPathInfoCACompatibility/new_structured_format_-_text285=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method286=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive287=== CONT TestPathInfoCACompatibility/old_string_format_-_text288--- PASS: TestPathInfoCACompatibility (0.00s)289 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)290 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)291 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)292 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)293 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)294=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths295=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths296--- PASS: TestParsePathInfoJSONMultiplePaths (0.00s)297 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)298 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)299=== CONT TestParsePathInfoJSON/Nix_format300=== CONT TestParsePathInfoJSON/invalid_JSON301=== CONT TestParsePathInfoJSON/whitespace_only302=== CONT TestParsePathInfoJSON/empty_input303=== CONT TestParsePathInfoJSON/Lix_format304--- PASS: TestParsePathInfoJSON (0.00s)305 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)306 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)307 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)308 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)309 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)310=== CONT TestUploadMultipart_SupersededByPeer/exists3112026/09/23 12:21:43 WARN Rate limiter enabled after throttle name=server-test rate=53122026/09/23 12:21:43 WARN Request returned retryable status, retrying attempt=1 max_attempts=2 backoff=0s status=503 url=http://127.0.0.1:566653132026/09/23 12:21:43 WARN Rate limiter backed off name=server-test rate=5314--- PASS: TestRateLimiterFeedback (0.00s)315 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.00s)316 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s)317 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.00s)318 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s)319=== CONT TestEncodeNixBase32/test_string_hash320=== CONT TestUploadMultipart_SupersededByPeer/missing321=== CONT TestEncodeNixBase32/empty_input322--- PASS: TestEncodeNixBase32 (0.00s)323 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)324 --- PASS: TestEncodeNixBase32/empty_input (0.00s)325=== CONT TestPartSizeForNAR/zero_stays_at_minimum326=== CONT TestPartSizeForNAR/capped_at_5_GiB327=== CONT TestPartSizeForNAR/5_TiB_S3_max_object328=== CONT TestPartSizeForNAR/1_TiB329=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts330=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum331=== CONT TestPartSizeForNAR/small_stays_at_minimum332--- PASS: TestPartSizeForNAR (0.00s)333 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)334 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)335 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)336 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)337 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)338 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)339 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)340=== CONT TestSetClientTLSErrors/missing_cert_file341=== CONT TestConvertHashToNix32/SRI_format_to_Nix32342=== CONT TestConvertHashToNix32/invalid_format343=== CONT TestConvertHashToNix32/already_Nix32_format344--- PASS: TestConvertHashToNix32 (0.00s)345 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)346 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)347 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)348=== CONT TestSetClientTLSErrors/missing_ca_file349=== CONT TestSetClientTLSErrors/invalid_ca_file350--- PASS: TestUploadMultipart_SupersededByPeer (0.00s)351 --- PASS: TestUploadMultipart_SupersededByPeer/exists (0.00s)352 --- PASS: TestUploadMultipart_SupersededByPeer/missing (0.00s)353=== CONT TestSetClientTLSErrors/missing_key_file354=== CONT TestGetStorePathHash/valid_store_path355=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error356=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error357=== CONT TestGetStorePathHash/basename_without_hyphen_should_error358--- PASS: TestGetStorePathHash (0.00s)359 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)360 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)361 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)362 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)363=== CONT TestSetClientTLS/rejects_connection_without_client_cert364=== CONT TestSetClientTLS/preserves_debug_logging_transport365--- PASS: TestRegisterUploadedObjectReusesConnections (0.02s)366=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA367--- PASS: TestSetClientTLSErrors (0.00s)368 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)369 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)370 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)371 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)372=== CONT TestFilterOversizedClosures/no_limit_keeps_everything373=== CONT TestFilterOversizedClosures/all_closures_skipped3742026/09/23 12:21:43 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50375=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3762026/09/23 12:21:43 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000377--- PASS: TestFilterOversizedClosures (0.00s)378 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)379 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)380 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)381--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.04s)3822026/09/23 12:21:43 http: TLS handshake error from 127.0.0.1:56671: remote error: tls: bad certificate383--- PASS: TestSetClientTLS (0.01s)384 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.00s)385 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.00s)386 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.01s)387--- PASS: TestDumpPathSingleFile (0.05s)388--- PASS: TestCaseHackSuffix (0.05s)389--- PASS: TestDumpPathMatchesNix (0.08s)390--- PASS: TestStreamPushBatchesUnderLoad (0.10s)391--- PASS: TestUploadMultipart_PartsInParallel (0.61s)392--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)393PASS394Running server tests...395The files belonging to this database system will be owned by user "_nixbld1".396This user must also own the server process.397398The database cluster will be initialized with locale "C".399The default database encoding has accordingly been set to "SQL_ASCII".400The default text search configuration will be set to "english".401402Data page checksums are enabled.403404creating directory /nix/var/nix/builds/nix-43626-243968142/postgres1398793126/data ... ok405creating subdirectories ... ok406selecting dynamic shared memory implementation ... posix407selecting default "max_connections" ... 100408selecting default "shared_buffers" ... 128MB409selecting default time zone ... UTC410creating configuration files ... ok411running bootstrap script ... ok412performing post-bootstrap initialization ... ok413syncing data to disk ... ok414415initdb: warning: enabling "trust" authentication for local connections416initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.417418Success. You can now start the database server using:419420 pg_ctl -D /nix/var/nix/builds/nix-43626-243968142/postgres1398793126/data -l logfile start421422/nix/var/nix/builds/nix-43626-243968142/postgres1398793126:5432 - no response4232026-09-23 12:21:44.945 UTC [43664] LOG: starting PostgreSQL 18.6 on aarch64-apple-darwin25.6.0, compiled by clang version 21.1.8, 64-bit4242026-09-23 12:21:44.945 UTC [43664] LOG: listening on Unix socket "/nix/var/nix/builds/nix-43626-243968142/postgres1398793126/.s.PGSQL.5432"4252026-09-23 12:21:44.947 UTC [43671] LOG: database system was shut down at 2026-09-23 12:21:44 UTC4262026-09-23 12:21:44.948 UTC [43664] LOG: database system is ready to accept connections427/nix/var/nix/builds/nix-43626-243968142/postgres1398793126:5432 - accepting connections428{"timestamp":"2026-09-23T12:21:45.161299Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"e14791cc-02e6-492f-82ba-0895944f084c","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":463,"threadName":"rustfs-worker","threadId":"ThreadId(10)"}429=== RUN TestService_AuthMiddleware430=== PAUSE TestService_AuthMiddleware431=== RUN TestService_AuthMiddleware_MTLSProxyHeader432=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader433=== RUN TestService_AuthMiddleware_MTLSBoundSubjects434=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects435=== RUN TestService_ReadAuthMiddleware436=== PAUSE TestService_ReadAuthMiddleware437=== RUN TestService_AuthMiddleware_OIDC438=== PAUSE TestService_AuthMiddleware_OIDC439=== RUN TestService_RequireScope_OIDC440=== PAUSE TestService_RequireScope_OIDC441=== RUN TestService_ReadScope_PublicByDefault442=== PAUSE TestService_ReadScope_PublicByDefault443=== RUN TestCacheConfigHandler444=== PAUSE TestCacheConfigHandler445=== RUN TestCacheStatsHandler446=== PAUSE TestCacheStatsHandler447=== RUN TestClientCADerivations448=== PAUSE TestClientCADerivations449=== RUN TestClientErrorHandling450=== PAUSE TestClientErrorHandling451=== RUN TestClientIntegration452=== PAUSE TestClientIntegration453=== RUN TestClientMultipleUploads454=== PAUSE TestClientMultipleUploads455=== RUN TestClientWithDependencies456=== PAUSE TestClientWithDependencies457=== RUN TestClientSharedPathCommittedMidPush458=== PAUSE TestClientSharedPathCommittedMidPush459=== RUN TestPinProtectsFromGC460=== PAUSE TestPinProtectsFromGC461=== RUN TestClientPushesUseOnePush462=== PAUSE TestClientPushesUseOnePush463=== RUN TestClientFallsBackToClosures464=== PAUSE TestClientFallsBackToClosures465=== RUN TestResolveDBConnectionString466=== PAUSE TestResolveDBConnectionString467=== RUN TestLeadElectsOneAndHandsOver468=== PAUSE TestLeadElectsOneAndHandsOver469=== RUN TestLeadIncumbentWinsAfterRestart4702026-09-23 12:21:45.602 UTC [43701] ERROR: relation "goose_db_version" does not exist at character 364712026-09-23 12:21:45.602 UTC [43701] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4722026/09/23 12:21:45 OK 20241026095416_initial_model.sql (9.77ms)4732026/09/23 12:21:45 OK 20251210153512_drop_unused_gin_index.sql (6.99ms)4742026/09/23 12:21:45 OK 20251218171726_add_pins.sql (5.97ms)4752026/09/23 12:21:45 OK 20260628120000_add_object_size_and_stats.sql (11.18ms)4762026/09/23 12:21:45 OK 20260905000000_add_claims.sql (9.34ms)4772026/09/23 12:21:45 OK 20260920000000_drop_claims.sql (10.68ms)4782026/09/23 12:21:45 OK 20260923120000_add_pushes.sql (1.44ms)4792026/09/23 12:21:45 goose: successfully migrated database to version: 202609231200004802026/09/23 12:21:45 OK 1_commit_pending_closure.sql (2.85ms)4812026/09/23 12:21:45 OK 2_object_stats_trigger.sql (634.54µs)4822026/09/23 12:21:45 OK 3_commit_push.sql (421.17µs)4832026/09/23 12:21:45 goose: up to current file version: 34842026/09/23 12:21:45 INFO lead: acquired remote=192.0.2.1:12344852026/09/23 12:21:46 INFO lead: released remote=192.0.2.1:12344862026/09/23 12:21:46 INFO lead: acquired remote=192.0.2.1:12344872026/09/23 12:21:46 INFO lead: released remote=192.0.2.1:1234488--- PASS: TestLeadIncumbentWinsAfterRestart (1.18s)489=== RUN TestLeadEndsOnShutdown490=== PAUSE TestLeadEndsOnShutdown491=== RUN TestGCAdvisoryLockBlocksConcurrentRun4922026-09-23 12:21:46.822 UTC [43705] ERROR: relation "goose_db_version" does not exist at character 364932026-09-23 12:21:46.822 UTC [43705] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4942026/09/23 12:21:46 OK 20241026095416_initial_model.sql (59.31ms)4952026/09/23 12:21:46 OK 20251210153512_drop_unused_gin_index.sql (8.88ms)4962026/09/23 12:21:46 OK 20251218171726_add_pins.sql (16.9ms)4972026/09/23 12:21:46 OK 20260628120000_add_object_size_and_stats.sql (9.58ms)4982026/09/23 12:21:46 OK 20260905000000_add_claims.sql (25.23ms)4992026/09/23 12:21:46 OK 20260920000000_drop_claims.sql (18.3ms)5002026/09/23 12:21:46 OK 20260923120000_add_pushes.sql (2.75ms)5012026/09/23 12:21:46 goose: successfully migrated database to version: 202609231200005022026/09/23 12:21:46 OK 1_commit_pending_closure.sql (3.99ms)5032026/09/23 12:21:46 OK 2_object_stats_trigger.sql (895.88µs)5042026/09/23 12:21:47 OK 3_commit_push.sql (820.79µs)5052026/09/23 12:21:47 goose: up to current file version: 3506--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (0.66s)507=== RUN TestGCBugBareHashReferences508=== PAUSE TestGCBugBareHashReferences509=== RUN TestGCMetrics510=== PAUSE TestGCMetrics511=== RUN TestGCTaskStore_StartNew512=== PAUSE TestGCTaskStore_StartNew513=== RUN TestGCTaskStore_DeduplicateSameParams514=== PAUSE TestGCTaskStore_DeduplicateSameParams515=== RUN TestGCTaskStore_ConflictDifferentParams516=== PAUSE TestGCTaskStore_ConflictDifferentParams517=== RUN TestGCTaskStore_GetEmpty518=== PAUSE TestGCTaskStore_GetEmpty519=== RUN TestGCTaskStore_GetReturnsLatest520=== PAUSE TestGCTaskStore_GetReturnsLatest521=== RUN TestGCTaskStore_CompletedAllowsNewTask522=== PAUSE TestGCTaskStore_CompletedAllowsNewTask523=== RUN TestGCTaskStore_PhaseUpdates524=== PAUSE TestGCTaskStore_PhaseUpdates525=== RUN TestGCTaskStore_Fail526=== PAUSE TestGCTaskStore_Fail527=== RUN TestGracefulShutdownDrainsInflight528=== PAUSE TestGracefulShutdownDrainsInflight529=== RUN TestService_healthCheckHandler530=== PAUSE TestService_healthCheckHandler531=== RUN TestService_readinessHandler532=== PAUSE TestService_readinessHandler533=== RUN TestGenerateLandingPage534=== PAUSE TestGenerateLandingPage535=== RUN TestCacheConfigHandlerMaxNarSize536=== PAUSE TestCacheConfigHandlerMaxNarSize537=== RUN TestCreatePendingClosureRejectsOversizedNAR538=== PAUSE TestCreatePendingClosureRejectsOversizedNAR539=== RUN TestNARDeduplicationMetadataUploadBug540=== PAUSE TestNARDeduplicationMetadataUploadBug541=== RUN TestMetricsInventory542=== PAUSE TestMetricsInventory543=== RUN TestService_NativeMTLS544=== PAUSE TestService_NativeMTLS545=== RUN TestServerTLSConfig546=== PAUSE TestServerTLSConfig547=== RUN TestMultipartCleanup548=== PAUSE TestMultipartCleanup549=== RUN TestObjectStatsTrigger550=== PAUSE TestObjectStatsTrigger551=== RUN TestOrphanedObjectsGC552=== PAUSE TestOrphanedObjectsGC553=== RUN TestOrphanedObjectsGCStressTest554=== PAUSE TestOrphanedObjectsGCStressTest555=== RUN TestResurrectedObjectNotDeleted556=== PAUSE TestResurrectedObjectNotDeleted557=== RUN TestCreatePin_ReservedPins558=== PAUSE TestCreatePin_ReservedPins559=== RUN TestParseSingleRange560=== PAUSE TestParseSingleRange561=== RUN TestProxyHeadersOnlyTrustedOnSocket562=== PAUSE TestProxyHeadersOnlyTrustedOnSocket563=== RUN TestIsValidCachePath564=== PAUSE TestIsValidCachePath565=== RUN TestReadProxyNarinfo566=== PAUSE TestReadProxyNarinfo567=== RUN TestReadProxyNarinfoAlreadyDecompressed568=== PAUSE TestReadProxyNarinfoAlreadyDecompressed569=== RUN TestReadProxyNarStreaming570=== PAUSE TestReadProxyNarStreaming571=== RUN TestReadProxy404572=== PAUSE TestReadProxy404573=== RUN TestReadProxyInvalidPath574=== PAUSE TestReadProxyInvalidPath575=== RUN TestReadProxyHead576=== PAUSE TestReadProxyHead577=== RUN TestReadProxyConditionalGet578=== PAUSE TestReadProxyConditionalGet579=== RUN TestReadProxyRootRedirectsToIndexHTML580=== PAUSE TestReadProxyRootRedirectsToIndexHTML581=== RUN TestReadProxyDisabled582=== PAUSE TestReadProxyDisabled583=== RUN TestReadRedirectNar584=== PAUSE TestReadRedirectNar585=== RUN TestReadRedirectKeepsNarinfoProxied586=== PAUSE TestReadRedirectKeepsNarinfoProxied587=== RUN TestReadProxyRangeRequest588=== PAUSE TestReadProxyRangeRequest589=== RUN TestReadRedirectUsesPublicS3URL590=== PAUSE TestReadRedirectUsesPublicS3URL591=== RUN TestPush_OverlappingRootsStoreOneRowPerKey592=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey593=== RUN TestPush_CompleteCommitsEveryRoot594=== PAUSE TestPush_CompleteCommitsEveryRoot595=== RUN TestPush_CommitFailsWhenSkippedKeyWasCollected596=== PAUSE TestPush_CommitFailsWhenSkippedKeyWasCollected597=== RUN TestPush_RejectsBadRequests598=== PAUSE TestPush_RejectsBadRequests599=== RUN TestPush_SignsNarinfosOfItsPendingObjects600=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects601=== RUN TestRedundantMultipartUpload602=== PAUSE TestRedundantMultipartUpload603=== RUN TestCompleteMultipartUpload_ErrorButObjectExists604=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists605=== RUN TestCompletedNarNotReofferedAcrossClosures606=== PAUSE TestCompletedNarNotReofferedAcrossClosures607=== RUN TestPresignedUploadRegisteredBeforeCommit608=== PAUSE TestPresignedUploadRegisteredBeforeCommit609=== RUN TestService_Rustfstest610=== PAUSE TestService_Rustfstest611=== RUN TestParseSize612=== PAUSE TestParseSize613=== RUN TestSkippedUploadsHandler614=== PAUSE TestSkippedUploadsHandler615=== RUN TestSystemdListenerNotActivated616--- PASS: TestSystemdListenerNotActivated (0.00s)617=== RUN TestWatchdogBeatsWhenHealthy618--- PASS: TestWatchdogBeatsWhenHealthy (0.03s)619=== RUN TestWatchdogSkipsWhenUnhealthy6202026/09/23 12:21:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6212026/09/23 12:21:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6222026/09/23 12:21:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6232026/09/23 12:21:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6242026/09/23 12:21:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6252026/09/23 12:21:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6262026/09/23 12:21:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6272026/09/23 12:21:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6282026/09/23 12:21:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6292026/09/23 12:21:47 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"630--- PASS: TestWatchdogSkipsWhenUnhealthy (0.22s)631=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle632=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle633=== RUN TestProxyWriteTimeout634=== PAUSE TestProxyWriteTimeout635=== RUN TestIsValidUploadKey636=== PAUSE TestIsValidUploadKey637=== RUN TestUploadHandlersRejectInvalidKeys638=== PAUSE TestUploadHandlersRejectInvalidKeys639=== RUN TestUploadHandlersRejectOversizedBody640=== PAUSE TestUploadHandlersRejectOversizedBody641=== RUN TestService_cleanupPendingClosuresHandler642=== PAUSE TestService_cleanupPendingClosuresHandler643=== RUN TestService_createPendingClosureHandler644=== PAUSE TestService_createPendingClosureHandler645=== RUN TestService_verifyS3Integrity646=== PAUSE TestService_verifyS3Integrity647=== RUN TestCompleteMultipartUnregistered648=== PAUSE TestCompleteMultipartUnregistered649=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT650=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT651=== CONT TestReadProxyHead652=== CONT TestService_AuthMiddleware653=== CONT TestCompleteMultipartUnregistered654=== CONT TestIsValidUploadKey655=== RUN TestIsValidUploadKey/narinfo656=== PAUSE TestIsValidUploadKey/narinfo657=== RUN TestIsValidUploadKey/nar_zst658=== PAUSE TestIsValidUploadKey/nar_zst659=== RUN TestIsValidUploadKey/nar_xz660=== PAUSE TestIsValidUploadKey/nar_xz661=== RUN TestIsValidUploadKey/nar_plain662=== PAUSE TestIsValidUploadKey/nar_plain663=== CONT TestSkippedUploadsHandler664=== CONT TestService_Rustfstest665=== RUN TestIsValidUploadKey/listing666=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT667=== CONT TestService_cleanupPendingClosuresHandler668=== CONT TestUploadHandlersRejectOversizedBody669=== CONT TestCompletedNarNotReofferedAcrossClosures670=== PAUSE TestIsValidUploadKey/listing671=== RUN TestIsValidUploadKey/build_log672=== PAUSE TestIsValidUploadKey/build_log673=== RUN TestIsValidUploadKey/build_log_home-manager_file674=== PAUSE TestIsValidUploadKey/build_log_home-manager_file675=== RUN TestIsValidUploadKey/build_log_plus_in_name676=== PAUSE TestIsValidUploadKey/build_log_plus_in_name677=== RUN TestIsValidUploadKey/build_log_question_mark678=== PAUSE TestIsValidUploadKey/build_log_question_mark679=== RUN TestIsValidUploadKey/build_log_equals680=== PAUSE TestIsValidUploadKey/build_log_equals681=== RUN TestIsValidUploadKey/realisation682=== PAUSE TestIsValidUploadKey/realisation683=== RUN TestIsValidUploadKey/realisation_plus_in_output684=== PAUSE TestIsValidUploadKey/realisation_plus_in_output685=== RUN TestIsValidUploadKey/nix-cache-info686=== PAUSE TestIsValidUploadKey/nix-cache-info687=== RUN TestIsValidUploadKey/index.html688=== PAUSE TestIsValidUploadKey/index.html689=== RUN TestIsValidUploadKey/narinfo_key,_nar_type690=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type691=== RUN TestIsValidUploadKey/nar_key,_narinfo_type692=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type693=== RUN TestIsValidUploadKey/listing_key,_narinfo_type694=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type695=== RUN TestIsValidUploadKey/traversal696=== PAUSE TestIsValidUploadKey/traversal697=== RUN TestIsValidUploadKey/traversal_nar698=== PAUSE TestIsValidUploadKey/traversal_nar699=== RUN TestIsValidUploadKey/absolute700=== PAUSE TestIsValidUploadKey/absolute701=== RUN TestIsValidUploadKey/empty_key702=== PAUSE TestIsValidUploadKey/empty_key703=== RUN TestIsValidUploadKey/unknown_type704=== PAUSE TestIsValidUploadKey/unknown_type705=== CONT TestUploadHandlersRejectInvalidKeys706=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info707=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info708=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal7092026/09/23 12:21:47 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000710=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal711=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key712=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key713=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key714=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key715=== CONT TestProxyWriteTimeout716=== RUN TestProxyWriteTimeout/narinfo717=== PAUSE TestProxyWriteTimeout/narinfo718=== RUN TestProxyWriteTimeout/1_GiB_nar719=== PAUSE TestProxyWriteTimeout/1_GiB_nar720=== RUN TestProxyWriteTimeout/10_GiB_nar721=== PAUSE TestProxyWriteTimeout/10_GiB_nar722=== RUN TestProxyWriteTimeout/unknown_size723=== PAUSE TestProxyWriteTimeout/unknown_size724=== CONT TestGCTaskStore_CompletedAllowsNewTask725--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)726=== CONT TestReadProxyInvalidPath727--- PASS: TestSkippedUploadsHandler (0.04s)728=== CONT TestReadProxy404729=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure730=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure731=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart732=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart733=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts734=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts735=== CONT TestReadProxyNarStreaming7362026-09-23 12:21:47.791 UTC [43727] ERROR: relation "goose_db_version" does not exist at character 367372026-09-23 12:21:47.791 UTC [43727] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7382026-09-23 12:21:47.793 UTC [43728] ERROR: relation "goose_db_version" does not exist at character 367392026-09-23 12:21:47.793 UTC [43728] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7402026-09-23 12:21:47.793 UTC [43729] ERROR: relation "goose_db_version" does not exist at character 367412026-09-23 12:21:47.793 UTC [43729] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7422026-09-23 12:21:47.794 UTC [43730] ERROR: relation "goose_db_version" does not exist at character 367432026-09-23 12:21:47.794 UTC [43730] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7442026-09-23 12:21:47.795 UTC [43731] ERROR: relation "goose_db_version" does not exist at character 367452026-09-23 12:21:47.795 UTC [43731] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7462026-09-23 12:21:47.796 UTC [43735] ERROR: relation "goose_db_version" does not exist at character 367472026-09-23 12:21:47.796 UTC [43735] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7482026-09-23 12:21:47.796 UTC [43733] ERROR: relation "goose_db_version" does not exist at character 367492026-09-23 12:21:47.796 UTC [43733] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7502026-09-23 12:21:47.796 UTC [43732] ERROR: relation "goose_db_version" does not exist at character 367512026-09-23 12:21:47.796 UTC [43732] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7522026-09-23 12:21:47.797 UTC [43736] ERROR: relation "goose_db_version" does not exist at character 367532026-09-23 12:21:47.797 UTC [43736] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7542026-09-23 12:21:47.797 UTC [43734] ERROR: relation "goose_db_version" does not exist at character 367552026-09-23 12:21:47.797 UTC [43734] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC7562026/09/23 12:21:47 OK 20241026095416_initial_model.sql (10.33ms)7572026/09/23 12:21:47 OK 20241026095416_initial_model.sql (25.31ms)7582026/09/23 12:21:47 OK 20251210153512_drop_unused_gin_index.sql (14.79ms)7592026/09/23 12:21:47 OK 20251210153512_drop_unused_gin_index.sql (8.64ms)7602026/09/23 12:21:47 OK 20251218171726_add_pins.sql (12.38ms)7612026/09/23 12:21:47 OK 20241026095416_initial_model.sql (38.1ms)7622026/09/23 12:21:47 OK 20251210153512_drop_unused_gin_index.sql (7.92ms)7632026/09/23 12:21:47 OK 20251218171726_add_pins.sql (19.42ms)7642026/09/23 12:21:47 OK 20260628120000_add_object_size_and_stats.sql (14.56ms)7652026/09/23 12:21:47 OK 20251218171726_add_pins.sql (6.68ms)7662026/09/23 12:21:47 OK 20260628120000_add_object_size_and_stats.sql (16.46ms)7672026/09/23 12:21:47 OK 20241026095416_initial_model.sql (68.2ms)7682026/09/23 12:21:47 OK 20260905000000_add_claims.sql (24.2ms)7692026/09/23 12:21:47 OK 20251210153512_drop_unused_gin_index.sql (7.45ms)7702026/09/23 12:21:47 OK 20260628120000_add_object_size_and_stats.sql (25.01ms)7712026/09/23 12:21:47 OK 20241026095416_initial_model.sql (81.54ms)7722026/09/23 12:21:47 OK 20260920000000_drop_claims.sql (7.93ms)7732026/09/23 12:21:47 OK 20241026095416_initial_model.sql (80.61ms)7742026/09/23 12:21:47 OK 20251210153512_drop_unused_gin_index.sql (1.24ms)7752026/09/23 12:21:47 OK 20260905000000_add_claims.sql (16.86ms)7762026/09/23 12:21:47 OK 20260923120000_add_pushes.sql (1.31ms)7772026/09/23 12:21:47 goose: successfully migrated database to version: 202609231200007782026/09/23 12:21:47 OK 20241026095416_initial_model.sql (81.75ms)7792026/09/23 12:21:47 OK 20251210153512_drop_unused_gin_index.sql (1.28ms)7802026/09/23 12:21:47 OK 20251218171726_add_pins.sql (9.15ms)7812026/09/23 12:21:47 OK 20260905000000_add_claims.sql (9.03ms)7822026/09/23 12:21:47 OK 20241026095416_initial_model.sql (76.98ms)7832026/09/23 12:21:47 OK 20241026095416_initial_model.sql (82.04ms)7842026/09/23 12:21:47 OK 20251210153512_drop_unused_gin_index.sql (826.13µs)7852026/09/23 12:21:47 OK 20251218171726_add_pins.sql (2.09ms)7862026/09/23 12:21:47 OK 20241026095416_initial_model.sql (82.63ms)7872026/09/23 12:21:47 OK 20260920000000_drop_claims.sql (1.39ms)7882026/09/23 12:21:47 OK 1_commit_pending_closure.sql (1.34ms)7892026/09/23 12:21:47 OK 20251210153512_drop_unused_gin_index.sql (791.63µs)7902026/09/23 12:21:47 OK 20251210153512_drop_unused_gin_index.sql (651.46µs)7912026/09/23 12:21:47 OK 20260920000000_drop_claims.sql (1.28ms)7922026/09/23 12:21:47 OK 20251210153512_drop_unused_gin_index.sql (608.54µs)7932026/09/23 12:21:47 OK 2_object_stats_trigger.sql (603µs)7942026/09/23 12:21:47 OK 20260628120000_add_object_size_and_stats.sql (1.72ms)7952026/09/23 12:21:47 OK 20260923120000_add_pushes.sql (818.21µs)7962026/09/23 12:21:47 goose: successfully migrated database to version: 202609231200007972026/09/23 12:21:47 OK 3_commit_push.sql (308.17µs)7982026/09/23 12:21:47 goose: up to current file version: 37992026/09/23 12:21:47 OK 20260923120000_add_pushes.sql (1.01ms)8002026/09/23 12:21:47 goose: successfully migrated database to version: 202609231200008012026/09/23 12:21:47 OK 20251218171726_add_pins.sql (2.8ms)8022026/09/23 12:21:47 OK 20251218171726_add_pins.sql (1.3ms)8032026/09/23 12:21:47 OK 20251218171726_add_pins.sql (1.8ms)8042026/09/23 12:21:47 OK 20251218171726_add_pins.sql (1.88ms)8052026/09/23 12:21:47 OK 20251218171726_add_pins.sql (2.47ms)8062026/09/23 12:21:47 OK 20260628120000_add_object_size_and_stats.sql (2.47ms)8072026/09/23 12:21:47 OK 1_commit_pending_closure.sql (1.67ms)8082026/09/23 12:21:47 OK 1_commit_pending_closure.sql (1.07ms)8092026/09/23 12:21:47 OK 20260905000000_add_claims.sql (1.95ms)8102026/09/23 12:21:47 OK 2_object_stats_trigger.sql (445.67µs)8112026/09/23 12:21:47 OK 2_object_stats_trigger.sql (640.67µs)8122026/09/23 12:21:47 OK 20260628120000_add_object_size_and_stats.sql (1.57ms)8132026/09/23 12:21:47 OK 3_commit_push.sql (396.13µs)8142026/09/23 12:21:47 goose: up to current file version: 38152026/09/23 12:21:47 OK 3_commit_push.sql (479.67µs)8162026/09/23 12:21:47 goose: up to current file version: 38172026/09/23 12:21:47 OK 20260628120000_add_object_size_and_stats.sql (1.91ms)8182026/09/23 12:21:47 OK 20260628120000_add_object_size_and_stats.sql (1.72ms)8192026/09/23 12:21:47 OK 20260905000000_add_claims.sql (1.5ms)8202026/09/23 12:21:47 OK 20260628120000_add_object_size_and_stats.sql (1.93ms)8212026/09/23 12:21:47 OK 20260920000000_drop_claims.sql (1.29ms)8222026/09/23 12:21:47 OK 20260628120000_add_object_size_and_stats.sql (2.26ms)8232026/09/23 12:21:47 OK 20260923120000_add_pushes.sql (10.64ms)8242026/09/23 12:21:47 goose: successfully migrated database to version: 202609231200008252026/09/23 12:21:47 OK 1_commit_pending_closure.sql (849.46µs)8262026/09/23 12:21:47 OK 2_object_stats_trigger.sql (227.21µs)8272026/09/23 12:21:47 OK 3_commit_push.sql (192.54µs)8282026/09/23 12:21:47 goose: up to current file version: 38292026/09/23 12:21:47 OK 20260920000000_drop_claims.sql (32.11ms)8302026/09/23 12:21:47 OK 20260905000000_add_claims.sql (32.32ms)8312026/09/23 12:21:47 OK 20260905000000_add_claims.sql (39.08ms)8322026/09/23 12:21:47 OK 20260905000000_add_claims.sql (39.72ms)8332026/09/23 12:21:47 OK 20260923120000_add_pushes.sql (7.05ms)8342026/09/23 12:21:47 goose: successfully migrated database to version: 202609231200008352026/09/23 12:21:47 OK 1_commit_pending_closure.sql (912.58µs)8362026/09/23 12:21:47 OK 2_object_stats_trigger.sql (254.33µs)8372026/09/23 12:21:47 OK 3_commit_push.sql (206.38µs)8382026/09/23 12:21:47 goose: up to current file version: 38392026/09/23 12:21:47 OK 20260905000000_add_claims.sql (42.92ms)8402026/09/23 12:21:47 OK 20260905000000_add_claims.sql (49.94ms)8412026/09/23 12:21:47 OK 20260920000000_drop_claims.sql (15.1ms)8422026/09/23 12:21:47 OK 20260920000000_drop_claims.sql (22.18ms)8432026/09/23 12:21:47 OK 20260920000000_drop_claims.sql (16.4ms)8442026/09/23 12:21:47 OK 20260920000000_drop_claims.sql (16.32ms)8452026/09/23 12:21:47 OK 20260923120000_add_pushes.sql (5.18ms)8462026/09/23 12:21:47 goose: successfully migrated database to version: 202609231200008472026/09/23 12:21:47 OK 20260923120000_add_pushes.sql (5.43ms)8482026/09/23 12:21:47 goose: successfully migrated database to version: 202609231200008492026/09/23 12:21:47 OK 1_commit_pending_closure.sql (1ms)8502026/09/23 12:21:47 OK 2_object_stats_trigger.sql (246.29µs)8512026/09/23 12:21:47 OK 3_commit_push.sql (234.58µs)8522026/09/23 12:21:47 goose: up to current file version: 38532026/09/23 12:21:47 OK 1_commit_pending_closure.sql (1.02ms)8542026/09/23 12:21:47 OK 2_object_stats_trigger.sql (252.46µs)8552026/09/23 12:21:47 OK 3_commit_push.sql (228.17µs)8562026/09/23 12:21:47 goose: up to current file version: 38572026/09/23 12:21:47 OK 20260923120000_add_pushes.sql (11.75ms)8582026/09/23 12:21:47 goose: successfully migrated database to version: 202609231200008592026/09/23 12:21:47 OK 20260920000000_drop_claims.sql (16.92ms)8602026/09/23 12:21:47 OK 20260923120000_add_pushes.sql (8.06ms)8612026/09/23 12:21:47 goose: successfully migrated database to version: 202609231200008622026/09/23 12:21:47 OK 1_commit_pending_closure.sql (1.04ms)8632026/09/23 12:21:47 OK 1_commit_pending_closure.sql (1.03ms)8642026/09/23 12:21:47 OK 2_object_stats_trigger.sql (236.17µs)8652026/09/23 12:21:47 OK 2_object_stats_trigger.sql (222.88µs)8662026/09/23 12:21:47 OK 3_commit_push.sql (217.46µs)8672026/09/23 12:21:47 goose: up to current file version: 38682026/09/23 12:21:47 OK 3_commit_push.sql (207.46µs)8692026/09/23 12:21:47 goose: up to current file version: 38702026/09/23 12:21:47 OK 20260923120000_add_pushes.sql (11.11ms)8712026/09/23 12:21:47 goose: successfully migrated database to version: 202609231200008722026/09/23 12:21:47 OK 1_commit_pending_closure.sql (1.08ms)8732026/09/23 12:21:47 OK 2_object_stats_trigger.sql (250.83µs)8742026/09/23 12:21:47 OK 3_commit_push.sql (217.67µs)8752026/09/23 12:21:47 goose: up to current file version: 38762026/09/23 12:21:48 INFO Received cleanup request method=DELETE path=/api/pending_closures8772026/09/23 12:21:48 INFO Aborted multipart uploads count=08782026/09/23 12:21:48 INFO Received uploads request method=POST path=/api/pending_closures8792026/09/23 12:21:48 INFO Received cleanup request method=DELETE path=/api/pending_closures8802026/09/23 12:21:48 INFO Aborted multipart uploads count=18812026/09/23 12:21:48 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete8822026-09-23 12:21:48.103 UTC [43728] ERROR: Closure does not exist: id=18832026-09-23 12:21:48.103 UTC [43728] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 16 at RAISE8842026-09-23 12:21:48.103 UTC [43728] STATEMENT: -- name: CommitPendingClosure :exec885 SELECT commit_pending_closure($1::bigint)886 887--- PASS: TestService_cleanupPendingClosuresHandler (0.75s)888=== CONT TestReadProxyNarinfoAlreadyDecompressed8892026/09/23 12:21:48 INFO Received uploads request method=POST path=/api/pending_closures890--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (0.94s)891=== CONT TestReadProxyNarinfo8922026/09/23 12:21:48 INFO Received complete multipart upload request method=POST path=/api/multipart/complete8932026/09/23 12:21:48 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst894--- PASS: TestCompleteMultipartUnregistered (1.05s)895=== CONT TestIsValidCachePath896=== RUN TestIsValidCachePath/narinfo897=== PAUSE TestIsValidCachePath/narinfo898=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars899=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars900=== RUN TestIsValidCachePath/nar_zst901=== PAUSE TestIsValidCachePath/nar_zst902=== RUN TestIsValidCachePath/nar_xz903=== PAUSE TestIsValidCachePath/nar_xz904=== RUN TestIsValidCachePath/nar_bz2905=== PAUSE TestIsValidCachePath/nar_bz2906=== RUN TestIsValidCachePath/nar_uncompressed907=== PAUSE TestIsValidCachePath/nar_uncompressed908=== RUN TestIsValidCachePath/ls909=== PAUSE TestIsValidCachePath/ls910=== RUN TestIsValidCachePath/log911=== PAUSE TestIsValidCachePath/log912=== RUN TestIsValidCachePath/realisation913=== PAUSE TestIsValidCachePath/realisation914=== RUN TestIsValidCachePath/nix-cache-info915=== PAUSE TestIsValidCachePath/nix-cache-info916=== RUN TestIsValidCachePath/index.html917=== PAUSE TestIsValidCachePath/index.html918=== RUN TestIsValidCachePath/traversal_parent919=== PAUSE TestIsValidCachePath/traversal_parent920=== RUN TestIsValidCachePath/traversal_in_middle921=== PAUSE TestIsValidCachePath/traversal_in_middle922=== RUN TestIsValidCachePath/invalid_char_e923=== PAUSE TestIsValidCachePath/invalid_char_e924=== RUN TestIsValidCachePath/invalid_char_u925=== PAUSE TestIsValidCachePath/invalid_char_u926=== RUN TestIsValidCachePath/random_path927=== PAUSE TestIsValidCachePath/random_path928=== RUN TestIsValidCachePath/empty929=== PAUSE TestIsValidCachePath/empty930=== RUN TestIsValidCachePath/leading_slash931=== PAUSE TestIsValidCachePath/leading_slash932=== RUN TestIsValidCachePath/wrong_extension933=== PAUSE TestIsValidCachePath/wrong_extension934=== RUN TestIsValidCachePath/short_hash935=== PAUSE TestIsValidCachePath/short_hash936=== CONT TestProxyHeadersOnlyTrustedOnSocket937--- PASS: TestService_Rustfstest (1.25s)938=== CONT TestParseSingleRange939=== RUN TestParseSingleRange/none940=== PAUSE TestParseSingleRange/none941=== RUN TestParseSingleRange/unknown_unit942=== PAUSE TestParseSingleRange/unknown_unit943=== RUN TestParseSingleRange/multi-range_ignored944=== PAUSE TestParseSingleRange/multi-range_ignored945=== RUN TestParseSingleRange/malformed_no_dash946=== PAUSE TestParseSingleRange/malformed_no_dash947=== RUN TestParseSingleRange/malformed_both_empty948=== PAUSE TestParseSingleRange/malformed_both_empty949=== RUN TestParseSingleRange/malformed_end_before_start950=== PAUSE TestParseSingleRange/malformed_end_before_start951=== RUN TestParseSingleRange/closed952=== PAUSE TestParseSingleRange/closed953=== RUN TestParseSingleRange/open-ended954=== PAUSE TestParseSingleRange/open-ended955=== RUN TestParseSingleRange/end_clamped_to_size956=== PAUSE TestParseSingleRange/end_clamped_to_size957=== RUN TestParseSingleRange/suffix958=== PAUSE TestParseSingleRange/suffix959=== RUN TestParseSingleRange/suffix_exceeds_size960=== PAUSE TestParseSingleRange/suffix_exceeds_size961=== RUN TestParseSingleRange/single_byte962=== PAUSE TestParseSingleRange/single_byte963=== RUN TestParseSingleRange/start_past_EOF964=== PAUSE TestParseSingleRange/start_past_EOF965=== RUN TestParseSingleRange/start_far_past_EOF966=== PAUSE TestParseSingleRange/start_far_past_EOF967=== CONT TestGCTaskStore_GetReturnsLatest968--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)969=== CONT TestCreatePin_ReservedPins9702026/09/23 12:21:48 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56708/oidc9712026/09/23 12:21:48 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"972--- PASS: TestService_AuthMiddleware (1.39s)973=== CONT TestResurrectedObjectNotDeleted9742026-09-23 12:21:48.794 UTC [43747] ERROR: relation "goose_db_version" does not exist at character 369752026-09-23 12:21:48.794 UTC [43747] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC9762026-09-23 12:21:48.824 UTC [43748] ERROR: relation "goose_db_version" does not exist at character 369772026-09-23 12:21:48.824 UTC [43748] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC9782026/09/23 12:21:48 OK 20241026095416_initial_model.sql (45.23ms)9792026/09/23 12:21:48 OK 20251210153512_drop_unused_gin_index.sql (5.15ms)9802026/09/23 12:21:48 OK 20251218171726_add_pins.sql (11.71ms)9812026/09/23 12:21:48 OK 20260628120000_add_object_size_and_stats.sql (11.41ms)9822026/09/23 12:21:48 OK 20260905000000_add_claims.sql (22.43ms)983--- PASS: TestReadProxyInvalidPath (1.52s)984=== CONT TestGCTaskStore_GetEmpty985--- PASS: TestGCTaskStore_GetEmpty (0.00s)9862026/09/23 12:21:48 OK 20260920000000_drop_claims.sql (8.72ms)987=== CONT TestOrphanedObjectsGCStressTest9882026/09/23 12:21:48 OK 20260923120000_add_pushes.sql (14.13ms)9892026/09/23 12:21:48 goose: successfully migrated database to version: 202609231200009902026/09/23 12:21:48 OK 1_commit_pending_closure.sql (1.07ms)9912026/09/23 12:21:48 OK 2_object_stats_trigger.sql (274.96µs)9922026/09/23 12:21:48 OK 3_commit_push.sql (246.63µs)9932026/09/23 12:21:48 goose: up to current file version: 39942026/09/23 12:21:48 OK 20241026095416_initial_model.sql (80.72ms)9952026/09/23 12:21:48 OK 20251210153512_drop_unused_gin_index.sql (5.06ms)9962026-09-23 12:21:48.944 UTC [43750] ERROR: relation "goose_db_version" does not exist at character 369972026-09-23 12:21:48.944 UTC [43750] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC9982026/09/23 12:21:48 OK 20251218171726_add_pins.sql (13.42ms)9992026/09/23 12:21:48 OK 20260628120000_add_object_size_and_stats.sql (7.65ms)10002026/09/23 12:21:48 OK 20260905000000_add_claims.sql (13.56ms)10012026/09/23 12:21:48 OK 20260920000000_drop_claims.sql (13.61ms)10022026/09/23 12:21:48 OK 20260923120000_add_pushes.sql (1.36ms)10032026/09/23 12:21:48 goose: successfully migrated database to version: 2026092312000010042026/09/23 12:21:48 OK 1_commit_pending_closure.sql (1.58ms)10052026/09/23 12:21:49 OK 2_object_stats_trigger.sql (28.29ms)10062026/09/23 12:21:49 OK 3_commit_push.sql (611.71µs)10072026/09/23 12:21:49 goose: up to current file version: 310082026/09/23 12:21:49 OK 20241026095416_initial_model.sql (64.49ms)10092026/09/23 12:21:49 OK 20251210153512_drop_unused_gin_index.sql (7.12ms)10102026/09/23 12:21:49 OK 20251218171726_add_pins.sql (12.78ms)10112026/09/23 12:21:49 OK 20260628120000_add_object_size_and_stats.sql (20.69ms)1012--- PASS: TestReadProxy404 (1.68s)1013=== CONT TestGCTaskStore_ConflictDifferentParams1014--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)1015=== CONT TestOrphanedObjectsGC10162026/09/23 12:21:49 OK 20260905000000_add_claims.sql (34.41ms)10172026/09/23 12:21:49 OK 20260920000000_drop_claims.sql (18.08ms)10182026/09/23 12:21:49 OK 20260923120000_add_pushes.sql (2.09ms)10192026/09/23 12:21:49 goose: successfully migrated database to version: 2026092312000010202026/09/23 12:21:49 OK 1_commit_pending_closure.sql (2.25ms)10212026/09/23 12:21:49 OK 2_object_stats_trigger.sql (398.96µs)10222026/09/23 12:21:49 OK 3_commit_push.sql (318.75µs)10232026/09/23 12:21:49 goose: up to current file version: 310242026/09/23 12:21:49 INFO Received uploads request method=POST path=/api/pending_closures1025--- PASS: TestReadProxyHead (2.11s)1026=== CONT TestObjectStatsTrigger10272026-09-23 12:21:49.566 UTC [43756] ERROR: relation "goose_db_version" does not exist at character 3610282026-09-23 12:21:49.566 UTC [43756] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC10292026-09-23 12:21:49.613 UTC [43757] ERROR: relation "goose_db_version" does not exist at character 3610302026-09-23 12:21:49.613 UTC [43757] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC1031--- PASS: TestReadProxyNarStreaming (2.28s)1032=== CONT TestMultipartCleanup10332026/09/23 12:21:49 OK 20241026095416_initial_model.sql (131.64ms)10342026/09/23 12:21:49 OK 20251210153512_drop_unused_gin_index.sql (11.02ms)10352026/09/23 12:21:49 OK 20251218171726_add_pins.sql (27.05ms)10362026/09/23 12:21:49 OK 20241026095416_initial_model.sql (158.48ms)10372026/09/23 12:21:49 OK 20251210153512_drop_unused_gin_index.sql (5.32ms)10382026/09/23 12:21:49 OK 20260628120000_add_object_size_and_stats.sql (42ms)10392026/09/23 12:21:49 OK 20251218171726_add_pins.sql (26.74ms)10402026/09/23 12:21:49 OK 20260628120000_add_object_size_and_stats.sql (14.54ms)10412026/09/23 12:21:49 OK 20260905000000_add_claims.sql (31.53ms)10422026/09/23 12:21:49 OK 20260920000000_drop_claims.sql (16.27ms)10432026/09/23 12:21:49 OK 20260923120000_add_pushes.sql (13.4ms)10442026/09/23 12:21:49 goose: successfully migrated database to version: 2026092312000010452026/09/23 12:21:49 OK 1_commit_pending_closure.sql (2.22ms)10462026/09/23 12:21:49 OK 2_object_stats_trigger.sql (408.29µs)10472026/09/23 12:21:49 OK 3_commit_push.sql (353.92µs)10482026/09/23 12:21:49 goose: up to current file version: 310492026/09/23 12:21:49 OK 20260905000000_add_claims.sql (45ms)10502026/09/23 12:21:49 OK 20260920000000_drop_claims.sql (25.38ms)10512026/09/23 12:21:49 OK 20260923120000_add_pushes.sql (21.09ms)10522026/09/23 12:21:49 goose: successfully migrated database to version: 2026092312000010532026/09/23 12:21:49 OK 1_commit_pending_closure.sql (2.45ms)10542026/09/23 12:21:49 OK 2_object_stats_trigger.sql (555.33µs)10552026/09/23 12:21:49 OK 3_commit_push.sql (411µs)10562026/09/23 12:21:49 goose: up to current file version: 31057--- PASS: TestReadProxyNarinfoAlreadyDecompressed (1.85s)1058=== CONT TestServerTLSConfig1059=== RUN TestServerTLSConfig/no_client_CA1060=== PAUSE TestServerTLSConfig/no_client_CA1061=== RUN TestServerTLSConfig/missing_CA_file1062=== PAUSE TestServerTLSConfig/missing_CA_file1063=== RUN TestServerTLSConfig/not_a_PEM_file1064=== PAUSE TestServerTLSConfig/not_a_PEM_file1065=== CONT TestService_NativeMTLS10662026-09-23 12:21:50.105 UTC [43762] ERROR: relation "goose_db_version" does not exist at character 3610672026-09-23 12:21:50.105 UTC [43762] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC1068--- PASS: TestReadProxyNarinfo (1.89s)1069=== CONT TestMetricsInventory10702026/09/23 12:21:50 OK 20241026095416_initial_model.sql (146.22ms)10712026/09/23 12:21:50 OK 20251210153512_drop_unused_gin_index.sql (5.07ms)10722026/09/23 12:21:50 OK 20251218171726_add_pins.sql (20.63ms)10732026/09/23 12:21:50 OK 20260628120000_add_object_size_and_stats.sql (49.23ms)10742026/09/23 12:21:50 INFO Starting HTTP server address=127.0.0.1:5673510752026/09/23 12:21:50 INFO Starting HTTP server address=/nix/var/nix/builds/nix-43626-243968142/TestProxyHeadersOnlyTrustedOnSocket2226407890/001/proxy.sock10762026/09/23 12:21:50 WARN mTLS auth: subject not in bound subjects subject="CN=someone"10772026/09/23 12:21:50 INFO Shutdown signal received, draining in-flight requests timeout=10s1078--- PASS: TestProxyHeadersOnlyTrustedOnSocket (2.04s)1079=== CONT TestNARDeduplicationMetadataUploadBug10802026/09/23 12:21:50 OK 20260905000000_add_claims.sql (64.76ms)10812026/09/23 12:21:50 OK 20260920000000_drop_claims.sql (35.23ms)10822026-09-23 12:21:50.500 UTC [43766] ERROR: relation "goose_db_version" does not exist at character 3610832026-09-23 12:21:50.500 UTC [43766] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC10842026/09/23 12:21:50 OK 20260923120000_add_pushes.sql (2.26ms)10852026/09/23 12:21:50 goose: successfully migrated database to version: 2026092312000010862026/09/23 12:21:50 OK 1_commit_pending_closure.sql (1.99ms)10872026/09/23 12:21:50 OK 2_object_stats_trigger.sql (524.13µs)10882026/09/23 12:21:50 OK 3_commit_push.sql (313.38µs)10892026/09/23 12:21:50 goose: up to current file version: 310902026/09/23 12:21:50 OK 20241026095416_initial_model.sql (100.4ms)10912026/09/23 12:21:50 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux10922026/09/23 12:21:50 WARN Refused reserved pin name=worker-x86_64-linux10932026/09/23 12:21:50 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux10942026/09/23 12:21:50 INFO Received create pin request method=POST path=/api/pins/my-app10952026/09/23 12:21:50 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux10962026/09/23 12:21:50 OK 20251210153512_drop_unused_gin_index.sql (17.73ms)1097=== CONT TestCreatePendingClosureRejectsOversizedNAR1098--- PASS: TestCreatePin_ReservedPins (2.07s)10992026/09/23 12:21:50 INFO Received uploads request method=POST path=/api/pending_closures1100--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.00s)1101=== CONT TestCacheConfigHandlerMaxNarSize1102--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1103=== CONT TestGenerateLandingPage1104--- PASS: TestGenerateLandingPage (0.00s)1105=== CONT TestService_readinessHandler11062026/09/23 12:21:50 INFO Received complete multipart upload request method=POST path=/api/multipart/complete11072026/09/23 12:21:50 OK 20251218171726_add_pins.sql (34.99ms)11082026/09/23 12:21:50 OK 20260628120000_add_object_size_and_stats.sql (16.55ms)11092026/09/23 12:21:50 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=MDE5OTY4Y2UtNTJmOC00OWY1LTgxMDQtZTZiMWMwMDlkNzEyLjI0ODIzMTgxLWZmZDUtNGQ4MS1hZTkxLTUxZDFjNTA4ZjE3Y3gxNzkwMTY2MTA5MjQ2NjA2MDAw parts=1211102026/09/23 12:21:50 INFO Received uploads request method=POST path=/api/pending_closures1111--- PASS: TestCompletedNarNotReofferedAcrossClosures (3.37s)1112=== CONT TestService_healthCheckHandler11132026/09/23 12:21:50 OK 20260905000000_add_claims.sql (47.54ms)11142026/09/23 12:21:50 OK 20260920000000_drop_claims.sql (16.49ms)11152026/09/23 12:21:50 OK 20260923120000_add_pushes.sql (2.24ms)11162026/09/23 12:21:50 goose: successfully migrated database to version: 2026092312000011172026/09/23 12:21:50 OK 1_commit_pending_closure.sql (1.21ms)11182026/09/23 12:21:50 OK 2_object_stats_trigger.sql (273.58µs)11192026/09/23 12:21:50 OK 3_commit_push.sql (264.17µs)11202026/09/23 12:21:50 goose: up to current file version: 311212026-09-23 12:21:50.965 UTC [43772] ERROR: relation "goose_db_version" does not exist at character 3611222026-09-23 12:21:50.965 UTC [43772] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC1123--- PASS: TestResurrectedObjectNotDeleted (2.29s)1124=== CONT TestGracefulShutdownDrainsInflight11252026/09/23 12:21:51 INFO Starting HTTP server address=127.0.0.1:5674111262026/09/23 12:21:51 INFO Shutdown signal received, draining in-flight requests timeout=10s1127--- PASS: TestGracefulShutdownDrainsInflight (0.07s)1128=== CONT TestGCTaskStore_Fail1129--- PASS: TestGCTaskStore_Fail (0.00s)1130=== CONT TestGCTaskStore_PhaseUpdates1131--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)1132=== CONT TestParseSize1133--- PASS: TestParseSize (0.00s)1134=== CONT TestPush_OverlappingRootsStoreOneRowPerKey11352026/09/23 12:21:51 OK 20241026095416_initial_model.sql (118.59ms)11362026/09/23 12:21:51 OK 20251210153512_drop_unused_gin_index.sql (1.4ms)11372026-09-23 12:21:51.160 UTC [43775] ERROR: relation "goose_db_version" does not exist at character 3611382026-09-23 12:21:51.160 UTC [43775] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC11392026/09/23 12:21:51 OK 20251218171726_add_pins.sql (17.7ms)11402026/09/23 12:21:51 OK 20260628120000_add_object_size_and_stats.sql (37.4ms)11412026/09/23 12:21:51 OK 20260905000000_add_claims.sql (59.42ms)11422026/09/23 12:21:51 OK 20260920000000_drop_claims.sql (39.28ms)11432026/09/23 12:21:51 OK 20260923120000_add_pushes.sql (19.49ms)11442026/09/23 12:21:51 goose: successfully migrated database to version: 2026092312000011452026/09/23 12:21:51 OK 1_commit_pending_closure.sql (4.2ms)11462026/09/23 12:21:51 OK 2_object_stats_trigger.sql (778.63µs)11472026/09/23 12:21:51 OK 3_commit_push.sql (526.42µs)11482026/09/23 12:21:51 goose: up to current file version: 311492026/09/23 12:21:51 OK 20241026095416_initial_model.sql (170.56ms)11502026/09/23 12:21:51 OK 20251210153512_drop_unused_gin_index.sql (3.38ms)11512026/09/23 12:21:51 OK 20251218171726_add_pins.sql (26.78ms)11522026/09/23 12:21:51 OK 20260628120000_add_object_size_and_stats.sql (37.13ms)11532026/09/23 12:21:51 OK 20260905000000_add_claims.sql (47.66ms)11542026/09/23 12:21:51 OK 20260920000000_drop_claims.sql (35.32ms)11552026-09-23 12:21:51.542 UTC [43776] ERROR: relation "goose_db_version" does not exist at character 3611562026-09-23 12:21:51.542 UTC [43776] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC11572026/09/23 12:21:51 OK 20260923120000_add_pushes.sql (13.8ms)11582026/09/23 12:21:51 goose: successfully migrated database to version: 2026092312000011592026/09/23 12:21:51 OK 1_commit_pending_closure.sql (3.78ms)11602026/09/23 12:21:51 OK 2_object_stats_trigger.sql (781.71µs)11612026/09/23 12:21:51 OK 3_commit_push.sql (454.54µs)11622026/09/23 12:21:51 goose: up to current file version: 31163--- PASS: TestObjectStatsTrigger (2.27s)1164=== CONT TestCompleteMultipartUpload_ErrorButObjectExists11652026/09/23 12:21:51 OK 20241026095416_initial_model.sql (183.64ms)11662026/09/23 12:21:51 OK 20251210153512_drop_unused_gin_index.sql (10.06ms)11672026/09/23 12:21:51 OK 20251218171726_add_pins.sql (24.74ms)11682026/09/23 12:21:51 OK 20260628120000_add_object_size_and_stats.sql (34.8ms)11692026-09-23 12:21:51.871 UTC [43779] ERROR: relation "goose_db_version" does not exist at character 3611702026-09-23 12:21:51.871 UTC [43779] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC11712026/09/23 12:21:51 OK 20260905000000_add_claims.sql (69.92ms)11722026/09/23 12:21:51 OK 20260920000000_drop_claims.sql (45.73ms)11732026/09/23 12:21:52 INFO Received uploads request method=POST path=/api/pending_closures11742026/09/23 12:21:52 OK 20260923120000_add_pushes.sql (61.38ms)11752026/09/23 12:21:52 goose: successfully migrated database to version: 2026092312000011762026/09/23 12:21:52 OK 1_commit_pending_closure.sql (5.76ms)11772026/09/23 12:21:52 OK 2_object_stats_trigger.sql (912.08µs)11782026/09/23 12:21:52 OK 3_commit_push.sql (548.54µs)11792026/09/23 12:21:52 goose: up to current file version: 311802026/09/23 12:21:52 OK 20241026095416_initial_model.sql (170.85ms)1181=== NAME TestOrphanedObjectsGC1182 orphaned_objects_gc_test.go:290: GC Test Summary:1183 orphaned_objects_gc_test.go:291: - Kept: 2 objects from closure A1184 orphaned_objects_gc_test.go:292: - Deleted: 2 objects from closure B1185 orphaned_objects_gc_test.go:293: - Deleted: 6 orphaned chain objects (X1->X2->X3)1186 orphaned_objects_gc_test.go:294: - Deleted: 2 orphaned single objects (Y)1187 orphaned_objects_gc_test.go:295: - Total deleted: 10 objects1188--- PASS: TestOrphanedObjectsGC (3.05s)1189=== CONT TestRedundantMultipartUpload11902026/09/23 12:21:52 OK 20251210153512_drop_unused_gin_index.sql (19.23ms)11912026/09/23 12:21:52 INFO Received cleanup request method=DELETE path=/api/pending_closures11922026/09/23 12:21:52 INFO Aborted multipart uploads count=111932026/09/23 12:21:52 OK 20251218171726_add_pins.sql (28.98ms)1194--- PASS: TestMultipartCleanup (2.49s)1195=== CONT TestPush_SignsNarinfosOfItsPendingObjects11962026/09/23 12:21:52 OK 20260628120000_add_object_size_and_stats.sql (35.93ms)11972026-09-23 12:21:52.217 UTC [43781] ERROR: relation "goose_db_version" does not exist at character 3611982026-09-23 12:21:52.217 UTC [43781] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC11992026/09/23 12:21:52 OK 20260905000000_add_claims.sql (50.48ms)12002026/09/23 12:21:52 OK 20260920000000_drop_claims.sql (33.87ms)12012026/09/23 12:21:52 OK 20260923120000_add_pushes.sql (13.55ms)12022026/09/23 12:21:52 goose: successfully migrated database to version: 2026092312000012032026/09/23 12:21:52 OK 1_commit_pending_closure.sql (2.49ms)12042026/09/23 12:21:52 OK 2_object_stats_trigger.sql (519.83µs)12052026/09/23 12:21:52 OK 3_commit_push.sql (500.04µs)12062026/09/23 12:21:52 goose: up to current file version: 312072026-09-23 12:21:52.320 UTC [43785] ERROR: relation "goose_db_version" does not exist at character 3612082026-09-23 12:21:52.320 UTC [43785] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC12092026/09/23 12:21:52 WARN mTLS auth: subject not in bound subjects subject="CN=reader"12102026/09/23 12:21:52 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1211--- PASS: TestService_NativeMTLS (2.39s)1212=== CONT TestPush_RejectsBadRequests12132026-09-23 12:21:52.406 UTC [43787] ERROR: relation "goose_db_version" does not exist at character 3612142026-09-23 12:21:52.406 UTC [43787] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC12152026/09/23 12:21:52 OK 20241026095416_initial_model.sql (128.33ms)12162026/09/23 12:21:52 OK 20251210153512_drop_unused_gin_index.sql (7.82ms)12172026/09/23 12:21:52 OK 20251218171726_add_pins.sql (33.43ms)12182026/09/23 12:21:52 OK 20260628120000_add_object_size_and_stats.sql (28.89ms)12192026/09/23 12:21:52 OK 20241026095416_initial_model.sql (155.65ms)12202026/09/23 12:21:52 OK 20260905000000_add_claims.sql (56.85ms)12212026/09/23 12:21:52 OK 20251210153512_drop_unused_gin_index.sql (14.42ms)12222026/09/23 12:21:52 OK 20260920000000_drop_claims.sql (24.91ms)12232026/09/23 12:21:52 OK 20251218171726_add_pins.sql (37.64ms)12242026/09/23 12:21:52 OK 20260923120000_add_pushes.sql (28.59ms)12252026/09/23 12:21:52 goose: successfully migrated database to version: 2026092312000012262026/09/23 12:21:52 OK 1_commit_pending_closure.sql (4.46ms)12272026/09/23 12:21:52 OK 2_object_stats_trigger.sql (4.73ms)12282026/09/23 12:21:52 OK 3_commit_push.sql (1.61ms)12292026/09/23 12:21:52 goose: up to current file version: 312302026/09/23 12:21:52 OK 20260628120000_add_object_size_and_stats.sql (33.6ms)12312026/09/23 12:21:52 OK 20241026095416_initial_model.sql (166.71ms)12322026/09/23 12:21:52 OK 20251210153512_drop_unused_gin_index.sql (16.17ms)12332026/09/23 12:21:52 OK 20251218171726_add_pins.sql (44.01ms)12342026/09/23 12:21:52 OK 20260905000000_add_claims.sql (77.54ms)12352026/09/23 12:21:52 OK 20260920000000_drop_claims.sql (33.3ms)12362026/09/23 12:21:52 OK 20260628120000_add_object_size_and_stats.sql (37.12ms)1237--- PASS: TestMetricsInventory (2.53s)1238=== CONT TestPush_CommitFailsWhenSkippedKeyWasCollected12392026/09/23 12:21:52 OK 20260923120000_add_pushes.sql (20.35ms)12402026/09/23 12:21:52 goose: successfully migrated database to version: 2026092312000012412026/09/23 12:21:52 OK 1_commit_pending_closure.sql (3.89ms)12422026/09/23 12:21:52 OK 2_object_stats_trigger.sql (775.63µs)12432026/09/23 12:21:52 OK 3_commit_push.sql (521.96µs)12442026/09/23 12:21:52 goose: up to current file version: 312452026/09/23 12:21:52 OK 20260905000000_add_claims.sql (55.69ms)12462026/09/23 12:21:52 OK 20260920000000_drop_claims.sql (17.41ms)12472026/09/23 12:21:52 OK 20260923120000_add_pushes.sql (16.5ms)12482026/09/23 12:21:52 goose: successfully migrated database to version: 2026092312000012492026/09/23 12:21:52 OK 1_commit_pending_closure.sql (2.77ms)12502026/09/23 12:21:52 OK 2_object_stats_trigger.sql (632.17µs)12512026/09/23 12:21:52 OK 3_commit_push.sql (385.13µs)12522026/09/23 12:21:52 goose: up to current file version: 312532026-09-23 12:21:52.903 UTC [43791] ERROR: relation "goose_db_version" does not exist at character 3612542026-09-23 12:21:52.903 UTC [43791] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC12552026/09/23 12:21:53 OK 20241026095416_initial_model.sql (151.49ms)12562026/09/23 12:21:53 OK 20251210153512_drop_unused_gin_index.sql (1.02ms)12572026/09/23 12:21:53 OK 20251218171726_add_pins.sql (73.31ms)12582026/09/23 12:21:53 WARN readiness check failed error="closed pool"1259--- PASS: TestService_readinessHandler (2.55s)1260=== CONT TestPush_CompleteCommitsEveryRoot12612026/09/23 12:21:53 OK 20260628120000_add_object_size_and_stats.sql (45.03ms)1262=== NAME TestNARDeduplicationMetadataUploadBug1263 metadata_upload_test.go:48: First store path: /nix/var/nix/builds/nix-43626-243968142/TestNARDeduplicationMetadataUploadBug949424868/001/store/b8xisdh30yp2gbzab8j9mc47nzavm9ab-file1.txt12642026/09/23 12:21:53 OK 20260905000000_add_claims.sql (44.95ms)12652026/09/23 12:21:53 OK 20260920000000_drop_claims.sql (29.04ms)12662026/09/23 12:21:53 OK 20260923120000_add_pushes.sql (13.73ms)12672026/09/23 12:21:53 goose: successfully migrated database to version: 2026092312000012682026/09/23 12:21:53 OK 1_commit_pending_closure.sql (1.12ms)12692026/09/23 12:21:53 OK 2_object_stats_trigger.sql (250.5µs)12702026/09/23 12:21:53 OK 3_commit_push.sql (190.46µs)12712026/09/23 12:21:53 goose: up to current file version: 312722026/09/23 12:21:53 INFO Received push request method=POST path=/api/pushes12732026/09/23 12:21:53 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)12742026/09/23 12:21:53 INFO Uploading b8xisdh30yp2gbzab8j9mc47nzavm9ab-file1.txt (160B)12752026/09/23 12:21:53 WARN Failed to register uploaded object key=b8xisdh30yp2gbzab8j9mc47nzavm9ab.ls error="server returned 404: 404 page not found\n"12762026/09/23 12:21:53 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"12772026/09/23 12:21:53 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12782026/09/23 12:21:53 INFO Signed narinfos id=1 count=112792026/09/23 12:21:53 INFO Uploading 1 narinfos12802026/09/23 12:21:53 INFO Received complete push request method=POST path=/api/pushes/1/complete12812026/09/23 12:21:53 WARN Failed to register uploaded object key=b8xisdh30yp2gbzab8j9mc47nzavm9ab.narinfo error="server returned 404: 404 page not found\n"12822026/09/23 12:21:53 INFO Upload complete. (158ms)1283 metadata_upload_test.go:54: Retrieved narinfo from S3:1284 StorePath: /nix/var/nix/builds/nix-43626-243968142/TestNARDeduplicationMetadataUploadBug949424868/001/store/b8xisdh30yp2gbzab8j9mc47nzavm9ab-file1.txt1285 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1286 Compression: zstd1287 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1288 NarSize: 1601289 References: 1290 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1291 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1292 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1293 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}1294--- PASS: TestService_healthCheckHandler (2.75s)1295=== CONT TestPresignedUploadRegisteredBeforeCommit12962026-09-23 12:21:53.543 UTC [43802] ERROR: relation "goose_db_version" does not exist at character 3612972026-09-23 12:21:53.543 UTC [43802] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC1298=== NAME TestNARDeduplicationMetadataUploadBug1299 metadata_upload_test.go:64: Second store path (same content): /nix/var/nix/builds/nix-43626-243968142/TestNARDeduplicationMetadataUploadBug949424868/001/store/zcn8svb5j0878p9920z510qlqn814qd7-file2.txt13002026/09/23 12:21:53 INFO Received push request method=POST path=/api/pushes13012026/09/23 12:21:53 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)13022026/09/23 12:21:53 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign13032026/09/23 12:21:53 INFO Signed narinfos id=2 count=113042026/09/23 12:21:53 INFO Uploading 1 narinfos13052026/09/23 12:21:53 WARN Failed to register uploaded object key=zcn8svb5j0878p9920z510qlqn814qd7.ls error="server returned 404: 404 page not found\n"13062026/09/23 12:21:53 INFO Received complete push request method=POST path=/api/pushes/2/complete13072026/09/23 12:21:53 WARN Failed to register uploaded object key=zcn8svb5j0878p9920z510qlqn814qd7.narinfo error="server returned 404: 404 page not found\n"13082026/09/23 12:21:53 INFO Upload complete. (122ms)1309 metadata_upload_test.go:76: Retrieved narinfo from S3:1310 StorePath: /nix/var/nix/builds/nix-43626-243968142/TestNARDeduplicationMetadataUploadBug949424868/001/store/zcn8svb5j0878p9920z510qlqn814qd7-file2.txt1311 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1312 Compression: zstd1313 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1314 NarSize: 1601315 References: 1316 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1317 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)1318 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):1319 {"version":1,"root":{"type":"regular","size":44}}1320--- PASS: TestNARDeduplicationMetadataUploadBug (3.36s)1321=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle13222026/09/23 12:21:53 INFO Received push request method=POST path=/api/pushes13232026/09/23 12:21:53 OK 20241026095416_initial_model.sql (216.67ms)13242026/09/23 12:21:53 OK 20251210153512_drop_unused_gin_index.sql (8.65ms)13252026/09/23 12:21:53 OK 20251218171726_add_pins.sql (47.22ms)1326--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (2.80s)1327=== CONT TestClientMultipleUploads13282026/09/23 12:21:53 OK 20260628120000_add_object_size_and_stats.sql (46.87ms)13292026/09/23 12:21:53 OK 20260905000000_add_claims.sql (49.23ms)13302026/09/23 12:21:54 OK 20260920000000_drop_claims.sql (52.11ms)13312026/09/23 12:21:54 OK 20260923120000_add_pushes.sql (34.6ms)13322026/09/23 12:21:54 goose: successfully migrated database to version: 2026092312000013332026/09/23 12:21:54 OK 1_commit_pending_closure.sql (3.44ms)13342026/09/23 12:21:54 OK 2_object_stats_trigger.sql (956.38µs)13352026/09/23 12:21:54 OK 3_commit_push.sql (692.75µs)13362026/09/23 12:21:54 goose: up to current file version: 313372026-09-23 12:21:54.341 UTC [43815] ERROR: relation "goose_db_version" does not exist at character 3613382026-09-23 12:21:54.341 UTC [43815] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13392026-09-23 12:21:54.346 UTC [43814] ERROR: relation "goose_db_version" does not exist at character 3613402026-09-23 12:21:54.346 UTC [43814] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13412026-09-23 12:21:54.404 UTC [43816] ERROR: relation "goose_db_version" does not exist at character 3613422026-09-23 12:21:54.404 UTC [43816] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13432026/09/23 12:21:54 INFO Received uploads request method=POST path=/api/pending_closures13442026/09/23 12:21:54 OK 20241026095416_initial_model.sql (220.43ms)13452026/09/23 12:21:54 OK 20241026095416_initial_model.sql (201.31ms)13462026/09/23 12:21:54 OK 20251210153512_drop_unused_gin_index.sql (10.77ms)13472026/09/23 12:21:54 OK 20251210153512_drop_unused_gin_index.sql (12.74ms)13482026/09/23 12:21:54 OK 20251218171726_add_pins.sql (43.91ms)13492026/09/23 12:21:54 OK 20251218171726_add_pins.sql (44.46ms)13502026/09/23 12:21:54 OK 20241026095416_initial_model.sql (226.73ms)13512026/09/23 12:21:54 OK 20251210153512_drop_unused_gin_index.sql (8.09ms)13522026/09/23 12:21:54 OK 20260628120000_add_object_size_and_stats.sql (27.99ms)13532026/09/23 12:21:54 OK 20260628120000_add_object_size_and_stats.sql (44.06ms)13542026/09/23 12:21:54 OK 20251218171726_add_pins.sql (34.92ms)13552026/09/23 12:21:54 OK 20260628120000_add_object_size_and_stats.sql (46.1ms)13562026/09/23 12:21:54 OK 20260905000000_add_claims.sql (88.89ms)13572026/09/23 12:21:54 OK 20260905000000_add_claims.sql (104.47ms)13582026/09/23 12:21:54 OK 20260920000000_drop_claims.sql (37.71ms)13592026/09/23 12:21:54 OK 20260923120000_add_pushes.sql (32.64ms)13602026/09/23 12:21:54 goose: successfully migrated database to version: 2026092312000013612026/09/23 12:21:54 OK 1_commit_pending_closure.sql (4.47ms)13622026/09/23 12:21:54 OK 2_object_stats_trigger.sql (799.88µs)13632026/09/23 12:21:54 OK 3_commit_push.sql (626.42µs)13642026/09/23 12:21:54 goose: up to current file version: 313652026/09/23 12:21:54 OK 20260920000000_drop_claims.sql (54.3ms)13662026/09/23 12:21:54 OK 20260905000000_add_claims.sql (105.93ms)13672026/09/23 12:21:54 INFO Received complete multipart upload request method=POST path=/api/multipart/complete13682026/09/23 12:21:54 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MDE5OTY4Y2UtNTJmOC00OWY1LTgxMDQtZTZiMWMwMDlkNzEyLjkxNjhkMjVmLWVjYWUtNDk1Yy1hNGYzLWNjMzkzZWJiNjc3MXgxNzkwMTY2MTE0NTgzOTMwMDAw13692026/09/23 12:21:54 OK 20260923120000_add_pushes.sql (35.69ms)13702026/09/23 12:21:54 goose: successfully migrated database to version: 2026092312000013712026/09/23 12:21:54 OK 1_commit_pending_closure.sql (3.71ms)13722026/09/23 12:21:54 OK 2_object_stats_trigger.sql (844.25µs)13732026/09/23 12:21:54 OK 3_commit_push.sql (473.54µs)13742026/09/23 12:21:54 goose: up to current file version: 313752026/09/23 12:21:54 OK 20260920000000_drop_claims.sql (39.35ms)13762026/09/23 12:21:54 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=MDE5OTY4Y2UtNTJmOC00OWY1LTgxMDQtZTZiMWMwMDlkNzEyLjkxNjhkMjVmLWVjYWUtNDk1Yy1hNGYzLWNjMzkzZWJiNjc3MXgxNzkwMTY2MTE0NTgzOTMwMDAw parts=11377--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (3.21s)1378=== CONT TestGCTaskStore_StartNew1379--- PASS: TestGCTaskStore_StartNew (0.00s)1380=== CONT TestGCMetrics13812026/09/23 12:21:54 OK 20260923120000_add_pushes.sql (19.76ms)13822026/09/23 12:21:54 goose: successfully migrated database to version: 2026092312000013832026/09/23 12:21:54 OK 1_commit_pending_closure.sql (2.19ms)13842026/09/23 12:21:54 OK 2_object_stats_trigger.sql (544.96µs)13852026/09/23 12:21:54 OK 3_commit_push.sql (428.38µs)13862026/09/23 12:21:54 goose: up to current file version: 313872026-09-23 12:21:55.148 UTC [43819] ERROR: relation "goose_db_version" does not exist at character 3613882026-09-23 12:21:55.148 UTC [43819] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC13892026/09/23 12:21:55 INFO Received uploads request method=POST path=/api/pending_closures13902026/09/23 12:21:55 INFO Received uploads request method=POST path=/api/pending_closures13912026/09/23 12:21:55 OK 20241026095416_initial_model.sql (255.72ms)13922026/09/23 12:21:55 OK 20251210153512_drop_unused_gin_index.sql (14.59ms)13932026/09/23 12:21:55 OK 20251218171726_add_pins.sql (32.82ms)13942026/09/23 12:21:55 OK 20260628120000_add_object_size_and_stats.sql (44.67ms)13952026/09/23 12:21:55 OK 20260905000000_add_claims.sql (56.97ms)13962026/09/23 12:21:55 INFO Received push request method=POST path=/api/pushes13972026/09/23 12:21:55 OK 20260920000000_drop_claims.sql (57.19ms)13982026/09/23 12:21:55 OK 20260923120000_add_pushes.sql (33.41ms)13992026/09/23 12:21:55 goose: successfully migrated database to version: 2026092312000014002026/09/23 12:21:55 OK 1_commit_pending_closure.sql (4.23ms)14012026/09/23 12:21:55 OK 2_object_stats_trigger.sql (838.92µs)14022026/09/23 12:21:55 OK 3_commit_push.sql (962.92µs)14032026/09/23 12:21:55 goose: up to current file version: 314042026/09/23 12:21:55 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14052026/09/23 12:21:55 INFO Signed narinfos id=1 count=11406--- PASS: TestPush_SignsNarinfosOfItsPendingObjects (3.61s)1407=== CONT TestGCBugBareHashReferences14082026-09-23 12:21:55.868 UTC [43820] ERROR: relation "goose_db_version" does not exist at character 3614092026-09-23 12:21:55.868 UTC [43820] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC1410=== RUN TestPush_RejectsBadRequests/root_not_in_objects1411=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects1412=== RUN TestPush_RejectsBadRequests/no_roots1413=== PAUSE TestPush_RejectsBadRequests/no_roots1414=== RUN TestPush_RejectsBadRequests/no_objects1415=== PAUSE TestPush_RejectsBadRequests/no_objects1416=== RUN TestPush_RejectsBadRequests/bad_root1417=== PAUSE TestPush_RejectsBadRequests/bad_root1418=== CONT TestService_ReadScope_PublicByDefault14192026-09-23 12:21:56.140 UTC [43824] ERROR: relation "goose_db_version" does not exist at character 3614202026-09-23 12:21:56.140 UTC [43824] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC14212026/09/23 12:21:56 OK 20241026095416_initial_model.sql (214.56ms)14222026/09/23 12:21:56 OK 20251210153512_drop_unused_gin_index.sql (14.23ms)14232026/09/23 12:21:56 OK 20251218171726_add_pins.sql (31.29ms)14242026/09/23 12:21:56 OK 20260628120000_add_object_size_and_stats.sql (59.42ms)14252026/09/23 12:21:56 OK 20260905000000_add_claims.sql (78.87ms)14262026/09/23 12:21:56 OK 20260920000000_drop_claims.sql (32.67ms)14272026/09/23 12:21:56 OK 20260923120000_add_pushes.sql (36.39ms)14282026/09/23 12:21:56 goose: successfully migrated database to version: 2026092312000014292026/09/23 12:21:56 OK 1_commit_pending_closure.sql (4.97ms)14302026/09/23 12:21:56 OK 2_object_stats_trigger.sql (1.39ms)14312026/09/23 12:21:56 OK 3_commit_push.sql (1.27ms)14322026/09/23 12:21:56 goose: up to current file version: 314332026/09/23 12:21:56 INFO Received push request method=POST path=/api/pushes14342026/09/23 12:21:56 INFO Received complete push request method=POST path=/api/pushes/1/complete14352026/09/23 12:21:56 OK 20241026095416_initial_model.sql (312.88ms)14362026/09/23 12:21:56 OK 20251210153512_drop_unused_gin_index.sql (18.18ms)14372026/09/23 12:21:56 INFO Received push request method=POST path=/api/pushes14382026-09-23 12:21:56.583 UTC [43827] ERROR: relation "goose_db_version" does not exist at character 3614392026-09-23 12:21:56.583 UTC [43827] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC14402026/09/23 12:21:56 INFO Received complete push request method=POST path=/api/pushes/2/complete14412026-09-23 12:21:56.626 UTC [43826] ERROR: Push object missing: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.narinfo14422026-09-23 12:21:56.626 UTC [43826] CONTEXT: PL/pgSQL function commit_push(bigint) line 31 at RAISE14432026-09-23 12:21:56.626 UTC [43826] STATEMENT: -- name: CommitPush :exec1444 SELECT commit_push($1::bigint)1445 1446--- PASS: TestPush_CommitFailsWhenSkippedKeyWasCollected (3.90s)1447=== CONT TestLeadEndsOnShutdown14482026/09/23 12:21:56 OK 20251218171726_add_pins.sql (49.44ms)14492026/09/23 12:21:56 OK 20260628120000_add_object_size_and_stats.sql (44.01ms)14502026-09-23 12:21:56.677 UTC [43828] ERROR: relation "goose_db_version" does not exist at character 3614512026-09-23 12:21:56.677 UTC [43828] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC14522026/09/23 12:21:56 OK 20260905000000_add_claims.sql (65.34ms)14532026/09/23 12:21:56 INFO Received push request method=POST path=/api/pushes14542026/09/23 12:21:56 OK 20260920000000_drop_claims.sql (57.23ms)14552026/09/23 12:21:56 OK 20260923120000_add_pushes.sql (37.92ms)14562026/09/23 12:21:56 goose: successfully migrated database to version: 2026092312000014572026/09/23 12:21:56 OK 20241026095416_initial_model.sql (174.91ms)14582026/09/23 12:21:56 OK 20251210153512_drop_unused_gin_index.sql (3.64ms)14592026/09/23 12:21:56 OK 1_commit_pending_closure.sql (6.35ms)14602026/09/23 12:21:56 OK 2_object_stats_trigger.sql (760.42µs)14612026/09/23 12:21:56 OK 3_commit_push.sql (445.83µs)14622026/09/23 12:21:56 goose: up to current file version: 314632026/09/23 12:21:56 OK 20251218171726_add_pins.sql (60.13ms)14642026/09/23 12:21:56 INFO Received complete push request method=POST path=/api/pushes/1/complete1465--- PASS: TestPush_CompleteCommitsEveryRoot (3.70s)1466=== CONT TestClientIntegration14672026/09/23 12:21:56 OK 20241026095416_initial_model.sql (189.25ms)14682026/09/23 12:21:56 OK 20260628120000_add_object_size_and_stats.sql (45.28ms)14692026/09/23 12:21:56 OK 20251210153512_drop_unused_gin_index.sql (19.93ms)14702026/09/23 12:21:57 OK 20251218171726_add_pins.sql (37.79ms)14712026/09/23 12:21:57 OK 20260905000000_add_claims.sql (69.24ms)14722026/09/23 12:21:57 OK 20260628120000_add_object_size_and_stats.sql (30.41ms)14732026/09/23 12:21:57 OK 20260920000000_drop_claims.sql (43.21ms)14742026/09/23 12:21:57 OK 20260923120000_add_pushes.sql (17.57ms)14752026/09/23 12:21:57 goose: successfully migrated database to version: 2026092312000014762026/09/23 12:21:57 OK 1_commit_pending_closure.sql (2.61ms)14772026/09/23 12:21:57 OK 2_object_stats_trigger.sql (608.71µs)14782026/09/23 12:21:57 OK 3_commit_push.sql (413.79µs)14792026/09/23 12:21:57 goose: up to current file version: 314802026/09/23 12:21:57 OK 20260905000000_add_claims.sql (93.64ms)14812026/09/23 12:21:57 OK 20260920000000_drop_claims.sql (45.98ms)14822026/09/23 12:21:57 OK 20260923120000_add_pushes.sql (27.5ms)14832026/09/23 12:21:57 goose: successfully migrated database to version: 2026092312000014842026/09/23 12:21:57 OK 1_commit_pending_closure.sql (4.77ms)14852026/09/23 12:21:57 OK 2_object_stats_trigger.sql (1.32ms)14862026/09/23 12:21:57 OK 3_commit_push.sql (720.63µs)14872026/09/23 12:21:57 goose: up to current file version: 314882026/09/23 12:21:57 INFO Received uploads request method=POST path=/api/pending_closures14892026/09/23 12:21:57 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst14902026/09/23 12:21:57 INFO Received uploads request method=POST path=/api/pending_closures1491--- PASS: TestPresignedUploadRegisteredBeforeCommit (3.83s)1492=== CONT TestClientErrorHandling1493=== RUN TestClientErrorHandling/InvalidStorePath1494=== PAUSE TestClientErrorHandling/InvalidStorePath1495=== RUN TestClientErrorHandling/InvalidAuthToken1496=== PAUSE TestClientErrorHandling/InvalidAuthToken1497=== RUN TestClientErrorHandling/ServerNotAvailable1498=== PAUSE TestClientErrorHandling/ServerNotAvailable1499=== CONT TestClientCADerivations15002026/09/23 12:21:57 INFO Received complete multipart upload request method=POST path=/api/multipart/complete15012026/09/23 12:21:57 INFO Received uploads request method=POST path=/api/pending_closures15022026/09/23 12:21:57 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=MDE5OTY4Y2UtNTJmOC00OWY1LTgxMDQtZTZiMWMwMDlkNzEyLmIwZDc3M2I5LWQwNjktNGZjZC1hMGU0LWQ1NTI5NzkzYTNjMHgxNzkwMTY2MTE1MzI5NTkwMDAw parts=121503--- PASS: TestRedundantMultipartUpload (5.50s)1504=== CONT TestLeadElectsOneAndHandsOver15052026-09-23 12:21:57.752 UTC [43837] ERROR: relation "goose_db_version" does not exist at character 3615062026-09-23 12:21:57.752 UTC [43837] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC15072026/09/23 12:21:58 INFO Received complete multipart upload request method=POST path=/api/multipart/complete15082026/09/23 12:21:58 OK 20241026095416_initial_model.sql (243.77ms)15092026/09/23 12:21:58 OK 20251210153512_drop_unused_gin_index.sql (58.19ms)1510=== NAME TestOrphanedObjectsGCStressTest1511 orphaned_objects_gc_test.go:425: Created 10 active closures, 5 to-delete closures, 20 orphaned chains15122026/09/23 12:21:58 OK 20251218171726_add_pins.sql (49.49ms)15132026/09/23 12:21:58 OK 20260628120000_add_object_size_and_stats.sql (38.39ms)1514 orphaned_objects_gc_test.go:446: Marked 210 objects for deletion15152026/09/23 12:21:58 OK 20260905000000_add_claims.sql (43.94ms)15162026/09/23 12:21:58 OK 20260920000000_drop_claims.sql (18.58ms)1517=== NAME TestClientMultipleUploads1518 client_integration_test.go:358: Created store path 0: /nix/var/nix/builds/nix-43626-243968142/TestClientMultipleUploads3968740888/001/store/j9f9k3wh8xr48xbk7gqn2kf6vlyksm2p-test-file-0.txt15192026/09/23 12:21:58 OK 20260923120000_add_pushes.sql (9.92ms)15202026/09/23 12:21:58 goose: successfully migrated database to version: 2026092312000015212026/09/23 12:21:58 OK 1_commit_pending_closure.sql (2.26ms)15222026/09/23 12:21:58 OK 2_object_stats_trigger.sql (305.17µs)15232026/09/23 12:21:58 OK 3_commit_push.sql (248.5µs)15242026/09/23 12:21:58 goose: up to current file version: 315252026-09-23 12:21:58.307 UTC [43841] ERROR: relation "goose_db_version" does not exist at character 3615262026-09-23 12:21:58.307 UTC [43841] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC1527 client_integration_test.go:358: Created store path 1: /nix/var/nix/builds/nix-43626-243968142/TestClientMultipleUploads3968740888/001/store/6965pi6wvhqvd1gf7hjsbwsygwlh9ph4-test-file-1.txt15282026-09-23 12:21:58.383 UTC [43844] ERROR: relation "goose_db_version" does not exist at character 3615292026-09-23 12:21:58.383 UTC [43844] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC1530 client_integration_test.go:358: Created store path 2: /nix/var/nix/builds/nix-43626-243968142/TestClientMultipleUploads3968740888/001/store/pkcmh2dn8y8fc9y35skhpy5ry0ly2hbq-test-file-2.txt15312026/09/23 12:21:58 OK 20241026095416_initial_model.sql (112.07ms)15322026/09/23 12:21:58 OK 20251210153512_drop_unused_gin_index.sql (10.75ms)15332026/09/23 12:21:58 INFO Received push request method=POST path=/api/pushes15342026/09/23 12:21:58 OK 20251218171726_add_pins.sql (17.66ms)15352026/09/23 12:21:58 OK 20260628120000_add_object_size_and_stats.sql (28.3ms)15362026/09/23 12:21:58 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)15372026/09/23 12:21:58 INFO Uploading j9f9k3wh8xr48xbk7gqn2kf6vlyksm2p-test-file-0.txt (160B)15382026/09/23 12:21:58 INFO Uploading pkcmh2dn8y8fc9y35skhpy5ry0ly2hbq-test-file-2.txt (160B)15392026/09/23 12:21:58 INFO Uploading 6965pi6wvhqvd1gf7hjsbwsygwlh9ph4-test-file-1.txt (160B)15402026/09/23 12:21:58 INFO Aborted multipart uploads count=015412026/09/23 12:21:58 WARN Force mode enabled - objects will be deleted immediately without grace period15422026/09/23 12:21:58 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=015432026/09/23 12:21:58 INFO Vacuumed table table=pending_closures15442026/09/23 12:21:58 INFO Vacuumed table table=pending_objects15452026/09/23 12:21:58 INFO Vacuumed table table=multipart_uploads15462026/09/23 12:21:58 INFO Vacuumed table table=closures15472026/09/23 12:21:58 INFO Vacuumed table table=objects15482026/09/23 12:21:58 WARN Failed to register uploaded object key=6965pi6wvhqvd1gf7hjsbwsygwlh9ph4.ls error="server returned 404: 404 page not found\n"15492026/09/23 12:21:58 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"1550--- PASS: TestGCMetrics (3.62s)1551=== CONT TestResolveDBConnectionString15522026/09/23 12:21:58 WARN Failed to register uploaded object key=pkcmh2dn8y8fc9y35skhpy5ry0ly2hbq.ls error="server returned 404: 404 page not found\n"15532026/09/23 12:21:58 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"15542026/09/23 12:21:58 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign15552026/09/23 12:21:58 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"15562026/09/23 12:21:58 WARN Failed to register uploaded object key=j9f9k3wh8xr48xbk7gqn2kf6vlyksm2p.ls error="server returned 404: 404 page not found\n"15572026/09/23 12:21:58 INFO Signed narinfos id=1 count=315582026/09/23 12:21:58 INFO Uploading 3 narinfos1559=== RUN TestResolveDBConnectionString/flag_wins1560=== PAUSE TestResolveDBConnectionString/flag_wins1561=== RUN TestResolveDBConnectionString/file_when_flag_empty1562=== PAUSE TestResolveDBConnectionString/file_when_flag_empty1563=== RUN TestResolveDBConnectionString/missing_file_is_an_error1564=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error1565=== RUN TestResolveDBConnectionString/PGHOST_allows_empty1566=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty1567=== RUN TestResolveDBConnectionString/nothing_configured1568=== PAUSE TestResolveDBConnectionString/nothing_configured1569=== CONT TestClientFallsBackToClosures15702026/09/23 12:21:58 OK 20260905000000_add_claims.sql (67.75ms)15712026/09/23 12:21:58 WARN Failed to register uploaded object key=j9f9k3wh8xr48xbk7gqn2kf6vlyksm2p.narinfo error="server returned 404: 404 page not found\n"15722026/09/23 12:21:58 WARN Failed to register uploaded object key=6965pi6wvhqvd1gf7hjsbwsygwlh9ph4.narinfo error="server returned 404: 404 page not found\n"15732026/09/23 12:21:58 INFO Received complete push request method=POST path=/api/pushes/1/complete15742026/09/23 12:21:58 WARN Failed to register uploaded object key=pkcmh2dn8y8fc9y35skhpy5ry0ly2hbq.narinfo error="server returned 404: 404 page not found\n"15752026/09/23 12:21:58 OK 20260920000000_drop_claims.sql (22.86ms)15762026/09/23 12:21:58 OK 20241026095416_initial_model.sql (173.99ms)15772026/09/23 12:21:58 INFO Upload complete. (177ms)1578=== NAME TestClientMultipleUploads1579 client_integration_test.go:369: Uploaded 3 paths in 208.712417ms15802026/09/23 12:21:58 OK 20251210153512_drop_unused_gin_index.sql (20ms)15812026/09/23 12:21:58 OK 20260923120000_add_pushes.sql (25.98ms)15822026/09/23 12:21:58 goose: successfully migrated database to version: 2026092312000015832026/09/23 12:21:58 OK 1_commit_pending_closure.sql (1.05ms)15842026/09/23 12:21:58 OK 2_object_stats_trigger.sql (250.04µs)15852026/09/23 12:21:58 OK 3_commit_push.sql (214.33µs)15862026/09/23 12:21:58 goose: up to current file version: 315872026/09/23 12:21:58 OK 20251218171726_add_pins.sql (48.43ms)15882026/09/23 12:21:58 OK 20260628120000_add_object_size_and_stats.sql (45.74ms)1589--- PASS: TestClientMultipleUploads (4.84s)1590=== CONT TestClientPushesUseOnePush15912026/09/23 12:21:58 OK 20260905000000_add_claims.sql (72.01ms)15922026/09/23 12:21:58 OK 20260920000000_drop_claims.sql (32.59ms)15932026/09/23 12:21:58 OK 20260923120000_add_pushes.sql (18.07ms)15942026/09/23 12:21:58 goose: successfully migrated database to version: 2026092312000015952026/09/23 12:21:58 OK 1_commit_pending_closure.sql (2.88ms)15962026/09/23 12:21:58 OK 2_object_stats_trigger.sql (622.71µs)15972026/09/23 12:21:58 OK 3_commit_push.sql (448.88µs)15982026/09/23 12:21:58 goose: up to current file version: 315992026-09-23 12:21:58.976 UTC [43855] ERROR: relation "goose_db_version" does not exist at character 3616002026-09-23 12:21:58.976 UTC [43855] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC16012026/09/23 12:21:59 OK 20241026095416_initial_model.sql (220.08ms)1602--- PASS: TestGCBugBareHashReferences (3.49s)1603=== CONT TestCacheStatsHandler16042026/09/23 12:21:59 OK 20251210153512_drop_unused_gin_index.sql (19.34ms)16052026/09/23 12:21:59 OK 20251218171726_add_pins.sql (35.78ms)16062026/09/23 12:21:59 OK 20260628120000_add_object_size_and_stats.sql (32.19ms)1607--- PASS: TestService_ReadScope_PublicByDefault (3.37s)1608=== CONT TestCacheConfigHandler1609=== RUN TestCacheConfigHandler/full_config,_no_issuer1610=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1611=== RUN TestCacheConfigHandler/no_cache_url_configured1612=== PAUSE TestCacheConfigHandler/no_cache_url_configured1613=== RUN TestCacheConfigHandler/no_signing_keys1614=== PAUSE TestCacheConfigHandler/no_signing_keys1615=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1616=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1617=== CONT TestPinProtectsFromGC16182026/09/23 12:21:59 OK 20260905000000_add_claims.sql (83.15ms)16192026/09/23 12:21:59 OK 20260920000000_drop_claims.sql (17.88ms)16202026/09/23 12:21:59 OK 20260923120000_add_pushes.sql (23.23ms)16212026/09/23 12:21:59 goose: successfully migrated database to version: 2026092312000016222026-09-23 12:21:59.485 UTC [43860] ERROR: relation "goose_db_version" does not exist at character 3616232026-09-23 12:21:59.485 UTC [43860] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC16242026/09/23 12:21:59 OK 1_commit_pending_closure.sql (3.6ms)16252026/09/23 12:21:59 OK 2_object_stats_trigger.sql (588.63µs)16262026/09/23 12:21:59 OK 3_commit_push.sql (357.29µs)16272026/09/23 12:21:59 goose: up to current file version: 316282026-09-23 12:21:59.611 UTC [43861] ERROR: relation "goose_db_version" does not exist at character 3616292026-09-23 12:21:59.611 UTC [43861] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC16302026/09/23 12:21:59 OK 20241026095416_initial_model.sql (167.59ms)16312026/09/23 12:21:59 OK 20251210153512_drop_unused_gin_index.sql (8.85ms)16322026/09/23 12:21:59 OK 20251218171726_add_pins.sql (38.32ms)16332026-09-23 12:21:59.784 UTC [43862] ERROR: relation "goose_db_version" does not exist at character 3616342026-09-23 12:21:59.784 UTC [43862] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC16352026/09/23 12:21:59 OK 20260628120000_add_object_size_and_stats.sql (59.44ms)16362026/09/23 12:21:59 INFO lead: acquired remote=192.0.2.1:123416372026/09/23 12:21:59 INFO lead: released remote=192.0.2.1:12341638--- PASS: TestLeadEndsOnShutdown (3.22s)1639=== CONT TestClientSharedPathCommittedMidPush16402026/09/23 12:21:59 OK 20260905000000_add_claims.sql (74.51ms)16412026/09/23 12:21:59 OK 20260920000000_drop_claims.sql (20.38ms)16422026/09/23 12:21:59 OK 20241026095416_initial_model.sql (216.29ms)16432026/09/23 12:21:59 OK 20260923120000_add_pushes.sql (5.2ms)16442026/09/23 12:21:59 goose: successfully migrated database to version: 2026092312000016452026/09/23 12:21:59 OK 20251210153512_drop_unused_gin_index.sql (7.47ms)16462026/09/23 12:21:59 OK 1_commit_pending_closure.sql (3.61ms)16472026/09/23 12:21:59 OK 2_object_stats_trigger.sql (623.54µs)16482026/09/23 12:21:59 OK 3_commit_push.sql (386.13µs)16492026/09/23 12:21:59 goose: up to current file version: 316502026/09/23 12:21:59 OK 20251218171726_add_pins.sql (26.77ms)16512026/09/23 12:21:59 OK 20241026095416_initial_model.sql (120.91ms)16522026/09/23 12:21:59 OK 20251210153512_drop_unused_gin_index.sql (14.41ms)16532026/09/23 12:22:00 OK 20260628120000_add_object_size_and_stats.sql (31.1ms)16542026/09/23 12:22:00 OK 20251218171726_add_pins.sql (15.12ms)16552026/09/23 12:22:00 OK 20260905000000_add_claims.sql (13.73ms)16562026/09/23 12:22:00 OK 20260628120000_add_object_size_and_stats.sql (9.13ms)16572026/09/23 12:22:00 OK 20260920000000_drop_claims.sql (14.29ms)16582026/09/23 12:22:00 OK 20260923120000_add_pushes.sql (9.77ms)16592026/09/23 12:22:00 goose: successfully migrated database to version: 2026092312000016602026/09/23 12:22:00 OK 1_commit_pending_closure.sql (2.26ms)16612026/09/23 12:22:00 OK 2_object_stats_trigger.sql (417.83µs)16622026/09/23 12:22:00 OK 3_commit_push.sql (325.17µs)16632026/09/23 12:22:00 goose: up to current file version: 316642026/09/23 12:22:00 OK 20260905000000_add_claims.sql (35.35ms)16652026/09/23 12:22:00 OK 20260920000000_drop_claims.sql (12.16ms)16662026/09/23 12:22:00 OK 20260923120000_add_pushes.sql (10.19ms)16672026/09/23 12:22:00 goose: successfully migrated database to version: 2026092312000016682026/09/23 12:22:00 OK 1_commit_pending_closure.sql (2.11ms)16692026/09/23 12:22:00 OK 2_object_stats_trigger.sql (486.88µs)16702026/09/23 12:22:00 OK 3_commit_push.sql (333.13µs)16712026/09/23 12:22:00 goose: up to current file version: 316722026-09-23 12:22:00.363 UTC [43870] ERROR: relation "goose_db_version" does not exist at character 3616732026-09-23 12:22:00.363 UTC [43870] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC1674=== NAME TestClientIntegration1675 client_integration_test.go:286: Created store path: /nix/var/nix/builds/nix-43626-243968142/TestClientIntegration1967160117/002/store/42lnpbk5x504hjz9gn3y0x8ns3w935il-test-file.txt16762026/09/23 12:22:00 OK 20241026095416_initial_model.sql (70.5ms)16772026/09/23 12:22:00 OK 20251210153512_drop_unused_gin_index.sql (7.35ms)16782026/09/23 12:22:00 INFO Received push request method=POST path=/api/pushes16792026/09/23 12:22:00 OK 20251218171726_add_pins.sql (20.18ms)16802026-09-23 12:22:00.517 UTC [43874] ERROR: relation "goose_db_version" does not exist at character 3616812026-09-23 12:22:00.517 UTC [43874] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC16822026/09/23 12:22:00 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)16832026/09/23 12:22:00 INFO Uploading 42lnpbk5x504hjz9gn3y0x8ns3w935il-test-file.txt (152B)16842026/09/23 12:22:00 WARN Failed to register uploaded object key=42lnpbk5x504hjz9gn3y0x8ns3w935il.ls error="server returned 404: 404 page not found\n"16852026/09/23 12:22:00 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16862026/09/23 12:22:00 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"16872026/09/23 12:22:00 INFO Signed narinfos id=1 count=116882026/09/23 12:22:00 INFO Uploading 1 narinfos16892026/09/23 12:22:00 OK 20260628120000_add_object_size_and_stats.sql (59.3ms)16902026/09/23 12:22:00 INFO Received complete push request method=POST path=/api/pushes/1/complete16912026/09/23 12:22:00 WARN Failed to register uploaded object key=42lnpbk5x504hjz9gn3y0x8ns3w935il.narinfo error="server returned 404: 404 page not found\n"16922026/09/23 12:22:00 INFO Upload complete. (136ms)16932026/09/23 12:22:00 OK 20260905000000_add_claims.sql (53.55ms)16942026/09/23 12:22:00 INFO All 1 paths already cached1695 client_integration_test.go:312: Retrieved narinfo from S3:1696 StorePath: /nix/var/nix/builds/nix-43626-243968142/TestClientIntegration1967160117/002/store/42lnpbk5x504hjz9gn3y0x8ns3w935il-test-file.txt1697 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1698 Compression: zstd1699 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11700 NarSize: 1521701 References: 1702 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11703 client_integration_test.go:313: Retrieved .ls file from S3 (compressed size: 77 bytes)1704 client_integration_test.go:313: Decompressed .ls content (64 bytes):1705 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}1706 client_integration_test.go:316: Testing garbage collection...17072026/09/23 12:22:00 INFO Starting cleanup of old closures method=DELETE path=/api/closures17082026/09/23 12:22:00 INFO Garbage collection started17092026/09/23 12:22:00 INFO Aborted multipart uploads count=017102026/09/23 12:22:00 WARN Force mode enabled - objects will be deleted immediately without grace period17112026/09/23 12:22:00 OK 20260920000000_drop_claims.sql (43.75ms)17122026/09/23 12:22:00 OK 20260923120000_add_pushes.sql (15.91ms)17132026/09/23 12:22:00 goose: successfully migrated database to version: 2026092312000017142026/09/23 12:22:00 OK 1_commit_pending_closure.sql (806.38µs)17152026/09/23 12:22:00 OK 2_object_stats_trigger.sql (201.88µs)17162026/09/23 12:22:00 OK 3_commit_push.sql (187.92µs)17172026/09/23 12:22:00 goose: up to current file version: 317182026/09/23 12:22:00 OK 20241026095416_initial_model.sql (126.27ms)17192026/09/23 12:22:00 OK 20251210153512_drop_unused_gin_index.sql (8.83ms)17202026/09/23 12:22:00 OK 20251218171726_add_pins.sql (7.68ms)17212026/09/23 12:22:00 OK 20260628120000_add_object_size_and_stats.sql (14.87ms)17222026/09/23 12:22:00 OK 20260905000000_add_claims.sql (36.28ms)17232026/09/23 12:22:00 INFO lead: acquired remote=192.0.2.1:123417242026/09/23 12:22:00 OK 20260920000000_drop_claims.sql (8.76ms)17252026/09/23 12:22:00 OK 20260923120000_add_pushes.sql (3.18ms)17262026/09/23 12:22:00 goose: successfully migrated database to version: 2026092312000017272026/09/23 12:22:00 OK 1_commit_pending_closure.sql (1.02ms)17282026/09/23 12:22:00 OK 2_object_stats_trigger.sql (269.38µs)17292026/09/23 12:22:00 OK 3_commit_push.sql (169.96µs)17302026/09/23 12:22:00 goose: up to current file version: 317312026-09-23 12:22:00.842 UTC [43887] ERROR: relation "goose_db_version" does not exist at character 3617322026-09-23 12:22:00.842 UTC [43887] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC17332026-09-23 12:22:00.857 UTC [43889] ERROR: relation "goose_db_version" does not exist at character 3617342026-09-23 12:22:00.857 UTC [43889] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC17352026/09/23 12:22:00 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=2001 objects-failed-to-delete=017362026/09/23 12:22:00 INFO Vacuumed table table=pending_closures17372026/09/23 12:22:00 INFO Vacuumed table table=pending_objects17382026/09/23 12:22:00 INFO Vacuumed table table=multipart_uploads17392026/09/23 12:22:00 INFO Vacuumed table table=closures1740=== NAME TestClientCADerivations1741 client_ca_test.go:136: Built CA derivation: /nix/var/nix/builds/nix-43626-243968142/TestClientCADerivations1424462887/001/store/j28m82v0dhbn0nmj2yz8vsbhnvrr050h-ca-test17422026/09/23 12:22:00 INFO Vacuumed table table=objects17432026/09/23 12:22:00 INFO lead: released remote=192.0.2.1:12341744 client_ca_test.go:139: Found 1 dependencies (including self)17452026/09/23 12:22:00 INFO lead: acquired remote=192.0.2.1:123417462026/09/23 12:22:00 INFO lead: released remote=192.0.2.1:12341747--- PASS: TestLeadElectsOneAndHandsOver (3.37s)1748=== CONT TestClientWithDependencies17492026-09-23 12:22:01.017 UTC [43893] ERROR: relation "goose_db_version" does not exist at character 3617502026-09-23 12:22:01.017 UTC [43893] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC17512026/09/23 12:22:01 OK 20241026095416_initial_model.sql (142.28ms)17522026/09/23 12:22:01 OK 20251210153512_drop_unused_gin_index.sql (11.89ms)17532026/09/23 12:22:01 OK 20251218171726_add_pins.sql (19.96ms)17542026/09/23 12:22:01 OK 20241026095416_initial_model.sql (153.26ms)17552026/09/23 12:22:01 OK 20251210153512_drop_unused_gin_index.sql (5.27ms)17562026/09/23 12:22:01 OK 20260628120000_add_object_size_and_stats.sql (14.43ms)17572026/09/23 12:22:01 OK 20251218171726_add_pins.sql (14.99ms)17582026/09/23 12:22:01 INFO Received push request method=POST path=/api/pushes17592026/09/23 12:22:01 OK 20260905000000_add_claims.sql (44.11ms)17602026/09/23 12:22:01 OK 20260628120000_add_object_size_and_stats.sql (39.44ms)17612026/09/23 12:22:01 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)17622026/09/23 12:22:01 INFO Uploading j28m82v0dhbn0nmj2yz8vsbhnvrr050h-ca-test (144B)17632026/09/23 12:22:01 OK 20260920000000_drop_claims.sql (19.61ms)17642026/09/23 12:22:01 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"17652026/09/23 12:22:01 WARN Failed to register uploaded object key=j28m82v0dhbn0nmj2yz8vsbhnvrr050h.ls error="server returned 404: 404 page not found\n"17662026/09/23 12:22:01 OK 20260923120000_add_pushes.sql (18.62ms)17672026/09/23 12:22:01 goose: successfully migrated database to version: 2026092312000017682026/09/23 12:22:01 OK 1_commit_pending_closure.sql (1.2ms)17692026/09/23 12:22:01 OK 20260905000000_add_claims.sql (31.17ms)17702026/09/23 12:22:01 OK 2_object_stats_trigger.sql (499.38µs)17712026/09/23 12:22:01 OK 3_commit_push.sql (187.13µs)17722026/09/23 12:22:01 goose: up to current file version: 317732026/09/23 12:22:01 WARN Failed to register uploaded object key=log/nl747l2mgza3n8zs8i0l5ww16vs2iy8a-ca-test.drv error="server returned 404: 404 page not found\n"17742026/09/23 12:22:01 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign17752026/09/23 12:22:01 INFO Signed narinfos id=1 count=117762026/09/23 12:22:01 INFO Uploading 1 narinfos17772026/09/23 12:22:01 OK 20260920000000_drop_claims.sql (36.33ms)17782026/09/23 12:22:01 INFO Received complete push request method=POST path=/api/pushes/1/complete17792026/09/23 12:22:01 WARN Failed to register uploaded object key=j28m82v0dhbn0nmj2yz8vsbhnvrr050h.narinfo error="server returned 404: 404 page not found\n"17802026/09/23 12:22:01 OK 20260923120000_add_pushes.sql (11.81ms)17812026/09/23 12:22:01 goose: successfully migrated database to version: 2026092312000017822026/09/23 12:22:01 OK 1_commit_pending_closure.sql (1.28ms)17832026/09/23 12:22:01 OK 2_object_stats_trigger.sql (214.46µs)17842026/09/23 12:22:01 OK 3_commit_push.sql (170µs)17852026/09/23 12:22:01 goose: up to current file version: 317862026/09/23 12:22:01 OK 20241026095416_initial_model.sql (155.8ms)17872026/09/23 12:22:01 INFO Upload complete. (224ms)1788=== NAME TestClientCADerivations1789 client_ca_test.go:180: Narinfo contains CA field: StorePath: /nix/var/nix/builds/nix-43626-243968142/TestClientCADerivations1424462887/001/store/j28m82v0dhbn0nmj2yz8vsbhnvrr050h-ca-test1790 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1791 Compression: zstd1792 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1793 NarSize: 1441794 References: 1795 Deriver: /nix/var/nix/builds/nix-43626-243968142/TestClientCADerivations1424462887/001/store/nl747l2mgza3n8zs8i0l5ww16vs2iy8a-ca-test.drv1796 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1797 client_ca_test.go:185: Checking for realisation files in S3...1798 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1799 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache18002026/09/23 12:22:01 OK 20251210153512_drop_unused_gin_index.sql (13.56ms)18012026/09/23 12:22:01 OK 20251218171726_add_pins.sql (30.94ms)18022026/09/23 12:22:01 OK 20260628120000_add_object_size_and_stats.sql (28.33ms)1803 client_ca_test.go:258: nix copy output: error: binary cache 's3://bucket42?endpoint=http://localhost:56682®ion=eu-west-1' is for Nix stores with prefix '/nix/store', not '/nix/var/nix/builds/nix-43626-243968142/TestClientCADerivations1424462887/001/store'1804 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 118052026/09/23 12:22:01 OK 20260905000000_add_claims.sql (35.04ms)18062026/09/23 12:22:01 OK 20260920000000_drop_claims.sql (19.29ms)1807--- PASS: TestClientCADerivations (4.04s)1808=== CONT TestService_ReadAuthMiddleware18092026/09/23 12:22:01 OK 20260923120000_add_pushes.sql (12.32ms)18102026/09/23 12:22:01 goose: successfully migrated database to version: 2026092312000018112026/09/23 12:22:01 OK 1_commit_pending_closure.sql (952µs)18122026/09/23 12:22:01 OK 2_object_stats_trigger.sql (286.08µs)18132026/09/23 12:22:01 OK 3_commit_push.sql (195.38µs)18142026/09/23 12:22:01 goose: up to current file version: 31815=== NAME TestOrphanedObjectsGCStressTest1816 orphaned_objects_gc_test.go:509: Stress test completed successfully:1817 orphaned_objects_gc_test.go:510: - Active objects preserved: 201818 orphaned_objects_gc_test.go:511: - Objects deleted: 2101819 orphaned_objects_gc_test.go:512: - Total GC'd: 2101820--- PASS: TestOrphanedObjectsGCStressTest (12.52s)1821=== CONT TestReadRedirectNar18222026/09/23 12:22:01 INFO Received uploads request method=POST path=/api/pending_closures18232026/09/23 12:22:01 INFO Received uploads request method=POST path=/api/pending_closures18242026/09/23 12:22:01 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)18252026/09/23 12:22:01 INFO Uploading k9m99qx073pxmds4w5dzwy240l2sx3j6-b (248B)18262026/09/23 12:22:01 INFO Uploading kj7jsxh11q6v9dy834sxmfl0fppm63qf-shared-dep (136B)18272026/09/23 12:22:01 WARN Failed to register uploaded object key=j84hi5cn84gwrsrg513idwxp97kvwpq5.ls error="server returned 404: 404 page not found\n"18282026/09/23 12:22:01 WARN Failed to register uploaded object key=k9m99qx073pxmds4w5dzwy240l2sx3j6.ls error="server returned 404: 404 page not found\n"18292026/09/23 12:22:01 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"18302026/09/23 12:22:01 WARN Failed to register uploaded object key=kj7jsxh11q6v9dy834sxmfl0fppm63qf.ls error="server returned 404: 404 page not found\n"18312026/09/23 12:22:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign18322026/09/23 12:22:01 WARN Failed to register uploaded object key=nar/0x1icc0xn1i33800scw27i5fq9nv6nqcl34gwwj2fy02ngaj5571.nar.zst error="server returned 404: 404 page not found\n"18332026/09/23 12:22:01 INFO Signed narinfos id=1 count=218342026/09/23 12:22:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign18352026/09/23 12:22:01 INFO Signed narinfos id=2 count=218362026/09/23 12:22:01 INFO Uploading 4 narinfos18372026/09/23 12:22:01 WARN Failed to register uploaded object key=kj7jsxh11q6v9dy834sxmfl0fppm63qf.narinfo error="server returned 404: 404 page not found\n"18382026/09/23 12:22:01 WARN Failed to register uploaded object key=k9m99qx073pxmds4w5dzwy240l2sx3j6.narinfo error="server returned 404: 404 page not found\n"18392026/09/23 12:22:01 WARN Failed to register uploaded object key=j84hi5cn84gwrsrg513idwxp97kvwpq5.narinfo error="server returned 404: 404 page not found\n"18402026/09/23 12:22:01 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete18412026/09/23 12:22:01 WARN Failed to register uploaded object key=kj7jsxh11q6v9dy834sxmfl0fppm63qf.narinfo error="server returned 404: 404 page not found\n"18422026/09/23 12:22:01 INFO Completed upload id=118432026/09/23 12:22:01 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete18442026/09/23 12:22:01 INFO Completed upload id=218452026/09/23 12:22:01 INFO Upload complete. (141ms)1846=== NAME TestClientFallsBackToClosures1847 client_pushes_test.go:112: Retrieved narinfo from S3:1848 StorePath: /nix/var/nix/builds/nix-43626-243968142/TestClientFallsBackToClosures1406305601/001/store/kj7jsxh11q6v9dy834sxmfl0fppm63qf-shared-dep1849 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1850 Compression: zstd1851 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821852 NarSize: 1361853 References: 1854 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1855 client_pushes_test.go:112: Retrieved narinfo from S3:1856 StorePath: /nix/var/nix/builds/nix-43626-243968142/TestClientFallsBackToClosures1406305601/001/store/j84hi5cn84gwrsrg513idwxp97kvwpq5-a1857 URL: nar/0x1icc0xn1i33800scw27i5fq9nv6nqcl34gwwj2fy02ngaj5571.nar.zst1858 Compression: zstd1859 NarHash: sha256:0x1icc0xn1i33800scw27i5fq9nv6nqcl34gwwj2fy02ngaj55711860 NarSize: 2481861 References: /nix/var/nix/builds/nix-43626-243968142/TestClientFallsBackToClosures1406305601/001/store/kj7jsxh11q6v9dy834sxmfl0fppm63qf-shared-dep1862 CA: text:sha256:0ah8xkrmfgpinf31706p6f6y0bwfygcq70mznxv0plsghx7lpfx21863 client_pushes_test.go:112: Retrieved narinfo from S3:1864 StorePath: /nix/var/nix/builds/nix-43626-243968142/TestClientFallsBackToClosures1406305601/001/store/k9m99qx073pxmds4w5dzwy240l2sx3j6-b1865 URL: nar/0x1icc0xn1i33800scw27i5fq9nv6nqcl34gwwj2fy02ngaj5571.nar.zst1866 Compression: zstd1867 NarHash: sha256:0x1icc0xn1i33800scw27i5fq9nv6nqcl34gwwj2fy02ngaj55711868 NarSize: 2481869 References: /nix/var/nix/builds/nix-43626-243968142/TestClientFallsBackToClosures1406305601/001/store/kj7jsxh11q6v9dy834sxmfl0fppm63qf-shared-dep1870 CA: text:sha256:0ah8xkrmfgpinf31706p6f6y0bwfygcq70mznxv0plsghx7lpfx21871--- PASS: TestCacheStatsHandler (2.35s)1872=== CONT TestService_RequireScope_OIDC1873--- PASS: TestClientFallsBackToClosures (3.07s)1874=== CONT TestService_AuthMiddleware_OIDC18752026/09/23 12:22:01 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56828/oidc18762026/09/23 12:22:01 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56831/oidc18772026/09/23 12:22:01 INFO Received push request method=POST path=/api/pushes18782026/09/23 12:22:01 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)18792026/09/23 12:22:01 INFO Uploading 5q2fdh9hb2ir38yln5880ai5civh81ij-shared-dep (136B)18802026/09/23 12:22:01 INFO Uploading 24kipw4g28y30nc2j0lf97n4karhng1y-b (248B)18812026/09/23 12:22:01 WARN Failed to register uploaded object key=24kipw4g28y30nc2j0lf97n4karhng1y.ls error="server returned 404: 404 page not found\n"18822026/09/23 12:22:01 WARN Failed to register uploaded object key=m08s3d7jv6xpx43s6hfim13rbk00fs8n.ls error="server returned 404: 404 page not found\n"18832026/09/23 12:22:01 WARN Failed to register uploaded object key=nar/1rk6nfv8wz86ljhlmzyfa2sjpalqaz9qr8kb5s1bnd4h41qsdmv3.nar.zst error="server returned 404: 404 page not found\n"18842026/09/23 12:22:01 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"18852026/09/23 12:22:01 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18862026/09/23 12:22:01 WARN Failed to register uploaded object key=5q2fdh9hb2ir38yln5880ai5civh81ij.ls error="server returned 404: 404 page not found\n"18872026/09/23 12:22:01 INFO Signed narinfos id=1 count=318882026/09/23 12:22:01 INFO Uploading 3 narinfos18892026/09/23 12:22:01 WARN Failed to register uploaded object key=m08s3d7jv6xpx43s6hfim13rbk00fs8n.narinfo error="server returned 404: 404 page not found\n"18902026/09/23 12:22:01 WARN Failed to register uploaded object key=5q2fdh9hb2ir38yln5880ai5civh81ij.narinfo error="server returned 404: 404 page not found\n"18912026/09/23 12:22:01 INFO Received complete push request method=POST path=/api/pushes/1/complete18922026/09/23 12:22:01 WARN Failed to register uploaded object key=24kipw4g28y30nc2j0lf97n4karhng1y.narinfo error="server returned 404: 404 page not found\n"18932026/09/23 12:22:01 INFO Upload complete. (107ms)1894=== NAME TestClientPushesUseOnePush1895 client_pushes_test.go:97: Retrieved narinfo from S3:1896 StorePath: /nix/var/nix/builds/nix-43626-243968142/TestClientPushesUseOnePush4078082828/001/store/5q2fdh9hb2ir38yln5880ai5civh81ij-shared-dep1897 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1898 Compression: zstd1899 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821900 NarSize: 1361901 References: 1902 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1903 client_pushes_test.go:97: Retrieved narinfo from S3:1904 StorePath: /nix/var/nix/builds/nix-43626-243968142/TestClientPushesUseOnePush4078082828/001/store/m08s3d7jv6xpx43s6hfim13rbk00fs8n-a1905 URL: nar/1rk6nfv8wz86ljhlmzyfa2sjpalqaz9qr8kb5s1bnd4h41qsdmv3.nar.zst1906 Compression: zstd1907 NarHash: sha256:1rk6nfv8wz86ljhlmzyfa2sjpalqaz9qr8kb5s1bnd4h41qsdmv31908 NarSize: 2481909 References: /nix/var/nix/builds/nix-43626-243968142/TestClientPushesUseOnePush4078082828/001/store/5q2fdh9hb2ir38yln5880ai5civh81ij-shared-dep1910 CA: text:sha256:0mz8pvkgr4fbmi317xw9a81bbinam4rhk9x5zzqm1ddmgr46c3g41911 client_pushes_test.go:97: Retrieved narinfo from S3:1912 StorePath: /nix/var/nix/builds/nix-43626-243968142/TestClientPushesUseOnePush4078082828/001/store/24kipw4g28y30nc2j0lf97n4karhng1y-b1913 URL: nar/1rk6nfv8wz86ljhlmzyfa2sjpalqaz9qr8kb5s1bnd4h41qsdmv3.nar.zst1914 Compression: zstd1915 NarHash: sha256:1rk6nfv8wz86ljhlmzyfa2sjpalqaz9qr8kb5s1bnd4h41qsdmv31916 NarSize: 2481917 References: /nix/var/nix/builds/nix-43626-243968142/TestClientPushesUseOnePush4078082828/001/store/5q2fdh9hb2ir38yln5880ai5civh81ij-shared-dep1918 CA: text:sha256:0mz8pvkgr4fbmi317xw9a81bbinam4rhk9x5zzqm1ddmgr46c3g41919--- PASS: TestClientPushesUseOnePush (3.10s)1920=== CONT TestReadRedirectUsesPublicS3URL1921=== NAME TestPinProtectsFromGC1922 client_integration_test.go:731: Pinned store path: /nix/var/nix/builds/nix-43626-243968142/TestPinProtectsFromGC3385078389/001/store/q0vn0kj4163bmqxjwa588az6nj657cc4-pinned-file.txt1923 client_integration_test.go:732: Unpinned store path: /nix/var/nix/builds/nix-43626-243968142/TestPinProtectsFromGC3385078389/001/store/zngjlxryqx7hrp5gaqcnzg906f23i8wc-unpinned-file.txt19242026-09-23 12:22:02.033 UTC [43943] ERROR: relation "goose_db_version" does not exist at character 3619252026-09-23 12:22:02.033 UTC [43943] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC19262026/09/23 12:22:02 INFO Received push request method=POST path=/api/pushes19272026/09/23 12:22:02 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)19282026/09/23 12:22:02 INFO Uploading q0vn0kj4163bmqxjwa588az6nj657cc4-pinned-file.txt (128B)19292026/09/23 12:22:02 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign19302026/09/23 12:22:02 WARN Failed to register uploaded object key=q0vn0kj4163bmqxjwa588az6nj657cc4.ls error="server returned 404: 404 page not found\n"19312026/09/23 12:22:02 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"19322026/09/23 12:22:02 INFO Signed narinfos id=1 count=119332026/09/23 12:22:02 INFO Uploading 1 narinfos19342026/09/23 12:22:02 INFO Received complete push request method=POST path=/api/pushes/1/complete19352026/09/23 12:22:02 WARN Failed to register uploaded object key=q0vn0kj4163bmqxjwa588az6nj657cc4.narinfo error="server returned 404: 404 page not found\n"19362026/09/23 12:22:02 INFO Upload complete. (83ms)19372026/09/23 12:22:02 OK 20241026095416_initial_model.sql (52.27ms)19382026/09/23 12:22:02 OK 20251210153512_drop_unused_gin_index.sql (1.24ms)19392026/09/23 12:22:02 OK 20251218171726_add_pins.sql (7.89ms)19402026/09/23 12:22:02 OK 20260628120000_add_object_size_and_stats.sql (8.68ms)19412026/09/23 12:22:02 OK 20260905000000_add_claims.sql (1.37ms)19422026/09/23 12:22:02 OK 20260920000000_drop_claims.sql (1.34ms)19432026/09/23 12:22:02 OK 20260923120000_add_pushes.sql (1.07ms)19442026/09/23 12:22:02 goose: successfully migrated database to version: 2026092312000019452026/09/23 12:22:02 OK 1_commit_pending_closure.sql (1.04ms)19462026/09/23 12:22:02 OK 2_object_stats_trigger.sql (520.88µs)19472026/09/23 12:22:02 OK 3_commit_push.sql (376.29µs)19482026/09/23 12:22:02 goose: up to current file version: 319492026-09-23 12:22:02.153 UTC [43951] ERROR: relation "goose_db_version" does not exist at character 3619502026-09-23 12:22:02.153 UTC [43951] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC19512026/09/23 12:22:02 INFO Received push request method=POST path=/api/pushes19522026/09/23 12:22:02 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)19532026/09/23 12:22:02 INFO Uploading zngjlxryqx7hrp5gaqcnzg906f23i8wc-unpinned-file.txt (128B)19542026/09/23 12:22:02 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"19552026/09/23 12:22:02 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign19562026/09/23 12:22:02 WARN Failed to register uploaded object key=zngjlxryqx7hrp5gaqcnzg906f23i8wc.ls error="server returned 404: 404 page not found\n"19572026/09/23 12:22:02 INFO Signed narinfos id=2 count=119582026/09/23 12:22:02 INFO Uploading 1 narinfos19592026/09/23 12:22:02 INFO Received complete push request method=POST path=/api/pushes/2/complete19602026/09/23 12:22:02 WARN Failed to register uploaded object key=zngjlxryqx7hrp5gaqcnzg906f23i8wc.narinfo error="server returned 404: 404 page not found\n"19612026/09/23 12:22:02 INFO Upload complete. (68ms)19622026/09/23 12:22:02 INFO Received push request method=POST path=/api/pushes19632026/09/23 12:22:02 INFO Received create pin request method=POST path=/api/pins/myapp19642026/09/23 12:22:02 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)19652026/09/23 12:22:02 INFO Uploading i6m6ma4qg3l17qhjildj6g4jwi02j009-shared-dep (136B)19662026/09/23 12:22:02 INFO Uploading pny79d9pjh3ngzgwk515n79zpkspfz89-top (256B)19672026/09/23 12:22:02 INFO Created/updated pin name=myapp store_path=/nix/var/nix/builds/nix-43626-243968142/TestPinProtectsFromGC3385078389/001/store/q0vn0kj4163bmqxjwa588az6nj657cc4-pinned-file.txt narinfo_key=q0vn0kj4163bmqxjwa588az6nj657cc4.narinfo19682026/09/23 12:22:02 WARN Failed to register uploaded object key=i6m6ma4qg3l17qhjildj6g4jwi02j009.ls error="server returned 404: 404 page not found\n"19692026/09/23 12:22:02 INFO Starting cleanup of old closures method=DELETE path=/api/closures19702026/09/23 12:22:02 INFO Garbage collection started19712026/09/23 12:22:02 INFO Aborted multipart uploads count=019722026/09/23 12:22:02 WARN Force mode enabled - objects will be deleted immediately without grace period19732026/09/23 12:22:02 WARN Failed to register uploaded object key=nar/185zznx6rzmn45a21zmlc214s5klvlmd28r5qj0f1krjkax23bq0.nar.zst error="server returned 404: 404 page not found\n"19742026/09/23 12:22:02 WARN Failed to register uploaded object key=pny79d9pjh3ngzgwk515n79zpkspfz89.ls error="server returned 404: 404 page not found\n"19752026/09/23 12:22:02 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign19762026/09/23 12:22:02 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"19772026/09/23 12:22:02 INFO Signed narinfos id=1 count=219782026/09/23 12:22:02 INFO Uploading 2 narinfos19792026/09/23 12:22:02 WARN Failed to register uploaded object key=i6m6ma4qg3l17qhjildj6g4jwi02j009.narinfo error="server returned 404: 404 page not found\n"19802026/09/23 12:22:02 INFO Received complete push request method=POST path=/api/pushes/1/complete19812026/09/23 12:22:02 WARN Failed to register uploaded object key=pny79d9pjh3ngzgwk515n79zpkspfz89.narinfo error="server returned 404: 404 page not found\n"19822026/09/23 12:22:02 INFO Upload complete. (142ms)1983=== NAME TestClientSharedPathCommittedMidPush1984 client_integration_test.go:680: Retrieved narinfo from S3:1985 StorePath: /nix/var/nix/builds/nix-43626-243968142/TestClientSharedPathCommittedMidPush2837163524/001/store/i6m6ma4qg3l17qhjildj6g4jwi02j009-shared-dep1986 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1987 Compression: zstd1988 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821989 NarSize: 1361990 References: 1991 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1992 client_integration_test.go:680: Retrieved narinfo from S3:1993 StorePath: /nix/var/nix/builds/nix-43626-243968142/TestClientSharedPathCommittedMidPush2837163524/001/store/pny79d9pjh3ngzgwk515n79zpkspfz89-top1994 URL: nar/185zznx6rzmn45a21zmlc214s5klvlmd28r5qj0f1krjkax23bq0.nar.zst1995 Compression: zstd1996 NarHash: sha256:185zznx6rzmn45a21zmlc214s5klvlmd28r5qj0f1krjkax23bq01997 NarSize: 2561998 References: /nix/var/nix/builds/nix-43626-243968142/TestClientSharedPathCommittedMidPush2837163524/001/store/i6m6ma4qg3l17qhjildj6g4jwi02j009-shared-dep1999 CA: text:sha256:1hw3a7c70w5cf8p31y3vrwly2jd645xn0k4r7ishfiiqgc7bnmw520002026/09/23 12:22:02 WARN Rate limiter enabled after throttle name=s3-test rate=520012026/09/23 12:22:02 WARN S3 rate limit hit operation="complete multipart upload" error="Please reduce your request rate."2002=== NAME TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle2003 throttle_test.go:213: Proxy stats: total=13, throttled=10, completeMultipart=102004 throttle_test.go:215: Rate limiter: enabled=true, rate=5.002005--- PASS: TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle (8.50s)2006=== CONT TestService_verifyS3Integrity20072026/09/23 12:22:02 OK 20241026095416_initial_model.sql (132.07ms)20082026/09/23 12:22:02 OK 20251210153512_drop_unused_gin_index.sql (7.22ms)2009--- PASS: TestClientSharedPathCommittedMidPush (2.50s)2010=== CONT TestReadProxyRangeRequest20112026/09/23 12:22:02 OK 20251218171726_add_pins.sql (16.44ms)20122026/09/23 12:22:02 OK 20260628120000_add_object_size_and_stats.sql (16.88ms)20132026/09/23 12:22:02 OK 20260905000000_add_claims.sql (29.28ms)20142026/09/23 12:22:02 OK 20260920000000_drop_claims.sql (9.14ms)20152026/09/23 12:22:02 OK 20260923120000_add_pushes.sql (7.76ms)20162026/09/23 12:22:02 goose: successfully migrated database to version: 2026092312000020172026/09/23 12:22:02 OK 1_commit_pending_closure.sql (1.25ms)20182026/09/23 12:22:02 OK 2_object_stats_trigger.sql (331µs)20192026/09/23 12:22:02 OK 3_commit_push.sql (225µs)20202026/09/23 12:22:02 goose: up to current file version: 320212026-09-23 12:22:02.431 UTC [43965] ERROR: relation "goose_db_version" does not exist at character 3620222026-09-23 12:22:02.431 UTC [43965] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC20232026/09/23 12:22:02 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=2001 objects-failed-to-delete=020242026/09/23 12:22:02 INFO Vacuumed table table=pending_closures20252026/09/23 12:22:02 INFO Vacuumed table table=pending_objects20262026/09/23 12:22:02 INFO Vacuumed table table=multipart_uploads20272026/09/23 12:22:02 INFO Vacuumed table table=closures20282026/09/23 12:22:02 INFO Vacuumed table table=objects20292026/09/23 12:22:02 OK 20241026095416_initial_model.sql (140.14ms)20302026/09/23 12:22:02 OK 20251210153512_drop_unused_gin_index.sql (13.93ms)2031--- PASS: TestService_ReadAuthMiddleware (1.30s)20322026/09/23 12:22:02 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=2001 objects_failed=02033=== CONT TestReadProxyRootRedirectsToIndexHTML20342026/09/23 12:22:02 OK 20251218171726_add_pins.sql (17.09ms)2035=== NAME TestClientIntegration2036 client_integration_test.go:323: Objects in database after GC:2037 client_integration_test.go:323: Successfully deleted all objects with GC --force2038--- PASS: TestClientIntegration (5.75s)2039=== CONT TestReadRedirectKeepsNarinfoProxied20402026/09/23 12:22:02 OK 20260628120000_add_object_size_and_stats.sql (28.64ms)20412026/09/23 12:22:02 OK 20260905000000_add_claims.sql (21.6ms)20422026/09/23 12:22:02 OK 20260920000000_drop_claims.sql (21.81ms)2043=== NAME TestClientWithDependencies2044 client_integration_test.go:613: Built derivation: /nix/var/nix/builds/nix-43626-243968142/TestClientWithDependencies1038564153/001/store/a1js391kxzv1jr0kl6lyy08ypk87sp2g-test-script20452026/09/23 12:22:02 OK 20260923120000_add_pushes.sql (11.36ms)20462026/09/23 12:22:02 goose: successfully migrated database to version: 2026092312000020472026/09/23 12:22:02 OK 1_commit_pending_closure.sql (1.59ms)20482026/09/23 12:22:02 OK 2_object_stats_trigger.sql (368.29µs)20492026/09/23 12:22:02 OK 3_commit_push.sql (402.25µs)20502026/09/23 12:22:02 goose: up to current file version: 320512026-09-23 12:22:02.735 UTC [43975] ERROR: relation "goose_db_version" does not exist at character 3620522026-09-23 12:22:02.735 UTC [43975] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC20532026-09-23 12:22:02.736 UTC [43976] ERROR: relation "goose_db_version" does not exist at character 3620542026-09-23 12:22:02.736 UTC [43976] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC2055 client_integration_test.go:615: Found 1 dependencies (including self)20562026/09/23 12:22:02 OK 20241026095416_initial_model.sql (45.89ms)20572026/09/23 12:22:02 OK 20241026095416_initial_model.sql (46.26ms)20582026/09/23 12:22:02 OK 20251210153512_drop_unused_gin_index.sql (8.54ms)20592026/09/23 12:22:02 OK 20251210153512_drop_unused_gin_index.sql (8.15ms)20602026/09/23 12:22:02 OK 20251218171726_add_pins.sql (11.3ms)20612026/09/23 12:22:02 INFO Received push request method=POST path=/api/pushes20622026/09/23 12:22:02 OK 20251218171726_add_pins.sql (11.81ms)20632026/09/23 12:22:02 OK 20260628120000_add_object_size_and_stats.sql (17.75ms)20642026/09/23 12:22:02 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)20652026/09/23 12:22:02 INFO Uploading a1js391kxzv1jr0kl6lyy08ypk87sp2g-test-script (136B)20662026/09/23 12:22:02 OK 20260628120000_add_object_size_and_stats.sql (19.2ms)20672026/09/23 12:22:02 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"20682026/09/23 12:22:02 WARN Failed to register uploaded object key=a1js391kxzv1jr0kl6lyy08ypk87sp2g.ls error="server returned 404: 404 page not found\n"20692026/09/23 12:22:02 WARN Failed to register uploaded object key=log/h78dx6lnjajw1vjv5pphaf7sd2pq2lgp-test-script.drv error="server returned 404: 404 page not found\n"20702026/09/23 12:22:02 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign20712026/09/23 12:22:02 INFO Signed narinfos id=1 count=120722026/09/23 12:22:02 INFO Uploading 1 narinfos20732026/09/23 12:22:02 OK 20260905000000_add_claims.sql (48.82ms)20742026/09/23 12:22:02 INFO Received complete push request method=POST path=/api/pushes/1/complete20752026/09/23 12:22:02 WARN Failed to register uploaded object key=a1js391kxzv1jr0kl6lyy08ypk87sp2g.narinfo error="server returned 404: 404 page not found\n"20762026/09/23 12:22:02 INFO Upload complete. (139ms)20772026/09/23 12:22:02 OK 20260905000000_add_claims.sql (64.55ms)2078 client_integration_test.go:617: Skipping nix copy test - isolated store (/nix/var/nix/builds/nix-43626-243968142/TestClientWithDependencies1038564153/001/store) requires matching store prefix20792026/09/23 12:22:02 OK 20260920000000_drop_claims.sql (32.88ms)20802026/09/23 12:22:02 OK 20260923120000_add_pushes.sql (5.77ms)20812026/09/23 12:22:02 goose: successfully migrated database to version: 2026092312000020822026/09/23 12:22:02 OK 1_commit_pending_closure.sql (1.19ms)20832026/09/23 12:22:02 OK 2_object_stats_trigger.sql (239.96µs)20842026/09/23 12:22:02 OK 3_commit_push.sql (205.21µs)20852026/09/23 12:22:02 goose: up to current file version: 320862026/09/23 12:22:02 OK 20260920000000_drop_claims.sql (26.91ms)20872026-09-23 12:22:02.973 UTC [43982] ERROR: relation "goose_db_version" does not exist at character 3620882026-09-23 12:22:02.973 UTC [43982] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC2089--- PASS: TestReadRedirectNar (1.54s)2090=== CONT TestReadProxyDisabled20912026/09/23 12:22:02 OK 20260923120000_add_pushes.sql (22.67ms)20922026/09/23 12:22:02 goose: successfully migrated database to version: 2026092312000020932026/09/23 12:22:02 OK 1_commit_pending_closure.sql (1.93ms)20942026/09/23 12:22:02 OK 2_object_stats_trigger.sql (242.63µs)20952026/09/23 12:22:02 OK 3_commit_push.sql (207.38µs)20962026/09/23 12:22:02 goose: up to current file version: 32097--- PASS: TestClientWithDependencies (2.02s)2098=== CONT TestGCTaskStore_DeduplicateSameParams2099--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)2100=== CONT TestReadProxyConditionalGet21012026/09/23 12:22:03 OK 20241026095416_initial_model.sql (58.52ms)21022026/09/23 12:22:03 OK 20251210153512_drop_unused_gin_index.sql (6.51ms)21032026/09/23 12:22:03 OK 20251218171726_add_pins.sql (8.01ms)21042026/09/23 12:22:03 OK 20260628120000_add_object_size_and_stats.sql (27.51ms)21052026/09/23 12:22:03 OK 20260905000000_add_claims.sql (34.11ms)2106=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token2107=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token2108=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected2109=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected2110=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected2111=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected2112=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured2113=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured2114=== CONT TestService_AuthMiddleware_MTLSBoundSubjects21152026/09/23 12:22:03 OK 20260920000000_drop_claims.sql (26.91ms)21162026/09/23 12:22:03 OK 20260923120000_add_pushes.sql (22.19ms)21172026/09/23 12:22:03 goose: successfully migrated database to version: 2026092312000021182026/09/23 12:22:03 OK 1_commit_pending_closure.sql (1.34ms)21192026/09/23 12:22:03 OK 2_object_stats_trigger.sql (349.63µs)21202026/09/23 12:22:03 OK 3_commit_push.sql (306.96µs)21212026/09/23 12:22:03 goose: up to current file version: 32122=== RUN TestService_RequireScope_OIDC/builder_may_write2123=== PAUSE TestService_RequireScope_OIDC/builder_may_write2124=== RUN TestService_RequireScope_OIDC/builder_may_not_admin2125=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin2126=== RUN TestService_RequireScope_OIDC/ops_may_admin2127=== PAUSE TestService_RequireScope_OIDC/ops_may_admin2128=== RUN TestService_RequireScope_OIDC/ops_may_not_write2129=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write2130=== RUN TestService_RequireScope_OIDC/reader_may_not_write2131=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write2132=== RUN TestService_RequireScope_OIDC/static_token_may_admin2133=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin2134=== RUN TestService_RequireScope_OIDC/static_token_may_write2135=== PAUSE TestService_RequireScope_OIDC/static_token_may_write2136=== RUN TestService_RequireScope_OIDC/reader_may_read2137=== PAUSE TestService_RequireScope_OIDC/reader_may_read2138=== RUN TestService_RequireScope_OIDC/writer_implies_read2139=== PAUSE TestService_RequireScope_OIDC/writer_implies_read2140=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read2141=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read2142=== CONT TestService_createPendingClosureHandler21432026-09-23 12:22:03.522 UTC [43991] ERROR: relation "goose_db_version" does not exist at character 3621442026-09-23 12:22:03.522 UTC [43991] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC2145--- PASS: TestReadRedirectUsesPublicS3URL (1.75s)2146=== CONT TestService_AuthMiddleware_MTLSProxyHeader21472026-09-23 12:22:03.612 UTC [43992] ERROR: relation "goose_db_version" does not exist at character 3621482026-09-23 12:22:03.612 UTC [43992] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC21492026/09/23 12:22:03 OK 20241026095416_initial_model.sql (109.19ms)21502026/09/23 12:22:03 OK 20251210153512_drop_unused_gin_index.sql (1.76ms)21512026/09/23 12:22:03 OK 20251218171726_add_pins.sql (4.89ms)21522026-09-23 12:22:03.687 UTC [43995] ERROR: relation "goose_db_version" does not exist at character 3621532026-09-23 12:22:03.687 UTC [43995] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC21542026/09/23 12:22:03 OK 20260628120000_add_object_size_and_stats.sql (9.76ms)21552026-09-23 12:22:03.690 UTC [43996] ERROR: relation "goose_db_version" does not exist at character 3621562026-09-23 12:22:03.690 UTC [43996] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC21572026/09/23 12:22:03 OK 20241026095416_initial_model.sql (47.58ms)21582026/09/23 12:22:03 OK 20251210153512_drop_unused_gin_index.sql (525.46µs)21592026/09/23 12:22:03 OK 20251218171726_add_pins.sql (5.45ms)21602026/09/23 12:22:03 OK 20260905000000_add_claims.sql (10.52ms)21612026/09/23 12:22:03 OK 20260920000000_drop_claims.sql (1.64ms)21622026/09/23 12:22:03 OK 20260628120000_add_object_size_and_stats.sql (5.29ms)21632026/09/23 12:22:03 OK 20260923120000_add_pushes.sql (1.09ms)21642026/09/23 12:22:03 goose: successfully migrated database to version: 2026092312000021652026/09/23 12:22:03 OK 1_commit_pending_closure.sql (1.6ms)21662026/09/23 12:22:03 OK 2_object_stats_trigger.sql (273.79µs)21672026/09/23 12:22:03 OK 3_commit_push.sql (217.04µs)21682026/09/23 12:22:03 goose: up to current file version: 321692026/09/23 12:22:03 OK 20260905000000_add_claims.sql (3.52ms)21702026/09/23 12:22:03 OK 20260920000000_drop_claims.sql (1.79ms)21712026/09/23 12:22:03 OK 20260923120000_add_pushes.sql (33.19ms)21722026/09/23 12:22:03 goose: successfully migrated database to version: 2026092312000021732026/09/23 12:22:03 OK 1_commit_pending_closure.sql (1.33ms)21742026/09/23 12:22:03 OK 2_object_stats_trigger.sql (296.75µs)21752026/09/23 12:22:03 OK 3_commit_push.sql (235.83µs)21762026/09/23 12:22:03 goose: up to current file version: 321772026/09/23 12:22:03 OK 20241026095416_initial_model.sql (59.7ms)21782026/09/23 12:22:03 OK 20251210153512_drop_unused_gin_index.sql (7.65ms)21792026/09/23 12:22:03 OK 20251218171726_add_pins.sql (17.41ms)21802026/09/23 12:22:03 OK 20241026095416_initial_model.sql (85.06ms)21812026/09/23 12:22:03 OK 20251210153512_drop_unused_gin_index.sql (13.57ms)21822026/09/23 12:22:03 OK 20260628120000_add_object_size_and_stats.sql (24.69ms)21832026/09/23 12:22:03 OK 20251218171726_add_pins.sql (8.61ms)21842026/09/23 12:22:03 OK 20260628120000_add_object_size_and_stats.sql (16.21ms)21852026/09/23 12:22:03 OK 20260905000000_add_claims.sql (23.71ms)21862026/09/23 12:22:03 OK 20260920000000_drop_claims.sql (13.92ms)21872026/09/23 12:22:03 OK 20260905000000_add_claims.sql (23.53ms)21882026/09/23 12:22:03 OK 20260923120000_add_pushes.sql (10.83ms)21892026/09/23 12:22:03 goose: successfully migrated database to version: 2026092312000021902026/09/23 12:22:03 OK 1_commit_pending_closure.sql (2.63ms)21912026/09/23 12:22:03 OK 2_object_stats_trigger.sql (444.54µs)21922026/09/23 12:22:03 OK 3_commit_push.sql (359.67µs)21932026/09/23 12:22:03 goose: up to current file version: 321942026/09/23 12:22:03 OK 20260920000000_drop_claims.sql (17.7ms)21952026/09/23 12:22:03 OK 20260923120000_add_pushes.sql (16.09ms)21962026/09/23 12:22:03 goose: successfully migrated database to version: 2026092312000021972026/09/23 12:22:03 OK 1_commit_pending_closure.sql (3.25ms)21982026/09/23 12:22:03 OK 2_object_stats_trigger.sql (1.27ms)21992026/09/23 12:22:03 OK 3_commit_push.sql (368.13µs)22002026/09/23 12:22:03 goose: up to current file version: 322012026/09/23 12:22:03 INFO Received uploads request method=POST path=/api/pending_closures22022026/09/23 12:22:04 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=2001 objects_failed=02203=== NAME TestPinProtectsFromGC2204 client_integration_test.go:794: Pin successfully protected closure from garbage collection2205--- PASS: TestReadProxyRangeRequest (1.92s)2206=== CONT TestIsValidUploadKey/narinfo2207=== CONT TestIsValidUploadKey/nar_key,_narinfo_type2208=== CONT TestIsValidUploadKey/narinfo_key,_nar_type2209=== CONT TestIsValidUploadKey/index.html2210=== CONT TestIsValidUploadKey/nix-cache-info2211=== CONT TestIsValidUploadKey/realisation_plus_in_output2212=== CONT TestIsValidUploadKey/realisation2213=== CONT TestIsValidUploadKey/listing_key,_narinfo_type2214=== CONT TestIsValidUploadKey/build_log_equals2215=== CONT TestIsValidUploadKey/build_log_question_mark2216=== CONT TestIsValidUploadKey/build_log_plus_in_name2217=== CONT TestIsValidUploadKey/build_log_home-manager_file2218=== CONT TestIsValidUploadKey/build_log2219=== CONT TestIsValidUploadKey/listing2220=== CONT TestIsValidUploadKey/nar_plain2221=== CONT TestIsValidUploadKey/nar_xz2222=== CONT TestIsValidUploadKey/nar_zst2223=== CONT TestIsValidUploadKey/absolute2224=== CONT TestIsValidUploadKey/unknown_type2225=== CONT TestIsValidUploadKey/empty_key2226=== CONT TestIsValidUploadKey/traversal_nar2227=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info22282026/09/23 12:22:04 INFO Received uploads request method=POST path=/2229=== CONT TestProxyWriteTimeout/narinfo2230=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key22312026/09/23 12:22:04 INFO Received request for more parts method=POST path=/2232=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key22332026/09/23 12:22:04 INFO Received complete multipart upload request method=POST path=/2234=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal22352026/09/23 12:22:04 INFO Received uploads request method=POST path=/2236--- PASS: TestUploadHandlersRejectInvalidKeys (0.00s)2237 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)2238 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)2239 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)2240 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)2241=== CONT TestProxyWriteTimeout/10_GiB_nar2242=== CONT TestProxyWriteTimeout/unknown_size2243=== CONT TestProxyWriteTimeout/1_GiB_nar2244--- PASS: TestProxyWriteTimeout (0.00s)2245 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)2246 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)2247 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)2248 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)2249=== CONT TestIsValidUploadKey/traversal2250--- PASS: TestIsValidUploadKey (0.03s)2251 --- PASS: TestIsValidUploadKey/narinfo (0.00s)2252 --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s)2253 --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s)2254 --- PASS: TestIsValidUploadKey/index.html (0.00s)2255 --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s)2256 --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s)2257 --- PASS: TestIsValidUploadKey/realisation (0.00s)2258 --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s)2259 --- PASS: TestIsValidUploadKey/build_log_equals (0.00s)2260 --- PASS: TestIsValidUploadKey/build_log_question_mark (0.00s)2261 --- PASS: TestIsValidUploadKey/build_log_plus_in_name (0.00s)2262 --- PASS: TestIsValidUploadKey/build_log_home-manager_file (0.00s)2263 --- PASS: TestIsValidUploadKey/build_log (0.00s)2264 --- PASS: TestIsValidUploadKey/listing (0.00s)2265 --- PASS: TestIsValidUploadKey/nar_plain (0.00s)2266 --- PASS: TestIsValidUploadKey/nar_xz (0.00s)2267 --- PASS: TestIsValidUploadKey/nar_zst (0.00s)2268 --- PASS: TestIsValidUploadKey/absolute (0.00s)2269 --- PASS: TestIsValidUploadKey/unknown_type (0.00s)2270 --- PASS: TestIsValidUploadKey/empty_key (0.00s)2271 --- PASS: TestIsValidUploadKey/traversal_nar (0.00s)2272 --- PASS: TestIsValidUploadKey/traversal (0.00s)2273=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure22742026/09/23 12:22:04 INFO Received uploads request method=POST path=/2275--- PASS: TestPinProtectsFromGC (4.94s)2276=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts22772026/09/23 12:22:04 INFO Received request for more parts method=POST path=/22782026-09-23 12:22:04.351 UTC [43997] ERROR: relation "goose_db_version" does not exist at character 3622792026-09-23 12:22:04.351 UTC [43997] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC2280=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart22812026/09/23 12:22:04 INFO Received complete multipart upload request method=POST path=/22822026-09-23 12:22:04.379 UTC [43998] ERROR: relation "goose_db_version" does not exist at character 3622832026-09-23 12:22:04.379 UTC [43998] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC2284=== CONT TestIsValidCachePath/narinfo2285=== CONT TestIsValidCachePath/index.html2286=== CONT TestIsValidCachePath/nix-cache-info2287=== CONT TestIsValidCachePath/realisation2288=== CONT TestIsValidCachePath/log2289=== CONT TestIsValidCachePath/ls2290=== CONT TestIsValidCachePath/nar_uncompressed2291=== CONT TestIsValidCachePath/nar_bz22292=== CONT TestIsValidCachePath/nar_xz2293=== CONT TestIsValidCachePath/nar_zst2294=== CONT TestIsValidCachePath/narinfo_all_nix_base32_chars2295=== CONT TestIsValidCachePath/empty2296=== CONT TestIsValidCachePath/wrong_extension2297=== CONT TestIsValidCachePath/leading_slash2298=== CONT TestIsValidCachePath/invalid_char_u2299=== CONT TestIsValidCachePath/random_path2300=== CONT TestIsValidCachePath/short_hash2301=== CONT TestIsValidCachePath/invalid_char_e2302=== CONT TestIsValidCachePath/traversal_in_middle2303=== CONT TestIsValidCachePath/traversal_parent2304--- PASS: TestIsValidCachePath (0.00s)2305 --- PASS: TestIsValidCachePath/narinfo (0.00s)2306 --- PASS: TestIsValidCachePath/index.html (0.00s)2307 --- PASS: TestIsValidCachePath/nix-cache-info (0.00s)2308 --- PASS: TestIsValidCachePath/realisation (0.00s)2309 --- PASS: TestIsValidCachePath/log (0.00s)2310 --- PASS: TestIsValidCachePath/ls (0.00s)2311 --- PASS: TestIsValidCachePath/nar_uncompressed (0.00s)2312 --- PASS: TestIsValidCachePath/nar_bz2 (0.00s)2313 --- PASS: TestIsValidCachePath/nar_xz (0.00s)2314 --- PASS: TestIsValidCachePath/nar_zst (0.00s)2315 --- PASS: TestIsValidCachePath/narinfo_all_nix_base32_chars (0.00s)2316 --- PASS: TestIsValidCachePath/empty (0.00s)2317 --- PASS: TestIsValidCachePath/wrong_extension (0.00s)2318 --- PASS: TestIsValidCachePath/leading_slash (0.00s)2319 --- PASS: TestIsValidCachePath/invalid_char_u (0.00s)2320 --- PASS: TestIsValidCachePath/random_path (0.00s)2321 --- PASS: TestIsValidCachePath/short_hash (0.00s)2322 --- PASS: TestIsValidCachePath/invalid_char_e (0.00s)2323 --- PASS: TestIsValidCachePath/traversal_in_middle (0.00s)2324 --- PASS: TestIsValidCachePath/traversal_parent (0.00s)2325=== CONT TestParseSingleRange/none2326=== CONT TestParseSingleRange/suffix2327=== CONT TestParseSingleRange/end_clamped_to_size2328=== CONT TestParseSingleRange/suffix_exceeds_size2329=== CONT TestParseSingleRange/start_past_EOF2330=== CONT TestParseSingleRange/start_far_past_EOF2331=== CONT TestParseSingleRange/malformed_both_empty2332=== CONT TestParseSingleRange/closed2333=== CONT TestParseSingleRange/malformed_end_before_start2334=== CONT TestParseSingleRange/multi-range_ignored2335=== CONT TestParseSingleRange/malformed_no_dash2336=== CONT TestParseSingleRange/single_byte2337=== CONT TestParseSingleRange/unknown_unit2338=== CONT TestParseSingleRange/open-ended2339--- PASS: TestParseSingleRange (0.00s)2340 --- PASS: TestParseSingleRange/none (0.00s)2341 --- PASS: TestParseSingleRange/suffix (0.00s)2342 --- PASS: TestParseSingleRange/end_clamped_to_size (0.00s)2343 --- PASS: TestParseSingleRange/suffix_exceeds_size (0.00s)2344 --- PASS: TestParseSingleRange/start_past_EOF (0.00s)2345 --- PASS: TestParseSingleRange/start_far_past_EOF (0.00s)2346 --- PASS: TestParseSingleRange/malformed_both_empty (0.00s)2347 --- PASS: TestParseSingleRange/closed (0.00s)2348 --- PASS: TestParseSingleRange/malformed_end_before_start (0.00s)2349 --- PASS: TestParseSingleRange/multi-range_ignored (0.00s)2350 --- PASS: TestParseSingleRange/malformed_no_dash (0.00s)2351 --- PASS: TestParseSingleRange/single_byte (0.00s)2352 --- PASS: TestParseSingleRange/unknown_unit (0.00s)2353 --- PASS: TestParseSingleRange/open-ended (0.00s)2354=== CONT TestServerTLSConfig/no_client_CA2355=== CONT TestServerTLSConfig/not_a_PEM_file2356=== CONT TestServerTLSConfig/missing_CA_file2357--- PASS: TestServerTLSConfig (0.00s)2358 --- PASS: TestServerTLSConfig/no_client_CA (0.00s)2359 --- PASS: TestServerTLSConfig/not_a_PEM_file (0.02s)2360 --- PASS: TestServerTLSConfig/missing_CA_file (0.00s)2361=== CONT TestPush_RejectsBadRequests/root_not_in_objects23622026/09/23 12:22:04 INFO Received push request method=POST path=/api/pushes2363=== CONT TestPush_RejectsBadRequests/no_objects23642026/09/23 12:22:04 INFO Received push request method=POST path=/api/pushes2365=== CONT TestPush_RejectsBadRequests/bad_root23662026/09/23 12:22:04 INFO Received push request method=POST path=/api/pushes2367=== CONT TestPush_RejectsBadRequests/no_roots23682026/09/23 12:22:04 INFO Received push request method=POST path=/api/pushes2369=== CONT TestClientErrorHandling/InvalidStorePath2370--- PASS: TestPush_RejectsBadRequests (3.70s)2371 --- PASS: TestPush_RejectsBadRequests/root_not_in_objects (0.00s)2372 --- PASS: TestPush_RejectsBadRequests/no_objects (0.00s)2373 --- PASS: TestPush_RejectsBadRequests/bad_root (0.00s)2374 --- PASS: TestPush_RejectsBadRequests/no_roots (0.00s)23752026/09/23 12:22:04 OK 20241026095416_initial_model.sql (103.87ms)23762026-09-23 12:22:04.503 UTC [44001] ERROR: relation "goose_db_version" does not exist at character 3623772026-09-23 12:22:04.503 UTC [44001] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC23782026/09/23 12:22:04 OK 20251210153512_drop_unused_gin_index.sql (7.03ms)2379--- PASS: TestReadProxyRootRedirectsToIndexHTML (1.88s)2380=== CONT TestClientErrorHandling/ServerNotAvailable23812026/09/23 12:22:04 OK 20241026095416_initial_model.sql (105.6ms)23822026/09/23 12:22:04 OK 20251218171726_add_pins.sql (30.77ms)23832026/09/23 12:22:04 OK 20251210153512_drop_unused_gin_index.sql (8.53ms)23842026/09/23 12:22:04 OK 20251218171726_add_pins.sql (18.56ms)2385=== CONT TestClientErrorHandling/InvalidAuthToken2386--- PASS: TestUploadHandlersRejectOversizedBody (0.05s)2387 --- PASS: TestUploadHandlersRejectOversizedBody/request_more_parts (0.02s)2388 --- PASS: TestUploadHandlersRejectOversizedBody/complete_multipart (0.02s)2389 --- PASS: TestUploadHandlersRejectOversizedBody/create_pending_closure (0.30s)23902026/09/23 12:22:04 OK 20260628120000_add_object_size_and_stats.sql (38.48ms)23912026/09/23 12:22:04 OK 20260628120000_add_object_size_and_stats.sql (54.84ms)23922026/09/23 12:22:04 OK 20260905000000_add_claims.sql (70.79ms)23932026/09/23 12:22:04 OK 20260905000000_add_claims.sql (44.51ms)23942026/09/23 12:22:04 OK 20260920000000_drop_claims.sql (16.48ms)23952026/09/23 12:22:04 OK 20260923120000_add_pushes.sql (8.12ms)23962026/09/23 12:22:04 goose: successfully migrated database to version: 2026092312000023972026/09/23 12:22:04 OK 1_commit_pending_closure.sql (1.21ms)23982026/09/23 12:22:04 OK 2_object_stats_trigger.sql (229.46µs)23992026/09/23 12:22:04 OK 3_commit_push.sql (170.46µs)24002026/09/23 12:22:04 goose: up to current file version: 324012026/09/23 12:22:04 OK 20260920000000_drop_claims.sql (21.11ms)24022026/09/23 12:22:04 OK 20260923120000_add_pushes.sql (1.62ms)24032026/09/23 12:22:04 goose: successfully migrated database to version: 2026092312000024042026/09/23 12:22:04 OK 1_commit_pending_closure.sql (892.38µs)24052026/09/23 12:22:04 OK 2_object_stats_trigger.sql (458.71µs)24062026/09/23 12:22:04 OK 3_commit_push.sql (257.29µs)24072026/09/23 12:22:04 goose: up to current file version: 324082026/09/23 12:22:04 OK 20241026095416_initial_model.sql (144.64ms)24092026/09/23 12:22:04 OK 20251210153512_drop_unused_gin_index.sql (3.99ms)24102026/09/23 12:22:04 WARN Request failed, retrying attempt=1 max_attempts=6 backoff=100ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present24112026/09/23 12:22:04 OK 20251218171726_add_pins.sql (18.75ms)24122026-09-23 12:22:04.751 UTC [44008] ERROR: relation "goose_db_version" does not exist at character 3624132026-09-23 12:22:04.751 UTC [44008] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC24142026/09/23 12:22:04 OK 20260628120000_add_object_size_and_stats.sql (20.22ms)2415--- PASS: TestReadRedirectKeepsNarinfoProxied (2.10s)2416=== CONT TestResolveDBConnectionString/flag_wins2417=== CONT TestResolveDBConnectionString/PGHOST_allows_empty2418=== CONT TestResolveDBConnectionString/nothing_configured2419=== CONT TestResolveDBConnectionString/missing_file_is_an_error2420=== CONT TestResolveDBConnectionString/file_when_flag_empty2421=== CONT TestCacheConfigHandler/full_config,_no_issuer2422=== CONT TestCacheConfigHandler/no_signing_keys2423=== CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator2424=== CONT TestCacheConfigHandler/no_cache_url_configured2425--- PASS: TestCacheConfigHandler (0.00s)2426 --- PASS: TestCacheConfigHandler/full_config,_no_issuer (0.00s)2427 --- PASS: TestCacheConfigHandler/no_signing_keys (0.00s)2428 --- PASS: TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator (0.00s)2429 --- PASS: TestCacheConfigHandler/no_cache_url_configured (0.00s)2430=== CONT TestService_AuthMiddleware_OIDC/valid_OIDC_token2431=== CONT TestService_AuthMiddleware_OIDC/malformed_token_rejected24322026/09/23 12:22:04 WARN Authentication failed token_preview=not-a-valid-jwt token_length=15 oidc_error="no provider could verify the token (signature or issuer mismatch)" oidc_provider="" tried_providers=[test]2433=== CONT TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected24342026/09/23 12:22:04 WARN Authentication failed token_preview=eyJhbGciOi...8pe41BFMCg token_length=701 oidc_error="no rule matched: claim \"repository_owner\" value [otherorg] not in allowed patterns [myorg]" oidc_provider=test tried_providers=[test]2435=== CONT TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured2436=== CONT TestService_RequireScope_OIDC/builder_may_write2437=== CONT TestService_RequireScope_OIDC/static_token_may_admin2438=== CONT TestService_RequireScope_OIDC/reader_may_not_write2439=== CONT TestService_RequireScope_OIDC/ops_may_not_write2440=== CONT TestService_RequireScope_OIDC/ops_may_admin2441=== CONT TestService_RequireScope_OIDC/builder_may_not_admin2442=== CONT TestService_RequireScope_OIDC/anonymous_may_not_read2443=== CONT TestService_RequireScope_OIDC/static_token_may_write2444=== CONT TestService_RequireScope_OIDC/writer_implies_read2445=== CONT TestService_RequireScope_OIDC/reader_may_read2446--- PASS: TestResolveDBConnectionString (0.03s)2447 --- PASS: TestResolveDBConnectionString/flag_wins (0.00s)2448 --- PASS: TestResolveDBConnectionString/PGHOST_allows_empty (0.00s)2449 --- PASS: TestResolveDBConnectionString/nothing_configured (0.00s)2450 --- PASS: TestResolveDBConnectionString/missing_file_is_an_error (0.00s)2451 --- PASS: TestResolveDBConnectionString/file_when_flag_empty (0.00s)2452--- PASS: TestService_AuthMiddleware_OIDC (1.51s)2453 --- PASS: TestService_AuthMiddleware_OIDC/valid_OIDC_token (0.00s)2454 --- PASS: TestService_AuthMiddleware_OIDC/malformed_token_rejected (0.00s)2455 --- PASS: TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected (0.00s)2456 --- PASS: TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured (0.00s)2457--- PASS: TestService_RequireScope_OIDC (1.73s)2458 --- PASS: TestService_RequireScope_OIDC/builder_may_write (0.00s)2459 --- PASS: TestService_RequireScope_OIDC/static_token_may_admin (0.00s)2460 --- PASS: TestService_RequireScope_OIDC/reader_may_not_write (0.00s)2461 --- PASS: TestService_RequireScope_OIDC/ops_may_not_write (0.00s)2462 --- PASS: TestService_RequireScope_OIDC/ops_may_admin (0.00s)2463 --- PASS: TestService_RequireScope_OIDC/builder_may_not_admin (0.00s)2464 --- PASS: TestService_RequireScope_OIDC/anonymous_may_not_read (0.00s)2465 --- PASS: TestService_RequireScope_OIDC/static_token_may_write (0.00s)2466 --- PASS: TestService_RequireScope_OIDC/writer_implies_read (0.00s)2467 --- PASS: TestService_RequireScope_OIDC/reader_may_read (0.00s)24682026/09/23 12:22:04 OK 20260905000000_add_claims.sql (38.47ms)24692026/09/23 12:22:04 OK 20260920000000_drop_claims.sql (27.28ms)24702026/09/23 12:22:04 WARN Request failed, retrying attempt=2 max_attempts=6 backoff=196.21119ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present24712026/09/23 12:22:04 OK 20260923120000_add_pushes.sql (21.37ms)24722026/09/23 12:22:04 goose: successfully migrated database to version: 2026092312000024732026/09/23 12:22:04 OK 1_commit_pending_closure.sql (970.92µs)24742026/09/23 12:22:04 OK 2_object_stats_trigger.sql (239.92µs)24752026/09/23 12:22:04 OK 3_commit_push.sql (211.08µs)24762026/09/23 12:22:04 goose: up to current file version: 324772026/09/23 12:22:04 OK 20241026095416_initial_model.sql (146.3ms)24782026/09/23 12:22:04 OK 20251210153512_drop_unused_gin_index.sql (13.93ms)24792026/09/23 12:22:04 OK 20251218171726_add_pins.sql (16.19ms)24802026-09-23 12:22:04.975 UTC [44009] ERROR: relation "goose_db_version" does not exist at character 3624812026-09-23 12:22:04.975 UTC [44009] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC24822026/09/23 12:22:04 OK 20260628120000_add_object_size_and_stats.sql (19.14ms)24832026/09/23 12:22:04 OK 20260905000000_add_claims.sql (22.99ms)2484--- PASS: TestReadProxyDisabled (2.03s)24852026/09/23 12:22:05 WARN Request failed, retrying attempt=3 max_attempts=6 backoff=428.658366ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present24862026/09/23 12:22:05 OK 20260920000000_drop_claims.sql (54.16ms)24872026/09/23 12:22:05 OK 20260923120000_add_pushes.sql (17.68ms)24882026/09/23 12:22:05 goose: successfully migrated database to version: 2026092312000024892026/09/23 12:22:05 OK 1_commit_pending_closure.sql (4.49ms)24902026/09/23 12:22:05 OK 2_object_stats_trigger.sql (659µs)24912026/09/23 12:22:05 OK 3_commit_push.sql (442.96µs)24922026/09/23 12:22:05 goose: up to current file version: 324932026/09/23 12:22:05 OK 20241026095416_initial_model.sql (93.98ms)24942026/09/23 12:22:05 OK 20251210153512_drop_unused_gin_index.sql (8.81ms)24952026/09/23 12:22:05 OK 20251218171726_add_pins.sql (25.33ms)24962026/09/23 12:22:05 OK 20260628120000_add_object_size_and_stats.sql (27.79ms)24972026/09/23 12:22:05 INFO Received complete multipart upload request method=POST path=/api/multipart/complete24982026/09/23 12:22:05 OK 20260905000000_add_claims.sql (43.99ms)24992026/09/23 12:22:05 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=MDE5OTY4Y2UtNTJmOC00OWY1LTgxMDQtZTZiMWMwMDlkNzEyLjc1NjZlOGVjLWY3YWItNGRlOC05NjRlLWFlYTExMmM3ZGNlZHgxNzkwMTY2MTIzOTc1NDAzMDAw parts=1025002026/09/23 12:22:05 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete25012026/09/23 12:22:05 OK 20260920000000_drop_claims.sql (29ms)25022026/09/23 12:22:05 INFO Completed upload id=125032026/09/23 12:22:05 INFO Received uploads request method=POST path=/api/pending_closures25042026/09/23 12:22:05 INFO Received uploads request method=POST path=/api/pending_closures25052026/09/23 12:22:05 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo25062026/09/23 12:22:05 WARN Found objects in DB but missing from S3, will re-upload count=12507--- PASS: TestService_verifyS3Integrity (2.94s)25082026/09/23 12:22:05 OK 20260923120000_add_pushes.sql (15.45ms)25092026/09/23 12:22:05 goose: successfully migrated database to version: 2026092312000025102026/09/23 12:22:05 OK 1_commit_pending_closure.sql (3.05ms)25112026/09/23 12:22:05 OK 2_object_stats_trigger.sql (643.17µs)25122026/09/23 12:22:05 OK 3_commit_push.sql (405.42µs)25132026/09/23 12:22:05 goose: up to current file version: 32514--- PASS: TestReadProxyConditionalGet (2.25s)25152026/09/23 12:22:05 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"25162026/09/23 12:22:05 WARN mTLS auth: bound subjects configured but subject DN unavailable25172026/09/23 12:22:05 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"2518--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (2.30s)25192026/09/23 12:22:05 WARN Request failed, retrying attempt=4 max_attempts=6 backoff=855.426854ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present25202026/09/23 12:22:05 INFO Received uploads request method=POST path=/api/pending_closures25212026/09/23 12:22:05 INFO Received uploads request method=POST path=/api/pending_closures25222026/09/23 12:22:05 INFO Received uploads request method=POST path=/api/pending_closures25232026-09-23 12:22:05.750 UTC [44010] ERROR: relation "goose_db_version" does not exist at character 3625242026-09-23 12:22:05.750 UTC [44010] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC25252026-09-23 12:22:05.863 UTC [44011] ERROR: relation "goose_db_version" does not exist at character 3625262026-09-23 12:22:05.863 UTC [44011] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC25272026/09/23 12:22:05 OK 20241026095416_initial_model.sql (129.25ms)25282026/09/23 12:22:05 OK 20251210153512_drop_unused_gin_index.sql (5.84ms)2529--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (2.32s)25302026/09/23 12:22:05 OK 20251218171726_add_pins.sql (16.35ms)25312026/09/23 12:22:05 OK 20260628120000_add_object_size_and_stats.sql (23ms)25322026/09/23 12:22:05 OK 20260905000000_add_claims.sql (20.5ms)25332026/09/23 12:22:05 OK 20260920000000_drop_claims.sql (3.69ms)25342026/09/23 12:22:05 OK 20241026095416_initial_model.sql (77.72ms)25352026/09/23 12:22:05 OK 20260923120000_add_pushes.sql (2.28ms)25362026/09/23 12:22:05 goose: successfully migrated database to version: 2026092312000025372026/09/23 12:22:05 OK 20251210153512_drop_unused_gin_index.sql (1.72ms)25382026/09/23 12:22:05 OK 1_commit_pending_closure.sql (3.51ms)25392026/09/23 12:22:05 OK 2_object_stats_trigger.sql (851.08µs)25402026/09/23 12:22:05 OK 20251218171726_add_pins.sql (3.57ms)25412026/09/23 12:22:05 OK 3_commit_push.sql (648.54µs)25422026/09/23 12:22:05 goose: up to current file version: 325432026/09/23 12:22:05 OK 20260628120000_add_object_size_and_stats.sql (3.88ms)25442026/09/23 12:22:06 OK 20260905000000_add_claims.sql (17.4ms)25452026/09/23 12:22:06 OK 20260920000000_drop_claims.sql (32.35ms)25462026/09/23 12:22:06 OK 20260923120000_add_pushes.sql (10.52ms)25472026/09/23 12:22:06 goose: successfully migrated database to version: 2026092312000025482026/09/23 12:22:06 OK 1_commit_pending_closure.sql (3.44ms)25492026/09/23 12:22:06 OK 2_object_stats_trigger.sql (732.67µs)25502026/09/23 12:22:06 OK 3_commit_push.sql (543.79µs)25512026/09/23 12:22:06 goose: up to current file version: 325522026/09/23 12:22:06 WARN Request failed, retrying attempt=5 max_attempts=6 backoff=1.670577373s error="Post \"http://localhost:19999/api/objects/present\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/objects/present25532026/09/23 12:22:06 WARN Authentication failed token_preview=invalid-token token_length=13 reason="static API token mismatch"25542026/09/23 12:22:06 WARN Authentication failed token_preview=invalid-token token_length=13 reason="static API token mismatch"25552026/09/23 12:22:06 INFO Received complete multipart upload request method=POST path=/api/multipart/complete25562026/09/23 12:22:06 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=MDE5OTY4Y2UtNTJmOC00OWY1LTgxMDQtZTZiMWMwMDlkNzEyLjg0ZDAwZmNkLTY2NTctNGI4OC05YTI1LWQ4ZTcyMDliZjllN3gxNzkwMTY2MTI1Njg5NzA4MDAw parts=1025572026/09/23 12:22:06 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete25582026/09/23 12:22:06 INFO Completed upload id=125592026/09/23 12:22:06 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000025602026/09/23 12:22:06 INFO Received uploads request method=POST path=/api/pending_closures25612026/09/23 12:22:06 INFO Starting cleanup of old closures method=DELETE path=/api/closures25622026/09/23 12:22:06 INFO Aborted multipart uploads count=025632026/09/23 12:22:06 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=0 objects-failed-to-delete=025642026/09/23 12:22:06 INFO Vacuumed table table=pending_closures25652026/09/23 12:22:06 INFO Vacuumed table table=pending_objects25662026/09/23 12:22:06 INFO Vacuumed table table=multipart_uploads25672026/09/23 12:22:06 INFO Vacuumed table table=closures25682026/09/23 12:22:06 INFO Vacuumed table table=objects25692026/09/23 12:22:06 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000002570--- PASS: TestService_createPendingClosureHandler (3.43s)25712026/09/23 12:22:08 WARN Request failed, retrying attempt=1 max_attempts=6 backoff=100ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config25722026/09/23 12:22:08 WARN Request failed, retrying attempt=2 max_attempts=6 backoff=185.43039ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config25732026/09/23 12:22:08 WARN Request failed, retrying attempt=3 max_attempts=6 backoff=416.217518ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config25742026/09/23 12:22:08 WARN Request failed, retrying attempt=4 max_attempts=6 backoff=801.878591ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config25752026/09/23 12:22:09 WARN Request failed, retrying attempt=5 max_attempts=6 backoff=1.576481937s error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config25762026/09/23 12:22:11 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: sending request: request failed after retries: Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused"25772026/09/23 12:22:11 WARN Request failed, retrying attempt=1 max_attempts=6 backoff=100ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config25782026/09/23 12:22:11 WARN Request failed, retrying attempt=2 max_attempts=6 backoff=198.856885ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config25792026/09/23 12:22:11 WARN Request failed, retrying attempt=3 max_attempts=6 backoff=393.543135ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config25802026/09/23 12:22:11 WARN Request failed, retrying attempt=4 max_attempts=6 backoff=750.865585ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config25812026/09/23 12:22:12 WARN Request failed, retrying attempt=5 max_attempts=6 backoff=1.493279092s error="Get \"http://localhost:19999/api/cache-config\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/cache-config25822026/09/23 12:22:14 WARN Request failed, retrying attempt=1 max_attempts=6 backoff=100ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures25832026/09/23 12:22:14 WARN Request failed, retrying attempt=2 max_attempts=6 backoff=207.019289ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures25842026/09/23 12:22:14 WARN Request failed, retrying attempt=3 max_attempts=6 backoff=364.404125ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures25852026/09/23 12:22:14 WARN Request failed, retrying attempt=4 max_attempts=6 backoff=813.225052ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures25862026/09/23 12:22:15 WARN Request failed, retrying attempt=5 max_attempts=6 backoff=1.670362077s error="Post \"http://localhost:19999/api/pending_closures\": dial tcp 127.0.0.1:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures2587--- PASS: TestClientErrorHandling (0.00s)2588 --- PASS: TestClientErrorHandling/InvalidStorePath (1.86s)2589 --- PASS: TestClientErrorHandling/InvalidAuthToken (1.96s)2590 --- PASS: TestClientErrorHandling/ServerNotAvailable (12.77s)2591PASS2592{"timestamp":"2026-09-23T12:22:17.301489Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:56908","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2260,"threadName":"rustfs-worker","threadId":"ThreadId(9)"}25932026-09-23 12:22:17.401 UTC [43664] LOG: received smart shutdown request25942026-09-23 12:22:17.402 UTC [43664] LOG: background worker "logical replication launcher" (PID 43674) exited with exit code 125952026-09-23 12:22:17.406 UTC [43669] LOG: shutting down25962026-09-23 12:22:17.406 UTC [43669] LOG: checkpoint starting: shutdown immediate25972026-09-23 12:22:18.536 UTC [43669] LOG: checkpoint complete: wrote 12622 buffers (77.0%), wrote 4 SLRU buffers; 0 WAL file(s) added, 0 removed, 18 recycled; write=0.711 s, sync=0.374 s, total=1.130 s; sync files=21875, longest=0.001 s, average=0.001 s; distance=302623 kB, estimate=302623 kB; lsn=0/13F14CD8, redo lsn=0/13F14CD825982026-09-23 12:22:18.541 UTC [43664] LOG: database system is shut down2599Running OIDC tests...2600=== RUN TestAudienceForIssuer2601=== PAUSE TestAudienceForIssuer2602=== RUN TestGlobMatch2603=== PAUSE TestGlobMatch2604=== RUN TestValidateToken_ValidToken2605=== PAUSE TestValidateToken_ValidToken2606=== RUN TestValidateToken_WrongAudience2607=== PAUSE TestValidateToken_WrongAudience2608=== RUN TestValidateToken_Expired2609=== PAUSE TestValidateToken_Expired2610=== RUN TestValidateToken_BoundClaimsMismatch2611=== PAUSE TestValidateToken_BoundClaimsMismatch2612=== RUN TestValidateToken_BoundSubjectMismatch2613=== PAUSE TestValidateToken_BoundSubjectMismatch2614=== RUN TestValidateToken_MultipleProviders2615=== PAUSE TestValidateToken_MultipleProviders2616=== RUN TestValidateToken_NoMatchingProvider2617=== PAUSE TestValidateToken_NoMatchingProvider2618=== RUN TestValidateToken_KubernetesServiceAccount2619=== PAUSE TestValidateToken_KubernetesServiceAccount2620=== RUN TestNewValidator_KubernetesRequiresCA2621=== PAUSE TestNewValidator_KubernetesRequiresCA2622=== RUN TestValidateToken_KubernetesIssuerFromOwnToken2623=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken2624=== RUN TestPins_ReservedForMatchingRule2625=== PAUSE TestPins_ReservedForMatchingRule2626=== RUN TestPins_TopLevelShorthand2627=== PAUSE TestPins_TopLevelShorthand2628=== RUN TestPins_ConfigValidation2629=== PAUSE TestPins_ConfigValidation2630=== RUN TestScopes_LegacyProviderDefaultsToWrite2631=== PAUSE TestScopes_LegacyProviderDefaultsToWrite2632=== RUN TestScopes_Rules2633=== PAUSE TestScopes_Rules2634=== RUN TestScopes_ConfigValidation2635=== PAUSE TestScopes_ConfigValidation2636=== CONT TestAudienceForIssuer2637--- PASS: TestAudienceForIssuer (0.00s)2638=== CONT TestValidateToken_NoMatchingProvider2639=== CONT TestPins_ConfigValidation2640=== CONT TestValidateToken_KubernetesServiceAccount2641=== CONT TestPins_ReservedForMatchingRule2642=== CONT TestScopes_ConfigValidation2643=== CONT TestPins_TopLevelShorthand2644=== CONT TestValidateToken_KubernetesIssuerFromOwnToken2645--- PASS: TestScopes_ConfigValidation (0.00s)2646=== CONT TestValidateToken_BoundClaimsMismatch2647=== CONT TestValidateToken_Expired2648--- PASS: TestPins_ConfigValidation (0.01s)2649=== CONT TestNewValidator_KubernetesRequiresCA2650=== CONT TestValidateToken_MultipleProviders2651=== CONT TestValidateToken_BoundSubjectMismatch26522026/09/23 12:22:19 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:5696326532026/09/23 12:22:19 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56968/oidc2654--- PASS: TestValidateToken_KubernetesServiceAccount (0.06s)2655=== CONT TestValidateToken_ValidToken26562026/09/23 12:22:19 http: TLS handshake error from 127.0.0.1:56966: read tcp 127.0.0.1:56965->127.0.0.1:56966: use of closed network connection2657--- PASS: TestNewValidator_KubernetesRequiresCA (0.05s)2658=== CONT TestValidateToken_WrongAudience2659--- PASS: TestPins_TopLevelShorthand (0.06s)2660=== CONT TestGlobMatch2661=== RUN TestGlobMatch/foo_foo2662=== PAUSE TestGlobMatch/foo_foo2663=== RUN TestGlobMatch/foo_bar2664=== PAUSE TestGlobMatch/foo_bar2665=== RUN TestGlobMatch/*_2666=== PAUSE TestGlobMatch/*_2667=== RUN TestGlobMatch/*_anything2668=== PAUSE TestGlobMatch/*_anything2669=== RUN TestGlobMatch/foo*_foo2670=== PAUSE TestGlobMatch/foo*_foo2671=== RUN TestGlobMatch/foo*_foobar2672=== PAUSE TestGlobMatch/foo*_foobar2673=== RUN TestGlobMatch/foo*_bar2674=== PAUSE TestGlobMatch/foo*_bar2675=== RUN TestGlobMatch/*bar_bar2676=== PAUSE TestGlobMatch/*bar_bar2677=== RUN TestGlobMatch/*bar_foobar2678=== PAUSE TestGlobMatch/*bar_foobar2679=== RUN TestGlobMatch/*bar_foo2680=== PAUSE TestGlobMatch/*bar_foo2681=== RUN TestGlobMatch/foo*bar_foobar2682=== PAUSE TestGlobMatch/foo*bar_foobar2683=== RUN TestGlobMatch/foo*bar_foo123bar2684=== PAUSE TestGlobMatch/foo*bar_foo123bar2685=== RUN TestGlobMatch/foo*bar_foobarbaz2686=== PAUSE TestGlobMatch/foo*bar_foobarbaz2687=== RUN TestGlobMatch/*/*_foo/bar2688=== PAUSE TestGlobMatch/*/*_foo/bar2689=== RUN TestGlobMatch/*/*_foo2690=== PAUSE TestGlobMatch/*/*_foo2691=== RUN TestGlobMatch/refs/heads/*_refs/heads/main2692=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main2693=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.02694=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.02695=== RUN TestGlobMatch/refs/*/main_refs/heads/main2696=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main2697=== RUN TestGlobMatch/fo?_foo2698=== PAUSE TestGlobMatch/fo?_foo2699=== RUN TestGlobMatch/fo?_fo2700=== PAUSE TestGlobMatch/fo?_fo2701=== RUN TestGlobMatch/fo?_fooo2702=== PAUSE TestGlobMatch/fo?_fooo2703=== RUN TestGlobMatch/?oo_foo2704=== PAUSE TestGlobMatch/?oo_foo2705=== RUN TestGlobMatch/?oo_boo2706=== PAUSE TestGlobMatch/?oo_boo2707=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main2708=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main2709=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main2710=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main2711=== CONT TestScopes_Rules27122026/09/23 12:22:19 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56971/oidc2713--- PASS: TestPins_ReservedForMatchingRule (0.07s)2714=== CONT TestScopes_LegacyProviderDefaultsToWrite27152026/09/23 12:22:19 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56973/oidc2716--- PASS: TestValidateToken_Expired (0.08s)2717=== CONT TestGlobMatch/foo_foo2718=== CONT TestGlobMatch/*bar_bar2719=== CONT TestGlobMatch/*/*_foo/bar2720=== CONT TestGlobMatch/foo*bar_foobarbaz2721=== CONT TestGlobMatch/foo*bar_foo123bar2722=== CONT TestGlobMatch/foo*bar_foobar2723=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main2724=== CONT TestGlobMatch/*bar_foo2725=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main2726=== CONT TestGlobMatch/*bar_foobar2727=== CONT TestGlobMatch/?oo_boo2728=== CONT TestGlobMatch/?oo_foo2729=== CONT TestGlobMatch/fo?_fooo2730=== CONT TestGlobMatch/fo?_fo2731=== CONT TestGlobMatch/fo?_foo2732=== CONT TestGlobMatch/refs/*/main_refs/heads/main2733=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.02734=== CONT TestGlobMatch/refs/heads/*_refs/heads/main2735=== CONT TestGlobMatch/*/*_foo2736=== CONT TestGlobMatch/foo*_foo2737=== CONT TestGlobMatch/foo*_bar2738=== CONT TestGlobMatch/foo*_foobar2739=== CONT TestGlobMatch/*_2740=== CONT TestGlobMatch/*_anything2741=== CONT TestGlobMatch/foo_bar2742--- PASS: TestGlobMatch (0.00s)2743 --- PASS: TestGlobMatch/foo_foo (0.00s)2744 --- PASS: TestGlobMatch/*bar_bar (0.00s)2745 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)2746 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)2747 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)2748 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)2749 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)2750 --- PASS: TestGlobMatch/*bar_foo (0.00s)2751 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)2752 --- PASS: TestGlobMatch/*bar_foobar (0.00s)2753 --- PASS: TestGlobMatch/?oo_boo (0.00s)2754 --- PASS: TestGlobMatch/?oo_foo (0.00s)2755 --- PASS: TestGlobMatch/fo?_fooo (0.00s)2756 --- PASS: TestGlobMatch/fo?_fo (0.00s)2757 --- PASS: TestGlobMatch/fo?_foo (0.00s)2758 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)2759 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)2760 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)2761 --- PASS: TestGlobMatch/*/*_foo (0.00s)2762 --- PASS: TestGlobMatch/foo*_foo (0.00s)2763 --- PASS: TestGlobMatch/foo*_bar (0.00s)2764 --- PASS: TestGlobMatch/foo*_foobar (0.00s)2765 --- PASS: TestGlobMatch/*_ (0.00s)2766 --- PASS: TestGlobMatch/*_anything (0.00s)2767 --- PASS: TestGlobMatch/foo_bar (0.00s)27682026/09/23 12:22:19 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56975/oidc27692026/09/23 12:22:19 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56978/oidc27702026/09/23 12:22:19 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC1232771--- PASS: TestValidateToken_ValidToken (0.03s)2772--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.02s)2773--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (0.09s)27742026/09/23 12:22:19 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:56967/oidc27752026/09/23 12:22:19 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:56982/oidc2776--- PASS: TestValidateToken_MultipleProviders (0.10s)27772026/09/23 12:22:19 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56985/oidc2778--- PASS: TestScopes_Rules (0.06s)27792026/09/23 12:22:19 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56987/oidc2780--- PASS: TestValidateToken_BoundSubjectMismatch (0.13s)27812026/09/23 12:22:19 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56989/oidc2782--- PASS: TestValidateToken_WrongAudience (0.09s)27832026/09/23 12:22:19 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:56980/oidc27842026/09/23 12:22:19 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56993/oidc2785--- PASS: TestValidateToken_NoMatchingProvider (0.16s)2786--- PASS: TestValidateToken_BoundClaimsMismatch (0.16s)2787PASS2788Running hook tests...2789=== RUN TestSendPathsEmpty2790=== PAUSE TestSendPathsEmpty2791=== RUN TestQueueEnqueueAndFetch2792=== PAUSE TestQueueEnqueueAndFetch2793=== RUN TestQueueDeduplication2794=== PAUSE TestQueueDeduplication2795=== RUN TestQueueRemove2796=== PAUSE TestQueueRemove2797=== RUN TestQueueFetchBatchLimit2798=== PAUSE TestQueueFetchBatchLimit2799=== RUN TestQueueRetryMovesToBack2800=== PAUSE TestQueueRetryMovesToBack2801=== RUN TestQueueFetchRemoveLifecycle2802=== PAUSE TestQueueFetchRemoveLifecycle2803=== RUN TestQueueConcurrentWriters2804=== PAUSE TestQueueConcurrentWriters2805=== RUN TestQueueRemoveLargeClosure2806=== PAUSE TestQueueRemoveLargeClosure2807=== RUN TestServerClientIntegration2808=== PAUSE TestServerClientIntegration2809=== RUN TestServerQueueError2810=== PAUSE TestServerQueueError2811=== RUN TestGetListenerSocketActivation2812 server_test.go:210: === RUN TestGetListenerSocketActivation2813 --- PASS: TestGetListenerSocketActivation (0.00s)2814 PASS2815 2816--- PASS: TestGetListenerSocketActivation (0.01s)2817=== RUN TestDrainIsolatesPoisonPath2818=== PAUSE TestDrainIsolatesPoisonPath2819=== RUN TestRunNotBlockedByPoisonHead2820=== PAUSE TestRunNotBlockedByPoisonHead2821=== RUN TestDrainGivesUpWhenServerDown2822=== PAUSE TestDrainGivesUpWhenServerDown2823=== RUN TestFailedPathPrunedByLaterClosure2824=== PAUSE TestFailedPathPrunedByLaterClosure2825=== RUN TestWorkerUploadsAndRemoves2826=== PAUSE TestWorkerUploadsAndRemoves2827=== RUN TestWorkerSkipsGCdPaths2828=== PAUSE TestWorkerSkipsGCdPaths2829=== RUN TestWorkerPrunesClosureDeps2830=== PAUSE TestWorkerPrunesClosureDeps2831=== RUN TestDrainTimeout2832=== PAUSE TestDrainTimeout2833=== CONT TestSendPathsEmpty2834=== CONT TestServerQueueError2835=== CONT TestWorkerUploadsAndRemoves2836=== CONT TestQueueFetchRemoveLifecycle2837=== CONT TestWorkerPrunesClosureDeps2838=== CONT TestQueueRemove2839=== CONT TestDrainTimeout2840=== CONT TestQueueDeduplication2841--- PASS: TestSendPathsEmpty (0.00s)2842=== CONT TestServerClientIntegration2843=== CONT TestQueueRemoveLargeClosure2844=== CONT TestQueueConcurrentWriters28452026/09/23 12:22:20 ERROR Failed to queue paths error="permission denied" count=12846--- PASS: TestServerClientIntegration (0.00s)2847=== CONT TestDrainGivesUpWhenServerDown2848--- PASS: TestServerQueueError (0.00s)2849=== CONT TestFailedPathPrunedByLaterClosure28502026/09/23 12:22:20 INFO Upload queue status pending=228512026/09/23 12:22:20 INFO Uploading batch count=228522026/09/23 12:22:20 INFO Uploading batch count=128532026/09/23 12:22:20 ERROR Upload failed error="upload failed" count=12854--- PASS: TestQueueRemove (0.01s)2855=== CONT TestWorkerSkipsGCdPaths2856--- PASS: TestQueueFetchRemoveLifecycle (0.01s)2857=== CONT TestRunNotBlockedByPoisonHead28582026/09/23 12:22:20 INFO Uploading batch count=228592026/09/23 12:22:20 ERROR Upload failed error="upload failed" count=228602026/09/23 12:22:20 ERROR Upload failed, will retry later error="upload failed" path=/nix/var/nix/builds/nix-43626-243968142/TestDrainGivesUpWhenServerDown3296112195/002/a28612026/09/23 12:22:20 INFO Uploading batch count=128622026/09/23 12:22:20 INFO Uploading batch count=228632026/09/23 12:22:20 ERROR Upload failed, will retry later error="upload failed" path=/nix/var/nix/builds/nix-43626-243968142/TestDrainGivesUpWhenServerDown3296112195/002/b28642026/09/23 12:22:20 INFO Uploading batch count=128652026/09/23 12:22:20 INFO Upload queue status pending=22866--- PASS: TestQueueDeduplication (0.01s)2867=== CONT TestQueueRetryMovesToBack28682026/09/23 12:22:20 INFO Uploading batch count=128692026/09/23 12:22:20 INFO Uploading batch count=228702026/09/23 12:22:20 ERROR Upload failed error="upload failed" count=228712026/09/23 12:22:20 ERROR Upload failed, will retry later error="upload failed" path=/nix/var/nix/builds/nix-43626-243968142/TestDrainGivesUpWhenServerDown3296112195/002/c28722026/09/23 12:22:20 ERROR Upload failed, will retry later error="upload failed" path=/nix/var/nix/builds/nix-43626-243968142/TestDrainGivesUpWhenServerDown3296112195/002/d28732026/09/23 12:22:20 INFO Uploading batch count=228742026/09/23 12:22:20 ERROR Upload failed error="upload failed" count=228752026/09/23 12:22:20 ERROR Upload failed, will retry later error="upload failed" path=/nix/var/nix/builds/nix-43626-243968142/TestDrainGivesUpWhenServerDown3296112195/002/e28762026/09/23 12:22:20 ERROR Upload failed, will retry later error="upload failed" path=/nix/var/nix/builds/nix-43626-243968142/TestDrainGivesUpWhenServerDown3296112195/002/f2877--- PASS: TestFailedPathPrunedByLaterClosure (0.01s)2878=== CONT TestQueueFetchBatchLimit28792026/09/23 12:22:20 INFO Upload queue status pending=228802026/09/23 12:22:20 ERROR Drain finished with paths left in queue remaining=1028812026/09/23 12:22:20 WARN Store path no longer exists (garbage collected?), removing from queue path=/nix/var/nix/builds/nix-43626-243968142/TestWorkerSkipsGCdPaths1432799772/002/nonexistent28822026/09/23 12:22:20 INFO Uploading batch count=128832026/09/23 12:22:20 INFO Upload queue status pending=328842026/09/23 12:22:20 INFO Uploading batch count=128852026/09/23 12:22:20 ERROR Upload failed error="upload failed" count=12886--- PASS: TestDrainGivesUpWhenServerDown (0.01s)2887=== CONT TestDrainIsolatesPoisonPath2888--- PASS: TestQueueFetchBatchLimit (0.00s)2889=== CONT TestQueueEnqueueAndFetch2890--- PASS: TestQueueRetryMovesToBack (0.00s)28912026/09/23 12:22:20 INFO Uploading batch count=428922026/09/23 12:22:20 ERROR Upload failed error="upload failed" count=428932026/09/23 12:22:20 ERROR Upload failed, will retry later error="upload failed" path=/nix/var/nix/builds/nix-43626-243968142/TestDrainIsolatesPoisonPath4071739915/002/bbb28942026/09/23 12:22:20 INFO Uploading batch count=128952026/09/23 12:22:20 ERROR Upload failed error="upload failed" count=12896--- PASS: TestQueueEnqueueAndFetch (0.00s)28972026/09/23 12:22:20 INFO Uploading batch count=128982026/09/23 12:22:20 ERROR Upload failed error="upload failed" count=128992026/09/23 12:22:20 INFO Uploading batch count=129002026/09/23 12:22:20 ERROR Upload failed error="upload failed" count=129012026/09/23 12:22:20 ERROR Drain finished with paths left in queue remaining=12902--- PASS: TestDrainIsolatesPoisonPath (0.00s)2903--- PASS: TestWorkerUploadsAndRemoves (0.03s)2904--- PASS: TestWorkerPrunesClosureDeps (0.03s)2905--- PASS: TestWorkerSkipsGCdPaths (0.02s)2906--- PASS: TestQueueRemoveLargeClosure (0.06s)2907--- PASS: TestQueueConcurrentWriters (0.16s)29082026/09/23 12:22:20 ERROR Upload failed error="context deadline exceeded" count=229092026/09/23 12:22:20 ERROR Drain finished with paths left in queue remaining=42910--- PASS: TestDrainTimeout (0.21s)29112026/09/23 12:22:21 INFO Uploading batch count=129122026/09/23 12:22:21 INFO Uploading batch count=129132026/09/23 12:22:21 INFO Uploading batch count=129142026/09/23 12:22:21 ERROR Upload failed error="upload failed" count=129152026/09/23 12:22:21 INFO Uploading batch count=129162026/09/23 12:22:21 ERROR Upload failed error="upload failed" count=129172026/09/23 12:22:21 INFO Uploading batch count=129182026/09/23 12:22:21 ERROR Upload failed error="upload failed" count=129192026/09/23 12:22:21 INFO Uploading batch count=129202026/09/23 12:22:21 ERROR Upload failed error="upload failed" count=129212026/09/23 12:22:21 ERROR Drain finished with paths left in queue remaining=12922--- PASS: TestRunNotBlockedByPoisonHead (1.01s)2923PASS