tribuchet: building on eliza Running client tests... === RUN TestDoServerRequestAttachesToken === PAUSE TestDoServerRequestAttachesToken === RUN TestUploadBuildLog_FileBodyReplayedOnRetry === PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry === RUN TestRegisterUploadedObjectReusesConnections === PAUSE TestRegisterUploadedObjectReusesConnections === RUN TestRunGarbageCollection_FinishedOnAnotherReplica === PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica === RUN TestRunGarbageCollection_NotFoundAfterLocalRun === PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun === RUN TestCaseHackSuffix === PAUSE TestCaseHackSuffix === RUN TestFilterOversizedClosures === PAUSE TestFilterOversizedClosures === RUN TestUploadMultipart_PartsInParallel === PAUSE TestUploadMultipart_PartsInParallel === RUN TestPartSizeForNAR === PAUSE TestPartSizeForNAR === RUN TestUploadMultipart_SupersededByPeer === PAUSE TestUploadMultipart_SupersededByPeer === RUN TestDumpPathCaseHackMatchesNix --- PASS: TestDumpPathCaseHackMatchesNix (0.05s) === RUN TestDumpPathCaseHackCollision --- PASS: TestDumpPathCaseHackCollision (0.00s) === RUN TestSupersededNARStillUploadsListing === PAUSE TestSupersededNARStillUploadsListing === RUN TestDumpPathMatchesNix === PAUSE TestDumpPathMatchesNix === RUN TestDumpPathSingleFile === PAUSE TestDumpPathSingleFile === RUN TestDumpPathWriterError === PAUSE TestDumpPathWriterError === RUN TestEncodeNixBase32 === PAUSE TestEncodeNixBase32 === RUN TestEncodeNixBase32WithRealHash === PAUSE TestEncodeNixBase32WithRealHash === RUN TestConvertHashToNix32 === PAUSE TestConvertHashToNix32 === RUN TestGetStorePathHash === PAUSE TestGetStorePathHash === RUN TestPathInfoHashCompatibility === PAUSE TestPathInfoHashCompatibility === RUN TestParsePathInfoJSON === PAUSE TestParsePathInfoJSON === RUN TestParsePathInfoJSONMultiplePaths === PAUSE TestParsePathInfoJSONMultiplePaths === RUN TestPathInfoCACompatibility === PAUSE TestPathInfoCACompatibility === RUN TestRateLimiterFeedback === PAUSE TestRateLimiterFeedback === RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess === PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess === RUN TestRegisterUploadedObject_BoundedAgainstSilentServer === PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer === RUN TestResolveStorePath === PAUSE TestResolveStorePath === RUN TestDoWithRetry_BodyReplayedViaGetBody === PAUSE TestDoWithRetry_BodyReplayedViaGetBody === RUN TestDoWithRetry_ClosesBodyWhenNoAttemptIsMade === PAUSE TestDoWithRetry_ClosesBodyWhenNoAttemptIsMade === RUN TestRetryTimeoutOutlastsMaxRetries === PAUSE TestRetryTimeoutOutlastsMaxRetries === RUN TestDoWithRetry_FinalResponseBodyReadable === PAUSE TestDoWithRetry_FinalResponseBodyReadable === RUN TestShellSplit === PAUSE TestShellSplit === RUN TestShellSplitErrors === PAUSE TestShellSplitErrors === RUN TestStreamPushReportsEveryPath === PAUSE TestStreamPushReportsEveryPath === RUN TestStreamPushBatchesUnderLoad === PAUSE TestStreamPushBatchesUnderLoad === RUN TestStreamPushIsolatesFailures === PAUSE TestStreamPushIsolatesFailures === RUN TestStreamPushGivesUpOnDeadServer === PAUSE TestStreamPushGivesUpOnDeadServer === RUN TestStreamPushRequestLine === PAUSE TestStreamPushRequestLine === RUN TestStreamPushReportsSignatures === PAUSE TestStreamPushReportsSignatures === RUN TestStreamPushReportsSkippedPaths === PAUSE TestStreamPushReportsSkippedPaths === RUN TestClientSignaturesByStorePath === PAUSE TestClientSignaturesByStorePath === RUN TestStreamPushStopsWhenCanceled === PAUSE TestStreamPushStopsWhenCanceled === RUN TestSetClientTLS === PAUSE TestSetClientTLS === RUN TestSetClientTLSDoesNotMutateDefaultTransport === PAUSE TestSetClientTLSDoesNotMutateDefaultTransport === RUN TestSetClientTLSErrors === PAUSE TestSetClientTLSErrors === RUN TestStaticToken === PAUSE TestStaticToken === RUN TestFileTokenReadsAndCaches === PAUSE TestFileTokenReadsAndCaches === RUN TestFileTokenMissing === PAUSE TestFileTokenMissing === RUN TestFileTokenEmpty === PAUSE TestFileTokenEmpty === RUN TestScriptTokenNoExpiryRerunsEveryCall === PAUSE TestScriptTokenNoExpiryRerunsEveryCall === RUN TestScriptTokenCachesUntilRefresh === PAUSE TestScriptTokenCachesUntilRefresh === RUN TestScriptTokenEmptyToken === PAUSE TestScriptTokenEmptyToken === RUN TestScriptTokenBadJSON === PAUSE TestScriptTokenBadJSON === RUN TestScriptTokenScriptFails === PAUSE TestScriptTokenScriptFails === RUN TestScriptTokenEmptyCommand === PAUSE TestScriptTokenEmptyCommand === CONT TestDoServerRequestAttachesToken === CONT TestEncodeNixBase32 === RUN TestEncodeNixBase32/test_string_hash === PAUSE TestEncodeNixBase32/test_string_hash === RUN TestEncodeNixBase32/empty_input === PAUSE TestEncodeNixBase32/empty_input === CONT TestDoWithRetry_ClosesBodyWhenNoAttemptIsMade === RUN TestDoWithRetry_ClosesBodyWhenNoAttemptIsMade/retries=0 === PAUSE TestDoWithRetry_ClosesBodyWhenNoAttemptIsMade/retries=0 === RUN TestDoWithRetry_ClosesBodyWhenNoAttemptIsMade/retries=5 === PAUSE TestDoWithRetry_ClosesBodyWhenNoAttemptIsMade/retries=5 === CONT TestStreamPushReportsSkippedPaths --- PASS: TestStreamPushReportsSkippedPaths (0.00s) === CONT TestPartSizeForNAR === RUN TestPartSizeForNAR/zero_stays_at_minimum === PAUSE TestPartSizeForNAR/zero_stays_at_minimum === RUN TestPartSizeForNAR/small_stays_at_minimum === PAUSE TestPartSizeForNAR/small_stays_at_minimum === RUN TestPartSizeForNAR/80_GiB_fits_at_minimum === PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum === RUN TestPartSizeForNAR/115_GiB_needs_larger_parts === PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts === RUN TestPartSizeForNAR/1_TiB === PAUSE TestPartSizeForNAR/1_TiB === RUN TestPartSizeForNAR/5_TiB_S3_max_object === PAUSE TestPartSizeForNAR/5_TiB_S3_max_object === RUN TestPartSizeForNAR/capped_at_5_GiB === PAUSE TestPartSizeForNAR/capped_at_5_GiB === CONT TestScriptTokenEmptyCommand --- PASS: TestScriptTokenEmptyCommand (0.00s) === CONT TestStreamPushGivesUpOnDeadServer === CONT TestStreamPushReportsSignatures === CONT TestStreamPushRequestLine === CONT TestScriptTokenScriptFails === CONT TestScriptTokenBadJSON === CONT TestScriptTokenEmptyToken === CONT TestScriptTokenCachesUntilRefresh === CONT TestScriptTokenNoExpiryRerunsEveryCall === CONT TestFileTokenEmpty === CONT TestFileTokenMissing === CONT TestFileTokenReadsAndCaches === CONT TestStaticToken === CONT TestUploadMultipart_PartsInParallel === CONT TestSetClientTLSErrors === CONT TestDumpPathWriterError === CONT TestSetClientTLSDoesNotMutateDefaultTransport === CONT TestDumpPathSingleFile === CONT TestSetClientTLS === CONT TestDumpPathMatchesNix === CONT TestStreamPushStopsWhenCanceled === CONT TestSupersededNARStillUploadsListing === CONT TestClientSignaturesByStorePath === CONT TestUploadMultipart_SupersededByPeer 2026/10/11 04:52:56 ERROR Upload failed error=boom count=1 2026/10/11 04:52:56 ERROR Upload failed error="connection refused" count=20 2026/10/11 04:52:56 ERROR Server seems unavailable, giving up on batch untried=17 --- PASS: TestStreamPushGivesUpOnDeadServer (0.00s) === CONT TestRunGarbageCollection_NotFoundAfterLocalRun === CONT TestStreamPushIsolatesFailures 2026/10/11 04:52:56 ERROR Upload failed error=boom count=1 --- PASS: TestScriptTokenScriptFails (0.00s) --- PASS: TestStreamPushReportsSignatures (0.00s) === CONT TestFilterOversizedClosures 2026/10/11 04:52:56 ERROR Upload failed error="bad path" count=3 --- PASS: TestStreamPushIsolatesFailures (0.00s) === CONT TestStreamPushBatchesUnderLoad --- PASS: TestStaticToken (0.00s) === CONT TestCaseHackSuffix --- PASS: TestDoServerRequestAttachesToken (0.01s) === CONT TestStreamPushReportsEveryPath --- PASS: TestFileTokenEmpty (0.00s) === CONT TestShellSplitErrors --- PASS: TestFileTokenReadsAndCaches (0.00s) === CONT TestDoWithRetry_FinalResponseBodyReadable --- PASS: TestStreamPushStopsWhenCanceled (0.00s) === CONT TestRetryTimeoutOutlastsMaxRetries === RUN TestSupersededNARStillUploadsListing/small_NAR === PAUSE TestSupersededNARStillUploadsListing/small_NAR === RUN TestSupersededNARStillUploadsListing/dump_cut_short 2026/10/11 04:52:56 INFO Garbage collection started === PAUSE TestSupersededNARStillUploadsListing/dump_cut_short === RUN TestSupersededNARStillUploadsListing/listing_upload_fails === PAUSE TestSupersededNARStillUploadsListing/listing_upload_fails === CONT TestShellSplit === CONT TestRegisterUploadedObject_BoundedAgainstSilentServer --- PASS: TestFileTokenMissing (0.02s) --- PASS: TestClientSignaturesByStorePath (0.00s) === CONT TestRegisterUploadedObjectReusesConnections === RUN TestUploadMultipart_SupersededByPeer/exists === PAUSE TestUploadMultipart_SupersededByPeer/exists === RUN TestUploadMultipart_SupersededByPeer/missing === PAUSE TestUploadMultipart_SupersededByPeer/missing 2026/10/11 04:52:56 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:37991 === CONT TestPathInfoHashCompatibility === RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) === PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) === RUN TestPathInfoHashCompatibility/old_string_format_with_colon === PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon === RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI === PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI === RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512 === PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha512 --- PASS: TestShellSplit (0.00s) 2026/10/11 04:52:56 WARN Rate limiter enabled after throttle name=server-test rate=5 --- PASS: TestScriptTokenEmptyToken (0.03s) --- PASS: TestScriptTokenCachesUntilRefresh (0.03s) === CONT TestDoWithRetry_BodyReplayedViaGetBody === CONT TestResolveStorePath --- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.06s) === CONT TestParsePathInfoJSONMultiplePaths === RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths === PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths === RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths === PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths === CONT TestGetStorePathHash === RUN TestGetStorePathHash/valid_store_path === PAUSE TestGetStorePathHash/valid_store_path === RUN TestGetStorePathHash/basename_without_hyphen_should_error 2026/10/11 04:52:56 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:43155 === PAUSE TestGetStorePathHash/basename_without_hyphen_should_error === RUN TestGetStorePathHash/hash_with_invalid_characters_should_error === PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error === RUN TestGetStorePathHash/hash_with_wrong_length_should_error === PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error === CONT TestParsePathInfoJSON === RUN TestParsePathInfoJSON/Nix_format === PAUSE TestParsePathInfoJSON/Nix_format === RUN TestParsePathInfoJSON/Lix_format === PAUSE TestParsePathInfoJSON/Lix_format === RUN TestParsePathInfoJSON/empty_input === PAUSE TestParsePathInfoJSON/empty_input === RUN TestParsePathInfoJSON/whitespace_only === PAUSE TestParsePathInfoJSON/whitespace_only === RUN TestParsePathInfoJSON/invalid_JSON === PAUSE TestParsePathInfoJSON/invalid_JSON === CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess --- PASS: TestResolveStorePath (0.00s) === CONT TestEncodeNixBase32WithRealHash --- PASS: TestEncodeNixBase32WithRealHash (0.01s) === CONT TestRateLimiterFeedback === RUN TestRateLimiterFeedback/429_enables_limiter === PAUSE TestRateLimiterFeedback/429_enables_limiter === RUN TestRateLimiterFeedback/503_enables_limiter === PAUSE TestRateLimiterFeedback/503_enables_limiter === RUN TestRateLimiterFeedback/200_does_not_enable_limiter === PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter === RUN TestRateLimiterFeedback/400_does_not_enable_limiter === PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter === RUN TestFilterOversizedClosures/no_limit_keeps_everything === PAUSE TestFilterOversizedClosures/no_limit_keeps_everything === RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped === PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped === RUN TestFilterOversizedClosures/all_closures_skipped === PAUSE TestFilterOversizedClosures/all_closures_skipped 2026/10/11 04:52:56 WARN Rate limiter enabled after throttle name=server-test rate=5 --- PASS: TestScriptTokenBadJSON (0.05s) === CONT TestDoWithRetry_ClosesBodyWhenNoAttemptIsMade/retries=0 2026/10/11 04:52:56 WARN Rate limiter enabled after throttle name=s3-test rate=5 === CONT TestEncodeNixBase32/test_string_hash === CONT TestPartSizeForNAR/zero_stays_at_minimum === CONT TestUploadBuildLog_FileBodyReplayedOnRetry --- PASS: TestShellSplitErrors (0.00s) === CONT TestPartSizeForNAR/capped_at_5_GiB === CONT TestPartSizeForNAR/5_TiB_S3_max_object === CONT TestPartSizeForNAR/1_TiB === CONT TestPartSizeForNAR/115_GiB_needs_larger_parts === CONT TestPartSizeForNAR/80_GiB_fits_at_minimum === CONT TestPartSizeForNAR/small_stays_at_minimum --- PASS: TestPartSizeForNAR (0.00s) --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s) --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s) --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s) --- PASS: TestPartSizeForNAR/1_TiB (0.00s) --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s) --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s) --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s) === CONT TestDoWithRetry_ClosesBodyWhenNoAttemptIsMade/retries=5 === CONT TestSupersededNARStillUploadsListing/dump_cut_short === CONT TestEncodeNixBase32/empty_input === CONT TestSupersededNARStillUploadsListing/listing_upload_fails 2026/10/11 04:52:56 WARN Rate limiter enabled after throttle name=s3-test rate=5 --- PASS: TestStreamPushReportsEveryPath (0.00s) --- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.07s) --- PASS: TestDoWithRetry_ClosesBodyWhenNoAttemptIsMade (0.00s) --- PASS: TestDoWithRetry_ClosesBodyWhenNoAttemptIsMade/retries=0 (0.00s) --- PASS: TestDoWithRetry_ClosesBodyWhenNoAttemptIsMade/retries=5 (0.00s) --- PASS: TestEncodeNixBase32 (0.00s) --- PASS: TestEncodeNixBase32/test_string_hash (0.00s) --- PASS: TestEncodeNixBase32/empty_input (0.00s) === CONT TestUploadMultipart_SupersededByPeer/exists === CONT TestRunGarbageCollection_FinishedOnAnotherReplica === CONT TestPathInfoCACompatibility 2026/10/11 04:52:56 INFO Uploading abc-hello (1.0GB) 2026/10/11 04:52:56 WARN Rate limiter enabled after throttle name=server-test rate=5 2026/10/11 04:52:56 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:42133 === RUN TestPathInfoCACompatibility/null_ca_field === CONT TestSupersededNARStillUploadsListing/small_NAR === CONT TestConvertHashToNix32 === PAUSE TestPathInfoCACompatibility/null_ca_field --- PASS: TestCaseHackSuffix (0.08s) === RUN TestPathInfoCACompatibility/old_string_format_-_text 2026/10/11 04:52:57 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:45159/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:45159/api/uploads/complete 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:37991 2026/10/11 04:52:57 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded" 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:37991 2026/10/11 04:52:57 INFO Uploading abc-hello (1.0GB) === RUN TestSetClientTLSErrors/missing_cert_file === PAUSE TestSetClientTLSErrors/missing_cert_file === RUN TestSetClientTLSErrors/missing_key_file === PAUSE TestSetClientTLSErrors/missing_key_file 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:37991 === RUN TestSetClientTLSErrors/missing_ca_file === PAUSE TestSetClientTLSErrors/missing_ca_file === RUN TestSetClientTLSErrors/invalid_ca_file === PAUSE TestSetClientTLSErrors/invalid_ca_file === CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths === CONT TestGetStorePathHash/valid_store_path === CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths --- PASS: TestParsePathInfoJSONMultiplePaths (0.00s) --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s) --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s) === CONT TestParsePathInfoJSON/Nix_format === CONT TestGetStorePathHash/hash_with_wrong_length_should_error === CONT TestGetStorePathHash/hash_with_invalid_characters_should_error === CONT TestGetStorePathHash/basename_without_hyphen_should_error --- PASS: TestGetStorePathHash (0.00s) --- PASS: TestGetStorePathHash/valid_store_path (0.00s) --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s) --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s) --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s) === CONT TestRateLimiterFeedback/429_enables_limiter 2026/10/11 04:52:57 WARN Rate limiter backed off name=server-test rate=5 --- PASS: TestDoWithRetry_FinalResponseBodyReadable (0.33s) === CONT TestUploadMultipart_SupersededByPeer/missing 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:37991 2026/10/11 04:52:57 WARN Rate limiter enabled after throttle name=server-test rate=5 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:40117 2026/10/11 04:52:57 WARN Rate limiter backed off name=server-test rate=5 === PAUSE TestPathInfoCACompatibility/old_string_format_-_text === RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive === PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive === RUN TestPathInfoCACompatibility/new_structured_format_-_text === PAUSE TestPathInfoCACompatibility/new_structured_format_-_text === RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method === PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method === CONT TestParsePathInfoJSON/whitespace_only === CONT TestParsePathInfoJSON/empty_input === CONT TestParsePathInfoJSON/Lix_format === CONT TestParsePathInfoJSON/invalid_JSON === CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped === CONT TestFilterOversizedClosures/all_closures_skipped 2026/10/11 04:52:57 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000 === CONT TestRateLimiterFeedback/200_does_not_enable_limiter 2026/10/11 04:52:57 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50 === CONT TestRateLimiterFeedback/503_enables_limiter 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:37991 --- PASS: TestParsePathInfoJSON (0.00s) --- PASS: TestParsePathInfoJSON/Nix_format (0.00s) --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s) --- PASS: TestParsePathInfoJSON/empty_input (0.00s) --- PASS: TestParsePathInfoJSON/Lix_format (0.00s) --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s) === RUN TestConvertHashToNix32/SRI_format_to_Nix32 === PAUSE TestConvertHashToNix32/SRI_format_to_Nix32 === RUN TestConvertHashToNix32/already_Nix32_format === PAUSE TestConvertHashToNix32/already_Nix32_format === RUN TestConvertHashToNix32/invalid_format === PAUSE TestConvertHashToNix32/invalid_format 2026/10/11 04:52:57 WARN Rate limiter enabled after throttle name=server-test rate=5 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:40299 === CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) 2026/10/11 04:52:57 INFO Garbage collection started === CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512 === CONT TestRateLimiterFeedback/400_does_not_enable_limiter === CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI === CONT TestSetClientTLSErrors/missing_cert_file === CONT TestPathInfoHashCompatibility/old_string_format_with_colon --- PASS: TestPathInfoHashCompatibility (0.00s) --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s) --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s) --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s) --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s) === CONT TestSetClientTLSErrors/invalid_ca_file 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:37991 === CONT TestSetClientTLSErrors/missing_key_file === CONT TestSetClientTLSErrors/missing_ca_file 2026/10/11 04:52:57 WARN Rate limiter backed off name=server-test rate=5 === CONT TestConvertHashToNix32/invalid_format === CONT TestConvertHashToNix32/already_Nix32_format 2026/10/11 04:52:57 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=0 --- PASS: TestRateLimiterFeedback (0.00s) --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.00s) --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.00s) --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s) --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s) --- PASS: TestSetClientTLSErrors (0.36s) --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s) --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s) --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s) --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s) === CONT TestPathInfoCACompatibility/null_ca_field === CONT TestConvertHashToNix32/SRI_format_to_Nix32 --- PASS: TestConvertHashToNix32 (0.00s) --- PASS: TestConvertHashToNix32/invalid_format (0.00s) --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s) --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s) 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:37991 2026/10/11 04:52:57 WARN Garbage collection finished on another replica. Its statistics are not available here --- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.29s) 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:37991 === CONT TestFilterOversizedClosures/no_limit_keeps_everything --- PASS: TestFilterOversizedClosures (0.00s) --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s) --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s) --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s) === RUN TestSetClientTLS/rejects_connection_without_client_cert === PAUSE TestSetClientTLS/rejects_connection_without_client_cert === RUN TestSetClientTLS/succeeds_with_client_cert_and_CA === PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA === RUN TestSetClientTLS/preserves_debug_logging_transport === PAUSE TestSetClientTLS/preserves_debug_logging_transport === CONT TestSetClientTLS/rejects_connection_without_client_cert === CONT TestPathInfoCACompatibility/new_structured_format_-_text === CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method 2026/10/11 04:52:57 WARN Rate limiter enabled after throttle name=s3-test rate=5 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:43273 === CONT TestSetClientTLS/preserves_debug_logging_transport === CONT TestSetClientTLS/succeeds_with_client_cert_and_CA --- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (0.30s) 2026/10/11 04:52:57 WARN Rate limiter backed off name=server-test rate=5 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:42133 --- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.31s) === CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive === CONT TestPathInfoCACompatibility/old_string_format_-_text --- PASS: TestPathInfoCACompatibility (0.28s) --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s) --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s) --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s) --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s) --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s) 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:37991 2026/10/11 04:52:57 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:45159/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:45159/api/uploads/complete 2026/10/11 04:52:57 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded" 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:37991 --- PASS: TestUploadMultipart_SupersededByPeer (0.00s) --- PASS: TestUploadMultipart_SupersededByPeer/missing (0.00s) --- PASS: TestUploadMultipart_SupersededByPeer/exists (0.30s) --- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.38s) --- PASS: TestDumpPathSingleFile (0.38s) --- PASS: TestStreamPushBatchesUnderLoad (0.38s) 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:37991 --- PASS: TestStreamPushRequestLine (0.38s) 2026/10/11 04:52:57 http: TLS handshake error from 127.0.0.1:58606: remote error: tls: bad certificate --- PASS: TestSetClientTLS (0.34s) --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.00s) --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.01s) --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.01s) 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:37991 2026/10/11 04:52:57 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:45159/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:45159/api/uploads/complete 2026/10/11 04:52:57 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded" === NAME TestRegisterUploadedObject_BoundedAgainstSilentServer register_upload_test.go:65: server saw 0 registrations, want 3 --- FAIL: TestRegisterUploadedObject_BoundedAgainstSilentServer (0.36s) 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:37991 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:37991 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:37991 --- PASS: TestRegisterUploadedObjectReusesConnections (0.36s) 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:37991 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:37991 2026/10/11 04:52:57 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:37991 --- PASS: TestRetryTimeoutOutlastsMaxRetries (0.37s) --- PASS: TestDumpPathWriterError (0.39s) 2026/10/11 04:52:57 INFO Uploading abc-hello (1.0GB) 2026/10/11 04:52:57 WARN Request failed, retrying attempt=1 backoff=100ms error="Put \"http://127.0.0.1:41229/part/2\": uploading part 1: server returned 404: no such upload\n" url=http://127.0.0.1:41229/part/2 --- PASS: TestDumpPathMatchesNix (0.41s) 2026/10/11 04:52:57 WARN Request failed, retrying attempt=1 backoff=100ms error="Put \"http://127.0.0.1:41229/part/3\": uploading part 1: server returned 404: no such upload\n" url=http://127.0.0.1:41229/part/3 2026/10/11 04:52:57 WARN Request failed, retrying attempt=1 backoff=100ms error="Put \"http://127.0.0.1:41229/part/4\": uploading part 1: server returned 404: no such upload\n" url=http://127.0.0.1:41229/part/4 2026/10/11 04:52:57 WARN Request failed, retrying attempt=1 backoff=100ms error="Put \"http://127.0.0.1:41229/part/5\": uploading part 1: server returned 404: no such upload\n" url=http://127.0.0.1:41229/part/5 2026/10/11 04:52:57 WARN Request failed, retrying attempt=1 backoff=100ms error="Put \"http://127.0.0.1:41229/part/6\": uploading part 1: server returned 404: no such upload\n" url=http://127.0.0.1:41229/part/6 2026/10/11 04:52:57 ERROR Failed to close zstd encoder error="io: read/write on closed pipe" --- PASS: TestSupersededNARStillUploadsListing (0.00s) --- PASS: TestSupersededNARStillUploadsListing/listing_upload_fails (0.29s) --- PASS: TestSupersededNARStillUploadsListing/small_NAR (0.30s) --- PASS: TestSupersededNARStillUploadsListing/dump_cut_short (0.85s) --- PASS: TestUploadMultipart_PartsInParallel (0.96s) --- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.12s) FAIL