niks3-go-unit-tests
checks.aarch64-linux.go-unit-tests
· build #359
· raw
1tribuchet: building on eliza2Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestRegisterUploadedObjectReusesConnections6=== PAUSE TestRegisterUploadedObjectReusesConnections7=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica8=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica9=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun10=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun11=== RUN TestCaseHackSuffix12=== PAUSE TestCaseHackSuffix13=== RUN TestFilterOversizedClosures14=== PAUSE TestFilterOversizedClosures15=== RUN TestUploadMultipart_PartsInParallel16=== PAUSE TestUploadMultipart_PartsInParallel17=== RUN TestPartSizeForNAR18=== PAUSE TestPartSizeForNAR19=== RUN TestUploadMultipart_SupersededByPeer20=== PAUSE TestUploadMultipart_SupersededByPeer21=== RUN TestDumpPathCaseHackMatchesNix22--- PASS: TestDumpPathCaseHackMatchesNix (0.03s)23=== RUN TestDumpPathCaseHackCollision24--- PASS: TestDumpPathCaseHackCollision (0.00s)25=== RUN TestDumpPathMatchesNix26=== PAUSE TestDumpPathMatchesNix27=== RUN TestDumpPathSingleFile28=== PAUSE TestDumpPathSingleFile29=== RUN TestDumpPathWriterError30=== PAUSE TestDumpPathWriterError31=== RUN TestEncodeNixBase3232=== PAUSE TestEncodeNixBase3233=== RUN TestEncodeNixBase32WithRealHash34=== PAUSE TestEncodeNixBase32WithRealHash35=== RUN TestConvertHashToNix3236=== PAUSE TestConvertHashToNix3237=== RUN TestGetStorePathHash38=== PAUSE TestGetStorePathHash39=== RUN TestPathInfoHashCompatibility40=== PAUSE TestPathInfoHashCompatibility41=== RUN TestParsePathInfoJSON42=== PAUSE TestParsePathInfoJSON43=== RUN TestParsePathInfoJSONMultiplePaths44=== PAUSE TestParsePathInfoJSONMultiplePaths45=== RUN TestPathInfoCACompatibility46=== PAUSE TestPathInfoCACompatibility47=== RUN TestRateLimiterFeedback48=== PAUSE TestRateLimiterFeedback49=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess50=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess51=== RUN TestResolveStorePath52=== PAUSE TestResolveStorePath53=== RUN TestDoWithRetry_BodyReplayedViaGetBody54=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody55=== RUN TestShellSplit56=== PAUSE TestShellSplit57=== RUN TestShellSplitErrors58=== PAUSE TestShellSplitErrors59=== RUN TestStreamPushReportsEveryPath60=== PAUSE TestStreamPushReportsEveryPath61=== RUN TestStreamPushBatchesUnderLoad62=== PAUSE TestStreamPushBatchesUnderLoad63=== RUN TestStreamPushIsolatesFailures64=== PAUSE TestStreamPushIsolatesFailures65=== RUN TestStreamPushGivesUpOnDeadServer66=== PAUSE TestStreamPushGivesUpOnDeadServer67=== RUN TestStreamPushRequestLine68=== PAUSE TestStreamPushRequestLine69=== RUN TestStreamPushReportsSignatures70=== PAUSE TestStreamPushReportsSignatures71=== RUN TestClientSignaturesByStorePath72=== PAUSE TestClientSignaturesByStorePath73=== RUN TestSetClientTLS74=== PAUSE TestSetClientTLS75=== RUN TestSetClientTLSDoesNotMutateDefaultTransport76=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport77=== RUN TestSetClientTLSErrors78=== PAUSE TestSetClientTLSErrors79=== RUN TestStaticToken80=== PAUSE TestStaticToken81=== RUN TestFileTokenReadsAndCaches82=== PAUSE TestFileTokenReadsAndCaches83=== RUN TestFileTokenMissing84=== PAUSE TestFileTokenMissing85=== RUN TestFileTokenEmpty86=== PAUSE TestFileTokenEmpty87=== RUN TestScriptTokenNoExpiryRerunsEveryCall88=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall89=== RUN TestScriptTokenCachesUntilRefresh90=== PAUSE TestScriptTokenCachesUntilRefresh91=== RUN TestScriptTokenEmptyToken92=== PAUSE TestScriptTokenEmptyToken93=== RUN TestScriptTokenBadJSON94=== PAUSE TestScriptTokenBadJSON95=== RUN TestScriptTokenScriptFails96=== PAUSE TestScriptTokenScriptFails97=== RUN TestScriptTokenEmptyCommand98=== PAUSE TestScriptTokenEmptyCommand99=== CONT TestDoServerRequestAttachesToken100=== CONT TestDoWithRetry_BodyReplayedViaGetBody101=== CONT TestScriptTokenEmptyCommand102--- PASS: TestScriptTokenEmptyCommand (0.00s)103=== CONT TestScriptTokenBadJSON104=== CONT TestEncodeNixBase32105=== RUN TestEncodeNixBase32/test_string_hash106=== CONT TestResolveStorePath107=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess108=== CONT TestRateLimiterFeedback109=== RUN TestRateLimiterFeedback/429_enables_limiter110=== PAUSE TestRateLimiterFeedback/429_enables_limiter1112026/10/05 11:42:43 WARN Rate limiter enabled after throttle name=server-test rate=5112=== RUN TestRateLimiterFeedback/503_enables_limiter113=== PAUSE TestRateLimiterFeedback/503_enables_limiter114=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter115=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter116=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter117=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter118=== CONT TestScriptTokenEmptyToken119=== CONT TestPathInfoCACompatibility120=== RUN TestPathInfoCACompatibility/null_ca_field121=== CONT TestParsePathInfoJSONMultiplePaths122=== CONT TestParsePathInfoJSON123=== CONT TestConvertHashToNix32124=== CONT TestEncodeNixBase32WithRealHash125=== CONT TestUploadMultipart_PartsInParallel126=== CONT TestDumpPathWriterError127=== CONT TestDumpPathSingleFile128=== CONT TestDumpPathMatchesNix129=== CONT TestUploadMultipart_SupersededByPeer130=== CONT TestPartSizeForNAR131=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun132=== CONT TestFilterOversizedClosures133=== CONT TestSetClientTLSDoesNotMutateDefaultTransport134=== CONT TestCaseHackSuffix135=== CONT TestScriptTokenScriptFails136=== CONT TestPathInfoHashCompatibility137=== PAUSE TestEncodeNixBase32/test_string_hash138--- PASS: TestResolveStorePath (0.00s)139=== CONT TestFileTokenMissing140=== PAUSE TestPathInfoCACompatibility/null_ca_field141=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths142=== RUN TestPathInfoCACompatibility/old_string_format_-_text143--- PASS: TestEncodeNixBase32WithRealHash (0.00s)144=== RUN TestParsePathInfoJSON/Nix_format145=== RUN TestConvertHashToNix32/SRI_format_to_Nix32146=== PAUSE TestParsePathInfoJSON/Nix_format147=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32148=== RUN TestParsePathInfoJSON/Lix_format149=== RUN TestPartSizeForNAR/zero_stays_at_minimum150=== RUN TestConvertHashToNix32/already_Nix32_format151=== RUN TestUploadMultipart_SupersededByPeer/exists152=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum153=== PAUSE TestConvertHashToNix32/already_Nix32_format154=== RUN TestPartSizeForNAR/small_stays_at_minimum155=== PAUSE TestUploadMultipart_SupersededByPeer/exists156=== RUN TestConvertHashToNix32/invalid_format157=== RUN TestUploadMultipart_SupersededByPeer/missing158=== RUN TestFilterOversizedClosures/no_limit_keeps_everything159=== CONT TestFileTokenReadsAndCaches160=== PAUSE TestUploadMultipart_SupersededByPeer/missing161=== CONT TestScriptTokenNoExpiryRerunsEveryCall162--- PASS: TestScriptTokenBadJSON (0.00s)163--- PASS: TestScriptTokenScriptFails (0.03s)164=== CONT TestScriptTokenCachesUntilRefresh165=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)166=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)167=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon168=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text169=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive170=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive171=== RUN TestPathInfoCACompatibility/new_structured_format_-_text172=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text173=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method174=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method175=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon176=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI177=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths178=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths179=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths180=== PAUSE TestPartSizeForNAR/small_stays_at_minimum181=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum182=== CONT TestFileTokenEmpty183=== PAUSE TestConvertHashToNix32/invalid_format184=== CONT TestStaticToken185=== CONT TestStreamPushGivesUpOnDeadServer1862026/10/05 11:42:43 WARN Rate limiter enabled after throttle name=server-test rate=5187=== CONT TestStreamPushIsolatesFailures188=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum1892026/10/05 11:42:43 WARN Request returned retryable status, retrying attempt=1 max_attempts=6 backoff=0s status=503 url=http://127.0.0.1:42265190=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts191=== CONT TestSetClientTLS1922026/10/05 11:42:43 ERROR Upload failed error="connection refused" count=20193=== CONT TestSetClientTLSErrors1942026/10/05 11:42:43 ERROR Server seems unavailable, giving up on batch untried=17195=== CONT TestClientSignaturesByStorePath196=== CONT TestStreamPushBatchesUnderLoad1972026/10/05 11:42:43 ERROR Upload failed error="bad path" count=3198=== RUN TestEncodeNixBase32/empty_input199=== PAUSE TestEncodeNixBase32/empty_input2002026/10/05 11:42:43 WARN Rate limiter backed off name=server-test rate=5201=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI202=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything203=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512204=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha5122052026/10/05 11:42:43 WARN Request returned retryable status, retrying attempt=2 max_attempts=6 backoff=0s status=503 url=http://127.0.0.1:42265206--- PASS: TestFileTokenMissing (0.00s)207=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica208=== CONT TestShellSplitErrors209=== CONT TestShellSplit210--- PASS: TestStaticToken (0.00s)211--- PASS: TestScriptTokenEmptyToken (0.03s)212=== PAUSE TestParsePathInfoJSON/Lix_format213=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts214--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.03s)215--- PASS: TestClientSignaturesByStorePath (0.00s)216--- PASS: TestStreamPushIsolatesFailures (0.00s)217=== RUN TestParsePathInfoJSON/empty_input218=== CONT TestRateLimiterFeedback/429_enables_limiter219=== RUN TestPartSizeForNAR/1_TiB220=== PAUSE TestPartSizeForNAR/1_TiB221=== RUN TestPartSizeForNAR/5_TiB_S3_max_object222=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object223=== CONT TestStreamPushRequestLine2242026/10/05 11:42:43 INFO Garbage collection started225=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped226=== CONT TestRegisterUploadedObjectReusesConnections2272026/10/05 11:42:43 INFO Garbage collection started228=== CONT TestStreamPushReportsEveryPath229=== CONT TestGetStorePathHash230=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter2312026/10/05 11:42:43 ERROR Upload failed error=boom count=1232--- PASS: TestStreamPushGivesUpOnDeadServer (0.00s)233--- PASS: TestFileTokenReadsAndCaches (0.00s)234--- PASS: TestShellSplitErrors (0.00s)235--- PASS: TestShellSplit (0.00s)236--- PASS: TestDoServerRequestAttachesToken (0.03s)237--- PASS: TestFileTokenEmpty (0.00s)2382026/10/05 11:42:43 WARN Rate limiter enabled after throttle name=server-test rate=5239--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.03s)240--- PASS: TestStreamPushReportsEveryPath (0.00s)2412026/10/05 11:42:43 WARN Request returned retryable status, retrying attempt=1 max_attempts=2 backoff=0s status=429 url=http://127.0.0.1:46121242=== PAUSE TestParsePathInfoJSON/empty_input243=== RUN TestParsePathInfoJSON/whitespace_only244=== PAUSE TestParsePathInfoJSON/whitespace_only245=== RUN TestSetClientTLSErrors/missing_cert_file246=== PAUSE TestSetClientTLSErrors/missing_cert_file247=== RUN TestSetClientTLSErrors/missing_key_file248=== PAUSE TestSetClientTLSErrors/missing_key_file249=== RUN TestSetClientTLSErrors/missing_ca_file250=== PAUSE TestSetClientTLSErrors/missing_ca_file2512026/10/05 11:42:43 WARN Rate limiter backed off name=server-test rate=5252=== RUN TestSetClientTLSErrors/invalid_ca_file253=== PAUSE TestSetClientTLSErrors/invalid_ca_file254=== RUN TestParsePathInfoJSON/invalid_JSON255=== PAUSE TestParsePathInfoJSON/invalid_JSON256=== CONT TestUploadMultipart_SupersededByPeer/exists257=== CONT TestPathInfoCACompatibility/null_ca_field258=== CONT TestPathInfoCACompatibility/new_structured_format_-_text259=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method260=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive261=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths262=== CONT TestUploadMultipart_SupersededByPeer/missing263=== CONT TestPathInfoCACompatibility/old_string_format_-_text264--- PASS: TestPathInfoCACompatibility (0.03s)265 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)266 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)267 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)268 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)269 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)270=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths271--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.03s)272--- PASS: TestParsePathInfoJSONMultiplePaths (0.03s)273 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)274 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)275=== RUN TestPartSizeForNAR/capped_at_5_GiB276=== PAUSE TestPartSizeForNAR/capped_at_5_GiB277=== CONT TestConvertHashToNix32/already_Nix32_format278=== CONT TestEncodeNixBase32/empty_input279=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)280=== CONT TestEncodeNixBase32/test_string_hash281=== RUN TestGetStorePathHash/valid_store_path282=== PAUSE TestGetStorePathHash/valid_store_path283=== RUN TestGetStorePathHash/basename_without_hyphen_should_error284=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error285=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error286=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error287=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error288=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error289=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512290=== CONT TestSetClientTLSErrors/missing_cert_file291=== CONT TestParsePathInfoJSON/Nix_format2922026/10/05 11:42:43 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=0293=== CONT TestParsePathInfoJSON/invalid_JSON294=== CONT TestParsePathInfoJSON/whitespace_only295=== CONT TestParsePathInfoJSON/empty_input296=== CONT TestParsePathInfoJSON/Lix_format297=== CONT TestSetClientTLSErrors/invalid_ca_file298=== CONT TestSetClientTLSErrors/missing_ca_file299=== CONT TestSetClientTLSErrors/missing_key_file300=== CONT TestPartSizeForNAR/zero_stays_at_minimum301=== CONT TestPartSizeForNAR/1_TiB302=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts303=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum304=== CONT TestPartSizeForNAR/small_stays_at_minimum305=== CONT TestPartSizeForNAR/capped_at_5_GiB306=== CONT TestGetStorePathHash/valid_store_path307=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error308=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error309=== CONT TestGetStorePathHash/basename_without_hyphen_should_error310=== CONT TestRateLimiterFeedback/503_enables_limiter311=== CONT TestPartSizeForNAR/5_TiB_S3_max_object312=== RUN TestSetClientTLS/rejects_connection_without_client_cert313=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert314=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter315=== CONT TestConvertHashToNix32/SRI_format_to_Nix32316=== CONT TestConvertHashToNix32/invalid_format317--- PASS: TestEncodeNixBase32 (0.03s)318 --- PASS: TestEncodeNixBase32/empty_input (0.00s)319 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)320=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI321=== CONT TestStreamPushReportsSignatures322=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon323--- PASS: TestParsePathInfoJSON (0.03s)324 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)325 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)326 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)327 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)328 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)329--- PASS: TestPathInfoHashCompatibility (0.00s)330 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)331 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)332 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)333 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)334--- PASS: TestUploadMultipart_SupersededByPeer (0.03s)335 --- PASS: TestUploadMultipart_SupersededByPeer/exists (0.00s)336 --- PASS: TestUploadMultipart_SupersededByPeer/missing (0.00s)337=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA338=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA3392026/10/05 11:42:43 ERROR Upload failed error=boom count=1340=== RUN TestSetClientTLS/preserves_debug_logging_transport341=== PAUSE TestSetClientTLS/preserves_debug_logging_transport342--- PASS: TestStreamPushReportsSignatures (0.00s)343=== CONT TestSetClientTLS/rejects_connection_without_client_cert344--- PASS: TestGetStorePathHash (0.00s)345 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)346 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)347 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)348 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)3492026/10/05 11:42:43 WARN Rate limiter enabled after throttle name=server-test rate=53502026/10/05 11:42:43 WARN Request returned retryable status, retrying attempt=1 max_attempts=2 backoff=0s status=503 url=http://127.0.0.1:41811351=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped352=== RUN TestFilterOversizedClosures/all_closures_skipped353=== PAUSE TestFilterOversizedClosures/all_closures_skipped354=== CONT TestFilterOversizedClosures/no_limit_keeps_everything355=== CONT TestSetClientTLS/preserves_debug_logging_transport3562026/10/05 11:42:43 WARN Garbage collection finished on another replica. Its statistics are not available here357--- PASS: TestSetClientTLSErrors (0.00s)358 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)359 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)360 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)361 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)362--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.00s)363--- PASS: TestConvertHashToNix32 (0.03s)364 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)365 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)366 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)3672026/10/05 11:42:43 WARN Rate limiter backed off name=server-test rate=5368--- PASS: TestPartSizeForNAR (0.03s)369 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)370 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)371 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)372 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)373 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)374 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)375 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)376=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3772026/10/05 11:42:43 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000378--- PASS: TestRateLimiterFeedback (0.00s)379 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.00s)380 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.00s)381 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.00s)382 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.00s)383=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA384=== CONT TestFilterOversizedClosures/all_closures_skipped3852026/10/05 11:42:43 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50386--- PASS: TestFilterOversizedClosures (0.04s)387 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)388 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)389 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)390--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.01s)391--- PASS: TestScriptTokenCachesUntilRefresh (0.01s)3922026/10/05 11:42:43 http: TLS handshake error from 127.0.0.1:49878: remote error: tls: bad certificate393--- PASS: TestSetClientTLS (0.00s)394 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.01s)395 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.01s)396 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.02s)397--- PASS: TestRegisterUploadedObjectReusesConnections (0.02s)398--- PASS: TestDumpPathSingleFile (0.06s)399--- PASS: TestStreamPushRequestLine (0.03s)400--- PASS: TestCaseHackSuffix (0.06s)401--- PASS: TestDumpPathWriterError (0.06s)402--- PASS: TestDumpPathMatchesNix (0.07s)403--- PASS: TestStreamPushBatchesUnderLoad (0.10s)404--- PASS: TestUploadMultipart_PartsInParallel (0.65s)405--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)406PASS407Running server tests...408The files belonging to this database system will be owned by user "nixbld".409This user must also own the server process.410411The database cluster will be initialized with locale "C".412The default database encoding has accordingly been set to "SQL_ASCII".413The default text search configuration will be set to "english".414415Data page checksums are enabled.416417creating directory /build/postgres905301565/data ... ok418creating subdirectories ... ok419selecting dynamic shared memory implementation ... posix420selecting default "max_connections" ... 100421selecting default "shared_buffers" ... 128MB422selecting default time zone ... UTC423creating configuration files ... ok424running bootstrap script ... ok425performing post-bootstrap initialization ... ok426syncing data to disk ... ok427428initdb: warning: enabling "trust" authentication for local connections429initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.430431Success. You can now start the database server using:432433 pg_ctl -D /build/postgres905301565/data -l logfile start434435/build/postgres905301565:5432 - no response4362026-10-05 11:42:45.610 UTC [129] LOG: starting PostgreSQL 18.6 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit4372026-10-05 11:42:45.611 UTC [129] LOG: listening on Unix socket "/build/postgres905301565/.s.PGSQL.5432"4382026-10-05 11:42:45.615 UTC [136] LOG: database system was shut down at 2026-10-05 11:42:45 UTC4392026-10-05 11:42:45.618 UTC [129] LOG: database system is ready to accept connections440/build/postgres905301565:5432 - accepting connections441{"timestamp":"2026-10-05T11:42:45.808875435Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"dc8a190f-218f-4f07-a07c-fe7d162bbc3a","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":501,"threadName":"rustfs-worker","threadId":"ThreadId(194)"}442=== RUN TestService_AuthMiddleware443=== PAUSE TestService_AuthMiddleware444=== RUN TestService_AuthMiddleware_MTLSProxyHeader445=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader446=== RUN TestService_AuthMiddleware_MTLSBoundSubjects447=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects448=== RUN TestService_ReadAuthMiddleware449=== PAUSE TestService_ReadAuthMiddleware450=== RUN TestService_AuthMiddleware_OIDC451=== PAUSE TestService_AuthMiddleware_OIDC452=== RUN TestService_RequireScope_OIDC453=== PAUSE TestService_RequireScope_OIDC454=== RUN TestService_ReadScope_PublicByDefault455=== PAUSE TestService_ReadScope_PublicByDefault456=== RUN TestCacheConfigHandler457=== PAUSE TestCacheConfigHandler458=== RUN TestCacheStatsHandler459=== PAUSE TestCacheStatsHandler460=== RUN TestClientCADerivations461=== PAUSE TestClientCADerivations462=== RUN TestClientErrorHandling463=== PAUSE TestClientErrorHandling464=== RUN TestClientIntegration465=== PAUSE TestClientIntegration466=== RUN TestClientMultipleUploads467=== PAUSE TestClientMultipleUploads468=== RUN TestClientWithDependencies469=== PAUSE TestClientWithDependencies470=== RUN TestClientSharedPathCommittedMidPush471=== PAUSE TestClientSharedPathCommittedMidPush472=== RUN TestPinProtectsFromGC473=== PAUSE TestPinProtectsFromGC474=== RUN TestClientPushesUseOnePush475=== PAUSE TestClientPushesUseOnePush476=== RUN TestClientFallsBackToClosures477=== PAUSE TestClientFallsBackToClosures478=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock479=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock480=== RUN TestResolveDBConnectionString481=== PAUSE TestResolveDBConnectionString482=== RUN TestConnectWaitsForAPeerMigration483=== PAUSE TestConnectWaitsForAPeerMigration484=== RUN TestLeadElectsOneAndHandsOver485=== PAUSE TestLeadElectsOneAndHandsOver486=== RUN TestLeadIncumbentWinsAfterRestart4872026/10/05 11:42:46 INFO lead: acquired remote=192.0.2.1:12344882026/10/05 11:42:46 INFO lead: released remote=192.0.2.1:12344892026/10/05 11:42:46 INFO lead: acquired remote=192.0.2.1:12344902026/10/05 11:42:47 INFO lead: released remote=192.0.2.1:1234491--- PASS: TestLeadIncumbentWinsAfterRestart (1.89s)492=== RUN TestLeadIncumbentAnnouncesAtOnce493=== PAUSE TestLeadIncumbentAnnouncesAtOnce494=== RUN TestLeadEndsOnShutdown495=== PAUSE TestLeadEndsOnShutdown496=== RUN TestLeadEndsWhenItsConnectionHangs4972026/10/05 11:42:47 INFO lead: acquired remote=192.0.2.1:12344982026-10-05 11:42:49.302 UTC [388] FATAL: terminating connection due to administrator command4992026/10/05 11:42:49 INFO lead: acquired remote=192.0.2.1:12345002026/10/05 11:42:50 WARN lead: lock connection lost error="timeout: context deadline exceeded"5012026/10/05 11:42:50 INFO lead: released remote=192.0.2.1:12345022026/10/05 11:42:50 INFO lead: released remote=192.0.2.1:1234503--- PASS: TestLeadEndsWhenItsConnectionHangs (3.10s)504=== RUN TestGCAdvisoryLockBlocksConcurrentRun505=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun506=== RUN TestGCBugBareHashReferences507=== PAUSE TestGCBugBareHashReferences508=== RUN TestGCMetrics509=== PAUSE TestGCMetrics510=== RUN TestPushDedupSurvivesConcurrentGC511=== PAUSE TestPushDedupSurvivesConcurrentGC512=== RUN TestDeduplicatedObjectsRecordedAsPending513=== PAUSE TestDeduplicatedObjectsRecordedAsPending514=== RUN TestGCSweepSkipsPendingObjects515=== PAUSE TestGCSweepSkipsPendingObjects516=== RUN TestTombstonedObjectOfferedWithoutWaiting517=== PAUSE TestTombstonedObjectOfferedWithoutWaiting518=== RUN TestGCSweepDeliversEachKeyOnce519=== PAUSE TestGCSweepDeliversEachKeyOnce520=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection521=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection522=== RUN TestForceGCDuringPushOffersSweptObject523=== PAUSE TestForceGCDuringPushOffersSweptObject524=== RUN TestFailedS3DeleteKeepsTombstone525=== PAUSE TestFailedS3DeleteKeepsTombstone526=== RUN TestSweepSkipsRowsLockedByPush527=== PAUSE TestSweepSkipsRowsLockedByPush528=== RUN TestRegisterCompletedObjectNeedsPendingRow529=== PAUSE TestRegisterCompletedObjectNeedsPendingRow530=== RUN TestGetExistingObjectsLocksRows531=== PAUSE TestGetExistingObjectsLocksRows532=== RUN TestSweepRechecksPendingAfterLocking533=== PAUSE TestSweepRechecksPendingAfterLocking534=== RUN TestPushWaitsForSweepHoldingRow535=== PAUSE TestPushWaitsForSweepHoldingRow536=== RUN TestCommitRacingPendingCleanupKeepsObjects537=== PAUSE TestCommitRacingPendingCleanupKeepsObjects538=== RUN TestGCEndsOnShutdown539=== PAUSE TestGCEndsOnShutdown540=== RUN TestGCTaskStore_StartNew541=== PAUSE TestGCTaskStore_StartNew542=== RUN TestGCTaskStore_DeduplicateSameParams543=== PAUSE TestGCTaskStore_DeduplicateSameParams544=== RUN TestGCTaskStore_ConflictDifferentParams545=== PAUSE TestGCTaskStore_ConflictDifferentParams546=== RUN TestGCTaskStore_GetEmpty547=== PAUSE TestGCTaskStore_GetEmpty548=== RUN TestGCTaskStore_GetReturnsLatest549=== PAUSE TestGCTaskStore_GetReturnsLatest550=== RUN TestGCTaskStore_CompletedAllowsNewTask551=== PAUSE TestGCTaskStore_CompletedAllowsNewTask552=== RUN TestGCTaskStore_PhaseUpdates553=== PAUSE TestGCTaskStore_PhaseUpdates554=== RUN TestGCTaskStore_Fail555=== PAUSE TestGCTaskStore_Fail556=== RUN TestGracefulShutdownDrainsInflight557=== PAUSE TestGracefulShutdownDrainsInflight558=== RUN TestService_healthCheckHandler559=== PAUSE TestService_healthCheckHandler560=== RUN TestService_readinessHandler561=== PAUSE TestService_readinessHandler562=== RUN TestGenerateLandingPage563=== PAUSE TestGenerateLandingPage564=== RUN TestCacheConfigHandlerMaxNarSize565=== PAUSE TestCacheConfigHandlerMaxNarSize566=== RUN TestCreatePendingClosureRejectsOversizedNAR567=== PAUSE TestCreatePendingClosureRejectsOversizedNAR568=== RUN TestNARDeduplicationMetadataUploadBug569=== PAUSE TestNARDeduplicationMetadataUploadBug570=== RUN TestMetricsInventory571=== PAUSE TestMetricsInventory572=== RUN TestService_NativeMTLS573=== PAUSE TestService_NativeMTLS574=== RUN TestServerTLSConfig575=== PAUSE TestServerTLSConfig576=== RUN TestMultipartCleanup577=== PAUSE TestMultipartCleanup578=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded579=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded580=== RUN TestPendingCleanupUsesOneCutoff581=== PAUSE TestPendingCleanupUsesOneCutoff582=== RUN TestObjectStatsTrigger583=== PAUSE TestObjectStatsTrigger584=== RUN TestOrphanedObjectsGC585=== PAUSE TestOrphanedObjectsGC586=== RUN TestOrphanedObjectsGCStressTest587=== PAUSE TestOrphanedObjectsGCStressTest588=== RUN TestCreatePin_ReservedPins589=== PAUSE TestCreatePin_ReservedPins590=== RUN TestPresentReportsOnlyClosureRoots591=== PAUSE TestPresentReportsOnlyClosureRoots592=== RUN TestPresentNotReportedWhileGCDeletesClosure593=== PAUSE TestPresentNotReportedWhileGCDeletesClosure594=== RUN TestParseSingleRange595=== PAUSE TestParseSingleRange596=== RUN TestProxyHeadersOnlyTrustedOnSocket597=== PAUSE TestProxyHeadersOnlyTrustedOnSocket598=== RUN TestIsValidCachePath599=== PAUSE TestIsValidCachePath600=== RUN TestReadProxyNarinfo601=== PAUSE TestReadProxyNarinfo602=== RUN TestReadProxyNarinfoAlreadyDecompressed603=== PAUSE TestReadProxyNarinfoAlreadyDecompressed604=== RUN TestReadProxyNarStreaming605=== PAUSE TestReadProxyNarStreaming606=== RUN TestReadProxy404607=== PAUSE TestReadProxy404608=== RUN TestReadProxyInvalidPath609=== PAUSE TestReadProxyInvalidPath610=== RUN TestReadProxyHead611=== PAUSE TestReadProxyHead612=== RUN TestReadProxyOutlastsServerWriteTimeout613=== PAUSE TestReadProxyOutlastsServerWriteTimeout614=== RUN TestReadProxyConditionalGet615=== PAUSE TestReadProxyConditionalGet616=== RUN TestReadProxyRootRedirectsToIndexHTML617=== PAUSE TestReadProxyRootRedirectsToIndexHTML618=== RUN TestReadProxyDisabled619=== PAUSE TestReadProxyDisabled620=== RUN TestReadRedirectNar621=== PAUSE TestReadRedirectNar622=== RUN TestReadRedirectKeepsNarinfoProxied623=== PAUSE TestReadRedirectKeepsNarinfoProxied624=== RUN TestReadProxyRangeRequest625=== PAUSE TestReadProxyRangeRequest626=== RUN TestReadRedirectUsesPublicS3URL627=== PAUSE TestReadRedirectUsesPublicS3URL628=== RUN TestPush_OverlappingRootsStoreOneRowPerKey629=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey630=== RUN TestPush_CompleteCommitsEveryRoot631=== PAUSE TestPush_CompleteCommitsEveryRoot632=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit633=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit634=== RUN TestPush_RejectsBadRequests635=== PAUSE TestPush_RejectsBadRequests636=== RUN TestPush_SignsNarinfosOfItsPendingObjects637=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects638=== RUN TestRedundantMultipartUpload639=== PAUSE TestRedundantMultipartUpload640=== RUN TestCompleteMultipartUpload_ErrorButObjectExists641=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists642=== RUN TestCompletedNarNotReofferedAcrossClosures643=== PAUSE TestCompletedNarNotReofferedAcrossClosures644=== RUN TestPresignedUploadRegisteredBeforeCommit645=== PAUSE TestPresignedUploadRegisteredBeforeCommit646=== RUN TestService_Rustfstest647=== PAUSE TestService_Rustfstest648=== RUN TestParseSize649=== PAUSE TestParseSize650=== RUN TestSkippedUploadsHandler651=== PAUSE TestSkippedUploadsHandler652=== RUN TestSystemdListenerNotActivated653--- PASS: TestSystemdListenerNotActivated (0.00s)654=== RUN TestWatchdogBeatsWhenHealthy655--- PASS: TestWatchdogBeatsWhenHealthy (0.02s)656=== RUN TestWatchdogSkipsWhenUnhealthy6572026/10/05 11:42:50 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6582026/10/05 11:42:50 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6592026/10/05 11:42:50 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6602026/10/05 11:42:51 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6612026/10/05 11:42:51 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6622026/10/05 11:42:51 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6632026/10/05 11:42:51 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6642026/10/05 11:42:51 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6652026/10/05 11:42:51 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"6662026/10/05 11:42:51 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"667--- PASS: TestWatchdogSkipsWhenUnhealthy (0.20s)668=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle669=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle670=== RUN TestProxyWriteTimeout671=== PAUSE TestProxyWriteTimeout672=== RUN TestPendingClosureWriteTimeout673=== PAUSE TestPendingClosureWriteTimeout674=== RUN TestIsValidUploadKey675=== PAUSE TestIsValidUploadKey676=== RUN TestUploadHandlersRejectInvalidKeys677=== PAUSE TestUploadHandlersRejectInvalidKeys678=== RUN TestUploadHandlersRejectOversizedBody679=== PAUSE TestUploadHandlersRejectOversizedBody680=== RUN TestService_cleanupPendingClosuresHandler681=== PAUSE TestService_cleanupPendingClosuresHandler682=== RUN TestService_createPendingClosureHandler683=== PAUSE TestService_createPendingClosureHandler684=== RUN TestService_verifyS3Integrity685=== PAUSE TestService_verifyS3Integrity686=== RUN TestCompleteMultipartUnregistered687=== PAUSE TestCompleteMultipartUnregistered688=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT689=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT690=== CONT TestProxyWriteTimeout691=== RUN TestProxyWriteTimeout/narinfo692=== PAUSE TestProxyWriteTimeout/narinfo693=== RUN TestProxyWriteTimeout/1_GiB_nar694=== PAUSE TestProxyWriteTimeout/1_GiB_nar695=== RUN TestProxyWriteTimeout/10_GiB_nar696=== PAUSE TestProxyWriteTimeout/10_GiB_nar697=== RUN TestProxyWriteTimeout/unknown_size698=== PAUSE TestProxyWriteTimeout/unknown_size699=== CONT TestService_AuthMiddleware700=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle701=== CONT TestSkippedUploadsHandler702=== CONT TestGCTaskStore_PhaseUpdates703--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)704=== CONT TestPush_RejectsBadRequests705=== CONT TestGCTaskStore_CompletedAllowsNewTask706--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)707=== CONT TestGCTaskStore_GetReturnsLatest708--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)709=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit710=== CONT TestClientPushesUseOnePush711=== CONT TestGCTaskStore_GetEmpty712=== CONT TestGCTaskStore_ConflictDifferentParams713=== CONT TestGCTaskStore_DeduplicateSameParams714=== CONT TestGCTaskStore_Fail7152026/10/05 11:42:51 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000716=== CONT TestGCTaskStore_StartNew717=== CONT TestParseSize718=== CONT TestGCEndsOnShutdown719=== CONT TestService_Rustfstest720=== CONT TestCommitRacingPendingCleanupKeepsObjects721=== CONT TestPresignedUploadRegisteredBeforeCommit722=== CONT TestPushWaitsForSweepHoldingRow723=== CONT TestCompletedNarNotReofferedAcrossClosures724=== CONT TestSweepRechecksPendingAfterLocking725=== CONT TestCompleteMultipartUpload_ErrorButObjectExists726=== CONT TestRedundantMultipartUpload727=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock728=== CONT TestPush_SignsNarinfosOfItsPendingObjects729=== CONT TestClientFallsBackToClosures730--- PASS: TestGCTaskStore_GetEmpty (0.00s)731=== RUN TestGCEndsOnShutdown/before_the_run732--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)733=== CONT TestReadRedirectUsesPublicS3URL734=== CONT TestPinProtectsFromGC735=== PAUSE TestGCEndsOnShutdown/before_the_run736--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)737=== CONT TestPush_OverlappingRootsStoreOneRowPerKey738=== CONT TestClientWithDependencies739=== CONT TestPush_CompleteCommitsEveryRoot740=== CONT TestClientSharedPathCommittedMidPush741--- PASS: TestGCTaskStore_Fail (0.00s)742--- PASS: TestParseSize (0.00s)743--- PASS: TestGCTaskStore_StartNew (0.00s)744=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete745=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete746=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark747=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark748=== CONT TestResolveDBConnectionString749=== RUN TestResolveDBConnectionString/flag_wins750=== PAUSE TestResolveDBConnectionString/flag_wins751=== RUN TestResolveDBConnectionString/file_when_flag_empty752=== PAUSE TestResolveDBConnectionString/file_when_flag_empty753=== RUN TestResolveDBConnectionString/missing_file_is_an_error754=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error755=== RUN TestResolveDBConnectionString/PGHOST_allows_empty756=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty757=== RUN TestResolveDBConnectionString/nothing_configured758=== PAUSE TestResolveDBConnectionString/nothing_configured759=== CONT TestGetExistingObjectsLocksRows760--- PASS: TestSkippedUploadsHandler (0.01s)761=== CONT TestClientMultipleUploads762=== RUN TestPush_RejectsBadRequests/no_roots763=== PAUSE TestPush_RejectsBadRequests/no_roots764=== RUN TestPush_RejectsBadRequests/no_objects765=== PAUSE TestPush_RejectsBadRequests/no_objects766=== RUN TestPush_RejectsBadRequests/bad_root767=== PAUSE TestPush_RejectsBadRequests/bad_root768=== RUN TestPush_RejectsBadRequests/root_not_in_objects769=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects770=== CONT TestRegisterCompletedObjectNeedsPendingRow7712026/10/05 11:42:51 INFO Received uploads request method=POST path=/api/pending_closures7722026/10/05 11:42:51 WARN Authentication failed token_preview=wrongtoken token_length=10 reason="static API token mismatch"773--- PASS: TestService_AuthMiddleware (0.38s)774=== CONT TestSweepSkipsRowsLockedByPush775--- PASS: TestService_Rustfstest (0.39s)776=== CONT TestFailedS3DeleteKeepsTombstone7772026/10/05 11:42:51 INFO Received uploads request method=POST path=/api/pending_closures7782026/10/05 11:42:51 INFO Received push request method=POST path=/api/pushes7792026/10/05 11:42:51 INFO Received complete multipart upload request method=POST path=/api/multipart/complete7802026/10/05 11:42:51 INFO Received complete multipart upload request method=POST path=/api/multipart/complete7812026/10/05 11:42:51 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=ZTVmOTI0ODUtZTJiNy00MGUzLThkYjUtYmY2ZGJhMTljMThiLjIzNGIyNmJlLTY4YmUtNGNhOS1iYzFjLTE1MDRjOTk0YzFmZXgxNzkxMjAwNTcxNTQ5NjA1MzQ37822026/10/05 11:42:51 INFO Received complete push request method=POST path=/api/pushes/1/complete7832026/10/05 11:42:51 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=ZTVmOTI0ODUtZTJiNy00MGUzLThkYjUtYmY2ZGJhMTljMThiLjIzNGIyNmJlLTY4YmUtNGNhOS1iYzFjLTE1MDRjOTk0YzFmZXgxNzkxMjAwNTcxNTQ5NjA1MzQ3 parts=1784--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (0.46s)785=== CONT TestClientIntegration7862026/10/05 11:42:51 INFO Received push request method=POST path=/api/pushes7872026/10/05 11:42:51 INFO Received push request method=POST path=/api/pushes7882026/10/05 11:42:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign7892026/10/05 11:42:51 INFO Signed narinfos id=1 count=1790--- PASS: TestPush_SignsNarinfosOfItsPendingObjects (0.49s)791=== CONT TestForceGCDuringPushOffersSweptObject792=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows793=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows794=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check795=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check796=== CONT TestClientErrorHandling797=== RUN TestClientErrorHandling/InvalidStorePath798=== PAUSE TestClientErrorHandling/InvalidStorePath799=== RUN TestClientErrorHandling/InvalidAuthToken800=== PAUSE TestClientErrorHandling/InvalidAuthToken801=== RUN TestClientErrorHandling/ServerNotAvailable802=== PAUSE TestClientErrorHandling/ServerNotAvailable803=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection8042026/10/05 11:42:51 INFO Received uploads request method=POST path=/api/pending_closures8052026/10/05 11:42:51 INFO Aborted multipart uploads count=0 kept=08062026/10/05 11:42:51 WARN Force mode enabled - objects will be deleted immediately without grace period8072026/10/05 11:42:51 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=08082026/10/05 11:42:51 INFO Vacuumed table table=pending_closures8092026/10/05 11:42:51 INFO Vacuumed table table=pending_objects8102026/10/05 11:42:51 INFO Vacuumed table table=multipart_uploads8112026/10/05 11:42:51 INFO Vacuumed table table=closures8122026/10/05 11:42:51 INFO Vacuumed table table=objects8132026/10/05 11:42:51 INFO Received complete push request method=POST path=/api/pushes/2/complete814--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (0.55s)815=== CONT TestClientCADerivations8162026/10/05 11:42:51 INFO Aborted multipart uploads count=0 kept=08172026/10/05 11:42:51 WARN Force mode enabled - objects will be deleted immediately without grace period8182026/10/05 11:42:51 INFO Aborted multipart uploads count=0 kept=08192026/10/05 11:42:51 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=08202026/10/05 11:42:51 WARN Force mode enabled - objects will be deleted immediately without grace period8212026/10/05 11:42:51 INFO Vacuumed table table=pending_closures8222026/10/05 11:42:51 INFO Received uploads request method=POST path=/api/pending_closures8232026/10/05 11:42:51 INFO Vacuumed table table=pending_objects8242026/10/05 11:42:51 INFO Vacuumed table table=multipart_uploads8252026/10/05 11:42:51 INFO Vacuumed table table=closures8262026/10/05 11:42:51 INFO Vacuumed table table=objects8272026/10/05 11:42:51 INFO Received uploads request method=POST path=/api/pending_closures8282026/10/05 11:42:51 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=0829--- PASS: TestCommitRacingPendingCleanupKeepsObjects (0.57s)830=== CONT TestGCSweepDeliversEachKeyOnce8312026/10/05 11:42:51 INFO Vacuumed table table=pending_closures8322026/10/05 11:42:51 INFO Vacuumed table table=pending_objects8332026/10/05 11:42:51 INFO Vacuumed table table=multipart_uploads8342026/10/05 11:42:51 INFO Vacuumed table table=closures8352026/10/05 11:42:51 INFO Vacuumed table table=objects836--- PASS: TestSweepRechecksPendingAfterLocking (0.58s)837=== CONT TestReadProxyRangeRequest8382026/10/05 11:42:51 INFO Received uploads request method=POST path=/api/pending_closures8392026/10/05 11:42:51 INFO Received push request method=POST path=/api/pushes8402026/10/05 11:42:51 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)8412026/10/05 11:42:51 INFO Uploading 4dyyvyfi56az6a6dmw4s1c98xfm6f8ln-shared-dep (136B)8422026/10/05 11:42:51 INFO Uploading 1c38g3np84zj357fnp9byz626kgsr7xr-b (208B)8432026/10/05 11:42:51 WARN Failed to register uploaded object key=4dyyvyfi56az6a6dmw4s1c98xfm6f8ln.ls error="server returned 404: 404 page not found\n"8442026/10/05 11:42:51 WARN Failed to register uploaded object key=k5ysbsiljj2h6ay51iy384ckizymwymq.ls error="server returned 404: 404 page not found\n"8452026/10/05 11:42:51 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"8462026/10/05 11:42:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign8472026/10/05 11:42:51 WARN Failed to register uploaded object key=1c38g3np84zj357fnp9byz626kgsr7xr.ls error="server returned 404: 404 page not found\n"8482026/10/05 11:42:51 WARN Failed to register uploaded object key=nar/0xqlwah0mirg22a5zv8sqfzndn7w2qrbvzq0j7h30nc349694w2y.nar.zst error="server returned 404: 404 page not found\n"8492026/10/05 11:42:51 INFO Signed narinfos id=1 count=38502026/10/05 11:42:51 INFO Uploading 3 narinfos8512026/10/05 11:42:51 WARN Failed to register uploaded object key=k5ysbsiljj2h6ay51iy384ckizymwymq.narinfo error="server returned 404: 404 page not found\n"8522026/10/05 11:42:51 INFO Received complete push request method=POST path=/api/pushes/1/complete8532026/10/05 11:42:51 WARN Failed to register uploaded object key=4dyyvyfi56az6a6dmw4s1c98xfm6f8ln.narinfo error="server returned 404: 404 page not found\n"8542026/10/05 11:42:51 WARN Failed to register uploaded object key=1c38g3np84zj357fnp9byz626kgsr7xr.narinfo error="server returned 404: 404 page not found\n"8552026/10/05 11:42:51 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst8562026/10/05 11:42:51 INFO Received uploads request method=POST path=/api/pending_closures857--- PASS: TestPresignedUploadRegisteredBeforeCommit (0.66s)858=== CONT TestTombstonedObjectOfferedWithoutWaiting8592026/10/05 11:42:51 INFO Upload complete. (64ms)860=== NAME TestClientPushesUseOnePush861 client_pushes_test.go:97: Retrieved narinfo from S3:862 StorePath: /build/TestClientPushesUseOnePush783938130/001/store/4dyyvyfi56az6a6dmw4s1c98xfm6f8ln-shared-dep863 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst864 Compression: zstd865 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82866 NarSize: 136867 References: 868 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n869 client_pushes_test.go:97: Retrieved narinfo from S3:870 StorePath: /build/TestClientPushesUseOnePush783938130/001/store/k5ysbsiljj2h6ay51iy384ckizymwymq-a871 URL: nar/0xqlwah0mirg22a5zv8sqfzndn7w2qrbvzq0j7h30nc349694w2y.nar.zst872 Compression: zstd873 NarHash: sha256:0xqlwah0mirg22a5zv8sqfzndn7w2qrbvzq0j7h30nc349694w2y874 NarSize: 208875 References: /build/TestClientPushesUseOnePush783938130/001/store/4dyyvyfi56az6a6dmw4s1c98xfm6f8ln-shared-dep876 CA: text:sha256:1b44c1g052gkms1nnv3agz6cvra4agpd9ykij2xf2ilw9g2h04pv877 client_pushes_test.go:97: Retrieved narinfo from S3:878 StorePath: /build/TestClientPushesUseOnePush783938130/001/store/1c38g3np84zj357fnp9byz626kgsr7xr-b879 URL: nar/0xqlwah0mirg22a5zv8sqfzndn7w2qrbvzq0j7h30nc349694w2y.nar.zst880 Compression: zstd881 NarHash: sha256:0xqlwah0mirg22a5zv8sqfzndn7w2qrbvzq0j7h30nc349694w2y882 NarSize: 208883 References: /build/TestClientPushesUseOnePush783938130/001/store/4dyyvyfi56az6a6dmw4s1c98xfm6f8ln-shared-dep884 CA: text:sha256:1b44c1g052gkms1nnv3agz6cvra4agpd9ykij2xf2ilw9g2h04pv885--- PASS: TestClientPushesUseOnePush (0.68s)886=== CONT TestCacheStatsHandler8872026/10/05 11:42:51 INFO Received push request method=POST path=/api/pushes888=== NAME TestPinProtectsFromGC889 client_integration_test.go:731: Pinned store path: /build/TestPinProtectsFromGC975776449/001/store/x709wf9whsqjxaaydjcdmcv65flx4d1s-pinned-file.txt890 client_integration_test.go:732: Unpinned store path: /build/TestPinProtectsFromGC975776449/001/store/xazhs9xj8j0gi5qbrw2axrjd5vad9n4j-unpinned-file.txt891--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (0.70s)892=== CONT TestGCSweepSkipsPendingObjects893--- PASS: TestGetExistingObjectsLocksRows (0.72s)894=== CONT TestCacheConfigHandler895=== RUN TestCacheConfigHandler/full_config,_no_issuer896=== PAUSE TestCacheConfigHandler/full_config,_no_issuer897=== RUN TestCacheConfigHandler/no_cache_url_configured898=== PAUSE TestCacheConfigHandler/no_cache_url_configured899=== RUN TestCacheConfigHandler/no_signing_keys900=== PAUSE TestCacheConfigHandler/no_signing_keys901=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator902=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator903=== CONT TestReadRedirectKeepsNarinfoProxied9042026/10/05 11:42:51 INFO Received uploads request method=POST path=/api/pending_closures9052026/10/05 11:42:51 INFO Received uploads request method=POST path=/api/pending_closures9062026/10/05 11:42:51 INFO Received push request method=POST path=/api/pushes9072026/10/05 11:42:51 INFO Received uploads request method=POST path=/api/pending_closures9082026/10/05 11:42:51 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)9092026/10/05 11:42:51 INFO Uploading x709wf9whsqjxaaydjcdmcv65flx4d1s-pinned-file.txt (128B)9102026/10/05 11:42:51 INFO Received push request method=POST path=/api/pushes9112026/10/05 11:42:51 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)9122026/10/05 11:42:51 INFO Uploading 6xvcrb68n2ff81cl5p3q61xbjxlg6n0p-b (216B)9132026/10/05 11:42:51 INFO Uploading 7a1gqczmq67va181mrdp7xxvvwzpgwqb-shared-dep (136B)9142026/10/05 11:42:51 WARN Failed to register uploaded object key=x709wf9whsqjxaaydjcdmcv65flx4d1s.ls error="server returned 404: 404 page not found\n"9152026/10/05 11:42:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9162026/10/05 11:42:51 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"9172026/10/05 11:42:51 INFO Signed narinfos id=1 count=19182026/10/05 11:42:51 INFO Uploading 1 narinfos9192026/10/05 11:42:51 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"9202026/10/05 11:42:51 WARN Failed to register uploaded object key=26kr4hrikyllqrc9118qsfb9nv57v8hx.ls error="server returned 404: 404 page not found\n"9212026/10/05 11:42:51 WARN Failed to register uploaded object key=7a1gqczmq67va181mrdp7xxvvwzpgwqb.ls error="server returned 404: 404 page not found\n"9222026/10/05 11:42:51 WARN Failed to register uploaded object key=6xvcrb68n2ff81cl5p3q61xbjxlg6n0p.ls error="server returned 404: 404 page not found\n"9232026/10/05 11:42:51 WARN Failed to register uploaded object key=nar/0mqm2agadyadlgv6l6l8vji05nj1yb7wvh0klfhh96shsjz95645.nar.zst error="server returned 404: 404 page not found\n"9242026/10/05 11:42:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign9252026/10/05 11:42:51 INFO Signed narinfos id=2 count=29262026/10/05 11:42:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign9272026/10/05 11:42:51 INFO Signed narinfos id=1 count=29282026/10/05 11:42:51 INFO Uploading 4 narinfos9292026/10/05 11:42:51 INFO Received complete push request method=POST path=/api/pushes/1/complete9302026/10/05 11:42:51 WARN Failed to register uploaded object key=x709wf9whsqjxaaydjcdmcv65flx4d1s.narinfo error="server returned 404: 404 page not found\n"9312026/10/05 11:42:51 WARN Failed to register uploaded object key=6xvcrb68n2ff81cl5p3q61xbjxlg6n0p.narinfo error="server returned 404: 404 page not found\n"9322026/10/05 11:42:51 WARN Failed to register uploaded object key=7a1gqczmq67va181mrdp7xxvvwzpgwqb.narinfo error="server returned 404: 404 page not found\n"9332026/10/05 11:42:51 WARN Failed to register uploaded object key=26kr4hrikyllqrc9118qsfb9nv57v8hx.narinfo error="server returned 404: 404 page not found\n"9342026/10/05 11:42:51 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete9352026/10/05 11:42:51 WARN Failed to register uploaded object key=7a1gqczmq67va181mrdp7xxvvwzpgwqb.narinfo error="server returned 404: 404 page not found\n"9362026/10/05 11:42:51 INFO Upload complete. (61ms)9372026/10/05 11:42:51 INFO Received complete push request method=POST path=/api/pushes/1/complete9382026/10/05 11:42:51 INFO Completed upload id=19392026/10/05 11:42:51 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete9402026/10/05 11:42:51 INFO Completed upload id=29412026/10/05 11:42:51 INFO Upload complete. (76ms)942=== NAME TestClientFallsBackToClosures943 client_pushes_test.go:112: Retrieved narinfo from S3:944 StorePath: /build/TestClientFallsBackToClosures4141176681/001/store/7a1gqczmq67va181mrdp7xxvvwzpgwqb-shared-dep945 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst946 Compression: zstd947 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82948 NarSize: 136949 References: 950 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n951--- PASS: TestPush_CompleteCommitsEveryRoot (0.80s)952=== CONT TestDeduplicatedObjectsRecordedAsPending953=== NAME TestClientFallsBackToClosures954 client_pushes_test.go:112: Retrieved narinfo from S3:955 StorePath: /build/TestClientFallsBackToClosures4141176681/001/store/26kr4hrikyllqrc9118qsfb9nv57v8hx-a956 URL: nar/0mqm2agadyadlgv6l6l8vji05nj1yb7wvh0klfhh96shsjz95645.nar.zst957 Compression: zstd958 NarHash: sha256:0mqm2agadyadlgv6l6l8vji05nj1yb7wvh0klfhh96shsjz95645959 NarSize: 216960 References: /build/TestClientFallsBackToClosures4141176681/001/store/7a1gqczmq67va181mrdp7xxvvwzpgwqb-shared-dep961 CA: text:sha256:0h65lm2kqnq6cdwfvn54xn9fl2nmd376q5z7j6b8a9nrx2yd168m962 client_pushes_test.go:112: Retrieved narinfo from S3:963 StorePath: /build/TestClientFallsBackToClosures4141176681/001/store/6xvcrb68n2ff81cl5p3q61xbjxlg6n0p-b964 URL: nar/0mqm2agadyadlgv6l6l8vji05nj1yb7wvh0klfhh96shsjz95645.nar.zst965 Compression: zstd966 NarHash: sha256:0mqm2agadyadlgv6l6l8vji05nj1yb7wvh0klfhh96shsjz95645967 NarSize: 216968 References: /build/TestClientFallsBackToClosures4141176681/001/store/7a1gqczmq67va181mrdp7xxvvwzpgwqb-shared-dep969 CA: text:sha256:0h65lm2kqnq6cdwfvn54xn9fl2nmd376q5z7j6b8a9nrx2yd168m9702026/10/05 11:42:51 INFO Received push request method=POST path=/api/pushes971--- PASS: TestClientFallsBackToClosures (0.81s)972=== CONT TestReadRedirectNar9732026/10/05 11:42:51 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)9742026/10/05 11:42:51 INFO Uploading 9d35ifhd703khzcs3887nw69s356fjcq-shared-dep (136B)9752026/10/05 11:42:51 INFO Uploading fhh71h15f5sd0ja7qd8g8dlm85hm0x0z-top (224B)9762026/10/05 11:42:51 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"9772026/10/05 11:42:51 WARN Failed to register uploaded object key=9d35ifhd703khzcs3887nw69s356fjcq.ls error="server returned 404: 404 page not found\n"9782026/10/05 11:42:51 WARN Failed to register uploaded object key=fhh71h15f5sd0ja7qd8g8dlm85hm0x0z.ls error="server returned 404: 404 page not found\n"9792026/10/05 11:42:51 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9802026/10/05 11:42:51 WARN Failed to register uploaded object key=nar/133dcb3d3n4ygl11qfhrpky13yivcxdv2prx065kjh0cbfzhq5yf.nar.zst error="server returned 404: 404 page not found\n"9812026/10/05 11:42:51 INFO Signed narinfos id=1 count=29822026/10/05 11:42:51 INFO Uploading 2 narinfos9832026/10/05 11:42:51 WARN Failed to register uploaded object key=9d35ifhd703khzcs3887nw69s356fjcq.narinfo error="server returned 404: 404 page not found\n"9842026/10/05 11:42:51 INFO Received complete push request method=POST path=/api/pushes/1/complete9852026/10/05 11:42:51 WARN Failed to register uploaded object key=fhh71h15f5sd0ja7qd8g8dlm85hm0x0z.narinfo error="server returned 404: 404 page not found\n"9862026/10/05 11:42:51 INFO Upload complete. (63ms)987=== NAME TestClientSharedPathCommittedMidPush988 client_integration_test.go:680: Retrieved narinfo from S3:989 StorePath: /build/TestClientSharedPathCommittedMidPush2087518458/001/store/9d35ifhd703khzcs3887nw69s356fjcq-shared-dep990 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst991 Compression: zstd992 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82993 NarSize: 136994 References: 995 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n996=== NAME TestClientWithDependencies997 client_integration_test.go:613: Built derivation: /build/TestClientWithDependencies3273703214/001/store/c8jdgr6qkfp7pz9mg6g68c0jqvl7cyzd-test-script998=== NAME TestClientMultipleUploads999 client_integration_test.go:358: Created store path 0: /build/TestClientMultipleUploads3003908281/001/store/xfvw99hm2pmbizs41hkshdyijxw8ni3q-test-file-0.txt1000=== NAME TestClientSharedPathCommittedMidPush1001 client_integration_test.go:680: Retrieved narinfo from S3:1002 StorePath: /build/TestClientSharedPathCommittedMidPush2087518458/001/store/fhh71h15f5sd0ja7qd8g8dlm85hm0x0z-top1003 URL: nar/133dcb3d3n4ygl11qfhrpky13yivcxdv2prx065kjh0cbfzhq5yf.nar.zst1004 Compression: zstd1005 NarHash: sha256:133dcb3d3n4ygl11qfhrpky13yivcxdv2prx065kjh0cbfzhq5yf1006 NarSize: 2241007 References: /build/TestClientSharedPathCommittedMidPush2087518458/001/store/9d35ifhd703khzcs3887nw69s356fjcq-shared-dep1008 CA: text:sha256:19nxplgiadwmwg6nba7x2mjvz6q1baaq0vy6xjxaxa9cagc10fsd1009--- PASS: TestReadRedirectUsesPublicS3URL (0.84s)1010=== CONT TestPushDedupSurvivesConcurrentGC1011--- PASS: TestClientSharedPathCommittedMidPush (0.84s)1012=== CONT TestService_ReadScope_PublicByDefault1013--- PASS: TestRegisterCompletedObjectNeedsPendingRow (0.51s)1014=== CONT TestReadProxyDisabled10152026/10/05 11:42:51 INFO Received push request method=POST path=/api/pushes1016=== NAME TestClientWithDependencies1017 client_integration_test.go:615: Found 1 dependencies (including self)1018=== NAME TestClientMultipleUploads1019 client_integration_test.go:358: Created store path 1: /build/TestClientMultipleUploads3003908281/001/store/zn2anp4znafcmxqbmmdi9jf823zmw2ib-test-file-1.txt10202026/10/05 11:42:52 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)10212026/10/05 11:42:52 INFO Uploading xazhs9xj8j0gi5qbrw2axrjd5vad9n4j-unpinned-file.txt (128B)10222026/10/05 11:42:52 WARN Failed to register uploaded object key=xazhs9xj8j0gi5qbrw2axrjd5vad9n4j.ls error="server returned 404: 404 page not found\n"10232026/10/05 11:42:52 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"10242026/10/05 11:42:52 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign10252026/10/05 11:42:52 INFO Signed narinfos id=2 count=110262026/10/05 11:42:52 INFO Uploading 1 narinfos10272026/10/05 11:42:52 INFO Received complete push request method=POST path=/api/pushes/2/complete10282026/10/05 11:42:52 WARN Failed to register uploaded object key=xazhs9xj8j0gi5qbrw2axrjd5vad9n4j.narinfo error="server returned 404: 404 page not found\n"10292026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures10302026/10/05 11:42:52 INFO Upload complete. (66ms)10312026/10/05 11:42:52 INFO Aborted multipart uploads count=0 kept=010322026/10/05 11:42:52 WARN Force mode enabled - objects will be deleted immediately without grace period10332026/10/05 11:42:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=01034 client_integration_test.go:358: Created store path 2: /build/TestClientMultipleUploads3003908281/001/store/qrzd83wpp0ry7nqsh8sgi87kn2acpda0-test-file-2.txt10352026/10/05 11:42:52 INFO Vacuumed table table=pending_closures10362026/10/05 11:42:52 INFO Vacuumed table table=pending_objects10372026/10/05 11:42:52 INFO Vacuumed table table=multipart_uploads10382026/10/05 11:42:52 INFO Vacuumed table table=closures10392026/10/05 11:42:52 INFO Vacuumed table table=objects10402026/10/05 11:42:52 INFO Aborted multipart uploads count=0 kept=010412026/10/05 11:42:52 WARN Force mode enabled - objects will be deleted immediately without grace period10422026/10/05 11:42:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=010432026/10/05 11:42:52 INFO Vacuumed table table=pending_closures10442026/10/05 11:42:52 INFO Vacuumed table table=pending_objects10452026/10/05 11:42:52 INFO Vacuumed table table=multipart_uploads10462026/10/05 11:42:52 INFO Vacuumed table table=closures10472026/10/05 11:42:52 INFO Starting cleanup of old closures method=DELETE path=/api/closures10482026/10/05 11:42:52 INFO Garbage collection started10492026/10/05 11:42:52 INFO Vacuumed table table=objects1050--- PASS: TestSweepSkipsRowsLockedByPush (0.56s)1051=== CONT TestGCMetrics10522026/10/05 11:42:52 INFO Received push request method=POST path=/api/pushes10532026/10/05 11:42:52 INFO Aborted multipart uploads count=0 kept=010542026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures10552026/10/05 11:42:52 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)10562026/10/05 11:42:52 INFO Uploading c8jdgr6qkfp7pz9mg6g68c0jqvl7cyzd-test-script (136B)10572026/10/05 11:42:52 WARN Force mode enabled - objects will be deleted immediately without grace period10582026/10/05 11:42:52 WARN Failed to register uploaded object key=c8jdgr6qkfp7pz9mg6g68c0jqvl7cyzd.ls error="server returned 404: 404 page not found\n"10592026/10/05 11:42:52 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"10602026/10/05 11:42:52 WARN Failed to register uploaded object key=log/nfgb20fip2sysmls1nyfv90klaas1i9r-test-script.drv error="server returned 404: 404 page not found\n"10612026/10/05 11:42:52 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign10622026/10/05 11:42:52 INFO Signed narinfos id=1 count=110632026/10/05 11:42:52 INFO Uploading 1 narinfos10642026/10/05 11:42:52 INFO Received complete push request method=POST path=/api/pushes/1/complete10652026/10/05 11:42:52 WARN Failed to register uploaded object key=c8jdgr6qkfp7pz9mg6g68c0jqvl7cyzd.narinfo error="server returned 404: 404 page not found\n"10662026/10/05 11:42:52 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1067=== NAME TestClientIntegration1068 client_integration_test.go:286: Created store path: /build/TestClientIntegration90471015/002/store/mmgk4rj44xg0x9s951vvald7b4b262sf-test-file.txt10692026/10/05 11:42:52 INFO Upload complete. (55ms)10702026/10/05 11:42:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=01071=== NAME TestClientWithDependencies1072 client_integration_test.go:617: Skipping nix copy test - isolated store (/build/TestClientWithDependencies3273703214/001/store) requires matching store prefix10732026/10/05 11:42:52 INFO Vacuumed table table=pending_closures1074--- PASS: TestClientWithDependencies (0.96s)1075=== CONT TestService_RequireScope_OIDC10762026/10/05 11:42:52 INFO Vacuumed table table=pending_objects10772026/10/05 11:42:52 INFO Vacuumed table table=multipart_uploads10782026/10/05 11:42:52 INFO Aborted multipart uploads count=0 kept=010792026/10/05 11:42:52 INFO Vacuumed table table=closures10802026/10/05 11:42:52 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=ZTVmOTI0ODUtZTJiNy00MGUzLThkYjUtYmY2ZGJhMTljMThiLmMwNDEyMDA3LTFiODItNDE4MC1iNmMxLWYzMjRkZWQwYzAxM3gxNzkxMjAwNTcxNjk2NDgzMTAx parts=1210812026/10/05 11:42:52 INFO Vacuumed table table=objects10822026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures10832026/10/05 11:42:52 WARN Force mode enabled - objects will be deleted immediately without grace period10842026/10/05 11:42:52 INFO Received push request method=POST path=/api/pushes10852026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures10862026/10/05 11:42:52 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)10872026/10/05 11:42:52 INFO Uploading zn2anp4znafcmxqbmmdi9jf823zmw2ib-test-file-1.txt (160B)10882026/10/05 11:42:52 INFO Uploading qrzd83wpp0ry7nqsh8sgi87kn2acpda0-test-file-2.txt (160B)10892026/10/05 11:42:52 INFO Uploading xfvw99hm2pmbizs41hkshdyijxw8ni3q-test-file-0.txt (160B)10902026/10/05 11:42:52 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."10912026/10/05 11:42:52 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."1092--- PASS: TestReadProxyRangeRequest (0.41s)1093=== CONT TestReadProxyRootRedirectsToIndexHTML10942026/10/05 11:42:52 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"10952026/10/05 11:42:52 WARN Failed to register uploaded object key=zn2anp4znafcmxqbmmdi9jf823zmw2ib.ls error="server returned 404: 404 page not found\n"10962026/10/05 11:42:52 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"10972026/10/05 11:42:52 WARN Failed to register uploaded object key=xfvw99hm2pmbizs41hkshdyijxw8ni3q.ls error="server returned 404: 404 page not found\n"10982026/10/05 11:42:52 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign10992026/10/05 11:42:52 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"11002026/10/05 11:42:52 WARN Failed to register uploaded object key=qrzd83wpp0ry7nqsh8sgi87kn2acpda0.ls error="server returned 404: 404 page not found\n"1101--- PASS: TestFailedS3DeleteKeepsTombstone (0.60s)11022026/10/05 11:42:52 INFO Signed narinfos id=1 count=31103=== CONT TestGCBugBareHashReferences11042026/10/05 11:42:52 INFO Uploading 3 narinfos11052026/10/05 11:42:52 WARN Failed to register uploaded object key=qrzd83wpp0ry7nqsh8sgi87kn2acpda0.narinfo error="server returned 404: 404 page not found\n"11062026/10/05 11:42:52 WARN Failed to register uploaded object key=zn2anp4znafcmxqbmmdi9jf823zmw2ib.narinfo error="server returned 404: 404 page not found\n"11072026/10/05 11:42:52 INFO Received complete push request method=POST path=/api/pushes/1/complete11082026/10/05 11:42:52 WARN Failed to register uploaded object key=xfvw99hm2pmbizs41hkshdyijxw8ni3q.narinfo error="server returned 404: 404 page not found\n"1109--- PASS: TestPushWaitsForSweepHoldingRow (1.00s)1110=== CONT TestReadProxyConditionalGet11112026/10/05 11:42:52 INFO Upload complete. (70ms)1112=== NAME TestClientMultipleUploads1113 client_integration_test.go:369: Uploaded 3 paths in 103.986291ms1114--- PASS: TestClientMultipleUploads (1.02s)1115=== CONT TestGCAdvisoryLockBlocksConcurrentRun1116--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (1.04s)1117=== CONT TestService_AuthMiddleware_OIDC11182026/10/05 11:42:52 INFO Aborted multipart uploads count=0 kept=011192026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures11202026/10/05 11:42:52 INFO Received push request method=POST path=/api/pushes11212026/10/05 11:42:52 WARN Force mode enabled - objects will be deleted immediately without grace period11222026/10/05 11:42:52 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)11232026/10/05 11:42:52 INFO Uploading mmgk4rj44xg0x9s951vvald7b4b262sf-test-file.txt (152B)1124--- PASS: TestTombstonedObjectOfferedWithoutWaiting (0.39s)1125=== CONT TestReadProxyOutlastsServerWriteTimeout11262026/10/05 11:42:52 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"11272026/10/05 11:42:52 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign11282026/10/05 11:42:52 WARN Failed to register uploaded object key=mmgk4rj44xg0x9s951vvald7b4b262sf.ls error="server returned 404: 404 page not found\n"11292026/10/05 11:42:52 INFO Signed narinfos id=1 count=111302026/10/05 11:42:52 INFO Uploading 1 narinfos1131=== NAME TestClientCADerivations1132 client_ca_test.go:136: Built CA derivation: /build/TestClientCADerivations2202518138/001/store/sb5g8vxllyxzdh6cd8gcsdz6v3b8sa8b-ca-test11332026/10/05 11:42:52 INFO Received complete push request method=POST path=/api/pushes/1/complete11342026/10/05 11:42:52 WARN Failed to register uploaded object key=mmgk4rj44xg0x9s951vvald7b4b262sf.narinfo error="server returned 404: 404 page not found\n"11352026/10/05 11:42:52 INFO Upload complete. (74ms)11362026/10/05 11:42:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=011372026/10/05 11:42:52 INFO Vacuumed table table=pending_closures11382026/10/05 11:42:52 INFO Vacuumed table table=pending_objects11392026/10/05 11:42:52 INFO Vacuumed table table=multipart_uploads11402026/10/05 11:42:52 INFO Vacuumed table table=closures11412026/10/05 11:42:52 INFO Vacuumed table table=objects1142--- PASS: TestGCSweepDeliversEachKeyOnce (0.51s)1143=== CONT TestLeadEndsOnShutdown1144--- PASS: TestCacheStatsHandler (0.41s)1145=== CONT TestService_ReadAuthMiddleware1146=== NAME TestClientCADerivations1147 client_ca_test.go:139: Found 1 dependencies (including self)11482026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures11492026/10/05 11:42:52 INFO All 1 paths already cached1150=== NAME TestClientIntegration1151 client_integration_test.go:312: Retrieved narinfo from S3:1152 StorePath: /build/TestClientIntegration90471015/002/store/mmgk4rj44xg0x9s951vvald7b4b262sf-test-file.txt1153 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1154 Compression: zstd1155 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11156 NarSize: 1521157 References: 1158 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk111592026/10/05 11:42:52 INFO Aborted multipart uploads count=0 kept=01160 client_integration_test.go:313: Retrieved .ls file from S3 (compressed size: 77 bytes)1161 client_integration_test.go:313: Decompressed .ls content (64 bytes):1162 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}11632026/10/05 11:42:52 WARN Force mode enabled - objects will be deleted immediately without grace period1164 client_integration_test.go:316: Testing garbage collection...11652026/10/05 11:42:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=01166--- PASS: TestReadRedirectKeepsNarinfoProxied (0.41s)1167=== CONT TestReadProxyHead11682026/10/05 11:42:52 INFO Vacuumed table table=pending_closures11692026/10/05 11:42:52 INFO Vacuumed table table=pending_objects11702026/10/05 11:42:52 INFO Vacuumed table table=multipart_uploads11712026/10/05 11:42:52 INFO Vacuumed table table=closures11722026/10/05 11:42:52 INFO Vacuumed table table=objects11732026/10/05 11:42:52 INFO Received complete multipart upload request method=POST path=/api/multipart/complete11742026/10/05 11:42:52 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:37181/oidc11752026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures1176--- PASS: TestGCSweepSkipsPendingObjects (0.45s)1177=== CONT TestLeadIncumbentAnnouncesAtOnce11782026/10/05 11:42:52 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=ZTVmOTI0ODUtZTJiNy00MGUzLThkYjUtYmY2ZGJhMTljMThiLmE3N2RkYzI5LTRiMWUtNDg3Mi05MGJkLTFmZGNkMjFjMDAyOXgxNzkxMjAwNTcxODY2NDU5Mzgy parts=1211792026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures1180--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (0.66s)1181=== CONT TestService_AuthMiddleware_MTLSBoundSubjects1182--- PASS: TestDeduplicatedObjectsRecordedAsPending (0.35s)1183=== CONT TestLeadElectsOneAndHandsOver11842026/10/05 11:42:52 INFO Starting cleanup of old closures method=DELETE path=/api/closures11852026/10/05 11:42:52 INFO Garbage collection started1186--- PASS: TestCompletedNarNotReofferedAcrossClosures (1.15s)1187=== CONT TestReadProxyInvalidPath11882026/10/05 11:42:52 INFO Aborted multipart uploads count=0 kept=011892026/10/05 11:42:52 WARN Force mode enabled - objects will be deleted immediately without grace period1190--- PASS: TestReadRedirectNar (0.36s)1191=== CONT TestService_cleanupPendingClosuresHandler11922026/10/05 11:42:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=011932026/10/05 11:42:52 INFO Vacuumed table table=pending_closures11942026/10/05 11:42:52 INFO Vacuumed table table=pending_objects11952026/10/05 11:42:52 INFO Vacuumed table table=multipart_uploads11962026/10/05 11:42:52 INFO Vacuumed table table=closures11972026/10/05 11:42:52 INFO Received push request method=POST path=/api/pushes11982026/10/05 11:42:52 INFO Vacuumed table table=objects11992026/10/05 11:42:52 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)12002026/10/05 11:42:52 INFO Uploading sb5g8vxllyxzdh6cd8gcsdz6v3b8sa8b-ca-test (144B)12012026/10/05 11:42:52 WARN Failed to register uploaded object key=nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst error="server returned 404: 404 page not found\n"12022026/10/05 11:42:52 WARN Failed to register uploaded object key=log/2q9wi0zi2mqrsdjicqr4cgpf2g34jx0p-ca-test.drv error="server returned 404: 404 page not found\n"12032026/10/05 11:42:52 WARN Failed to register uploaded object key=sb5g8vxllyxzdh6cd8gcsdz6v3b8sa8b.ls error="server returned 404: 404 page not found\n"12042026/10/05 11:42:52 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign12052026/10/05 11:42:52 INFO Signed narinfos id=1 count=112062026/10/05 11:42:52 INFO Uploading 1 narinfos12072026/10/05 11:42:52 INFO Received complete push request method=POST path=/api/pushes/1/complete12082026/10/05 11:42:52 WARN Failed to register uploaded object key=sb5g8vxllyxzdh6cd8gcsdz6v3b8sa8b.narinfo error="server returned 404: 404 page not found\n"1209--- PASS: TestReadProxyDisabled (0.37s)1210=== CONT TestReadProxy40412112026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures12122026/10/05 11:42:52 INFO Upload complete. (97ms)1213=== NAME TestClientCADerivations1214 client_ca_test.go:180: Narinfo contains CA field: StorePath: /build/TestClientCADerivations2202518138/001/store/sb5g8vxllyxzdh6cd8gcsdz6v3b8sa8b-ca-test1215 URL: nar/04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n.nar.zst1216 Compression: zstd1217 NarHash: sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1218 NarSize: 1441219 References: 1220 Deriver: /build/TestClientCADerivations2202518138/001/store/2q9wi0zi2mqrsdjicqr4cgpf2g34jx0p-ca-test.drv1221 CA: fixed:r:sha256:04jj7iiy79vhzp5kk3r2751n9x3xa44j98y5bk1ds4jp31kky74n1222 client_ca_test.go:185: Checking for realisation files in S3...1223 client_ca_test.go:207: No realisation files uploaded - this is expected for locally built CA derivations1224 client_ca_test.go:208: Locally built CA derivations don't have DrvOutput IDs until they're substituted from a cache1225--- PASS: TestService_ReadScope_PublicByDefault (0.40s)1226=== CONT TestConnectWaitsForAPeerMigration12272026/10/05 11:42:52 INFO Aborted multipart uploads count=0 kept=012282026/10/05 11:42:52 WARN Force mode enabled - objects will be deleted immediately without grace period12292026/10/05 11:42:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012302026/10/05 11:42:52 INFO Vacuumed table table=pending_closures12312026/10/05 11:42:52 INFO Vacuumed table table=pending_objects12322026/10/05 11:42:52 INFO Vacuumed table table=multipart_uploads12332026/10/05 11:42:52 INFO Vacuumed table table=closures12342026/10/05 11:42:52 INFO Vacuumed table table=objects12352026/10/05 11:42:52 INFO Aborted multipart uploads count=0 kept=01236--- PASS: TestGCMetrics (0.36s)1237=== CONT TestReadProxyNarStreaming12382026/10/05 11:42:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012392026/10/05 11:42:52 INFO Vacuumed table table=pending_closures12402026/10/05 11:42:52 INFO Vacuumed table table=pending_objects12412026/10/05 11:42:52 INFO Vacuumed table table=multipart_uploads12422026/10/05 11:42:52 INFO Vacuumed table table=closures12432026/10/05 11:42:52 INFO Vacuumed table table=objects12442026/10/05 11:42:52 INFO Aborted multipart uploads count=0 kept=012452026/10/05 11:42:52 WARN Force mode enabled - objects will be deleted immediately without grace period12462026/10/05 11:42:52 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012472026/10/05 11:42:52 INFO Vacuumed table table=pending_closures1248--- PASS: TestReadProxyRootRedirectsToIndexHTML (0.32s)1249=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT12502026/10/05 11:42:52 INFO Vacuumed table table=pending_objects12512026/10/05 11:42:52 INFO Vacuumed table table=multipart_uploads1252=== NAME TestClientCADerivations1253 client_ca_test.go:258: nix copy output: warning: you don't have Internet access; disabling some network-dependent features1254 warning: failed to create TLS context for AWS credential providers; SSO, STS WebIdentity, and ECS container authentication will be unavailable1255 error: binary cache 's3://bucket32?endpoint=http://localhost:32883®ion=eu-west-1' is for Nix stores with prefix '/nix/store', not '/build/TestClientCADerivations2202518138/001/store'1256 client_ca_test.go:259: nix copy failed (might be expected with isolated stores): exit status 112572026/10/05 11:42:52 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:33509/oidc12582026/10/05 11:42:52 INFO Vacuumed table table=closures12592026/10/05 11:42:52 INFO Vacuumed table table=objects1260--- PASS: TestClientCADerivations (0.78s)1261=== CONT TestCompleteMultipartUnregistered1262--- PASS: TestPushDedupSurvivesConcurrentGC (0.48s)1263=== CONT TestService_verifyS3Integrity1264--- PASS: TestReadProxyConditionalGet (0.34s)1265=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded12662026/10/05 11:42:52 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=nixbld database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1267--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (0.35s)1268=== CONT TestPendingCleanupUsesOneCutoff12692026/10/05 11:42:52 INFO Received complete multipart upload request method=POST path=/api/multipart/complete12702026/10/05 11:42:52 WARN Failed to abort redundant multipart upload, keeping its row object_key=nar/gggggggggggggggggggggggggggggg0200000000000000000000.nar.zst upload_id=ZTVmOTI0ODUtZTJiNy00MGUzLThkYjUtYmY2ZGJhMTljMThiLmI5YWNmZDIzLWVjMGMtNGI5OS05MjY2LTk5YzE4ZjRlMGE2OHgxNzkxMjAwNTcyMTIyMjgzODcy error="Get \"http://127.0.0.1:1/bucket14/?location=\": dial tcp 127.0.0.1:1: connect: connection refused"12712026/10/05 11:42:52 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0200000000000000000000.nar.zst upload_id=ZTVmOTI0ODUtZTJiNy00MGUzLThkYjUtYmY2ZGJhMTljMThiLjk2ZWM5MDJkLTk5ZDktNDE1Mi05YzA4LWEyNmJmMmZhOGM5YngxNzkxMjAwNTcyMTA3OTI0MTIz parts=1212722026/10/05 11:42:52 INFO Received cleanup request method=DELETE path=/api/pending_closures12732026/10/05 11:42:52 INFO Aborted multipart uploads count=1 kept=012742026/10/05 11:42:52 INFO lead: acquired remote=192.0.2.1:12341275--- PASS: TestRedundantMultipartUpload (1.40s)1276=== CONT TestMultipartCleanup1277--- PASS: TestService_ReadAuthMiddleware (0.33s)1278=== CONT TestService_createPendingClosureHandler1279--- PASS: TestReadProxyHead (0.31s)1280=== CONT TestReadProxyNarinfoAlreadyDecompressed12812026/10/05 11:42:52 INFO lead: acquired remote=192.0.2.1:12341282=== RUN TestService_RequireScope_OIDC/builder_may_write1283=== PAUSE TestService_RequireScope_OIDC/builder_may_write1284=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1285=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1286=== RUN TestService_RequireScope_OIDC/ops_may_admin1287=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1288=== RUN TestService_RequireScope_OIDC/ops_may_not_write1289=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1290=== RUN TestService_RequireScope_OIDC/reader_may_not_write1291=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1292=== RUN TestService_RequireScope_OIDC/static_token_may_admin1293=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1294=== RUN TestService_RequireScope_OIDC/static_token_may_write1295=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1296=== RUN TestService_RequireScope_OIDC/reader_may_read1297=== PAUSE TestService_RequireScope_OIDC/reader_may_read1298=== RUN TestService_RequireScope_OIDC/writer_implies_read1299=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1300=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1301=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1302=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1303=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1304=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1305=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1306=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1307=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1308=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1309=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1310=== CONT TestServerTLSConfig1311=== RUN TestServerTLSConfig/no_client_CA1312=== PAUSE TestServerTLSConfig/no_client_CA1313=== RUN TestServerTLSConfig/missing_CA_file1314=== PAUSE TestServerTLSConfig/missing_CA_file1315=== RUN TestServerTLSConfig/not_a_PEM_file1316=== PAUSE TestServerTLSConfig/not_a_PEM_file1317=== CONT TestPresentReportsOnlyClosureRoots13182026/10/05 11:42:52 INFO lead: acquired remote=192.0.2.1:123413192026/10/05 11:42:52 INFO lead: released remote=192.0.2.1:12341320--- PASS: TestLeadIncumbentAnnouncesAtOnce (0.35s)1321=== CONT TestService_NativeMTLS1322--- PASS: TestReadProxyInvalidPath (0.36s)1323=== CONT TestCreatePin_ReservedPins1324--- PASS: TestGCBugBareHashReferences (0.52s)1325=== CONT TestMetricsInventory13262026/10/05 11:42:52 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted,O=Other"13272026/10/05 11:42:52 WARN mTLS auth: bound subjects configured but subject DN unavailable13282026/10/05 11:42:52 WARN mTLS auth: subject not in bound subjects subject="CN=untrusted"1329--- PASS: TestService_AuthMiddleware_MTLSBoundSubjects (0.38s)1330=== CONT TestService_AuthMiddleware_MTLSProxyHeader13312026/10/05 11:42:52 INFO Received cleanup request method=DELETE path=/api/pending_closures13322026/10/05 11:42:52 INFO Aborted multipart uploads count=0 kept=013332026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures13342026/10/05 11:42:52 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41601/oidc13352026/10/05 11:42:52 INFO Received cleanup request method=DELETE path=/api/pending_closures13362026/10/05 11:42:52 INFO Aborted multipart uploads count=1 kept=013372026/10/05 11:42:52 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete13382026-10-05 11:42:52.705 UTC [1373] ERROR: Closure does not exist: id=113392026-10-05 11:42:52.705 UTC [1373] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 12 at RAISE13402026-10-05 11:42:52.705 UTC [1373] STATEMENT: -- name: CommitPendingClosure :exec1341 SELECT commit_pending_closure($1::bigint)1342 1343--- PASS: TestService_cleanupPendingClosuresHandler (0.40s)1344=== CONT TestPresentNotReportedWhileGCDeletesClosure1345--- PASS: TestReadProxy404 (0.38s)1346=== CONT TestOrphanedObjectsGCStressTest1347--- PASS: TestReadProxyNarStreaming (0.31s)1348=== CONT TestReadProxyNarinfo13492026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures1350--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (0.32s)1351=== CONT TestOrphanedObjectsGC1352=== RUN TestService_AuthMiddleware_OIDC/valid_OIDC_token1353=== PAUSE TestService_AuthMiddleware_OIDC/valid_OIDC_token1354=== RUN TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1355=== PAUSE TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected1356=== RUN TestService_AuthMiddleware_OIDC/malformed_token_rejected1357=== PAUSE TestService_AuthMiddleware_OIDC/malformed_token_rejected1358=== RUN TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1359=== PAUSE TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1360=== CONT TestGenerateLandingPage1361--- PASS: TestGenerateLandingPage (0.00s)1362=== CONT TestObjectStatsTrigger13632026/10/05 11:42:52 INFO Received complete multipart upload request method=POST path=/api/multipart/complete13642026/10/05 11:42:52 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst1365--- PASS: TestCompleteMultipartUnregistered (0.34s)1366=== CONT TestCreatePendingClosureRejectsOversizedNAR13672026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures1368--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.00s)1369=== CONT TestIsValidCachePath1370=== RUN TestIsValidCachePath/narinfo1371=== PAUSE TestIsValidCachePath/narinfo1372=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1373=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1374=== RUN TestIsValidCachePath/nar_zst1375=== PAUSE TestIsValidCachePath/nar_zst1376=== RUN TestIsValidCachePath/nar_xz1377=== PAUSE TestIsValidCachePath/nar_xz1378=== RUN TestIsValidCachePath/nar_bz21379=== PAUSE TestIsValidCachePath/nar_bz21380=== RUN TestIsValidCachePath/nar_uncompressed1381=== PAUSE TestIsValidCachePath/nar_uncompressed1382=== RUN TestIsValidCachePath/ls1383=== PAUSE TestIsValidCachePath/ls1384=== RUN TestIsValidCachePath/log1385=== PAUSE TestIsValidCachePath/log1386=== RUN TestIsValidCachePath/realisation1387=== PAUSE TestIsValidCachePath/realisation1388=== RUN TestIsValidCachePath/nix-cache-info1389=== PAUSE TestIsValidCachePath/nix-cache-info1390=== RUN TestIsValidCachePath/index.html1391=== PAUSE TestIsValidCachePath/index.html1392=== RUN TestIsValidCachePath/traversal_parent1393=== PAUSE TestIsValidCachePath/traversal_parent1394=== RUN TestIsValidCachePath/traversal_in_middle1395=== PAUSE TestIsValidCachePath/traversal_in_middle1396=== RUN TestIsValidCachePath/invalid_char_e1397=== PAUSE TestIsValidCachePath/invalid_char_e1398=== RUN TestIsValidCachePath/invalid_char_u1399=== PAUSE TestIsValidCachePath/invalid_char_u1400=== RUN TestIsValidCachePath/random_path1401=== PAUSE TestIsValidCachePath/random_path1402=== RUN TestIsValidCachePath/empty1403=== PAUSE TestIsValidCachePath/empty1404=== RUN TestIsValidCachePath/leading_slash1405=== PAUSE TestIsValidCachePath/leading_slash1406=== RUN TestIsValidCachePath/wrong_extension1407=== PAUSE TestIsValidCachePath/wrong_extension1408=== RUN TestIsValidCachePath/short_hash1409=== PAUSE TestIsValidCachePath/short_hash1410=== CONT TestService_healthCheckHandler14112026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures14122026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures1413--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (0.41s)1414=== CONT TestProxyHeadersOnlyTrustedOnSocket14152026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures14162026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures14172026/10/05 11:42:52 INFO Received cleanup request method=DELETE path=/api/pending_closures14182026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures14192026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures14202026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures14212026/10/05 11:42:52 INFO Received uploads request method=POST path=/api/pending_closures1422--- PASS: TestReadProxyNarinfoAlreadyDecompressed (0.38s)1423=== CONT TestService_readinessHandler1424--- PASS: TestPresentReportsOnlyClosureRoots (0.37s)1425=== CONT TestParseSingleRange1426=== RUN TestParseSingleRange/none1427=== PAUSE TestParseSingleRange/none1428=== RUN TestParseSingleRange/unknown_unit1429=== PAUSE TestParseSingleRange/unknown_unit1430=== RUN TestParseSingleRange/multi-range_ignored1431=== PAUSE TestParseSingleRange/multi-range_ignored1432=== RUN TestParseSingleRange/malformed_no_dash1433=== PAUSE TestParseSingleRange/malformed_no_dash1434=== RUN TestParseSingleRange/malformed_both_empty1435=== PAUSE TestParseSingleRange/malformed_both_empty1436=== RUN TestParseSingleRange/malformed_end_before_start1437=== PAUSE TestParseSingleRange/malformed_end_before_start1438=== RUN TestParseSingleRange/closed1439=== PAUSE TestParseSingleRange/closed1440=== RUN TestParseSingleRange/open-ended1441=== PAUSE TestParseSingleRange/open-ended1442=== RUN TestParseSingleRange/end_clamped_to_size1443=== PAUSE TestParseSingleRange/end_clamped_to_size1444=== RUN TestParseSingleRange/suffix1445=== PAUSE TestParseSingleRange/suffix1446=== RUN TestParseSingleRange/suffix_exceeds_size1447=== PAUSE TestParseSingleRange/suffix_exceeds_size1448=== RUN TestParseSingleRange/single_byte1449=== PAUSE TestParseSingleRange/single_byte1450=== RUN TestParseSingleRange/start_past_EOF1451=== PAUSE TestParseSingleRange/start_past_EOF1452=== RUN TestParseSingleRange/start_far_past_EOF1453=== PAUSE TestParseSingleRange/start_far_past_EOF1454=== CONT TestNARDeduplicationMetadataUploadBug14552026/10/05 11:42:52 WARN mTLS auth: subject not in bound subjects subject="CN=reader"14562026/10/05 11:42:52 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1457--- PASS: TestService_NativeMTLS (0.36s)1458=== CONT TestGracefulShutdownDrainsInflight14592026/10/05 11:42:52 INFO Starting HTTP server address=127.0.0.1:3950314602026/10/05 11:42:52 INFO Shutdown signal received, draining in-flight requests timeout=10s14612026/10/05 11:42:53 INFO Received cleanup request method=DELETE path=/api/pending_closures14622026/10/05 11:42:53 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket67/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""14632026/10/05 11:42:53 INFO Aborted multipart uploads count=0 kept=114642026/10/05 11:42:53 INFO Received cleanup request method=DELETE path=/api/pending_closures1465--- PASS: TestService_AuthMiddleware_MTLSProxyHeader (0.36s)1466=== CONT TestUploadHandlersRejectInvalidKeys1467=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1468=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1469=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal1470=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal1471=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key1472=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key1473=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key1474=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key1475=== CONT TestUploadHandlersRejectOversizedBody14762026/10/05 11:42:53 INFO Aborted multipart uploads count=1 kept=01477--- PASS: TestMetricsInventory (0.38s)1478=== CONT TestCacheConfigHandlerMaxNarSize1479--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1480=== CONT TestPendingClosureWriteTimeout1481=== RUN TestPendingClosureWriteTimeout/empty1482=== PAUSE TestPendingClosureWriteTimeout/empty1483=== RUN TestPendingClosureWriteTimeout/400_objects1484=== PAUSE TestPendingClosureWriteTimeout/400_objects1485=== RUN TestPendingClosureWriteTimeout/670k_objects1486=== PAUSE TestPendingClosureWriteTimeout/670k_objects1487=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1488=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1489=== CONT TestIsValidUploadKey1490=== RUN TestIsValidUploadKey/narinfo1491=== PAUSE TestIsValidUploadKey/narinfo1492=== RUN TestIsValidUploadKey/nar_zst1493=== PAUSE TestIsValidUploadKey/nar_zst1494=== RUN TestIsValidUploadKey/nar_xz1495=== PAUSE TestIsValidUploadKey/nar_xz1496=== RUN TestIsValidUploadKey/nar_plain1497=== PAUSE TestIsValidUploadKey/nar_plain1498=== RUN TestIsValidUploadKey/listing1499=== PAUSE TestIsValidUploadKey/listing1500=== RUN TestIsValidUploadKey/build_log1501=== PAUSE TestIsValidUploadKey/build_log1502=== RUN TestIsValidUploadKey/build_log_home-manager_file1503=== PAUSE TestIsValidUploadKey/build_log_home-manager_file1504=== RUN TestIsValidUploadKey/build_log_plus_in_name1505=== PAUSE TestIsValidUploadKey/build_log_plus_in_name1506=== RUN TestIsValidUploadKey/build_log_question_mark1507=== PAUSE TestIsValidUploadKey/build_log_question_mark1508=== RUN TestIsValidUploadKey/build_log_equals1509=== PAUSE TestIsValidUploadKey/build_log_equals1510=== RUN TestIsValidUploadKey/realisation1511=== PAUSE TestIsValidUploadKey/realisation1512=== RUN TestIsValidUploadKey/realisation_plus_in_output1513=== PAUSE TestIsValidUploadKey/realisation_plus_in_output1514=== RUN TestIsValidUploadKey/nix-cache-info1515=== PAUSE TestIsValidUploadKey/nix-cache-info1516=== RUN TestIsValidUploadKey/index.html1517=== PAUSE TestIsValidUploadKey/index.html1518=== RUN TestIsValidUploadKey/narinfo_key,_nar_type1519=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type1520=== RUN TestIsValidUploadKey/nar_key,_narinfo_type1521=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type1522=== RUN TestIsValidUploadKey/listing_key,_narinfo_type1523=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type1524=== RUN TestIsValidUploadKey/traversal1525=== PAUSE TestIsValidUploadKey/traversal1526=== RUN TestIsValidUploadKey/traversal_nar1527=== PAUSE TestIsValidUploadKey/traversal_nar1528=== RUN TestIsValidUploadKey/absolute1529=== PAUSE TestIsValidUploadKey/absolute1530=== RUN TestIsValidUploadKey/empty_key1531=== PAUSE TestIsValidUploadKey/empty_key1532=== RUN TestIsValidUploadKey/unknown_type1533=== PAUSE TestIsValidUploadKey/unknown_type1534=== CONT TestProxyWriteTimeout/narinfo1535=== CONT TestProxyWriteTimeout/unknown_size1536=== CONT TestProxyWriteTimeout/10_GiB_nar1537=== CONT TestProxyWriteTimeout/1_GiB_nar1538--- PASS: TestProxyWriteTimeout (0.00s)1539 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1540 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1541 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1542 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1543=== CONT TestGCEndsOnShutdown/before_the_run1544--- PASS: TestMultipartCleanup (0.50s)1545=== CONT TestResolveDBConnectionString/flag_wins1546=== CONT TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1547--- PASS: TestGracefulShutdownDrainsInflight (0.07s)1548=== CONT TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete15492026/10/05 11:42:53 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux15502026/10/05 11:42:53 WARN Refused reserved pin name=worker-x86_64-linux15512026/10/05 11:42:53 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux15522026/10/05 11:42:53 INFO Received create pin request method=POST path=/api/pins/my-app15532026/10/05 11:42:53 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux1554--- PASS: TestCreatePin_ReservedPins (0.41s)1555=== CONT TestResolveDBConnectionString/PGHOST_allows_empty1556=== CONT TestResolveDBConnectionString/nothing_configured1557=== CONT TestResolveDBConnectionString/missing_file_is_an_error1558=== CONT TestResolveDBConnectionString/file_when_flag_empty1559=== CONT TestPush_RejectsBadRequests/no_roots15602026/10/05 11:42:53 INFO Received push request method=POST path=/api/pushes1561=== CONT TestPush_RejectsBadRequests/bad_root15622026/10/05 11:42:53 INFO Received push request method=POST path=/api/pushes1563--- PASS: TestResolveDBConnectionString (0.00s)1564 --- PASS: TestResolveDBConnectionString/flag_wins (0.00s)1565 --- PASS: TestResolveDBConnectionString/PGHOST_allows_empty (0.00s)1566 --- PASS: TestResolveDBConnectionString/nothing_configured (0.00s)1567 --- PASS: TestResolveDBConnectionString/missing_file_is_an_error (0.00s)1568 --- PASS: TestResolveDBConnectionString/file_when_flag_empty (0.00s)1569=== CONT TestPush_RejectsBadRequests/no_objects15702026/10/05 11:42:53 INFO Received push request method=POST path=/api/pushes1571=== CONT TestPush_RejectsBadRequests/root_not_in_objects15722026/10/05 11:42:53 INFO Received push request method=POST path=/api/pushes1573=== CONT TestForceGCDuringPushOffersSweptObject/before_pending_rows1574--- PASS: TestPush_RejectsBadRequests (0.35s)1575 --- PASS: TestPush_RejectsBadRequests/no_roots (0.00s)1576 --- PASS: TestPush_RejectsBadRequests/bad_root (0.00s)1577 --- PASS: TestPush_RejectsBadRequests/no_objects (0.00s)1578 --- PASS: TestPush_RejectsBadRequests/root_not_in_objects (0.00s)1579=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure1580=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure1581=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart1582=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart1583=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts1584=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts1585=== CONT TestForceGCDuringPushOffersSweptObject/after_presence_check15862026/10/05 11:42:53 INFO Received complete multipart upload request method=POST path=/api/multipart/complete15872026/10/05 11:42:53 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=ZTVmOTI0ODUtZTJiNy00MGUzLThkYjUtYmY2ZGJhMTljMThiLjc2Mzg0ZjI5LWY3MmQtNDVhZi04NWVhLTk0NTQ3ODFjNDk3ZngxNzkxMjAwNTcyODI4Mjg4NDY5 parts=1015882026/10/05 11:42:53 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete15892026/10/05 11:42:53 INFO Completed upload id=115902026/10/05 11:42:53 INFO Received uploads request method=POST path=/api/pending_closures15912026/10/05 11:42:53 INFO Received uploads request method=POST path=/api/pending_closures15922026/10/05 11:42:53 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo15932026/10/05 11:42:53 WARN Found objects in DB but missing from S3, will re-upload count=11594--- PASS: TestService_verifyS3Integrity (0.76s)1595=== CONT TestClientErrorHandling/InvalidStorePath1596--- PASS: TestObjectStatsTrigger (0.46s)1597=== CONT TestClientErrorHandling/InvalidAuthToken15982026/10/05 11:42:53 ERROR Failed to decompress narinfo error="decompressed size exceeds configured limit"1599=== CONT TestClientErrorHandling/ServerNotAvailable1600--- PASS: TestService_healthCheckHandler (0.46s)1601=== CONT TestCacheConfigHandler/full_config,_no_issuer1602=== CONT TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1603=== CONT TestCacheConfigHandler/no_signing_keys1604=== CONT TestCacheConfigHandler/no_cache_url_configured1605--- PASS: TestCacheConfigHandler (0.00s)1606 --- PASS: TestCacheConfigHandler/full_config,_no_issuer (0.00s)1607 --- PASS: TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator (0.00s)1608 --- PASS: TestCacheConfigHandler/no_signing_keys (0.00s)1609 --- PASS: TestCacheConfigHandler/no_cache_url_configured (0.00s)1610=== CONT TestService_RequireScope_OIDC/builder_may_write1611=== CONT TestService_RequireScope_OIDC/reader_may_read1612=== CONT TestService_RequireScope_OIDC/empty_netrc_password_may_not_read1613=== CONT TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read16142026/10/05 11:42:53 WARN Authentication failed token_preview=wrong token_length=5 oidc_error="no provider could verify the token (signature or issuer mismatch)" oidc_provider="" tried_providers=[test]1615=== CONT TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1616=== CONT TestService_RequireScope_OIDC/writer_implies_read1617=== CONT TestService_RequireScope_OIDC/reader_may_not_write1618=== CONT TestService_RequireScope_OIDC/static_token_may_write1619=== CONT TestService_RequireScope_OIDC/static_token_may_admin1620=== CONT TestService_RequireScope_OIDC/anonymous_may_not_read1621=== CONT TestService_RequireScope_OIDC/reader_may_read_with_netrc1622=== CONT TestService_RequireScope_OIDC/ops_may_admin1623=== CONT TestService_RequireScope_OIDC/builder_may_not_admin1624=== CONT TestService_RequireScope_OIDC/ops_may_not_write1625=== CONT TestServerTLSConfig/no_client_CA1626=== CONT TestServerTLSConfig/not_a_PEM_file1627--- PASS: TestService_RequireScope_OIDC (0.51s)1628 --- PASS: TestService_RequireScope_OIDC/builder_may_write (0.00s)1629 --- PASS: TestService_RequireScope_OIDC/reader_may_read (0.00s)1630 --- PASS: TestService_RequireScope_OIDC/empty_netrc_password_may_not_read (0.00s)1631 --- PASS: TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read (0.00s)1632 --- PASS: TestService_RequireScope_OIDC/static_token_may_admin_with_netrc (0.00s)1633 --- PASS: TestService_RequireScope_OIDC/writer_implies_read (0.00s)1634 --- PASS: TestService_RequireScope_OIDC/reader_may_not_write (0.00s)1635 --- PASS: TestService_RequireScope_OIDC/static_token_may_write (0.00s)1636 --- PASS: TestService_RequireScope_OIDC/static_token_may_admin (0.00s)1637 --- PASS: TestService_RequireScope_OIDC/anonymous_may_not_read (0.00s)1638 --- PASS: TestService_RequireScope_OIDC/reader_may_read_with_netrc (0.00s)1639 --- PASS: TestService_RequireScope_OIDC/ops_may_admin (0.00s)1640 --- PASS: TestService_RequireScope_OIDC/builder_may_not_admin (0.00s)1641 --- PASS: TestService_RequireScope_OIDC/ops_may_not_write (0.00s)1642=== CONT TestServerTLSConfig/missing_CA_file1643--- PASS: TestServerTLSConfig (0.00s)1644 --- PASS: TestServerTLSConfig/no_client_CA (0.00s)1645 --- PASS: TestServerTLSConfig/not_a_PEM_file (0.00s)1646 --- PASS: TestServerTLSConfig/missing_CA_file (0.00s)1647=== CONT TestService_AuthMiddleware_OIDC/valid_OIDC_token1648=== CONT TestService_AuthMiddleware_OIDC/malformed_token_rejected16492026/10/05 11:42:53 WARN Authentication failed token_preview=not-a-valid-jwt token_length=15 oidc_error="no provider could verify the token (signature or issuer mismatch)" oidc_provider="" tried_providers=[test]1650=== CONT TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured1651=== CONT TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected16522026/10/05 11:42:53 WARN Authentication failed token_preview=eyJhbGciOi...ISj1nE0cfA token_length=701 oidc_error="no rule matched: claim \"repository_owner\" value [otherorg] not in allowed patterns [myorg]" oidc_provider=test tried_providers=[test]1653=== CONT TestIsValidCachePath/narinfo1654=== CONT TestIsValidCachePath/index.html1655=== CONT TestIsValidCachePath/short_hash1656=== CONT TestIsValidCachePath/wrong_extension1657=== CONT TestIsValidCachePath/leading_slash1658=== CONT TestIsValidCachePath/empty1659=== CONT TestIsValidCachePath/traversal_in_middle1660=== CONT TestIsValidCachePath/traversal_parent1661=== CONT TestIsValidCachePath/nar_uncompressed1662=== CONT TestIsValidCachePath/nix-cache-info1663=== CONT TestIsValidCachePath/invalid_char_e1664=== CONT TestIsValidCachePath/realisation1665=== CONT TestIsValidCachePath/random_path1666=== CONT TestIsValidCachePath/log1667=== CONT TestIsValidCachePath/invalid_char_u1668=== CONT TestIsValidCachePath/ls1669=== CONT TestIsValidCachePath/nar_xz1670=== CONT TestIsValidCachePath/nar_bz21671=== CONT TestIsValidCachePath/nar_zst1672=== CONT TestIsValidCachePath/narinfo_all_nix_base32_chars1673--- PASS: TestIsValidCachePath (0.00s)1674 --- PASS: TestIsValidCachePath/narinfo (0.00s)1675 --- PASS: TestIsValidCachePath/index.html (0.00s)1676 --- PASS: TestIsValidCachePath/short_hash (0.00s)1677 --- PASS: TestIsValidCachePath/wrong_extension (0.00s)1678 --- PASS: TestIsValidCachePath/leading_slash (0.00s)1679 --- PASS: TestIsValidCachePath/empty (0.00s)1680 --- PASS: TestIsValidCachePath/traversal_in_middle (0.00s)1681 --- PASS: TestIsValidCachePath/traversal_parent (0.00s)1682 --- PASS: TestIsValidCachePath/nar_uncompressed (0.00s)1683 --- PASS: TestIsValidCachePath/nix-cache-info (0.00s)1684 --- PASS: TestIsValidCachePath/invalid_char_e (0.00s)1685 --- PASS: TestIsValidCachePath/realisation (0.00s)1686 --- PASS: TestIsValidCachePath/random_path (0.00s)1687 --- PASS: TestIsValidCachePath/log (0.00s)1688 --- PASS: TestIsValidCachePath/invalid_char_u (0.00s)1689 --- PASS: TestIsValidCachePath/ls (0.00s)1690 --- PASS: TestIsValidCachePath/nar_xz (0.00s)1691 --- PASS: TestIsValidCachePath/nar_bz2 (0.00s)1692 --- PASS: TestIsValidCachePath/nar_zst (0.00s)1693 --- PASS: TestIsValidCachePath/narinfo_all_nix_base32_chars (0.00s)1694--- PASS: TestService_AuthMiddleware_OIDC (0.60s)1695 --- PASS: TestService_AuthMiddleware_OIDC/valid_OIDC_token (0.00s)1696 --- PASS: TestService_AuthMiddleware_OIDC/malformed_token_rejected (0.00s)1697 --- PASS: TestService_AuthMiddleware_OIDC/static_token_still_works_with_OIDC_configured (0.00s)1698 --- PASS: TestService_AuthMiddleware_OIDC/OIDC_token_with_wrong_org_rejected (0.00s)1699=== CONT TestParseSingleRange/none1700=== CONT TestParseSingleRange/start_far_past_EOF1701=== CONT TestParseSingleRange/start_past_EOF1702=== CONT TestParseSingleRange/single_byte1703=== CONT TestParseSingleRange/suffix_exceeds_size1704=== CONT TestParseSingleRange/suffix1705=== CONT TestParseSingleRange/end_clamped_to_size1706=== CONT TestParseSingleRange/open-ended1707=== CONT TestParseSingleRange/closed1708=== CONT TestParseSingleRange/malformed_end_before_start1709=== CONT TestParseSingleRange/malformed_both_empty1710=== CONT TestParseSingleRange/malformed_no_dash1711=== CONT TestParseSingleRange/multi-range_ignored1712=== CONT TestParseSingleRange/unknown_unit1713=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info1714--- PASS: TestParseSingleRange (0.00s)1715 --- PASS: TestParseSingleRange/none (0.00s)1716 --- PASS: TestParseSingleRange/start_far_past_EOF (0.00s)1717 --- PASS: TestParseSingleRange/start_past_EOF (0.00s)1718 --- PASS: TestParseSingleRange/single_byte (0.00s)1719 --- PASS: TestParseSingleRange/suffix_exceeds_size (0.00s)1720 --- PASS: TestParseSingleRange/suffix (0.00s)1721 --- PASS: TestParseSingleRange/end_clamped_to_size (0.00s)1722 --- PASS: TestParseSingleRange/open-ended (0.00s)1723 --- PASS: TestParseSingleRange/closed (0.00s)1724 --- PASS: TestParseSingleRange/malformed_end_before_start (0.00s)1725 --- PASS: TestParseSingleRange/malformed_both_empty (0.00s)1726 --- PASS: TestParseSingleRange/malformed_no_dash (0.00s)1727 --- PASS: TestParseSingleRange/multi-range_ignored (0.00s)1728 --- PASS: TestParseSingleRange/unknown_unit (0.00s)17292026/10/05 11:42:53 INFO Received uploads request method=POST path=/1730=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key17312026/10/05 11:42:53 INFO Received complete multipart upload request method=POST path=/1732=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key17332026/10/05 11:42:53 INFO Received request for more parts method=POST path=/1734=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal17352026/10/05 11:42:53 INFO Received uploads request method=POST path=/1736--- PASS: TestUploadHandlersRejectInvalidKeys (0.00s)1737 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1738 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1739 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1740 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.00s)1741=== CONT TestPendingClosureWriteTimeout/empty1742=== CONT TestPendingClosureWriteTimeout/670k_objects1743=== CONT TestPendingClosureWriteTimeout/400_objects1744=== CONT TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline17452026/10/05 11:42:53 INFO Starting HTTP server address=127.0.0.1:3772517462026/10/05 11:42:53 INFO Starting HTTP server address=/build/TestProxyHeadersOnlyTrustedOnSocket1026240123/001/proxy.sock17472026/10/05 11:42:53 WARN mTLS auth: subject not in bound subjects subject="CN=someone"17482026/10/05 11:42:53 INFO Shutdown signal received, draining in-flight requests timeout=10s1749--- PASS: TestProxyHeadersOnlyTrustedOnSocket (0.40s)1750=== CONT TestIsValidUploadKey/narinfo1751=== CONT TestIsValidUploadKey/unknown_type1752=== CONT TestIsValidUploadKey/empty_key1753=== CONT TestIsValidUploadKey/absolute1754=== CONT TestIsValidUploadKey/traversal_nar1755=== CONT TestIsValidUploadKey/traversal1756=== CONT TestIsValidUploadKey/listing_key,_narinfo_type1757=== CONT TestIsValidUploadKey/nar_key,_narinfo_type1758=== CONT TestIsValidUploadKey/narinfo_key,_nar_type1759=== CONT TestIsValidUploadKey/index.html1760=== CONT TestIsValidUploadKey/nix-cache-info1761=== CONT TestIsValidUploadKey/realisation_plus_in_output1762=== CONT TestIsValidUploadKey/realisation1763=== CONT TestIsValidUploadKey/build_log_equals1764=== CONT TestIsValidUploadKey/build_log_question_mark1765=== CONT TestIsValidUploadKey/build_log_plus_in_name1766=== CONT TestIsValidUploadKey/build_log_home-manager_file1767=== CONT TestIsValidUploadKey/build_log1768=== CONT TestIsValidUploadKey/listing1769=== CONT TestIsValidUploadKey/nar_plain1770=== CONT TestIsValidUploadKey/nar_xz1771=== CONT TestIsValidUploadKey/nar_zst1772=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure1773--- PASS: TestIsValidUploadKey (0.00s)1774 --- PASS: TestIsValidUploadKey/narinfo (0.00s)1775 --- PASS: TestIsValidUploadKey/unknown_type (0.00s)1776 --- PASS: TestIsValidUploadKey/empty_key (0.00s)1777 --- PASS: TestIsValidUploadKey/absolute (0.00s)1778 --- PASS: TestIsValidUploadKey/traversal_nar (0.00s)1779 --- PASS: TestIsValidUploadKey/traversal (0.00s)1780 --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s)1781 --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s)1782 --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s)1783 --- PASS: TestIsValidUploadKey/index.html (0.00s)1784 --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s)1785 --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s)1786 --- PASS: TestIsValidUploadKey/realisation (0.00s)1787 --- PASS: TestIsValidUploadKey/build_log_equals (0.00s)1788 --- PASS: TestIsValidUploadKey/build_log_question_mark (0.00s)1789 --- PASS: TestIsValidUploadKey/build_log_plus_in_name (0.00s)1790 --- PASS: TestIsValidUploadKey/build_log_home-manager_file (0.00s)1791 --- PASS: TestIsValidUploadKey/build_log (0.00s)1792 --- PASS: TestIsValidUploadKey/listing (0.00s)1793 --- PASS: TestIsValidUploadKey/nar_plain (0.00s)1794 --- PASS: TestIsValidUploadKey/nar_xz (0.00s)1795 --- PASS: TestIsValidUploadKey/nar_zst (0.00s)17962026/10/05 11:42:53 INFO Received uploads request method=POST path=/17972026/10/05 11:42:53 INFO Received complete multipart upload request method=POST path=/api/multipart/complete17982026/10/05 11:42:53 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=ZTVmOTI0ODUtZTJiNy00MGUzLThkYjUtYmY2ZGJhMTljMThiLjM1YTIzNDA4LWY2NzctNDlkMy05OTE4LTllYjE1Y2IyNDBmOHgxNzkxMjAwNTcyOTM2NDMwMzU3 parts=1017992026/10/05 11:42:53 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete18002026/10/05 11:42:53 WARN readiness check failed error="closed pool"1801--- PASS: TestService_readinessHandler (0.35s)1802=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts18032026/10/05 11:42:53 INFO Received request for more parts method=POST path=/18042026/10/05 11:42:53 INFO Completed upload id=118052026/10/05 11:42:53 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000018062026/10/05 11:42:53 INFO Received uploads request method=POST path=/api/pending_closures18072026/10/05 11:42:53 INFO Starting cleanup of old closures method=DELETE path=/api/closures18082026/10/05 11:42:53 INFO Aborted multipart uploads count=0 kept=018092026/10/05 11:42:53 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=018102026/10/05 11:42:53 WARN Request failed, retrying attempt=1 max_attempts=6 backoff=100ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/objects/present18112026/10/05 11:42:53 INFO Vacuumed table table=pending_closures18122026/10/05 11:42:53 INFO Vacuumed table table=pending_objects18132026/10/05 11:42:53 INFO Vacuumed table table=multipart_uploads18142026/10/05 11:42:53 INFO Vacuumed table table=closures18152026/10/05 11:42:53 INFO Vacuumed table table=objects18162026/10/05 11:42:53 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001817--- PASS: TestService_createPendingClosureHandler (0.82s)1818=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart18192026/10/05 11:42:53 INFO Received complete multipart upload request method=POST path=/18202026/10/05 11:42:53 INFO Aborted multipart uploads count=0 kept=01821=== NAME TestNARDeduplicationMetadataUploadBug1822 metadata_upload_test.go:48: First store path: /build/TestNARDeduplicationMetadataUploadBug2748332010/001/store/pzc3p3pwzc2q11crfiw3c63a8acvl19d-file1.txt18232026/10/05 11:42:53 WARN Force mode enabled - objects will be deleted immediately without grace period18242026/10/05 11:42:53 INFO Received uploads request method=POST path=/api/pending_closures18252026/10/05 11:42:53 INFO Aborted multipart uploads count=0 kept=018262026/10/05 11:42:53 WARN Force mode enabled - objects will be deleted immediately without grace period18272026-10-05 11:42:53.378 UTC [1632] ERROR: canceling statement due to user request18282026-10-05 11:42:53.378 UTC [1632] CONTEXT: PL/pgSQL function object_stats_apply() line 5 at statement block18292026-10-05 11:42:53.378 UTC [1632] STATEMENT: -- name: MarkStaleObjects :execrows1830 WITH RECURSIVE ct AS (1831 SELECT timezone('UTC', now()) AS now1832 ),1833 closure_reach AS (1834 -- Start with all closure keys1835 SELECT o.key, o.refs1836 FROM objects o1837 INNER JOIN closures c ON o.key = c.key1838 UNION1839 -- Recursively add all referenced objects1840 SELECT o.key, o.refs1841 FROM objects o1842 INNER JOIN closure_reach cr ON o.key = ANY(cr.refs)1843 ),1844 reachable_objects AS (1845 SELECT DISTINCT key FROM closure_reach1846 ),1847 stale_objects AS (1848 SELECT o.key1849 FROM objects AS o, ct1850 WHERE1851 NOT EXISTS (1852 SELECT 11853 FROM reachable_objects ro1854 WHERE ro.key = o.key1855 )1856 AND NOT EXISTS (1857 SELECT 11858 FROM pending_objects AS po1859 WHERE po.key = o.key1860 )1861 AND o.deleted_at IS NULL -- Only mark fresh objects1862 ORDER BY o.key COLLATE "C" -- lock in key order, like commit_pending_closure1863 FOR UPDATE1864 )1865 UPDATE objects1866 SET1867 deleted_at = ct.now,1868 first_deleted_at = COALESCE(first_deleted_at, ct.now)1869 FROM stale_objects, ct1870 WHERE objects.key = stale_objects.key1871 18722026/10/05 11:42:53 ERROR Refusing narinfo larger than the limit key=5hcdxyjf9yiq7qf3i4548drb6sjmwa1v.narinfo limit=167772161873--- PASS: TestReadProxyNarinfo (0.65s)18742026/10/05 11:42:53 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=1 objects-deleted-after-grace-period=1 objects-failed-to-delete=018752026/10/05 11:42:53 INFO Vacuumed table table=pending_closures18762026/10/05 11:42:53 INFO Vacuumed table table=pending_objects18772026/10/05 11:42:53 INFO Vacuumed table table=multipart_uploads18782026/10/05 11:42:53 INFO Vacuumed table table=closures18792026/10/05 11:42:53 INFO Aborted multipart uploads count=0 kept=018802026/10/05 11:42:53 INFO Vacuumed table table=objects18812026/10/05 11:42:53 WARN Force mode enabled - objects will be deleted immediately without grace period18822026/10/05 11:42:53 ERROR failed to remove object object=nar/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.nar.zst error="Post \"http://localhost:32883/bucket87/?delete=\": context canceled"18832026/10/05 11:42:53 INFO Received uploads request method=POST path=/api/pending_closures1884--- PASS: TestGCEndsOnShutdown (0.00s)1885 --- PASS: TestGCEndsOnShutdown/before_the_run (0.21s)1886 --- PASS: TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark (0.35s)1887 --- PASS: TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete (0.35s)18882026/10/05 11:42:53 INFO Aborted multipart uploads count=0 kept=018892026/10/05 11:42:53 WARN Force mode enabled - objects will be deleted immediately without grace period18902026/10/05 11:42:53 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=018912026/10/05 11:42:53 INFO Vacuumed table table=pending_closures18922026/10/05 11:42:53 INFO Vacuumed table table=pending_objects18932026/10/05 11:42:53 INFO Vacuumed table table=multipart_uploads18942026/10/05 11:42:53 WARN Request failed, retrying attempt=2 max_attempts=6 backoff=198.753492ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/objects/present18952026/10/05 11:42:53 INFO Vacuumed table table=closures18962026/10/05 11:42:53 INFO Vacuumed table table=objects1897--- PASS: TestForceGCDuringPushOffersSweptObject (0.00s)1898 --- PASS: TestForceGCDuringPushOffersSweptObject/before_pending_rows (0.35s)1899 --- PASS: TestForceGCDuringPushOffersSweptObject/after_presence_check (0.26s)19002026/10/05 11:42:53 INFO Received push request method=POST path=/api/pushes19012026/10/05 11:42:53 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)19022026/10/05 11:42:53 INFO Uploading pzc3p3pwzc2q11crfiw3c63a8acvl19d-file1.txt (160B)19032026/10/05 11:42:53 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"19042026/10/05 11:42:53 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign19052026/10/05 11:42:53 WARN Failed to register uploaded object key=pzc3p3pwzc2q11crfiw3c63a8acvl19d.ls error="server returned 404: 404 page not found\n"19062026/10/05 11:42:53 INFO Signed narinfos id=1 count=119072026/10/05 11:42:53 INFO Uploading 1 narinfos19082026/10/05 11:42:53 INFO Received complete push request method=POST path=/api/pushes/1/complete19092026/10/05 11:42:53 WARN Failed to register uploaded object key=pzc3p3pwzc2q11crfiw3c63a8acvl19d.narinfo error="server returned 404: 404 page not found\n"19102026/10/05 11:42:53 INFO Received uploads request method=POST path=/api/pending_closures19112026/10/05 11:42:53 INFO Upload complete. (53ms)1912=== NAME TestNARDeduplicationMetadataUploadBug1913 metadata_upload_test.go:54: Retrieved narinfo from S3:1914 StorePath: /build/TestNARDeduplicationMetadataUploadBug2748332010/001/store/pzc3p3pwzc2q11crfiw3c63a8acvl19d-file1.txt1915 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1916 Compression: zstd1917 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1918 NarSize: 1601919 References: 1920 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1921 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)1922 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):1923 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}1924=== NAME TestOrphanedObjectsGC1925 orphaned_objects_gc_test.go:290: GC Test Summary:1926 orphaned_objects_gc_test.go:291: - Kept: 2 objects from closure A1927 orphaned_objects_gc_test.go:292: - Deleted: 2 objects from closure B1928 orphaned_objects_gc_test.go:293: - Deleted: 6 orphaned chain objects (X1->X2->X3)1929 orphaned_objects_gc_test.go:294: - Deleted: 2 orphaned single objects (Y)1930 orphaned_objects_gc_test.go:295: - Total deleted: 10 objects1931--- PASS: TestOrphanedObjectsGC (0.72s)1932=== NAME TestNARDeduplicationMetadataUploadBug1933 metadata_upload_test.go:64: Second store path (same content): /build/TestNARDeduplicationMetadataUploadBug2748332010/001/store/rxqdd3hya8x2l9axs2g088qdfsmnqh11-file2.txt19342026/10/05 11:42:53 WARN Authentication failed token_preview=invalid-token token_length=13 reason="static API token mismatch"19352026/10/05 11:42:53 WARN Authentication failed token_preview=invalid-token token_length=13 reason="static API token mismatch"19362026/10/05 11:42:53 INFO Received push request method=POST path=/api/pushes19372026/10/05 11:42:53 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)19382026/10/05 11:42:53 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign19392026/10/05 11:42:53 WARN Failed to register uploaded object key=rxqdd3hya8x2l9axs2g088qdfsmnqh11.ls error="server returned 404: 404 page not found\n"19402026/10/05 11:42:53 INFO Signed narinfos id=2 count=119412026/10/05 11:42:53 INFO Uploading 1 narinfos19422026/10/05 11:42:53 INFO Received complete push request method=POST path=/api/pushes/2/complete19432026/10/05 11:42:53 WARN Failed to register uploaded object key=rxqdd3hya8x2l9axs2g088qdfsmnqh11.narinfo error="server returned 404: 404 page not found\n"19442026/10/05 11:42:53 INFO Upload complete. (43ms)1945 metadata_upload_test.go:76: Retrieved narinfo from S3:1946 StorePath: /build/TestNARDeduplicationMetadataUploadBug2748332010/001/store/rxqdd3hya8x2l9axs2g088qdfsmnqh11-file2.txt1947 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1948 Compression: zstd1949 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1950 NarSize: 1601951 References: 1952 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1953 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)1954 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):1955 {"version":1,"root":{"type":"regular","size":44}}1956--- PASS: TestNARDeduplicationMetadataUploadBug (0.59s)19572026/10/05 11:42:53 INFO lead: released remote=192.0.2.1:12341958--- PASS: TestPresentNotReportedWhileGCDeletesClosure (0.87s)1959--- PASS: TestLeadEndsOnShutdown (1.36s)19602026/10/05 11:42:53 WARN Request failed, retrying attempt=3 max_attempts=6 backoff=390.094966ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/objects/present19612026/10/05 11:42:53 INFO lead: released remote=192.0.2.1:123419622026/10/05 11:42:53 INFO lead: acquired remote=192.0.2.1:123419632026/10/05 11:42:54 WARN Request failed, retrying attempt=4 max_attempts=6 backoff=732.692655ms error="Post \"http://localhost:19999/api/objects/present\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/objects/present19642026/10/05 11:42:54 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=019652026/10/05 11:42:54 INFO Received create pin request method=POST path=/api/pins/myapp19662026/10/05 11:42:54 INFO Created/updated pin name=myapp store_path=/build/TestPinProtectsFromGC975776449/001/store/x709wf9whsqjxaaydjcdmcv65flx4d1s-pinned-file.txt narinfo_key=x709wf9whsqjxaaydjcdmcv65flx4d1s.narinfo19672026/10/05 11:42:54 INFO Starting cleanup of old closures method=DELETE path=/api/closures19682026/10/05 11:42:54 INFO Garbage collection started19692026/10/05 11:42:54 INFO Aborted multipart uploads count=0 kept=019702026/10/05 11:42:54 WARN Force mode enabled - objects will be deleted immediately without grace period19712026/10/05 11:42:54 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=019722026/10/05 11:42:54 INFO Vacuumed table table=pending_closures19732026/10/05 11:42:54 INFO Vacuumed table table=pending_objects19742026/10/05 11:42:54 INFO Vacuumed table table=multipart_uploads19752026/10/05 11:42:54 INFO Vacuumed table table=closures19762026/10/05 11:42:54 INFO Vacuumed table table=objects1977=== NAME TestOrphanedObjectsGCStressTest1978 orphaned_objects_gc_test.go:425: Created 10 active closures, 5 to-delete closures, 20 orphaned chains1979 orphaned_objects_gc_test.go:446: Marked 210 objects for deletion1980--- PASS: TestPendingClosureWriteTimeout (0.00s)1981 --- PASS: TestPendingClosureWriteTimeout/empty (0.00s)1982 --- PASS: TestPendingClosureWriteTimeout/670k_objects (0.00s)1983 --- PASS: TestPendingClosureWriteTimeout/400_objects (0.00s)1984 --- PASS: TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline (0.99s)19852026/10/05 11:42:54 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=01986=== NAME TestClientIntegration1987 client_integration_test.go:323: Objects in database after GC:1988 client_integration_test.go:323: Successfully deleted all objects with GC --force1989--- PASS: TestClientIntegration (2.71s)1990--- PASS: TestUploadHandlersRejectOversizedBody (0.15s)1991 --- PASS: TestUploadHandlersRejectOversizedBody/request_more_parts (0.07s)1992 --- PASS: TestUploadHandlersRejectOversizedBody/complete_multipart (0.07s)1993 --- PASS: TestUploadHandlersRejectOversizedBody/create_pending_closure (1.04s)1994=== NAME TestOrphanedObjectsGCStressTest1995 orphaned_objects_gc_test.go:509: Stress test completed successfully:1996 orphaned_objects_gc_test.go:510: - Active objects preserved: 201997 orphaned_objects_gc_test.go:511: - Objects deleted: 2101998 orphaned_objects_gc_test.go:512: - Total GC'd: 2101999--- PASS: TestOrphanedObjectsGCStressTest (1.67s)20002026/10/05 11:42:54 WARN Request failed, retrying attempt=5 max_attempts=6 backoff=1.680705s error="Post \"http://localhost:19999/api/objects/present\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/objects/present20012026/10/05 11:42:54 INFO lead: released remote=192.0.2.1:12342002--- PASS: TestLeadElectsOneAndHandsOver (2.68s)20032026/10/05 11:42:55 WARN Rate limiter enabled after throttle name=s3-test rate=520042026/10/05 11:42:55 WARN S3 rate limit hit operation="complete multipart upload" error="Please reduce your request rate."2005=== NAME TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle2006 throttle_test.go:213: Proxy stats: total=13, throttled=10, completeMultipart=102007 throttle_test.go:215: Rate limiter: enabled=true, rate=5.002008--- PASS: TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle (4.46s)2009=== NAME TestPinProtectsFromGC2010 client_integration_test.go:840: Pin successfully protected closure from garbage collection2011--- PASS: TestPinProtectsFromGC (4.95s)20122026/10/05 11:42:56 WARN Request failed, retrying attempt=1 max_attempts=6 backoff=100ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config20132026/10/05 11:42:56 WARN Request failed, retrying attempt=2 max_attempts=6 backoff=200.503464ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config2014--- PASS: TestReadProxyOutlastsServerWriteTimeout (4.53s)20152026/10/05 11:42:56 WARN Request failed, retrying attempt=3 max_attempts=6 backoff=405.688958ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config20162026/10/05 11:42:57 WARN Request failed, retrying attempt=4 max_attempts=6 backoff=818.942718ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config2017--- PASS: TestConnectWaitsForAPeerMigration (5.12s)20182026/10/05 11:42:57 WARN Request failed, retrying attempt=5 max_attempts=6 backoff=1.470892496s error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config20192026/10/05 11:42:58 INFO Aborted multipart uploads count=1 kept=020202026/10/05 11:42:58 INFO Received cleanup request method=DELETE path=/api/pending_closures20212026/10/05 11:42:58 INFO Aborted multipart uploads count=1 kept=02022--- PASS: TestPendingCleanupUsesOneCutoff (6.42s)20232026/10/05 11:42:59 WARN Failed to fetch cache config, uploading without size limit error="fetching cache config: sending request: request failed after retries: Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused"20242026/10/05 11:42:59 WARN Request failed, retrying attempt=1 max_attempts=6 backoff=100ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config20252026/10/05 11:42:59 WARN Request failed, retrying attempt=2 max_attempts=6 backoff=196.6438ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config20262026/10/05 11:42:59 WARN Request failed, retrying attempt=3 max_attempts=6 backoff=421.284067ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config20272026/10/05 11:43:00 WARN Request failed, retrying attempt=4 max_attempts=6 backoff=843.085984ms error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config20282026/10/05 11:43:01 WARN Request failed, retrying attempt=5 max_attempts=6 backoff=1.66646157s error="Get \"http://localhost:19999/api/cache-config\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/cache-config20292026/10/05 11:43:02 WARN Request failed, retrying attempt=1 max_attempts=6 backoff=100ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures20302026/10/05 11:43:02 WARN Request failed, retrying attempt=2 max_attempts=6 backoff=218.415702ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures20312026/10/05 11:43:03 WARN Request failed, retrying attempt=3 max_attempts=6 backoff=371.886928ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures20322026/10/05 11:43:03 WARN Request failed, retrying attempt=4 max_attempts=6 backoff=872.956217ms error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures20332026/10/05 11:43:04 WARN Request failed, retrying attempt=5 max_attempts=6 backoff=1.470789767s error="Post \"http://localhost:19999/api/pending_closures\": dial tcp [::1]:19999: connect: connection refused" url=http://localhost:19999/api/pending_closures2034--- PASS: TestClientErrorHandling (0.00s)2035 --- PASS: TestClientErrorHandling/InvalidStorePath (0.23s)2036 --- PASS: TestClientErrorHandling/InvalidAuthToken (0.31s)2037 --- PASS: TestClientErrorHandling/ServerNotAvailable (12.50s)2038PASS20392026-10-05 11:43:05.997 UTC [129] LOG: received smart shutdown request20402026-10-05 11:43:06.001 UTC [129] LOG: background worker "logical replication launcher" (PID 139) exited with exit code 120412026-10-05 11:43:06.013 UTC [134] LOG: shutting down20422026-10-05 11:43:06.013 UTC [134] LOG: checkpoint starting: shutdown immediate20432026-10-05 11:43:06.884 UTC [134] LOG: checkpoint complete: wrote 11191 buffers (68.3%), wrote 5 SLRU buffers; 0 WAL file(s) added, 0 removed, 26 recycled; write=0.189 s, sync=0.652 s, total=0.871 s; sync files=31000, longest=0.002 s, average=0.001 s; distance=428558 kB, estimate=428558 kB; lsn=0/1B9E9618, redo lsn=0/1B9E961820442026-10-05 11:43:06.962 UTC [129] LOG: database system is shut down2045Running OIDC tests...2046=== RUN TestAudienceForIssuer2047=== PAUSE TestAudienceForIssuer2048=== RUN TestGlobMatch2049=== PAUSE TestGlobMatch2050=== RUN TestValidateToken_ValidToken2051=== PAUSE TestValidateToken_ValidToken2052=== RUN TestValidateToken_WrongAudience2053=== PAUSE TestValidateToken_WrongAudience2054=== RUN TestValidateToken_Expired2055=== PAUSE TestValidateToken_Expired2056=== RUN TestValidateToken_BoundClaimsMismatch2057=== PAUSE TestValidateToken_BoundClaimsMismatch2058=== RUN TestValidateToken_BoundSubjectMismatch2059=== PAUSE TestValidateToken_BoundSubjectMismatch2060=== RUN TestValidateToken_MultipleProviders2061=== PAUSE TestValidateToken_MultipleProviders2062=== RUN TestValidateToken_NoMatchingProvider2063=== PAUSE TestValidateToken_NoMatchingProvider2064=== RUN TestValidateToken_KubernetesServiceAccount2065=== PAUSE TestValidateToken_KubernetesServiceAccount2066=== RUN TestNewValidator_KubernetesRequiresCA2067=== PAUSE TestNewValidator_KubernetesRequiresCA2068=== RUN TestValidateToken_KubernetesIssuerFromOwnToken2069=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken2070=== RUN TestPins_ReservedForMatchingRule2071=== PAUSE TestPins_ReservedForMatchingRule2072=== RUN TestPins_TopLevelShorthand2073=== PAUSE TestPins_TopLevelShorthand2074=== RUN TestPins_ConfigValidation2075=== PAUSE TestPins_ConfigValidation2076=== RUN TestScopes_LegacyProviderDefaultsToWrite2077=== PAUSE TestScopes_LegacyProviderDefaultsToWrite2078=== RUN TestScopes_Rules2079=== PAUSE TestScopes_Rules2080=== RUN TestScopes_ConfigValidation2081=== PAUSE TestScopes_ConfigValidation2082=== CONT TestAudienceForIssuer2083=== CONT TestValidateToken_KubernetesServiceAccount2084=== CONT TestValidateToken_Expired2085=== CONT TestPins_ConfigValidation2086=== CONT TestScopes_ConfigValidation2087--- PASS: TestAudienceForIssuer (0.00s)2088=== CONT TestScopes_Rules2089=== CONT TestValidateToken_NoMatchingProvider2090=== CONT TestValidateToken_MultipleProviders2091=== CONT TestValidateToken_BoundSubjectMismatch2092=== CONT TestScopes_LegacyProviderDefaultsToWrite2093=== CONT TestValidateToken_BoundClaimsMismatch2094=== CONT TestPins_TopLevelShorthand2095=== CONT TestValidateToken_KubernetesIssuerFromOwnToken2096--- PASS: TestPins_ConfigValidation (0.00s)2097--- PASS: TestScopes_ConfigValidation (0.00s)2098=== CONT TestValidateToken_ValidToken2099=== CONT TestValidateToken_WrongAudience2100=== CONT TestNewValidator_KubernetesRequiresCA2101=== CONT TestGlobMatch2102=== RUN TestGlobMatch/foo_foo2103=== PAUSE TestGlobMatch/foo_foo2104=== RUN TestGlobMatch/foo_bar2105=== PAUSE TestGlobMatch/foo_bar2106=== RUN TestGlobMatch/*_2107=== PAUSE TestGlobMatch/*_2108=== RUN TestGlobMatch/*_anything2109=== PAUSE TestGlobMatch/*_anything2110=== RUN TestGlobMatch/foo*_foo2111=== PAUSE TestGlobMatch/foo*_foo2112=== RUN TestGlobMatch/foo*_foobar2113=== PAUSE TestGlobMatch/foo*_foobar2114=== RUN TestGlobMatch/foo*_bar2115=== CONT TestPins_ReservedForMatchingRule2116=== PAUSE TestGlobMatch/foo*_bar2117=== RUN TestGlobMatch/*bar_bar2118=== PAUSE TestGlobMatch/*bar_bar2119=== RUN TestGlobMatch/*bar_foobar2120=== PAUSE TestGlobMatch/*bar_foobar2121=== RUN TestGlobMatch/*bar_foo2122=== PAUSE TestGlobMatch/*bar_foo2123=== RUN TestGlobMatch/foo*bar_foobar2124=== PAUSE TestGlobMatch/foo*bar_foobar2125=== RUN TestGlobMatch/foo*bar_foo123bar2126=== PAUSE TestGlobMatch/foo*bar_foo123bar2127=== RUN TestGlobMatch/foo*bar_foobarbaz2128=== PAUSE TestGlobMatch/foo*bar_foobarbaz2129=== RUN TestGlobMatch/*/*_foo/bar2130=== PAUSE TestGlobMatch/*/*_foo/bar2131=== RUN TestGlobMatch/*/*_foo2132=== PAUSE TestGlobMatch/*/*_foo2133=== RUN TestGlobMatch/refs/heads/*_refs/heads/main2134=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main2135=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.02136=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.02137=== RUN TestGlobMatch/refs/*/main_refs/heads/main2138=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main2139=== RUN TestGlobMatch/fo?_foo2140=== PAUSE TestGlobMatch/fo?_foo2141=== RUN TestGlobMatch/fo?_fo2142=== PAUSE TestGlobMatch/fo?_fo2143=== RUN TestGlobMatch/fo?_fooo2144=== PAUSE TestGlobMatch/fo?_fooo2145=== RUN TestGlobMatch/?oo_foo2146=== PAUSE TestGlobMatch/?oo_foo2147=== RUN TestGlobMatch/?oo_boo2148=== PAUSE TestGlobMatch/?oo_boo2149=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main2150=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main2151=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main2152=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main2153=== CONT TestGlobMatch/foo_foo2154=== CONT TestGlobMatch/foo*bar_foobarbaz2155=== CONT TestGlobMatch/foo*_foobar2156=== CONT TestGlobMatch/foo*_foo2157=== CONT TestGlobMatch/*_anything2158=== CONT TestGlobMatch/fo?_fooo2159=== CONT TestGlobMatch/fo?_foo2160=== CONT TestGlobMatch/*bar_foo2161=== CONT TestGlobMatch/foo*bar_foo123bar2162=== CONT TestGlobMatch/foo*bar_foobar2163=== CONT TestGlobMatch/*bar_foobar2164=== CONT TestGlobMatch/refs/heads/*_refs/heads/main2165=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.02166=== CONT TestGlobMatch/*/*_foo2167=== CONT TestGlobMatch/*bar_bar2168=== CONT TestGlobMatch/*/*_foo/bar2169=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main2170=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main2171=== CONT TestGlobMatch/?oo_boo2172=== CONT TestGlobMatch/?oo_foo2173=== CONT TestGlobMatch/*_2174=== CONT TestGlobMatch/foo_bar2175=== CONT TestGlobMatch/refs/*/main_refs/heads/main2176=== CONT TestGlobMatch/fo?_fo2177=== CONT TestGlobMatch/foo*_bar2178--- PASS: TestGlobMatch (0.00s)2179 --- PASS: TestGlobMatch/foo_foo (0.00s)2180 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)2181 --- PASS: TestGlobMatch/foo*_foobar (0.00s)2182 --- PASS: TestGlobMatch/foo*_foo (0.00s)2183 --- PASS: TestGlobMatch/*_anything (0.00s)2184 --- PASS: TestGlobMatch/fo?_fooo (0.00s)2185 --- PASS: TestGlobMatch/fo?_foo (0.00s)2186 --- PASS: TestGlobMatch/*bar_foo (0.00s)2187 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)2188 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)2189 --- PASS: TestGlobMatch/*bar_foobar (0.00s)2190 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)2191 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)2192 --- PASS: TestGlobMatch/*/*_foo (0.00s)2193 --- PASS: TestGlobMatch/*bar_bar (0.00s)2194 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)2195 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)2196 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)2197 --- PASS: TestGlobMatch/?oo_boo (0.00s)2198 --- PASS: TestGlobMatch/?oo_foo (0.00s)2199 --- PASS: TestGlobMatch/*_ (0.00s)2200 --- PASS: TestGlobMatch/foo_bar (0.00s)2201 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)2202 --- PASS: TestGlobMatch/fo?_fo (0.00s)2203 --- PASS: TestGlobMatch/foo*_bar (0.00s)22042026/10/05 11:43:08 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41543/oidc2205--- PASS: TestPins_TopLevelShorthand (0.03s)22062026/10/05 11:43:08 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:35143/oidc2207--- PASS: TestValidateToken_ValidToken (0.05s)22082026/10/05 11:43:08 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:41511/oidc2209--- PASS: TestValidateToken_WrongAudience (0.06s)22102026/10/05 11:43:08 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:32957/oidc2211--- PASS: TestValidateToken_BoundSubjectMismatch (0.09s)22122026/10/05 11:43:08 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:39183/oidc2213--- PASS: TestPins_ReservedForMatchingRule (0.11s)22142026/10/05 11:43:08 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:434492215--- PASS: TestValidateToken_KubernetesServiceAccount (0.12s)22162026/10/05 11:43:08 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:36127/oidc22172026/10/05 11:43:08 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:44293/oidc2218--- PASS: TestValidateToken_NoMatchingProvider (0.13s)2219--- PASS: TestValidateToken_BoundClaimsMismatch (0.13s)22202026/10/05 11:43:08 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:40257/oidc2221--- PASS: TestValidateToken_Expired (0.15s)22222026/10/05 11:43:08 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:43735/oidc22232026/10/05 11:43:08 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC1232224--- PASS: TestScopes_Rules (0.17s)2225--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (0.17s)22262026/10/05 11:43:08 http: TLS handshake error from 127.0.0.1:40918: remote error: tls: bad certificate2227--- PASS: TestNewValidator_KubernetesRequiresCA (0.18s)22282026/10/05 11:43:08 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:40251/oidc2229--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.19s)22302026/10/05 11:43:08 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:44879/oidc22312026/10/05 11:43:08 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:43977/oidc2232--- PASS: TestValidateToken_MultipleProviders (0.34s)2233PASS2234Running hook tests...2235=== RUN TestSendPathsEmpty2236=== PAUSE TestSendPathsEmpty2237=== RUN TestQueueEnqueueAndFetch2238=== PAUSE TestQueueEnqueueAndFetch2239=== RUN TestQueueDeduplication2240=== PAUSE TestQueueDeduplication2241=== RUN TestQueueRemove2242=== PAUSE TestQueueRemove2243=== RUN TestQueueFetchBatchLimit2244=== PAUSE TestQueueFetchBatchLimit2245=== RUN TestQueueRetryMovesToBack2246=== PAUSE TestQueueRetryMovesToBack2247=== RUN TestQueueFetchRemoveLifecycle2248=== PAUSE TestQueueFetchRemoveLifecycle2249=== RUN TestQueueConcurrentWriters2250=== PAUSE TestQueueConcurrentWriters2251=== RUN TestQueueRemoveLargeClosure2252=== PAUSE TestQueueRemoveLargeClosure2253=== RUN TestServerClientIntegration2254=== PAUSE TestServerClientIntegration2255=== RUN TestServerQueueError2256=== PAUSE TestServerQueueError2257=== RUN TestGetListenerSocketActivation2258 server_test.go:210: === RUN TestGetListenerSocketActivation2259 --- PASS: TestGetListenerSocketActivation (0.00s)2260 PASS2261 2262--- PASS: TestGetListenerSocketActivation (0.01s)2263=== RUN TestDrainIsolatesPoisonPath2264=== PAUSE TestDrainIsolatesPoisonPath2265=== RUN TestRunNotBlockedByPoisonHead2266=== PAUSE TestRunNotBlockedByPoisonHead2267=== RUN TestDrainGivesUpWhenServerDown2268=== PAUSE TestDrainGivesUpWhenServerDown2269=== RUN TestFailedPathPrunedByLaterClosure2270=== PAUSE TestFailedPathPrunedByLaterClosure2271=== RUN TestWorkerUploadsAndRemoves2272=== PAUSE TestWorkerUploadsAndRemoves2273=== RUN TestWorkerSkipsGCdPaths2274=== PAUSE TestWorkerSkipsGCdPaths2275=== RUN TestWorkerPrunesClosureDeps2276=== PAUSE TestWorkerPrunesClosureDeps2277=== RUN TestDrainTimeout2278=== PAUSE TestDrainTimeout2279=== CONT TestSendPathsEmpty2280=== CONT TestRunNotBlockedByPoisonHead2281=== CONT TestFailedPathPrunedByLaterClosure2282--- PASS: TestSendPathsEmpty (0.00s)2283=== CONT TestQueueRetryMovesToBack2284=== CONT TestQueueFetchBatchLimit2285=== CONT TestQueueRemove2286=== CONT TestQueueFetchRemoveLifecycle2287=== CONT TestDrainIsolatesPoisonPath2288=== CONT TestQueueDeduplication2289=== CONT TestServerQueueError2290=== CONT TestQueueEnqueueAndFetch2291=== CONT TestServerClientIntegration2292=== CONT TestQueueRemoveLargeClosure2293=== CONT TestQueueConcurrentWriters2294=== CONT TestWorkerSkipsGCdPaths2295=== CONT TestDrainTimeout22962026/10/05 11:43:08 ERROR Failed to queue paths error="permission denied" count=12297=== CONT TestWorkerPrunesClosureDeps2298=== CONT TestWorkerUploadsAndRemoves2299=== CONT TestDrainGivesUpWhenServerDown2300--- PASS: TestServerQueueError (0.00s)2301--- PASS: TestServerClientIntegration (0.00s)2302--- PASS: TestQueueFetchBatchLimit (0.01s)23032026/10/05 11:43:08 INFO Upload queue status pending=223042026/10/05 11:43:08 WARN Store path no longer exists (garbage collected?), removing from queue path=/build/TestWorkerSkipsGCdPaths1052680299/002/nonexistent23052026/10/05 11:43:08 INFO Upload queue status pending=223062026/10/05 11:43:08 INFO Uploading batch count=223072026/10/05 11:43:08 INFO Uploading batch count=223082026/10/05 11:43:08 ERROR Upload failed error="upload failed" count=223092026/10/05 11:43:08 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown3143289018/002/a23102026/10/05 11:43:08 INFO Uploading batch count=123112026/10/05 11:43:08 ERROR Upload failed error="upload failed" count=12312--- PASS: TestQueueRemove (0.01s)23132026/10/05 11:43:08 INFO Upload queue status pending=223142026/10/05 11:43:08 INFO Uploading batch count=123152026/10/05 11:43:08 INFO Uploading batch count=123162026/10/05 11:43:08 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown3143289018/002/b23172026/10/05 11:43:08 INFO Uploading batch count=123182026/10/05 11:43:08 INFO Uploading batch count=423192026/10/05 11:43:08 ERROR Upload failed error="upload failed" count=423202026/10/05 11:43:08 INFO Uploading batch count=123212026/10/05 11:43:08 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainIsolatesPoisonPath840970141/002/bbb23222026/10/05 11:43:08 INFO Uploading batch count=223232026/10/05 11:43:08 INFO Uploading batch count=223242026/10/05 11:43:08 ERROR Upload failed error="upload failed" count=223252026/10/05 11:43:08 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown3143289018/002/c2326--- PASS: TestQueueEnqueueAndFetch (0.01s)23272026/10/05 11:43:08 INFO Upload queue status pending=323282026/10/05 11:43:08 INFO Uploading batch count=123292026/10/05 11:43:08 ERROR Upload failed error="upload failed" count=123302026/10/05 11:43:08 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown3143289018/002/d2331--- PASS: TestQueueDeduplication (0.01s)2332--- PASS: TestQueueFetchRemoveLifecycle (0.01s)23332026/10/05 11:43:08 INFO Uploading batch count=223342026/10/05 11:43:08 ERROR Upload failed error="upload failed" count=223352026/10/05 11:43:08 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown3143289018/002/e23362026/10/05 11:43:08 INFO Uploading batch count=123372026/10/05 11:43:08 ERROR Upload failed error="upload failed" count=12338--- PASS: TestQueueRetryMovesToBack (0.02s)2339--- PASS: TestFailedPathPrunedByLaterClosure (0.02s)23402026/10/05 11:43:08 ERROR Upload failed, will retry later error="upload failed" path=/build/TestDrainGivesUpWhenServerDown3143289018/002/f23412026/10/05 11:43:08 INFO Uploading batch count=123422026/10/05 11:43:08 ERROR Upload failed error="upload failed" count=123432026/10/05 11:43:08 ERROR Drain finished with paths left in queue remaining=1023442026/10/05 11:43:08 INFO Uploading batch count=123452026/10/05 11:43:08 ERROR Upload failed error="upload failed" count=123462026/10/05 11:43:08 ERROR Drain finished with paths left in queue remaining=12347--- PASS: TestDrainGivesUpWhenServerDown (0.02s)2348--- PASS: TestDrainIsolatesPoisonPath (0.02s)2349--- PASS: TestWorkerSkipsGCdPaths (0.03s)2350--- PASS: TestWorkerUploadsAndRemoves (0.03s)2351--- PASS: TestWorkerPrunesClosureDeps (0.03s)2352--- PASS: TestQueueRemoveLargeClosure (0.13s)2353--- PASS: TestQueueConcurrentWriters (0.15s)23542026/10/05 11:43:08 ERROR Upload failed error="context deadline exceeded" count=223552026/10/05 11:43:08 ERROR Drain finished with paths left in queue remaining=42356--- PASS: TestDrainTimeout (0.21s)23572026/10/05 11:43:09 INFO Uploading batch count=123582026/10/05 11:43:09 INFO Uploading batch count=123592026/10/05 11:43:09 INFO Uploading batch count=123602026/10/05 11:43:09 ERROR Upload failed error="upload failed" count=123612026/10/05 11:43:09 INFO Uploading batch count=123622026/10/05 11:43:09 ERROR Upload failed error="upload failed" count=123632026/10/05 11:43:09 INFO Uploading batch count=123642026/10/05 11:43:09 ERROR Upload failed error="upload failed" count=123652026/10/05 11:43:09 INFO Uploading batch count=123662026/10/05 11:43:09 ERROR Upload failed error="upload failed" count=123672026/10/05 11:43:09 ERROR Drain finished with paths left in queue remaining=12368--- PASS: TestRunNotBlockedByPoisonHead (1.03s)2369PASS