niks3-go-unit-tests
checks.aarch64-darwin.go-unit-tests
· build #506
· raw
1tribuchet: building on mac022Running client tests...3=== RUN TestDoServerRequestAttachesToken4=== PAUSE TestDoServerRequestAttachesToken5=== RUN TestUploadBuildLog_FileBodyReplayedOnRetry6=== PAUSE TestUploadBuildLog_FileBodyReplayedOnRetry7=== RUN TestRegisterUploadedObjectReusesConnections8=== PAUSE TestRegisterUploadedObjectReusesConnections9=== RUN TestRunGarbageCollection_FinishedOnAnotherReplica10=== PAUSE TestRunGarbageCollection_FinishedOnAnotherReplica11=== RUN TestRunGarbageCollection_NotFoundAfterLocalRun12=== PAUSE TestRunGarbageCollection_NotFoundAfterLocalRun13=== RUN TestCaseHackSuffix14=== PAUSE TestCaseHackSuffix15=== RUN TestFilterOversizedClosures16=== PAUSE TestFilterOversizedClosures17=== RUN TestUploadMultipart_PartsInParallel18=== PAUSE TestUploadMultipart_PartsInParallel19=== RUN TestUploadMultipart_ReturnsAfterDroppedBody20=== PAUSE TestUploadMultipart_ReturnsAfterDroppedBody21=== RUN TestUploadMultipart_FailedProducerIsNotCompleted22=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted23=== RUN TestPartSizeForNAR24=== PAUSE TestPartSizeForNAR25=== RUN TestDumpPathCaseHackMatchesNix26--- PASS: TestDumpPathCaseHackMatchesNix (0.16s)27=== RUN TestDumpPathCaseHackCollision28--- PASS: TestDumpPathCaseHackCollision (0.00s)29=== RUN TestDumpPathMatchesNix30=== PAUSE TestDumpPathMatchesNix31=== RUN TestDumpPathSingleFile32=== PAUSE TestDumpPathSingleFile33=== RUN TestDumpPathWriterError34=== PAUSE TestDumpPathWriterError35=== RUN TestEncodeNixBase3236=== PAUSE TestEncodeNixBase3237=== RUN TestEncodeNixBase32WithRealHash38=== PAUSE TestEncodeNixBase32WithRealHash39=== RUN TestConvertHashToNix3240=== PAUSE TestConvertHashToNix3241=== RUN TestGetStorePathHash42=== PAUSE TestGetStorePathHash43=== RUN TestPathInfoHashCompatibility44=== PAUSE TestPathInfoHashCompatibility45=== RUN TestParsePathInfoJSON46=== PAUSE TestParsePathInfoJSON47=== RUN TestParsePathInfoJSONMultiplePaths48=== PAUSE TestParsePathInfoJSONMultiplePaths49=== RUN TestPathInfoCACompatibility50=== PAUSE TestPathInfoCACompatibility51=== RUN TestRateLimiterFeedback52=== PAUSE TestRateLimiterFeedback53=== RUN TestRateLimiterFeedback_400DoesNotCountAsSuccess54=== PAUSE TestRateLimiterFeedback_400DoesNotCountAsSuccess55=== RUN TestRegisterUploadedObject_BoundedAgainstSilentServer56=== PAUSE TestRegisterUploadedObject_BoundedAgainstSilentServer57=== RUN TestRegisterUploadedObject_OutlivesPush58=== PAUSE TestRegisterUploadedObject_OutlivesPush59=== RUN TestResolveStorePath60=== PAUSE TestResolveStorePath61=== RUN TestDoWithRetry_BodyReplayedViaGetBody62=== PAUSE TestDoWithRetry_BodyReplayedViaGetBody63=== RUN TestRetryTimeoutOutlastsMaxRetries64=== PAUSE TestRetryTimeoutOutlastsMaxRetries65=== RUN TestDoWithRetry_FinalResponseBodyReadable66=== PAUSE TestDoWithRetry_FinalResponseBodyReadable67=== RUN TestShellSplit68=== PAUSE TestShellSplit69=== RUN TestShellSplitErrors70=== PAUSE TestShellSplitErrors71=== RUN TestStreamPushReportsEveryPath72=== PAUSE TestStreamPushReportsEveryPath73=== RUN TestStreamPushBatchesUnderLoad74=== PAUSE TestStreamPushBatchesUnderLoad75=== RUN TestStreamPushIsolatesFailures76=== PAUSE TestStreamPushIsolatesFailures77=== RUN TestStreamPushGivesUpOnDeadServer78=== PAUSE TestStreamPushGivesUpOnDeadServer79=== RUN TestStreamPushRequestLine80=== PAUSE TestStreamPushRequestLine81=== RUN TestStreamPushReportsSignatures82=== PAUSE TestStreamPushReportsSignatures83=== RUN TestStreamPushReportsSkippedPaths84=== PAUSE TestStreamPushReportsSkippedPaths85=== RUN TestClientSignaturesByStorePath86=== PAUSE TestClientSignaturesByStorePath87=== RUN TestStreamPushAnswersEveryLineWhenCanceled88=== PAUSE TestStreamPushAnswersEveryLineWhenCanceled89=== RUN TestStreamPushStopsWhenCanceled90=== PAUSE TestStreamPushStopsWhenCanceled91=== RUN TestSetClientTLS92=== PAUSE TestSetClientTLS93=== RUN TestSetClientTLSDoesNotMutateDefaultTransport94=== PAUSE TestSetClientTLSDoesNotMutateDefaultTransport95=== RUN TestSetClientTLSErrors96=== PAUSE TestSetClientTLSErrors97=== RUN TestStaticToken98=== PAUSE TestStaticToken99=== RUN TestFileTokenReadsAndCaches100=== PAUSE TestFileTokenReadsAndCaches101=== RUN TestFileTokenMissing102=== PAUSE TestFileTokenMissing103=== RUN TestFileTokenEmpty104=== PAUSE TestFileTokenEmpty105=== RUN TestScriptTokenNoExpiryRerunsEveryCall106=== PAUSE TestScriptTokenNoExpiryRerunsEveryCall107=== RUN TestScriptTokenCachesUntilRefresh108=== PAUSE TestScriptTokenCachesUntilRefresh109=== RUN TestScriptTokenEmptyToken110=== PAUSE TestScriptTokenEmptyToken111=== RUN TestScriptTokenBadJSON112=== PAUSE TestScriptTokenBadJSON113=== RUN TestScriptTokenScriptFails114=== PAUSE TestScriptTokenScriptFails115=== RUN TestScriptTokenEmptyCommand116=== PAUSE TestScriptTokenEmptyCommand117=== CONT TestDoServerRequestAttachesToken118=== CONT TestStreamPushAnswersEveryLineWhenCanceled119=== CONT TestFileTokenEmpty120=== CONT TestScriptTokenEmptyCommand121--- PASS: TestScriptTokenEmptyCommand (0.00s)122=== CONT TestRegisterUploadedObject_OutlivesPush123=== CONT TestScriptTokenScriptFails124=== CONT TestScriptTokenBadJSON125=== CONT TestScriptTokenEmptyToken126=== CONT TestParsePathInfoJSONMultiplePaths127=== CONT TestStreamPushReportsSignatures128=== CONT TestScriptTokenCachesUntilRefresh129=== CONT TestStreamPushRequestLine130=== CONT TestClientSignaturesByStorePath131=== CONT TestStreamPushGivesUpOnDeadServer132=== CONT TestScriptTokenNoExpiryRerunsEveryCall133=== CONT TestStreamPushIsolatesFailures134=== CONT TestStreamPushBatchesUnderLoad135=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=false136=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=false137=== RUN TestStreamPushBatchesUnderLoad/oneAtATime=true138=== PAUSE TestStreamPushBatchesUnderLoad/oneAtATime=true139=== CONT TestStreamPushReportsEveryPath140=== CONT TestShellSplitErrors141=== CONT TestShellSplit142=== CONT TestDoWithRetry_FinalResponseBodyReadable143=== CONT TestRetryTimeoutOutlastsMaxRetries144=== CONT TestDoWithRetry_BodyReplayedViaGetBody145=== CONT TestResolveStorePath146=== CONT TestSetClientTLSErrors1472026/11/12 00:43:16 ERROR Upload failed error="bad path" count=31482000/01/01 00:00:00 ERROR Upload failed error="context canceled" count=21492026/11/12 00:43:16 ERROR Upload failed error=boom count=11502026/11/12 00:43:16 ERROR Upload failed error="connection refused" count=201512026/11/12 00:43:16 ERROR Server seems unavailable, giving up on batch untried=17152=== CONT TestRegisterUploadedObject_BoundedAgainstSilentServer153--- PASS: TestStreamPushIsolatesFailures (0.00s)154--- PASS: TestStreamPushAnswersEveryLineWhenCanceled (0.01s)155=== CONT TestPartSizeForNAR156=== RUN TestPartSizeForNAR/zero_stays_at_minimum157=== PAUSE TestPartSizeForNAR/zero_stays_at_minimum158=== RUN TestPartSizeForNAR/small_stays_at_minimum159=== PAUSE TestPartSizeForNAR/small_stays_at_minimum160=== RUN TestPartSizeForNAR/80_GiB_fits_at_minimum161=== PAUSE TestPartSizeForNAR/80_GiB_fits_at_minimum162=== RUN TestPartSizeForNAR/115_GiB_needs_larger_parts163=== PAUSE TestPartSizeForNAR/115_GiB_needs_larger_parts164=== RUN TestPartSizeForNAR/1_TiB165=== PAUSE TestPartSizeForNAR/1_TiB166=== RUN TestPartSizeForNAR/5_TiB_S3_max_object167=== PAUSE TestPartSizeForNAR/5_TiB_S3_max_object168=== RUN TestPartSizeForNAR/capped_at_5_GiB169=== PAUSE TestPartSizeForNAR/capped_at_5_GiB170=== CONT TestRateLimiterFeedback_400DoesNotCountAsSuccess171=== RUN TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths172=== CONT TestRateLimiterFeedback173=== CONT TestPathInfoCACompatibility174=== CONT TestCaseHackSuffix175=== CONT TestUploadMultipart_FailedProducerIsNotCompleted176=== CONT TestUploadMultipart_ReturnsAfterDroppedBody177--- PASS: TestShellSplit (0.00s)178--- PASS: TestFileTokenEmpty (0.01s)179=== CONT TestUploadMultipart_PartsInParallel180--- PASS: TestClientSignaturesByStorePath (0.00s)181--- PASS: TestShellSplitErrors (0.00s)182--- PASS: TestStreamPushReportsEveryPath (0.00s)183--- PASS: TestScriptTokenScriptFails (0.01s)1842026/11/12 00:43:16 WARN Rate limiter enabled after throttle name=server-test rate=5185=== RUN TestSetClientTLSErrors/missing_cert_file186=== PAUSE TestSetClientTLSErrors/missing_cert_file187=== CONT TestParsePathInfoJSON188=== RUN TestPathInfoCACompatibility/null_ca_field189=== PAUSE TestPathInfoCACompatibility/null_ca_field190=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part191=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part1922026/11/12 00:43:16 ERROR Upload failed error=boom count=11932026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:56598194--- PASS: TestRegisterUploadedObject_OutlivesPush (0.01s)195--- PASS: TestStreamPushGivesUpOnDeadServer (0.01s)196--- PASS: TestStreamPushReportsSignatures (0.01s)197=== CONT TestFilterOversizedClosures198=== CONT TestPathInfoHashCompatibility199=== PAUSE TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths200=== CONT TestGetStorePathHash201--- PASS: TestResolveStorePath (0.00s)202=== CONT TestFileTokenReadsAndCaches203--- PASS: TestFileTokenReadsAndCaches (0.00s)204=== CONT TestConvertHashToNix322052026/11/12 00:43:16 WARN Rate limiter enabled after throttle name=server-test rate=52062026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:56600207=== RUN TestRateLimiterFeedback/429_enables_limiter208=== PAUSE TestRateLimiterFeedback/429_enables_limiter209=== RUN TestSetClientTLSErrors/missing_key_file210=== PAUSE TestSetClientTLSErrors/missing_key_file211=== RUN TestSetClientTLSErrors/missing_ca_file212=== PAUSE TestSetClientTLSErrors/missing_ca_file213=== RUN TestSetClientTLSErrors/invalid_ca_file214=== PAUSE TestSetClientTLSErrors/invalid_ca_file2152026/11/12 00:43:16 WARN Rate limiter backed off name=server-test rate=5216=== RUN TestParsePathInfoJSON/Nix_format2172026/11/12 00:43:16 WARN Rate limiter enabled after throttle name=server-test rate=52182026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:56602219=== PAUSE TestParsePathInfoJSON/Nix_format220=== RUN TestParsePathInfoJSON/Lix_format221=== PAUSE TestParsePathInfoJSON/Lix_format222=== RUN TestParsePathInfoJSON/empty_input223=== PAUSE TestParsePathInfoJSON/empty_input224=== RUN TestParsePathInfoJSON/whitespace_only225=== PAUSE TestParsePathInfoJSON/whitespace_only226=== RUN TestParsePathInfoJSON/invalid_JSON227=== PAUSE TestParsePathInfoJSON/invalid_JSON228--- PASS: TestDoWithRetry_FinalResponseBodyReadable (0.01s)229--- PASS: TestDoServerRequestAttachesToken (0.02s)230=== CONT TestEncodeNixBase32231=== RUN TestEncodeNixBase32/test_string_hash232=== PAUSE TestEncodeNixBase32/test_string_hash233=== RUN TestEncodeNixBase32/empty_input234=== PAUSE TestEncodeNixBase32/empty_input235=== CONT TestRunGarbageCollection_FinishedOnAnotherReplica236=== CONT TestEncodeNixBase32WithRealHash237--- PASS: TestEncodeNixBase32WithRealHash (0.00s)238=== CONT TestRunGarbageCollection_NotFoundAfterLocalRun239=== CONT TestFileTokenMissing240--- PASS: TestScriptTokenBadJSON (0.02s)241=== CONT TestStreamPushReportsSkippedPaths242--- PASS: TestStreamPushReportsSkippedPaths (0.00s)243=== CONT TestDumpPathSingleFile244=== CONT TestDumpPathWriterError245=== RUN TestPathInfoCACompatibility/old_string_format_-_text246=== PAUSE TestPathInfoCACompatibility/old_string_format_-_text247=== RUN TestConvertHashToNix32/SRI_format_to_Nix32248=== PAUSE TestConvertHashToNix32/SRI_format_to_Nix32249=== RUN TestGetStorePathHash/valid_store_path250=== RUN TestFilterOversizedClosures/no_limit_keeps_everything251=== RUN TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths252=== RUN TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)253=== RUN TestRateLimiterFeedback/503_enables_limiter254=== RUN TestPathInfoCACompatibility/old_string_format_-_fixed_recursive255=== PAUSE TestPathInfoCACompatibility/old_string_format_-_fixed_recursive256=== RUN TestPathInfoCACompatibility/new_structured_format_-_text257=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_text258=== RUN TestPathInfoCACompatibility/new_structured_format_-_nar_method259=== PAUSE TestPathInfoCACompatibility/new_structured_format_-_nar_method260=== RUN TestConvertHashToNix32/already_Nix32_format261=== PAUSE TestConvertHashToNix32/already_Nix32_format262=== RUN TestConvertHashToNix32/invalid_format263=== PAUSE TestConvertHashToNix32/invalid_format264=== PAUSE TestGetStorePathHash/valid_store_path265=== RUN TestGetStorePathHash/basename_without_hyphen_should_error266=== PAUSE TestGetStorePathHash/basename_without_hyphen_should_error267=== RUN TestGetStorePathHash/hash_with_invalid_characters_should_error268=== PAUSE TestGetStorePathHash/hash_with_invalid_characters_should_error269=== RUN TestGetStorePathHash/hash_with_wrong_length_should_error270=== PAUSE TestGetStorePathHash/hash_with_wrong_length_should_error271=== PAUSE TestFilterOversizedClosures/no_limit_keeps_everything272=== RUN TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped273=== PAUSE TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped274=== RUN TestFilterOversizedClosures/all_closures_skipped275=== PAUSE TestFilterOversizedClosures/all_closures_skipped276=== PAUSE TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths277=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)278=== RUN TestPathInfoHashCompatibility/old_string_format_with_colon279=== PAUSE TestPathInfoHashCompatibility/old_string_format_with_colon280=== RUN TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI281=== PAUSE TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI282=== RUN TestPathInfoHashCompatibility/new_structured_format_with_sha512283=== PAUSE TestPathInfoHashCompatibility/new_structured_format_with_sha5122842026/11/12 00:43:16 WARN Rate limiter backed off name=server-test rate=52852026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=2 backoff=0s status=503 url=http://127.0.0.1:566022862026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:56598287=== PAUSE TestRateLimiterFeedback/503_enables_limiter288=== RUN TestRateLimiterFeedback/200_does_not_enable_limiter289=== PAUSE TestRateLimiterFeedback/200_does_not_enable_limiter290=== RUN TestRateLimiterFeedback/400_does_not_enable_limiter291--- PASS: TestUploadMultipart_ReturnsAfterDroppedBody (0.01s)292=== CONT TestSetClientTLSDoesNotMutateDefaultTransport293--- PASS: TestDoWithRetry_BodyReplayedViaGetBody (0.01s)294=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=false295=== RUN TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary296=== CONT TestStreamPushBatchesUnderLoad/oneAtATime=true297=== CONT TestRegisterUploadedObjectReusesConnections298=== CONT TestDumpPathMatchesNix299=== PAUSE TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary300=== CONT TestPartSizeForNAR/zero_stays_at_minimum301=== CONT TestPartSizeForNAR/1_TiB302=== CONT TestPartSizeForNAR/115_GiB_needs_larger_parts303=== CONT TestPartSizeForNAR/80_GiB_fits_at_minimum304=== CONT TestPartSizeForNAR/small_stays_at_minimum305=== CONT TestPartSizeForNAR/capped_at_5_GiB306=== CONT TestPartSizeForNAR/5_TiB_S3_max_object307--- PASS: TestPartSizeForNAR (0.00s)308 --- PASS: TestPartSizeForNAR/zero_stays_at_minimum (0.00s)309 --- PASS: TestPartSizeForNAR/1_TiB (0.00s)310 --- PASS: TestPartSizeForNAR/115_GiB_needs_larger_parts (0.00s)311 --- PASS: TestPartSizeForNAR/80_GiB_fits_at_minimum (0.00s)312 --- PASS: TestPartSizeForNAR/small_stays_at_minimum (0.00s)313 --- PASS: TestPartSizeForNAR/capped_at_5_GiB (0.00s)314 --- PASS: TestPartSizeForNAR/5_TiB_S3_max_object (0.00s)315=== CONT TestSetClientTLSErrors/missing_cert_file316=== CONT TestSetClientTLSErrors/missing_ca_file317=== CONT TestSetClientTLSErrors/invalid_ca_file318=== CONT TestSetClientTLSErrors/missing_key_file319--- PASS: TestSetClientTLSErrors (0.01s)320 --- PASS: TestSetClientTLSErrors/missing_cert_file (0.00s)321 --- PASS: TestSetClientTLSErrors/missing_ca_file (0.00s)322 --- PASS: TestSetClientTLSErrors/invalid_ca_file (0.00s)323 --- PASS: TestSetClientTLSErrors/missing_key_file (0.00s)324=== CONT TestParsePathInfoJSON/Nix_format325=== CONT TestEncodeNixBase32/test_string_hash326=== CONT TestParsePathInfoJSON/whitespace_only327=== CONT TestParsePathInfoJSON/empty_input328=== CONT TestParsePathInfoJSON/Lix_format329=== CONT TestEncodeNixBase32/empty_input330--- PASS: TestEncodeNixBase32 (0.00s)331 --- PASS: TestEncodeNixBase32/test_string_hash (0.00s)332 --- PASS: TestEncodeNixBase32/empty_input (0.00s)333=== CONT TestPathInfoCACompatibility/null_ca_field334=== CONT TestConvertHashToNix32/SRI_format_to_Nix32335=== CONT TestGetStorePathHash/valid_store_path336=== CONT TestFilterOversizedClosures/no_limit_keeps_everything337=== CONT TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths338=== CONT TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI)339=== CONT TestGetStorePathHash/hash_with_wrong_length_should_error340=== CONT TestPathInfoCACompatibility/old_string_format_-_fixed_recursive341=== CONT TestPathInfoCACompatibility/old_string_format_-_text342=== CONT TestPathInfoCACompatibility/new_structured_format_-_nar_method343=== CONT TestPathInfoCACompatibility/new_structured_format_-_text344--- PASS: TestPathInfoCACompatibility (0.01s)345 --- PASS: TestPathInfoCACompatibility/null_ca_field (0.00s)346 --- PASS: TestPathInfoCACompatibility/old_string_format_-_fixed_recursive (0.00s)347 --- PASS: TestPathInfoCACompatibility/old_string_format_-_text (0.00s)348 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_nar_method (0.00s)349 --- PASS: TestPathInfoCACompatibility/new_structured_format_-_text (0.00s)350=== CONT TestConvertHashToNix32/invalid_format351=== CONT TestConvertHashToNix32/already_Nix32_format352--- PASS: TestConvertHashToNix32 (0.01s)353 --- PASS: TestConvertHashToNix32/SRI_format_to_Nix32 (0.00s)354 --- PASS: TestConvertHashToNix32/invalid_format (0.00s)355 --- PASS: TestConvertHashToNix32/already_Nix32_format (0.00s)356=== CONT TestGetStorePathHash/hash_with_invalid_characters_should_error357=== CONT TestGetStorePathHash/basename_without_hyphen_should_error358--- PASS: TestGetStorePathHash (0.00s)359 --- PASS: TestGetStorePathHash/valid_store_path (0.00s)360 --- PASS: TestGetStorePathHash/hash_with_wrong_length_should_error (0.00s)361 --- PASS: TestGetStorePathHash/hash_with_invalid_characters_should_error (0.00s)362 --- PASS: TestGetStorePathHash/basename_without_hyphen_should_error (0.00s)363=== CONT TestFilterOversizedClosures/all_closures_skipped3642026/11/12 00:43:16 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper nar_size=100 max_nar_size=50365=== CONT TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped3662026/11/12 00:43:16 WARN Skipping closure: path exceeds server max NAR size top_level_path=/nix/store/cccccccccccccccccccccccccccccccc-wrapper oversized_path=/nix/store/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb-vm-image nar_size=5000 max_nar_size=2000367--- PASS: TestFilterOversizedClosures (0.00s)368 --- PASS: TestFilterOversizedClosures/no_limit_keeps_everything (0.00s)369 --- PASS: TestFilterOversizedClosures/all_closures_skipped (0.00s)370 --- PASS: TestFilterOversizedClosures/closure_with_oversized_dependency_is_skipped (0.00s)371=== CONT TestPathInfoHashCompatibility/new_structured_format_with_sha512372=== CONT TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths373--- PASS: TestParsePathInfoJSONMultiplePaths (0.02s)374 --- PASS: TestParsePathInfoJSONMultiplePaths/Nix_multiple_paths (0.00s)375 --- PASS: TestParsePathInfoJSONMultiplePaths/Lix_multiple_paths (0.00s)376=== CONT TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI377=== CONT TestPathInfoHashCompatibility/old_string_format_with_colon378--- PASS: TestPathInfoHashCompatibility (0.00s)379 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_dash_(SRI) (0.00s)380 --- PASS: TestPathInfoHashCompatibility/new_structured_format_with_sha512 (0.00s)381 --- PASS: TestPathInfoHashCompatibility/new_structured_format_-_converts_to_SRI (0.00s)382 --- PASS: TestPathInfoHashCompatibility/old_string_format_with_colon (0.00s)383=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part384=== CONT TestStaticToken385--- PASS: TestStaticToken (0.00s)386=== CONT TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary387=== CONT TestStreamPushStopsWhenCanceled388=== CONT TestUploadBuildLog_FileBodyReplayedOnRetry389=== CONT TestSetClientTLS390=== PAUSE TestRateLimiterFeedback/400_does_not_enable_limiter391=== CONT TestRateLimiterFeedback/429_enables_limiter392--- PASS: TestStreamPushBatchesUnderLoad (0.00s)393 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=false (0.00s)394 --- PASS: TestStreamPushBatchesUnderLoad/oneAtATime=true (0.00s)395=== CONT TestParsePathInfoJSON/invalid_JSON396--- PASS: TestFileTokenMissing (0.01s)397--- PASS: TestParsePathInfoJSON (0.01s)398 --- PASS: TestParsePathInfoJSON/Nix_format (0.00s)399 --- PASS: TestParsePathInfoJSON/whitespace_only (0.00s)400 --- PASS: TestParsePathInfoJSON/empty_input (0.00s)401 --- PASS: TestParsePathInfoJSON/Lix_format (0.00s)402 --- PASS: TestParsePathInfoJSON/invalid_JSON (0.00s)403=== CONT TestRateLimiterFeedback/400_does_not_enable_limiter404=== CONT TestRateLimiterFeedback/200_does_not_enable_limiter405--- PASS: TestScriptTokenEmptyToken (0.02s)406=== CONT TestRateLimiterFeedback/503_enables_limiter4072026/11/12 00:43:16 INFO Garbage collection started4082026/11/12 00:43:16 INFO Garbage collection started409--- PASS: TestStreamPushStopsWhenCanceled (0.00s)410=== RUN TestSetClientTLS/rejects_connection_without_client_cert411=== PAUSE TestSetClientTLS/rejects_connection_without_client_cert412=== RUN TestSetClientTLS/succeeds_with_client_cert_and_CA413=== PAUSE TestSetClientTLS/succeeds_with_client_cert_and_CA414=== RUN TestSetClientTLS/preserves_debug_logging_transport415=== PAUSE TestSetClientTLS/preserves_debug_logging_transport416=== CONT TestSetClientTLS/rejects_connection_without_client_cert417=== CONT TestSetClientTLS/preserves_debug_logging_transport4182026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=1 backoff=1ms status=502 url=http://127.0.0.1:56598419=== CONT TestSetClientTLS/succeeds_with_client_cert_and_CA4202026/11/12 00:43:16 WARN Rate limiter enabled after throttle name=s3-test rate=54212026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:56627422--- PASS: TestStreamPushRequestLine (0.03s)423--- PASS: TestUploadMultipart_FailedProducerIsNotCompleted (0.01s)424 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/inside_a_part (0.01s)425 --- PASS: TestUploadMultipart_FailedProducerIsNotCompleted/on_a_part_boundary (0.01s)426--- PASS: TestRunGarbageCollection_NotFoundAfterLocalRun (0.02s)4272026/11/12 00:43:16 WARN Rate limiter enabled after throttle name=server-test rate=54282026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=429 url=http://127.0.0.1:566224292026/11/12 00:43:16 INFO Garbage collection progress phase=running_on_another_replica failed_uploads_deleted=0 old_closures_deleted=0 objects_marked=0 objects_deleted=0 objects_failed=04302026/11/12 00:43:16 WARN Rate limiter backed off name=server-test rate=54312026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=2 backoff=1ms status=502 url=http://127.0.0.1:565984322026/11/12 00:43:16 WARN Rate limiter enabled after throttle name=server-test rate=54332026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=1 backoff=0s status=503 url=http://127.0.0.1:56628434--- PASS: TestUploadBuildLog_FileBodyReplayedOnRetry (0.02s)4352026/11/12 00:43:16 WARN Rate limiter backed off name=server-test rate=54362026/11/12 00:43:16 WARN Garbage collection finished on another replica. Its statistics are not available here437--- PASS: TestRunGarbageCollection_FinishedOnAnotherReplica (0.02s)438--- PASS: TestRateLimiterFeedback (0.01s)439 --- PASS: TestRateLimiterFeedback/400_does_not_enable_limiter (0.01s)440 --- PASS: TestRateLimiterFeedback/200_does_not_enable_limiter (0.01s)441 --- PASS: TestRateLimiterFeedback/429_enables_limiter (0.02s)442 --- PASS: TestRateLimiterFeedback/503_enables_limiter (0.02s)443--- PASS: TestSetClientTLSDoesNotMutateDefaultTransport (0.02s)4442026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=3 backoff=1ms status=502 url=http://127.0.0.1:565984452026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=4 backoff=1ms status=502 url=http://127.0.0.1:565984462026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=5 backoff=1ms status=502 url=http://127.0.0.1:565984472026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=6 backoff=1ms status=502 url=http://127.0.0.1:565984482026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=7 backoff=1ms status=502 url=http://127.0.0.1:565984492026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=8 backoff=1ms status=502 url=http://127.0.0.1:565984502026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=9 backoff=1ms status=502 url=http://127.0.0.1:565984512026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=10 backoff=1ms status=502 url=http://127.0.0.1:565984522026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=11 backoff=1ms status=502 url=http://127.0.0.1:565984532026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=12 backoff=1ms status=502 url=http://127.0.0.1:565984542026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=13 backoff=1ms status=502 url=http://127.0.0.1:565984552026/11/12 00:43:16 http: TLS handshake error from 127.0.0.1:56690: read tcp 127.0.0.1:56688->127.0.0.1:56690: use of closed network connection4562026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=14 backoff=1ms status=502 url=http://127.0.0.1:56598457--- PASS: TestSetClientTLS (0.01s)458 --- PASS: TestSetClientTLS/preserves_debug_logging_transport (0.01s)459 --- PASS: TestSetClientTLS/succeeds_with_client_cert_and_CA (0.01s)460 --- PASS: TestSetClientTLS/rejects_connection_without_client_cert (0.04s)461--- PASS: TestRegisterUploadedObjectReusesConnections (0.05s)4622026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=15 backoff=1ms status=502 url=http://127.0.0.1:565984632026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=16 backoff=1ms status=502 url=http://127.0.0.1:565984642026/11/12 00:43:16 WARN Request returned retryable status, retrying attempt=17 backoff=1ms status=502 url=http://127.0.0.1:56598465--- PASS: TestScriptTokenCachesUntilRefresh (0.09s)466--- PASS: TestScriptTokenNoExpiryRerunsEveryCall (0.09s)467--- PASS: TestRetryTimeoutOutlastsMaxRetries (0.08s)468--- PASS: TestDumpPathWriterError (0.07s)469--- PASS: TestDumpPathSingleFile (0.09s)470--- PASS: TestCaseHackSuffix (0.09s)4712026/11/12 00:43:16 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:56603/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:56603/api/uploads/complete4722026/11/12 00:43:16 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4732026/11/12 00:43:16 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:56603/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:56603/api/uploads/complete4742026/11/12 00:43:16 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"4752026/11/12 00:43:16 WARN Request failed, retrying attempt=1 backoff=100ms error="Post \"http://127.0.0.1:56603/api/uploads/complete\": context deadline exceeded" url=http://127.0.0.1:56603/api/uploads/complete4762026/11/12 00:43:16 WARN Failed to register uploaded object key=abc.narinfo error="sending request: context canceled during retry: context deadline exceeded"477--- PASS: TestRegisterUploadedObject_BoundedAgainstSilentServer (0.11s)478--- PASS: TestDumpPathMatchesNix (0.16s)479--- PASS: TestUploadMultipart_PartsInParallel (0.64s)480--- PASS: TestRateLimiterFeedback_400DoesNotCountAsSuccess (1.00s)481PASS482Running server tests...483The files belonging to this database system will be owned by user "_tribuchetbld3".484This user must also own the server process.485486The database cluster will be initialized with locale "C".487The default database encoding has accordingly been set to "SQL_ASCII".488The default text search configuration will be set to "english".489490Data page checksums are enabled.491492creating directory /private/var/tb/3/scratch/cf270c76132c2e86/build/postgres1704635087/data ... ok493creating subdirectories ... ok494selecting dynamic shared memory implementation ... posix495selecting default "max_connections" ... 100496selecting default "shared_buffers" ... 128MB497selecting default time zone ... UTC498creating configuration files ... ok499running bootstrap script ... ok500performing post-bootstrap initialization ... ok501syncing data to disk ... ok502503initdb: warning: enabling "trust" authentication for local connections504initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.505506Success. You can now start the database server using:507508 pg_ctl -D /private/var/tb/3/scratch/cf270c76132c2e86/build/postgres1704635087/data -l logfile start5095102026-11-12 00:43:18.247 UTC [40635] LOG: starting PostgreSQL 18.6 on aarch64-apple-darwin25.6.0, compiled by clang version 21.1.8, 64-bit5112026-11-12 00:43:18.248 UTC [40635] LOG: listening on Unix socket "/private/var/tb/3/scratch/cf270c76132c2e86/build/postgres1704635087/.s.PGSQL.5432"5122026-11-12 00:43:18.252 UTC [40642] LOG: database system was shut down at 2026-11-12 00:43:18 UTC5132026-11-12 00:43:18.252 UTC [40643] FATAL: the database system is starting up514/private/var/tb/3/scratch/cf270c76132c2e86/build/postgres1704635087:5432 - rejecting connections5152026-11-12 00:43:18.254 UTC [40635] LOG: database system is ready to accept connections516/private/var/tb/3/scratch/cf270c76132c2e86/build/postgres1704635087:5432 - accepting connections517{"timestamp":"2026-11-12T00:43:18.522892Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"1dd05367-fadc-4a2e-8bcd-76bf8afa91ef","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":12,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(8)"}518{"timestamp":"2026-11-12T00:43:18.629887Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0410c8f0-5203-4ac6-a205-de2c426d4c2f","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(8)"}519{"timestamp":"2026-11-12T00:43:18.732668Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"1a487ac7-d7f0-40a6-83ee-e5af1e64c2d2","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(4)"}520{"timestamp":"2026-11-12T00:43:18.834032Z","level":"ERROR","message":"HTTP request completed","event":"http_request_completed","component":"server","subsystem":"http","request_id":"0dd47d0f-30a6-4568-b87d-dfcd2d1433cd","trace_id":"unknown","span_id":"unknown","peer_addr":"127.0.0.1","method":"GET","uri":"/health/ready","status_code":503,"suppressed_errors":0,"duration_ms":0,"result":"server_error","target":"rustfs::server::http","filename":"rustfs/src/server/layer.rs","line_number":502,"threadName":"rustfs-worker","threadId":"ThreadId(5)"}521=== RUN TestAbortSucceeded522=== PAUSE TestAbortSucceeded523=== RUN TestService_AuthMiddleware524=== PAUSE TestService_AuthMiddleware525=== RUN TestService_AuthMiddleware_MTLSProxyHeader526=== PAUSE TestService_AuthMiddleware_MTLSProxyHeader527=== RUN TestService_AuthMiddleware_MTLSBoundSubjects528=== PAUSE TestService_AuthMiddleware_MTLSBoundSubjects529=== RUN TestService_ReadAuthMiddleware530=== PAUSE TestService_ReadAuthMiddleware531=== RUN TestService_AuthMiddleware_OIDC532=== PAUSE TestService_AuthMiddleware_OIDC533=== RUN TestService_RequireScope_OIDC534=== PAUSE TestService_RequireScope_OIDC535=== RUN TestService_ReadScope_PublicByDefault536=== PAUSE TestService_ReadScope_PublicByDefault537=== RUN TestCacheConfigHandler538=== PAUSE TestCacheConfigHandler539=== RUN TestCacheStatsHandler540=== PAUSE TestCacheStatsHandler541=== RUN TestClientCADerivations542=== PAUSE TestClientCADerivations543=== RUN TestClientErrorHandling544=== PAUSE TestClientErrorHandling545=== RUN TestClientIntegration546=== PAUSE TestClientIntegration547=== RUN TestClientMultipleUploads548=== PAUSE TestClientMultipleUploads549=== RUN TestClientWithDependencies550=== PAUSE TestClientWithDependencies551=== RUN TestClientSharedPathCommittedMidPush552=== PAUSE TestClientSharedPathCommittedMidPush553=== RUN TestPinProtectsFromGC554=== PAUSE TestPinProtectsFromGC555=== RUN TestClientPushesUseOnePush556=== PAUSE TestClientPushesUseOnePush557=== RUN TestClientFallsBackToClosures558=== PAUSE TestClientFallsBackToClosures559=== RUN TestDeleteClosuresRacingPins560=== PAUSE TestDeleteClosuresRacingPins561=== RUN TestConcurrentCommitsSharingObjectsDoNotDeadlock562=== PAUSE TestConcurrentCommitsSharingObjectsDoNotDeadlock563=== RUN TestResolveDBConnectionString564=== PAUSE TestResolveDBConnectionString565=== RUN TestConnectWaitsForAPeerMigration566=== PAUSE TestConnectWaitsForAPeerMigration567=== RUN TestLeadElectsOneAndHandsOver568=== PAUSE TestLeadElectsOneAndHandsOver569=== RUN TestLeadIncumbentWinsAfterRestart5702026/11/12 00:43:19 INFO lead: acquired remote=192.0.2.1:1234 generation=15712026/11/12 00:43:19 INFO lead: released remote=192.0.2.1:1234 generation=15722026/11/12 00:43:20 INFO lead: acquired remote=192.0.2.1:1234 generation=25732026/11/12 00:43:21 INFO lead: released remote=192.0.2.1:1234 generation=2574--- PASS: TestLeadIncumbentWinsAfterRestart (2.16s)575=== RUN TestLeadUnannouncedStandbyYieldsToIncumbent576=== PAUSE TestLeadUnannouncedStandbyYieldsToIncumbent577=== RUN TestLeadIncumbentAnnouncesAtOnce578=== PAUSE TestLeadIncumbentAnnouncesAtOnce579=== RUN TestLeadGenerationGrowsWithEveryGrant580=== PAUSE TestLeadGenerationGrowsWithEveryGrant581=== RUN TestLeadEndsOnShutdown582=== PAUSE TestLeadEndsOnShutdown583=== RUN TestLeadEndsWhenItsConnectionHangs5842026/11/12 00:43:21 INFO lead: acquired remote=192.0.2.1:1234 generation=15852026-11-12 00:43:22.729 UTC [40977] FATAL: terminating connection due to administrator command5862026/11/12 00:43:22 INFO lead: acquired remote=192.0.2.1:1234 generation=25872026/11/12 00:43:23 WARN lead: lock connection lost error="timeout: context deadline exceeded"5882026/11/12 00:43:23 INFO lead: released remote=192.0.2.1:1234 generation=15892026/11/12 00:43:24 INFO lead: released remote=192.0.2.1:1234 generation=2590--- PASS: TestLeadEndsWhenItsConnectionHangs (3.03s)591=== RUN TestGCAdvisoryLockBlocksConcurrentRun592=== PAUSE TestGCAdvisoryLockBlocksConcurrentRun593=== RUN TestGCBugBareHashReferences594=== PAUSE TestGCBugBareHashReferences595=== RUN TestGCMetrics596=== PAUSE TestGCMetrics597=== RUN TestPushDedupSurvivesConcurrentGC598=== PAUSE TestPushDedupSurvivesConcurrentGC599=== RUN TestDeduplicatedObjectsRecordedAsPending600=== PAUSE TestDeduplicatedObjectsRecordedAsPending601=== RUN TestGCSweepSkipsPendingObjects602=== PAUSE TestGCSweepSkipsPendingObjects603=== RUN TestTombstonedObjectOfferedWithoutWaiting604=== PAUSE TestTombstonedObjectOfferedWithoutWaiting605=== RUN TestGCSweepDeliversEachKeyOnce606=== PAUSE TestGCSweepDeliversEachKeyOnce607=== RUN TestCreatePendingClosureVerifyS3FailureReleasesConnection608=== PAUSE TestCreatePendingClosureVerifyS3FailureReleasesConnection609=== RUN TestForceGCDuringPushOffersSweptObject610=== PAUSE TestForceGCDuringPushOffersSweptObject611=== RUN TestFailedS3DeleteKeepsTombstone612=== PAUSE TestFailedS3DeleteKeepsTombstone613=== RUN TestSweepSkipsRowsLockedByPush614=== PAUSE TestSweepSkipsRowsLockedByPush615=== RUN TestRegisterCompletedObjectNeedsPendingRow616=== PAUSE TestRegisterCompletedObjectNeedsPendingRow617=== RUN TestGetExistingObjectsLocksRows618=== PAUSE TestGetExistingObjectsLocksRows619=== RUN TestSweepRechecksPendingAfterLocking620=== PAUSE TestSweepRechecksPendingAfterLocking621=== RUN TestPushWaitsForSweepHoldingRow622=== PAUSE TestPushWaitsForSweepHoldingRow623=== RUN TestCommitRacingPendingCleanupKeepsObjects624=== PAUSE TestCommitRacingPendingCleanupKeepsObjects625=== RUN TestGCEndsOnShutdown626=== PAUSE TestGCEndsOnShutdown627=== RUN TestGCTaskStore_StartNew628=== PAUSE TestGCTaskStore_StartNew629=== RUN TestGCTaskStore_DeduplicateSameParams630=== PAUSE TestGCTaskStore_DeduplicateSameParams631=== RUN TestGCTaskStore_ConflictDifferentParams632=== PAUSE TestGCTaskStore_ConflictDifferentParams633=== RUN TestGCTaskStore_GetEmpty634=== PAUSE TestGCTaskStore_GetEmpty635=== RUN TestGCTaskStore_GetReturnsLatest636=== PAUSE TestGCTaskStore_GetReturnsLatest637=== RUN TestGCTaskStore_CompletedAllowsNewTask638=== PAUSE TestGCTaskStore_CompletedAllowsNewTask639=== RUN TestGCTaskStore_PhaseUpdates640=== PAUSE TestGCTaskStore_PhaseUpdates641=== RUN TestGCTaskStore_Fail642=== PAUSE TestGCTaskStore_Fail643=== RUN TestGracefulShutdownDrainsInflight644=== PAUSE TestGracefulShutdownDrainsInflight645=== RUN TestService_healthCheckHandler646=== PAUSE TestService_healthCheckHandler647=== RUN TestService_readinessHandler648=== PAUSE TestService_readinessHandler649=== RUN TestGenerateLandingPage650=== PAUSE TestGenerateLandingPage651=== RUN TestEnsureAbortUploadsRule652=== PAUSE TestEnsureAbortUploadsRule653=== RUN TestCacheConfigHandlerMaxNarSize654=== PAUSE TestCacheConfigHandlerMaxNarSize655=== RUN TestCreatePendingClosureRejectsOversizedNAR656=== PAUSE TestCreatePendingClosureRejectsOversizedNAR657=== RUN TestNARDeduplicationMetadataUploadBug658=== PAUSE TestNARDeduplicationMetadataUploadBug659=== RUN TestMetricsInventory660=== PAUSE TestMetricsInventory661=== RUN TestMetricsCommit662=== PAUSE TestMetricsCommit663=== RUN TestService_NativeMTLS664=== PAUSE TestService_NativeMTLS665=== RUN TestServerTLSConfig666=== PAUSE TestServerTLSConfig667=== RUN TestMultipartCleanup668=== PAUSE TestMultipartCleanup669=== RUN TestMultipartUploadAbortedWhenCancelledBeforeRecorded670=== PAUSE TestMultipartUploadAbortedWhenCancelledBeforeRecorded671=== RUN TestPendingCleanupUsesOneCutoff672=== PAUSE TestPendingCleanupUsesOneCutoff673=== RUN TestObjectStatsTrigger674=== PAUSE TestObjectStatsTrigger675=== RUN TestObjectStatsUpdatedOncePerCommit676=== PAUSE TestObjectStatsUpdatedOncePerCommit677=== RUN TestOrphanedObjectsGC678=== PAUSE TestOrphanedObjectsGC679=== RUN TestOrphanedObjectsGCStressTest680=== PAUSE TestOrphanedObjectsGCStressTest681=== RUN TestCreatePin_ReservedPins682=== PAUSE TestCreatePin_ReservedPins683=== RUN TestPresentReportsOnlyClosureRoots684=== PAUSE TestPresentReportsOnlyClosureRoots685=== RUN TestPresentRefreshesOnlyStaleClosures686=== PAUSE TestPresentRefreshesOnlyStaleClosures687=== RUN TestPresentNotReportedWhileGCDeletesClosure688=== PAUSE TestPresentNotReportedWhileGCDeletesClosure689=== RUN TestParseSingleRange690=== PAUSE TestParseSingleRange691=== RUN TestProxyHeadersOnlyTrustedOnSocket6922026/11/12 00:43:24 INFO Starting HTTP server address=proxy.sock6932026/11/12 00:43:24 INFO Starting HTTP server address=127.0.0.1:567326942026/11/12 00:43:24 WARN mTLS auth: subject not in bound subjects subject="CN=someone"6952026/11/12 00:43:24 INFO Shutdown signal received, draining in-flight requests timeout=10s696--- PASS: TestProxyHeadersOnlyTrustedOnSocket (0.25s)697=== RUN TestIsValidCachePath698=== PAUSE TestIsValidCachePath699=== RUN TestReadProxyNarinfo700=== PAUSE TestReadProxyNarinfo701=== RUN TestReadProxyNarinfoAlreadyDecompressed702=== PAUSE TestReadProxyNarinfoAlreadyDecompressed703=== RUN TestReadProxyContentNegotiation704=== PAUSE TestReadProxyContentNegotiation705=== RUN TestReadProxyNarStreaming706=== PAUSE TestReadProxyNarStreaming707=== RUN TestReadProxy404708=== PAUSE TestReadProxy404709=== RUN TestReadProxyInvalidPath710=== PAUSE TestReadProxyInvalidPath711=== RUN TestReadProxyHead712=== PAUSE TestReadProxyHead713=== RUN TestReadProxyOutlastsServerWriteTimeout714=== PAUSE TestReadProxyOutlastsServerWriteTimeout715=== RUN TestReadProxyConditionalGet716=== PAUSE TestReadProxyConditionalGet717=== RUN TestReadProxyRootRedirectsToIndexHTML718=== PAUSE TestReadProxyRootRedirectsToIndexHTML719=== RUN TestReadProxyDisabled720=== PAUSE TestReadProxyDisabled721=== RUN TestReadRedirectNar722=== PAUSE TestReadRedirectNar723=== RUN TestReadRedirectKeepsNarinfoProxied724=== PAUSE TestReadRedirectKeepsNarinfoProxied725=== RUN TestReadProxyRangeRequest726=== PAUSE TestReadProxyRangeRequest727=== RUN TestReadRedirectUsesPublicS3URL728=== PAUSE TestReadRedirectUsesPublicS3URL729=== RUN TestPush_OverlappingRootsStoreOneRowPerKey730=== PAUSE TestPush_OverlappingRootsStoreOneRowPerKey731=== RUN TestPush_CompleteCommitsEveryRoot732=== PAUSE TestPush_CompleteCommitsEveryRoot733=== RUN TestPush_RepeatedRootCommitsOnce734=== PAUSE TestPush_RepeatedRootCommitsOnce735=== RUN TestPush_CompleteWaitsForCleanupOfThePush736=== PAUSE TestPush_CompleteWaitsForCleanupOfThePush737=== RUN TestPush_RootsAreLockedInByteOrder738=== PAUSE TestPush_RootsAreLockedInByteOrder739=== RUN TestPush_SkippedKeySurvivesGCBeforeCommit740=== PAUSE TestPush_SkippedKeySurvivesGCBeforeCommit741=== RUN TestPush_RejectsBadRequests742=== PAUSE TestPush_RejectsBadRequests743=== RUN TestPush_SignsNarinfosOfItsPendingObjects744=== PAUSE TestPush_SignsNarinfosOfItsPendingObjects745=== RUN TestCommitCanBeRepeatedUntilCleanup746=== PAUSE TestCommitCanBeRepeatedUntilCleanup747=== RUN TestSharedNarBothUploadsComplete748=== PAUSE TestSharedNarBothUploadsComplete749=== RUN TestCompleteMultipartUpload_ErrorButObjectExists750=== PAUSE TestCompleteMultipartUpload_ErrorButObjectExists751=== RUN TestCompletedNarNotReofferedAcrossClosures752=== PAUSE TestCompletedNarNotReofferedAcrossClosures753=== RUN TestPresignedUploadRegisteredBeforeCommit754=== PAUSE TestPresignedUploadRegisteredBeforeCommit755=== RUN TestService_Rustfstest756=== PAUSE TestService_Rustfstest757=== RUN TestParseSize758=== PAUSE TestParseSize759=== RUN TestSkippedUploadsHandler760=== PAUSE TestSkippedUploadsHandler761=== RUN TestSystemdListenerNotActivated762--- PASS: TestSystemdListenerNotActivated (0.00s)763=== RUN TestWatchdogBeatsWhenHealthy764--- PASS: TestWatchdogBeatsWhenHealthy (0.03s)765=== RUN TestWatchdogSkipsWhenUnhealthy7662026/11/12 00:43:24 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7672026/11/12 00:43:24 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7682026/11/12 00:43:24 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7692026/11/12 00:43:24 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7702026/11/12 00:43:24 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7712026/11/12 00:43:24 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7722026/11/12 00:43:24 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7732026/11/12 00:43:24 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7742026/11/12 00:43:24 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"7752026/11/12 00:43:24 WARN watchdog liveness check failed, skipping heartbeat error="context deadline exceeded"776--- PASS: TestWatchdogSkipsWhenUnhealthy (0.21s)777=== RUN TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle778=== PAUSE TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle779=== RUN TestProxyWriteTimeout780=== PAUSE TestProxyWriteTimeout781=== RUN TestPendingClosureWriteTimeout782=== PAUSE TestPendingClosureWriteTimeout783=== RUN TestIsValidUploadKey784=== PAUSE TestIsValidUploadKey785=== RUN TestUploadHandlersRejectInvalidKeys786=== PAUSE TestUploadHandlersRejectInvalidKeys787=== RUN TestUploadHandlersRejectOversizedBody788=== PAUSE TestUploadHandlersRejectOversizedBody789=== RUN TestService_cleanupPendingClosuresHandler790=== PAUSE TestService_cleanupPendingClosuresHandler791=== RUN TestService_createPendingClosureHandler792=== PAUSE TestService_createPendingClosureHandler793=== RUN TestService_verifyS3Integrity794=== PAUSE TestService_verifyS3Integrity795=== RUN TestService_verifyS3ReuploadsStaleNarinfo796=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo797=== RUN TestCompleteMultipartUnregistered798=== PAUSE TestCompleteMultipartUnregistered799=== RUN TestCreatePendingClosure_SmallNARUsesSimplePUT800=== PAUSE TestCreatePendingClosure_SmallNARUsesSimplePUT801=== CONT TestAbortSucceeded802--- PASS: TestAbortSucceeded (0.00s)803=== CONT TestCreatePendingClosure_SmallNARUsesSimplePUT804=== CONT TestNARDeduplicationMetadataUploadBug805=== CONT TestProxyWriteTimeout806=== CONT TestReadProxyOutlastsServerWriteTimeout807=== CONT TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle808=== CONT TestReadRedirectKeepsNarinfoProxied809=== CONT TestPush_SignsNarinfosOfItsPendingObjects810=== RUN TestProxyWriteTimeout/narinfo811=== CONT TestPresentNotReportedWhileGCDeletesClosure812=== CONT TestService_verifyS3ReuploadsStaleNarinfo813=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed814=== CONT TestService_verifyS3Integrity815=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed816=== RUN TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed817=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed818=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost819=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost820=== RUN TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed821=== CONT TestService_createPendingClosureHandler822=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed823=== RUN TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed824=== PAUSE TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed825=== CONT TestService_cleanupPendingClosuresHandler826=== CONT TestPresignedUploadRegisteredBeforeCommit827=== CONT TestUploadHandlersRejectOversizedBody828=== CONT TestUploadHandlersRejectInvalidKeys829=== CONT TestIsValidUploadKey830=== CONT TestPendingClosureWriteTimeout831=== RUN TestPendingClosureWriteTimeout/empty832=== PAUSE TestPendingClosureWriteTimeout/empty833=== RUN TestPendingClosureWriteTimeout/400_objects834=== PAUSE TestPendingClosureWriteTimeout/400_objects835=== RUN TestPendingClosureWriteTimeout/670k_objects836=== PAUSE TestPendingClosureWriteTimeout/670k_objects837=== RUN TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline838=== PAUSE TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline839=== CONT TestReadRedirectNar840=== CONT TestSkippedUploadsHandler841=== CONT TestReadProxyDisabled842=== CONT TestReadProxyRootRedirectsToIndexHTML843=== CONT TestReadProxyConditionalGet844=== CONT TestPush_RejectsBadRequests845=== CONT TestPush_SkippedKeySurvivesGCBeforeCommit8462026/11/12 00:43:24 INFO Client skipped oversized paths paths=3 nar_bytes=5000000000847=== CONT TestPush_RootsAreLockedInByteOrder848=== CONT TestCompleteMultipartUnregistered849=== PAUSE TestProxyWriteTimeout/narinfo850=== RUN TestPresentNotReportedWhileGCDeletesClosure/stale851=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info852=== RUN TestIsValidUploadKey/narinfo853--- PASS: TestSkippedUploadsHandler (0.01s)854=== CONT TestPush_CompleteWaitsForCleanupOfThePush855=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info856=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/stale857=== PAUSE TestIsValidUploadKey/narinfo858=== RUN TestProxyWriteTimeout/1_GiB_nar859=== PAUSE TestProxyWriteTimeout/1_GiB_nar860=== RUN TestProxyWriteTimeout/10_GiB_nar861=== PAUSE TestProxyWriteTimeout/10_GiB_nar862=== RUN TestProxyWriteTimeout/unknown_size863=== RUN TestPresentNotReportedWhileGCDeletesClosure/fresh864=== RUN TestIsValidUploadKey/nar_zst865=== PAUSE TestIsValidUploadKey/nar_zst866=== RUN TestIsValidUploadKey/nar_xz867=== PAUSE TestIsValidUploadKey/nar_xz868=== RUN TestIsValidUploadKey/nar_plain869=== PAUSE TestIsValidUploadKey/nar_plain870=== RUN TestIsValidUploadKey/listing871=== PAUSE TestIsValidUploadKey/listing872=== RUN TestIsValidUploadKey/build_log873=== PAUSE TestIsValidUploadKey/build_log874=== RUN TestIsValidUploadKey/build_log_home-manager_file875=== PAUSE TestIsValidUploadKey/build_log_home-manager_file876=== RUN TestIsValidUploadKey/build_log_plus_in_name877=== PAUSE TestIsValidUploadKey/build_log_plus_in_name878=== RUN TestIsValidUploadKey/build_log_question_mark879=== PAUSE TestIsValidUploadKey/build_log_question_mark880=== RUN TestIsValidUploadKey/build_log_equals881=== PAUSE TestIsValidUploadKey/build_log_equals882=== RUN TestIsValidUploadKey/realisation883=== PAUSE TestProxyWriteTimeout/unknown_size884=== PAUSE TestPresentNotReportedWhileGCDeletesClosure/fresh885=== CONT TestParseSize886--- PASS: TestParseSize (0.00s)887=== CONT TestService_Rustfstest888=== CONT TestPush_RepeatedRootCommitsOnce889=== PAUSE TestIsValidUploadKey/realisation890=== RUN TestIsValidUploadKey/realisation_plus_in_output891=== PAUSE TestIsValidUploadKey/realisation_plus_in_output892=== RUN TestIsValidUploadKey/nix-cache-info893=== PAUSE TestIsValidUploadKey/nix-cache-info894=== RUN TestIsValidUploadKey/index.html895=== PAUSE TestIsValidUploadKey/index.html896=== RUN TestIsValidUploadKey/narinfo_key,_nar_type897=== PAUSE TestIsValidUploadKey/narinfo_key,_nar_type898=== RUN TestIsValidUploadKey/nar_key,_narinfo_type899=== PAUSE TestIsValidUploadKey/nar_key,_narinfo_type900=== RUN TestIsValidUploadKey/listing_key,_narinfo_type901=== PAUSE TestIsValidUploadKey/listing_key,_narinfo_type902=== RUN TestIsValidUploadKey/traversal903=== PAUSE TestIsValidUploadKey/traversal904=== RUN TestIsValidUploadKey/traversal_nar905=== PAUSE TestIsValidUploadKey/traversal_nar906=== RUN TestIsValidUploadKey/absolute907=== PAUSE TestIsValidUploadKey/absolute908=== RUN TestIsValidUploadKey/empty_key909=== PAUSE TestIsValidUploadKey/empty_key910=== RUN TestIsValidUploadKey/unknown_type911=== PAUSE TestIsValidUploadKey/unknown_type912=== CONT TestPush_CompleteCommitsEveryRoot913=== RUN TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal914=== PAUSE TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal915=== RUN TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key916=== PAUSE TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key917=== RUN TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key918=== PAUSE TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key919=== CONT TestCompleteMultipartUpload_ErrorButObjectExists920=== RUN TestUploadHandlersRejectOversizedBody/create_pending_closure921=== PAUSE TestUploadHandlersRejectOversizedBody/create_pending_closure922=== RUN TestUploadHandlersRejectOversizedBody/complete_multipart923=== PAUSE TestUploadHandlersRejectOversizedBody/complete_multipart924=== RUN TestUploadHandlersRejectOversizedBody/request_more_parts925=== PAUSE TestUploadHandlersRejectOversizedBody/request_more_parts926=== CONT TestPush_OverlappingRootsStoreOneRowPerKey9272026/11/12 00:43:25 INFO Received uploads request method=POST path=/api/pending_closures9282026/11/12 00:43:25 INFO Received uploads request method=POST path=/api/pending_closures929--- PASS: TestCreatePendingClosure_SmallNARUsesSimplePUT (1.44s)930=== CONT TestCompletedNarNotReofferedAcrossClosures9312026/11/12 00:43:26 INFO Received complete multipart upload request method=POST path=/api/multipart/complete932--- PASS: TestReadRedirectKeepsNarinfoProxied (1.98s)933=== CONT TestReadRedirectUsesPublicS3URL9342026/11/12 00:43:26 INFO Received uploads request method=POST path=/api/pending_closures935=== NAME TestNARDeduplicationMetadataUploadBug936 metadata_upload_test.go:48: First store path: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestNARDeduplicationMetadataUploadBug1582196060/001/store/6xxz2bpc1gqi6wasmgnxsggjvc9z1xfh-file1.txt9372026/11/12 00:43:27 INFO Received uploads request method=POST path=/api/pending_closures9382026/11/12 00:43:27 INFO Registered completed upload object_key=nar/jjjjjjjjjjjjjjjjjjjjjjjjjjjjjj0100000000000000000000.nar.zst9392026/11/12 00:43:27 INFO Received uploads request method=POST path=/api/pending_closures940--- PASS: TestPresignedUploadRegisteredBeforeCommit (2.77s)941=== CONT TestReadProxyContentNegotiation9422026/11/12 00:43:27 INFO Received push request method=POST path=/api/pushes9432026/11/12 00:43:27 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)9442026/11/12 00:43:27 INFO Uploading 6xxz2bpc1gqi6wasmgnxsggjvc9z1xfh-file1.txt (160B)9452026/11/12 00:43:27 WARN Failed to register uploaded object key=nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst error="server returned 404: 404 page not found\n"9462026/11/12 00:43:27 WARN Failed to register uploaded object key=6xxz2bpc1gqi6wasmgnxsggjvc9z1xfh.ls error="server returned 404: 404 page not found\n"9472026/11/12 00:43:27 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9482026/11/12 00:43:27 INFO Signed narinfos id=1 count=19492026/11/12 00:43:27 INFO Uploading 1 narinfos9502026/11/12 00:43:27 INFO Received push request method=POST path=/api/pushes9512026/11/12 00:43:27 INFO Received complete push request method=POST path=/api/pushes/1/complete9522026/11/12 00:43:27 WARN Failed to register uploaded object key=6xxz2bpc1gqi6wasmgnxsggjvc9z1xfh.narinfo error="server returned 404: 404 page not found\n"9532026/11/12 00:43:27 INFO Completed push id=1 objects=3 duration=21.236ms9542026/11/12 00:43:27 INFO Upload complete. (253ms)9552026/11/12 00:43:27 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign956=== NAME TestNARDeduplicationMetadataUploadBug957 metadata_upload_test.go:54: Retrieved narinfo from S3:958 StorePath: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestNARDeduplicationMetadataUploadBug1582196060/001/store/6xxz2bpc1gqi6wasmgnxsggjvc9z1xfh-file1.txt959 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst960 Compression: zstd961 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf962 NarSize: 160963 References: 964 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf9652026/11/12 00:43:27 INFO Signed narinfos id=1 count=1966 metadata_upload_test.go:55: Retrieved .ls file from S3 (compressed size: 77 bytes)9672026/11/12 00:43:27 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign968 metadata_upload_test.go:55: Decompressed .ls content (64 bytes):969 {"version":1,"root":{"type":"regular","size":44,"narOffset":96}}9702026/11/12 00:43:27 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9712026/11/12 00:43:27 ERROR Failed to sign narinfos signer="index 1" error="signing key cannot be nil"9722026/11/12 00:43:27 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9732026/11/12 00:43:27 ERROR Failed to sign narinfos signer=niks3-test-1 error="signer failed"9742026/11/12 00:43:27 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9752026/11/12 00:43:27 ERROR Signer returned unexpected signature count signer=niks3-test-1 got=0 want=19762026/11/12 00:43:27 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign9772026/11/12 00:43:27 ERROR Signer returned unexpected object key signer=niks3-test-1 object_key=unexpected.narinfo978--- PASS: TestPush_SignsNarinfosOfItsPendingObjects (2.99s)979=== CONT TestReadProxyRangeRequest980=== NAME TestNARDeduplicationMetadataUploadBug981 metadata_upload_test.go:64: Second store path (same content): /private/var/tb/3/scratch/cf270c76132c2e86/build/TestNARDeduplicationMetadataUploadBug1582196060/001/store/z87bpjfy52qnhp1r36v2k5ji1mcgxfq3-file2.txt9822026/11/12 00:43:27 INFO Received cleanup request method=DELETE path=/api/pending_closures9832026/11/12 00:43:27 INFO Aborted multipart uploads count=0 kept=09842026/11/12 00:43:27 INFO Received uploads request method=POST path=/api/pending_closures9852026/11/12 00:43:27 INFO Received cleanup request method=DELETE path=/api/pending_closures9862026/11/12 00:43:27 INFO Received push request method=POST path=/api/pushes9872026/11/12 00:43:27 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)9882026/11/12 00:43:27 INFO Aborted multipart uploads count=1 kept=09892026/11/12 00:43:27 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete9902026-11-12 00:43:27.915 UTC [41760] ERROR: Closure does not exist: id=19912026-11-12 00:43:27.915 UTC [41760] CONTEXT: PL/pgSQL function commit_pending_closure(bigint) line 17 at RAISE9922026-11-12 00:43:27.915 UTC [41760] STATEMENT: -- name: CommitPendingClosure :one993 SELECT commit_pending_closure($1::bigint)::bigint994 995--- PASS: TestService_cleanupPendingClosuresHandler (3.28s)996=== CONT TestReadProxyHead9972026/11/12 00:43:27 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign9982026/11/12 00:43:27 WARN Failed to register uploaded object key=z87bpjfy52qnhp1r36v2k5ji1mcgxfq3.ls error="server returned 404: 404 page not found\n"9992026/11/12 00:43:27 INFO Signed narinfos id=2 count=110002026/11/12 00:43:27 INFO Uploading 1 narinfos10012026/11/12 00:43:28 INFO Received complete push request method=POST path=/api/pushes/2/complete10022026/11/12 00:43:28 WARN Failed to register uploaded object key=z87bpjfy52qnhp1r36v2k5ji1mcgxfq3.narinfo error="server returned 404: 404 page not found\n"10032026/11/12 00:43:28 INFO Completed push id=2 objects=3 duration=15.055458ms10042026/11/12 00:43:28 INFO Upload complete. (257ms)1005=== NAME TestNARDeduplicationMetadataUploadBug1006 metadata_upload_test.go:76: Retrieved narinfo from S3:1007 StorePath: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestNARDeduplicationMetadataUploadBug1582196060/001/store/z87bpjfy52qnhp1r36v2k5ji1mcgxfq3-file2.txt1008 URL: nar/1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf.nar.zst1009 Compression: zstd1010 NarHash: sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1011 NarSize: 1601012 References: 1013 CA: fixed:r:sha256:1byf36hhc5sn8lvwqfkjh3ybgn76slmx8naakslrypxs6mna71rf1014 metadata_upload_test.go:77: Retrieved .ls file from S3 (compressed size: 62 bytes)1015 metadata_upload_test.go:77: Decompressed .ls content (49 bytes):1016 {"version":1,"root":{"type":"regular","size":44}}10172026/11/12 00:43:28 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10182026/11/12 00:43:28 INFO Received push request method=POST path=/api/pushes1019--- PASS: TestNARDeduplicationMetadataUploadBug (3.44s)1020=== CONT TestReadProxyInvalidPath10212026/11/12 00:43:28 INFO Received complete push request method=POST path=/api/pushes/1/complete10222026/11/12 00:43:28 INFO Completed multipart upload object_key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst upload_id=YTFkOTlkNzctYjJiYi00MGQ4LTk3N2EtYzJmZjBkMTNhYzM2LjQzNWU3NDc3LTA4YWMtNGNmOS04YWJhLTViOTJkNTVlMzAwMXgxNzk0NDQ0MjA2ODM4MTI2MDAw parts=1010232026/11/12 00:43:28 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete10242026/11/12 00:43:28 INFO Completed upload id=1 objects=2 duration=7.231625ms10252026/11/12 00:43:28 INFO Completed push id=1 objects=6 duration=22.242833ms1026--- PASS: TestPush_CompleteCommitsEveryRoot (3.49s)1027=== CONT TestReadProxyNarinfo10282026/11/12 00:43:28 INFO Received uploads request method=POST path=/api/pending_closures10292026/11/12 00:43:28 INFO Received uploads request method=POST path=/api/pending_closures10302026/11/12 00:43:28 INFO Object in database but missing from S3 key=dadb44fdadb44fdadb44fdadb44f0000.narinfo10312026/11/12 00:43:28 WARN Found objects in DB but missing from S3, will re-upload count=11032--- PASS: TestService_verifyS3Integrity (3.57s)1033=== CONT TestReadProxy4041034--- PASS: TestReadRedirectNar (3.67s)1035=== CONT TestReadProxyNarinfoAlreadyDecompressed10362026/11/12 00:43:28 INFO Received push request method=POST path=/api/pushes10372026/11/12 00:43:28 INFO Received complete push request method=POST path=/api/pushes/1/complete10382026/11/12 00:43:28 INFO Completed push id=1 objects=4 duration=20.976292ms1039--- PASS: TestPush_RepeatedRootCommitsOnce (3.87s)1040=== CONT TestReadProxyNarStreaming1041--- PASS: TestReadProxyDisabled (4.06s)1042=== CONT TestSharedNarBothUploadsComplete10432026/11/12 00:43:28 INFO Received complete multipart upload request method=POST path=/api/multipart/complete10442026/11/12 00:43:28 ERROR Failed to get multipart upload error="no rows in result set" upload_id=does-not-exist object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst1045--- PASS: TestCompleteMultipartUnregistered (4.24s)1046=== CONT TestCommitCanBeRepeatedUntilCleanup1047=== RUN TestCommitCanBeRepeatedUntilCleanup/push1048=== PAUSE TestCommitCanBeRepeatedUntilCleanup/push1049=== RUN TestCommitCanBeRepeatedUntilCleanup/pending_closure1050=== PAUSE TestCommitCanBeRepeatedUntilCleanup/pending_closure1051=== CONT TestPushDedupSurvivesConcurrentGC10522026/11/12 00:43:29 INFO Received uploads request method=POST path=/api/pending_closures10532026/11/12 00:43:29 INFO Received uploads request method=POST path=/api/pending_closures10542026/11/12 00:43:29 INFO Received uploads request method=POST path=/api/pending_closures10552026/11/12 00:43:29 INFO Received push request method=POST path=/api/pushes10562026/11/12 00:43:29 INFO Received complete push request method=POST path=/api/pushes/1/complete10572026/11/12 00:43:29 INFO Completed push id=1 objects=4 duration=40.602667ms10582026/11/12 00:43:29 INFO Received push request method=POST path=/api/pushes10592026/11/12 00:43:29 INFO Aborted multipart uploads count=0 kept=010602026/11/12 00:43:29 WARN Force mode enabled - objects will be deleted immediately without grace period1061=== RUN TestPush_RejectsBadRequests/no_roots1062=== PAUSE TestPush_RejectsBadRequests/no_roots1063=== RUN TestPush_RejectsBadRequests/no_objects1064=== PAUSE TestPush_RejectsBadRequests/no_objects1065=== RUN TestPush_RejectsBadRequests/bad_root1066=== PAUSE TestPush_RejectsBadRequests/bad_root1067=== RUN TestPush_RejectsBadRequests/root_not_in_objects1068=== PAUSE TestPush_RejectsBadRequests/root_not_in_objects1069=== CONT TestCreatePendingClosureRejectsOversizedNAR10702026/11/12 00:43:29 INFO Received uploads request method=POST path=/api/pending_closures1071--- PASS: TestCreatePendingClosureRejectsOversizedNAR (0.00s)1072=== CONT TestIsValidCachePath1073=== RUN TestIsValidCachePath/narinfo1074=== PAUSE TestIsValidCachePath/narinfo1075=== RUN TestIsValidCachePath/narinfo_all_nix_base32_chars1076=== PAUSE TestIsValidCachePath/narinfo_all_nix_base32_chars1077=== RUN TestIsValidCachePath/nar_zst1078=== PAUSE TestIsValidCachePath/nar_zst1079=== RUN TestIsValidCachePath/nar_xz1080=== PAUSE TestIsValidCachePath/nar_xz1081=== RUN TestIsValidCachePath/nar_bz21082=== PAUSE TestIsValidCachePath/nar_bz21083=== RUN TestIsValidCachePath/nar_uncompressed1084=== PAUSE TestIsValidCachePath/nar_uncompressed1085=== RUN TestIsValidCachePath/ls1086=== PAUSE TestIsValidCachePath/ls1087=== RUN TestIsValidCachePath/log1088=== PAUSE TestIsValidCachePath/log1089=== RUN TestIsValidCachePath/realisation1090=== PAUSE TestIsValidCachePath/realisation10912026/11/12 00:43:29 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=2 objects-deleted-after-grace-period=2 objects-failed-to-delete=01092=== RUN TestIsValidCachePath/nix-cache-info1093=== PAUSE TestIsValidCachePath/nix-cache-info1094=== RUN TestIsValidCachePath/index.html1095=== PAUSE TestIsValidCachePath/index.html1096=== RUN TestIsValidCachePath/traversal_parent1097=== PAUSE TestIsValidCachePath/traversal_parent1098=== RUN TestIsValidCachePath/traversal_in_middle1099=== PAUSE TestIsValidCachePath/traversal_in_middle1100=== RUN TestIsValidCachePath/invalid_char_e1101=== PAUSE TestIsValidCachePath/invalid_char_e1102=== RUN TestIsValidCachePath/invalid_char_u1103=== PAUSE TestIsValidCachePath/invalid_char_u1104=== RUN TestIsValidCachePath/random_path1105=== PAUSE TestIsValidCachePath/random_path1106=== RUN TestIsValidCachePath/empty1107=== PAUSE TestIsValidCachePath/empty1108=== RUN TestIsValidCachePath/leading_slash1109=== PAUSE TestIsValidCachePath/leading_slash1110=== RUN TestIsValidCachePath/wrong_extension1111=== PAUSE TestIsValidCachePath/wrong_extension1112=== RUN TestIsValidCachePath/short_hash1113=== PAUSE TestIsValidCachePath/short_hash1114=== CONT TestCacheConfigHandlerMaxNarSize1115--- PASS: TestCacheConfigHandlerMaxNarSize (0.00s)1116=== CONT TestEnsureAbortUploadsRule11172026/11/12 00:43:29 INFO Vacuumed table table=pending_closures11182026/11/12 00:43:29 INFO Vacuumed table table=commit_receipts11192026/11/12 00:43:29 INFO Vacuumed table table=pending_objects11202026/11/12 00:43:29 INFO Vacuumed table table=multipart_uploads11212026/11/12 00:43:29 INFO Vacuumed table table=closures11222026/11/12 00:43:29 INFO Vacuumed table table=objects11232026/11/12 00:43:29 INFO Received complete push request method=POST path=/api/pushes/2/complete11242026/11/12 00:43:29 INFO Completed push id=2 objects=4 duration=14.563625ms1125--- PASS: TestPush_SkippedKeySurvivesGCBeforeCommit (5.11s)1126=== CONT TestGenerateLandingPage1127--- PASS: TestGenerateLandingPage (0.00s)1128=== CONT TestObjectStatsTrigger11292026/11/12 00:43:29 INFO Received uploads request method=POST path=/api/pending_closures11302026/11/12 00:43:30 INFO Received complete multipart upload request method=POST path=/api/multipart/complete11312026/11/12 00:43:30 WARN CompleteMultipartUpload errored but object exists; treating as success error="The specified key does not exist." object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=YTFkOTlkNzctYjJiYi00MGQ4LTk3N2EtYzJmZjBkMTNhYzM2LjlhMDQ2YTdkLTE0ODktNDgxZC04Yjk2LTlmMmIyNTU5OWEyYngxNzk0NDQ0MjA5ODY2MDY4MDAw11322026/11/12 00:43:30 INFO Received push request method=POST path=/api/pushes11332026/11/12 00:43:30 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0300000000000000000000.nar.zst upload_id=YTFkOTlkNzctYjJiYi00MGQ4LTk3N2EtYzJmZjBkMTNhYzM2LjlhMDQ2YTdkLTE0ODktNDgxZC04Yjk2LTlmMmIyNTU5OWEyYngxNzk0NDQ0MjA5ODY2MDY4MDAw parts=11134--- PASS: TestCompleteMultipartUpload_ErrorButObjectExists (5.44s)1135=== CONT TestService_readinessHandler11362026/11/12 00:43:30 WARN Rate limiter enabled after throttle name=s3-test rate=511372026/11/12 00:43:30 WARN S3 rate limit hit operation="complete multipart upload" error="Please reduce your request rate."1138=== NAME TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle1139 throttle_test.go:213: Proxy stats: total=13, throttled=10, completeMultipart=101140 throttle_test.go:215: Rate limiter: enabled=true, rate=5.001141--- PASS: TestCompleteMultipartUploadHandler_RateLimitTriggersThrottle (5.61s)1142=== CONT TestService_healthCheckHandler11432026-11-12 00:43:30.257 UTC [42608] ERROR: could not obtain lock on row in relation "closures"11442026-11-12 00:43:30.257 UTC [42608] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11452026-11-12 00:43:30.258 UTC [42608] ERROR: could not obtain lock on row in relation "closures"11462026-11-12 00:43:30.258 UTC [42608] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11472026-11-12 00:43:30.263 UTC [42608] ERROR: could not obtain lock on row in relation "closures"11482026-11-12 00:43:30.263 UTC [42608] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11492026-11-12 00:43:30.264 UTC [42608] ERROR: could not obtain lock on row in relation "closures"11502026-11-12 00:43:30.264 UTC [42608] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT11512026-11-12 00:43:30.271 UTC [42608] ERROR: could not obtain lock on row in relation "closures"11522026-11-12 00:43:30.271 UTC [42608] STATEMENT: SELECT 1 FROM closures WHERE key = $1 FOR UPDATE NOWAIT1153--- PASS: TestPush_RootsAreLockedInByteOrder (5.65s)1154=== CONT TestGracefulShutdownDrainsInflight11552026/11/12 00:43:30 INFO Starting HTTP server address=127.0.0.1:5683311562026/11/12 00:43:30 INFO Shutdown signal received, draining in-flight requests timeout=10s1157--- PASS: TestGracefulShutdownDrainsInflight (0.07s)1158=== CONT TestPresentRefreshesOnlyStaleClosures11592026/11/12 00:43:30 INFO Received push request method=POST path=/api/pushes11602026/11/12 00:43:30 INFO Received complete multipart upload request method=POST path=/api/multipart/complete11612026/11/12 00:43:30 INFO Completed multipart upload object_key=nar/0000000000000000000000000000000000000000000000000000.nar.zst upload_id=YTFkOTlkNzctYjJiYi00MGQ4LTk3N2EtYzJmZjBkMTNhYzM2LjQ1NGU3NmQyLWQ3Y2ItNDg3My05MTNiLTAyNDdlMGFhMGVlOXgxNzk0NDQ0MjA5MTM2OTUwMDAw parts=1011622026/11/12 00:43:30 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete11632026/11/12 00:43:30 INFO Completed upload id=1 objects=2 duration=23.543875ms11642026/11/12 00:43:30 INFO Received get closure request method=GET path=/api/closures/0000000000000000000000000000000011652026/11/12 00:43:30 INFO Received uploads request method=POST path=/api/pending_closures11662026/11/12 00:43:30 INFO Starting cleanup of old closures method=DELETE path=/api/closures11672026/11/12 00:43:30 INFO Aborted multipart uploads count=0 kept=011682026/11/12 00:43:30 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=011692026/11/12 00:43:30 INFO Vacuumed table table=pending_closures11702026/11/12 00:43:30 INFO Vacuumed table table=commit_receipts11712026/11/12 00:43:30 INFO Vacuumed table table=pending_objects1172--- PASS: TestReadProxyRootRedirectsToIndexHTML (5.98s)11732026/11/12 00:43:30 INFO Vacuumed table table=multipart_uploads1174=== CONT TestPresentReportsOnlyClosureRoots1175--- PASS: TestReadProxyOutlastsServerWriteTimeout (5.99s)1176=== CONT TestCreatePin_ReservedPins11772026/11/12 00:43:30 INFO Vacuumed table table=closures11782026/11/12 00:43:30 INFO Vacuumed table table=objects11792026/11/12 00:43:30 INFO Received get closure request method=GET path=/api/closures/000000000000000000000000000000001180--- PASS: TestService_createPendingClosureHandler (6.05s)1181=== CONT TestGCTaskStore_Fail1182--- PASS: TestGCTaskStore_Fail (0.00s)1183=== CONT TestOrphanedObjectsGCStressTest11842026/11/12 00:43:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56842/oidc11852026-11-12 00:43:30.766 UTC [42653] ERROR: Push does not exist: id=111862026-11-12 00:43:30.766 UTC [42653] CONTEXT: PL/pgSQL function commit_push(bigint) line 15 at RAISE11872026-11-12 00:43:30.766 UTC [42653] STATEMENT: SELECT commit_push($1)1188--- PASS: TestPush_CompleteWaitsForCleanupOfThePush (6.12s)1189=== CONT TestGCTaskStore_PhaseUpdates1190--- PASS: TestGCTaskStore_PhaseUpdates (0.00s)1191=== CONT TestOrphanedObjectsGC1192--- PASS: TestService_Rustfstest (6.14s)1193=== CONT TestGCTaskStore_CompletedAllowsNewTask1194--- PASS: TestGCTaskStore_CompletedAllowsNewTask (0.00s)1195=== CONT TestObjectStatsUpdatedOncePerCommit11962026/11/12 00:43:31 INFO Received push request method=POST path=/api/pushes1197--- PASS: TestPush_OverlappingRootsStoreOneRowPerKey (6.42s)1198=== CONT TestGCTaskStore_GetReturnsLatest1199--- PASS: TestGCTaskStore_GetReturnsLatest (0.00s)1200=== CONT TestRegisterCompletedObjectNeedsPendingRow1201--- PASS: TestReadProxyConditionalGet (6.72s)1202=== CONT TestSweepSkipsRowsLockedByPush12032026/11/12 00:43:31 INFO Received uploads request method=POST path=/api/pending_closures1204--- PASS: TestReadRedirectUsesPublicS3URL (5.32s)1205=== CONT TestGetExistingObjectsLocksRows1206--- PASS: TestReadProxyContentNegotiation (4.97s)1207=== CONT TestFailedS3DeleteKeepsTombstone1208--- PASS: TestReadProxyRangeRequest (4.91s)1209=== CONT TestGCTaskStore_GetEmpty1210--- PASS: TestGCTaskStore_GetEmpty (0.00s)1211=== CONT TestGCTaskStore_ConflictDifferentParams1212--- PASS: TestGCTaskStore_ConflictDifferentParams (0.00s)1213=== CONT TestForceGCDuringPushOffersSweptObject1214=== RUN TestForceGCDuringPushOffersSweptObject/before_pending_rows1215=== PAUSE TestForceGCDuringPushOffersSweptObject/before_pending_rows1216=== RUN TestForceGCDuringPushOffersSweptObject/after_presence_check1217=== PAUSE TestForceGCDuringPushOffersSweptObject/after_presence_check1218=== CONT TestGCTaskStore_DeduplicateSameParams1219--- PASS: TestGCTaskStore_DeduplicateSameParams (0.00s)1220=== CONT TestGCTaskStore_StartNew1221--- PASS: TestGCTaskStore_StartNew (0.00s)1222=== CONT TestCreatePendingClosureVerifyS3FailureReleasesConnection1223--- PASS: TestReadProxyHead (4.86s)1224=== CONT TestGCSweepDeliversEachKeyOnce1225--- PASS: TestReadProxyInvalidPath (4.97s)1226=== CONT TestTombstonedObjectOfferedWithoutWaiting12272026/11/12 00:43:33 INFO Received complete multipart upload request method=POST path=/api/multipart/complete12282026/11/12 00:43:33 INFO Completed multipart upload object_key=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhh0100000000000000000000.nar.zst upload_id=YTFkOTlkNzctYjJiYi00MGQ4LTk3N2EtYzJmZjBkMTNhYzM2LjIzMTkyZWQzLTNlN2MtNDIzMC1hM2VmLWM0YWUyZGQ1YWRhMngxNzk0NDQ0MjExNjE2MjMyMDAw parts=1212292026/11/12 00:43:33 INFO Received uploads request method=POST path=/api/pending_closures1230--- PASS: TestCompletedNarNotReofferedAcrossClosures (7.30s)1231=== CONT TestGCSweepSkipsPendingObjects12322026/11/12 00:43:33 ERROR Failed to decode narinfo key=3hcdxyjf9yiq7qf3i4548drb6sjmwa1v.narinfo limit=16777216 error=<nil>1233--- PASS: TestReadProxyNarinfo (5.29s)1234=== CONT TestDeduplicatedObjectsRecordedAsPending1235--- PASS: TestReadProxyNarinfoAlreadyDecompressed (5.73s)1236=== CONT TestGCEndsOnShutdown1237=== RUN TestGCEndsOnShutdown/before_the_run1238=== PAUSE TestGCEndsOnShutdown/before_the_run1239=== RUN TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1240=== PAUSE TestGCEndsOnShutdown/blocked_in_the_sweep's_S3_delete1241=== RUN TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1242=== PAUSE TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1243=== CONT TestCommitRacingPendingCleanupKeepsObjects1244--- PASS: TestReadProxy404 (6.01s)1245=== CONT TestPushWaitsForSweepHoldingRow12462026/11/12 00:43:34 INFO Received uploads request method=POST path=/api/pending_closures12472026/11/12 00:43:34 INFO Received uploads request method=POST path=/api/pending_closures1248--- PASS: TestReadProxyNarStreaming (6.14s)1249=== CONT TestSweepRechecksPendingAfterLocking12502026/11/12 00:43:35 INFO Received uploads request method=POST path=/api/pending_closures12512026/11/12 00:43:35 INFO Aborted multipart uploads count=0 kept=012522026/11/12 00:43:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012532026/11/12 00:43:35 INFO Vacuumed table table=pending_closures12542026/11/12 00:43:35 INFO Vacuumed table table=commit_receipts12552026/11/12 00:43:35 INFO Installed lifecycle rule aborting incomplete NAR uploads bucket=bucket40 days=21256--- PASS: TestEnsureAbortUploadsRule (5.97s)1257=== CONT TestMultipartUploadAbortedWhenCancelledBeforeRecorded12582026/11/12 00:43:35 INFO Vacuumed table table=pending_objects12592026/11/12 00:43:35 INFO Vacuumed table table=multipart_uploads12602026/11/12 00:43:35 INFO Vacuumed table table=closures12612026/11/12 00:43:35 INFO Vacuumed table table=objects12622026/11/12 00:43:35 INFO Aborted multipart uploads count=0 kept=012632026/11/12 00:43:35 WARN Force mode enabled - objects will be deleted immediately without grace period12642026/11/12 00:43:35 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=012652026/11/12 00:43:35 INFO Vacuumed table table=pending_closures12662026/11/12 00:43:35 INFO Vacuumed table table=commit_receipts12672026/11/12 00:43:35 INFO Vacuumed table table=pending_objects12682026/11/12 00:43:35 INFO Vacuumed table table=multipart_uploads12692026/11/12 00:43:35 INFO Vacuumed table table=closures12702026/11/12 00:43:35 INFO Vacuumed table table=objects1271--- PASS: TestPushDedupSurvivesConcurrentGC (6.94s)1272=== CONT TestPendingCleanupUsesOneCutoff1273--- PASS: TestObjectStatsTrigger (6.15s)1274=== CONT TestServerTLSConfig1275=== RUN TestServerTLSConfig/no_client_CA1276=== PAUSE TestServerTLSConfig/no_client_CA1277=== RUN TestServerTLSConfig/missing_CA_file1278=== PAUSE TestServerTLSConfig/missing_CA_file1279=== RUN TestServerTLSConfig/not_a_PEM_file1280=== PAUSE TestServerTLSConfig/not_a_PEM_file1281=== CONT TestMultipartCleanup12822026/11/12 00:43:36 WARN readiness check failed error="closed pool"1283--- PASS: TestService_readinessHandler (6.01s)1284=== CONT TestParseSingleRange1285=== RUN TestParseSingleRange/none1286=== PAUSE TestParseSingleRange/none1287=== RUN TestParseSingleRange/unknown_unit1288=== PAUSE TestParseSingleRange/unknown_unit1289=== RUN TestParseSingleRange/multi-range_ignored1290=== PAUSE TestParseSingleRange/multi-range_ignored1291=== RUN TestParseSingleRange/malformed_no_dash1292=== PAUSE TestParseSingleRange/malformed_no_dash1293=== RUN TestParseSingleRange/malformed_both_empty1294=== PAUSE TestParseSingleRange/malformed_both_empty1295=== RUN TestParseSingleRange/malformed_end_before_start1296=== PAUSE TestParseSingleRange/malformed_end_before_start1297=== RUN TestParseSingleRange/closed1298=== PAUSE TestParseSingleRange/closed1299=== RUN TestParseSingleRange/open-ended1300=== PAUSE TestParseSingleRange/open-ended1301=== RUN TestParseSingleRange/end_clamped_to_size1302=== PAUSE TestParseSingleRange/end_clamped_to_size1303=== RUN TestParseSingleRange/suffix1304=== PAUSE TestParseSingleRange/suffix1305=== RUN TestParseSingleRange/suffix_exceeds_size1306=== PAUSE TestParseSingleRange/suffix_exceeds_size1307=== RUN TestParseSingleRange/single_byte1308=== PAUSE TestParseSingleRange/single_byte1309=== RUN TestParseSingleRange/start_past_EOF1310=== PAUSE TestParseSingleRange/start_past_EOF1311=== RUN TestParseSingleRange/start_far_past_EOF1312=== PAUSE TestParseSingleRange/start_far_past_EOF1313=== CONT TestMetricsCommit13142026/11/12 00:43:36 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1315--- PASS: TestService_healthCheckHandler (6.22s)1316=== CONT TestService_NativeMTLS13172026/11/12 00:43:36 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=YTFkOTlkNzctYjJiYi00MGQ4LTk3N2EtYzJmZjBkMTNhYzM2LjlmN2UyMmNlLWFhOTAtNDFjZi1hZWVhLTI2N2UzNmZkMGFlN3gxNzk0NDQ0MjE0Mzc1Njk4MDAw parts=121318--- PASS: TestPresentRefreshesOnlyStaleClosures (6.54s)1319=== CONT TestMetricsInventory1320--- PASS: TestPresentReportsOnlyClosureRoots (6.55s)1321=== CONT TestPinProtectsFromGC13222026/11/12 00:43:37 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux13232026/11/12 00:43:37 WARN Refused reserved pin name=worker-x86_64-linux13242026/11/12 00:43:37 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux13252026/11/12 00:43:37 INFO Received create pin request method=POST path=/api/pins/my-app13262026/11/12 00:43:37 INFO Received create pin request method=POST path=/api/pins/worker-x86_64-linux1327--- PASS: TestCreatePin_ReservedPins (7.07s)1328=== CONT TestLeadUnannouncedStandbyYieldsToIncumbent13292026/11/12 00:43:38 INFO Received complete multipart upload request method=POST path=/api/multipart/complete1330--- PASS: TestObjectStatsUpdatedOncePerCommit (7.48s)1331=== CONT TestLeadElectsOneAndHandsOver13322026/11/12 00:43:38 INFO Completed multipart upload object_key=nar/gggggggggggggggggggggggggggggg0100000000000000000000.nar.zst upload_id=YTFkOTlkNzctYjJiYi00MGQ4LTk3N2EtYzJmZjBkMTNhYzM2LjU1Mzc2MWExLTVhNGItNGFlMy1hNzcxLTYyYmQ3NjY0YzAzN3gxNzk0NDQ0MjE0NDE4Mjc5MDAw parts=121333--- PASS: TestSharedNarBothUploadsComplete (9.67s)1334=== CONT TestGCMetrics1335--- PASS: TestRegisterCompletedObjectNeedsPendingRow (7.52s)1336=== CONT TestGCBugBareHashReferences13372026/11/12 00:43:39 INFO Aborted multipart uploads count=0 kept=013382026/11/12 00:43:39 WARN Force mode enabled - objects will be deleted immediately without grace period13392026/11/12 00:43:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013402026/11/12 00:43:39 INFO Vacuumed table table=pending_closures13412026/11/12 00:43:39 INFO Vacuumed table table=commit_receipts13422026/11/12 00:43:39 INFO Vacuumed table table=pending_objects13432026/11/12 00:43:39 INFO Vacuumed table table=multipart_uploads13442026/11/12 00:43:39 INFO Vacuumed table table=closures13452026/11/12 00:43:39 INFO Vacuumed table table=objects13462026/11/12 00:43:39 INFO Aborted multipart uploads count=0 kept=013472026/11/12 00:43:39 WARN Force mode enabled - objects will be deleted immediately without grace period1348--- PASS: TestGetExistingObjectsLocksRows (7.42s)1349=== CONT TestGCAdvisoryLockBlocksConcurrentRun13502026/11/12 00:43:39 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=013512026/11/12 00:43:39 INFO Vacuumed table table=pending_closures13522026/11/12 00:43:39 INFO Vacuumed table table=commit_receipts13532026/11/12 00:43:39 INFO Vacuumed table table=pending_objects13542026/11/12 00:43:39 INFO Vacuumed table table=multipart_uploads13552026/11/12 00:43:39 INFO Vacuumed table table=closures13562026/11/12 00:43:39 INFO Vacuumed table table=objects1357--- PASS: TestSweepSkipsRowsLockedByPush (8.22s)1358=== CONT TestLeadEndsOnShutdown13592026/11/12 00:43:39 INFO Received uploads request method=POST path=/api/pending_closures13602026/11/12 00:43:40 INFO Aborted multipart uploads count=0 kept=013612026/11/12 00:43:40 WARN Force mode enabled - objects will be deleted immediately without grace period13622026/11/12 00:43:40 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."13632026/11/12 00:43:40 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."1364--- PASS: TestFailedS3DeleteKeepsTombstone (7.83s)1365=== CONT TestLeadGenerationGrowsWithEveryGrant13662026/11/12 00:43:40 INFO Received uploads request method=POST path=/api/pending_closures1367=== NAME TestOrphanedObjectsGC1368 orphaned_objects_gc_test.go:287: GC Test Summary:1369 orphaned_objects_gc_test.go:288: - Kept: 2 objects from closure A1370 orphaned_objects_gc_test.go:289: - Deleted: 2 objects from closure B1371 orphaned_objects_gc_test.go:290: - Deleted: 6 orphaned chain objects (X1->X2->X3)1372 orphaned_objects_gc_test.go:291: - Deleted: 2 orphaned single objects (Y)1373 orphaned_objects_gc_test.go:292: - Total deleted: 10 objects1374--- PASS: TestOrphanedObjectsGC (9.62s)1375=== CONT TestLeadIncumbentAnnouncesAtOnce1376--- PASS: TestCreatePendingClosureVerifyS3FailureReleasesConnection (8.00s)1377=== CONT TestConnectWaitsForAPeerMigration13782026/11/12 00:43:40 INFO Aborted multipart uploads count=0 kept=013792026/11/12 00:43:40 WARN Force mode enabled - objects will be deleted immediately without grace period13802026/11/12 00:43:40 INFO Received uploads request method=POST path=/api/pending_closures1381--- PASS: TestTombstonedObjectOfferedWithoutWaiting (7.92s)1382=== CONT TestClientFallsBackToClosures13832026/11/12 00:43:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=013842026/11/12 00:43:41 INFO Vacuumed table table=pending_closures13852026/11/12 00:43:41 INFO Vacuumed table table=commit_receipts13862026/11/12 00:43:41 INFO Vacuumed table table=pending_objects13872026/11/12 00:43:41 INFO Received uploads request method=POST path=/api/pending_closures13882026/11/12 00:43:41 INFO Vacuumed table table=multipart_uploads13892026/11/12 00:43:41 INFO Vacuumed table table=closures13902026/11/12 00:43:41 INFO Vacuumed table table=objects1391--- PASS: TestGCSweepDeliversEachKeyOnce (8.58s)1392=== CONT TestDeleteClosuresRacingPins1393--- PASS: TestDeduplicatedObjectsRecordedAsPending (7.91s)1394=== CONT TestClientPushesUseOnePush13952026/11/12 00:43:41 INFO Received uploads request method=POST path=/api/pending_closures13962026/11/12 00:43:41 INFO Aborted multipart uploads count=0 kept=013972026/11/12 00:43:41 WARN Force mode enabled - objects will be deleted immediately without grace period13982026/11/12 00:43:41 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=013992026/11/12 00:43:41 INFO Vacuumed table table=pending_closures14002026/11/12 00:43:41 INFO Vacuumed table table=commit_receipts14012026/11/12 00:43:41 INFO Vacuumed table table=pending_objects14022026/11/12 00:43:41 INFO Vacuumed table table=multipart_uploads14032026/11/12 00:43:41 INFO Vacuumed table table=closures14042026/11/12 00:43:41 INFO Vacuumed table table=objects1405--- PASS: TestGCSweepSkipsPendingObjects (8.54s)1406=== CONT TestResolveDBConnectionString1407=== RUN TestResolveDBConnectionString/flag_wins1408=== PAUSE TestResolveDBConnectionString/flag_wins1409=== RUN TestResolveDBConnectionString/file_when_flag_empty1410=== PAUSE TestResolveDBConnectionString/file_when_flag_empty1411=== RUN TestResolveDBConnectionString/missing_file_is_an_error1412=== PAUSE TestResolveDBConnectionString/missing_file_is_an_error1413=== RUN TestResolveDBConnectionString/PGHOST_allows_empty1414=== PAUSE TestResolveDBConnectionString/PGHOST_allows_empty1415=== RUN TestResolveDBConnectionString/nothing_configured1416=== PAUSE TestResolveDBConnectionString/nothing_configured1417=== CONT TestConcurrentCommitsSharingObjectsDoNotDeadlock14182026/11/12 00:43:42 INFO Aborted multipart uploads count=0 kept=014192026/11/12 00:43:42 WARN Force mode enabled - objects will be deleted immediately without grace period14202026/11/12 00:43:42 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014212026/11/12 00:43:42 INFO Vacuumed table table=pending_closures14222026/11/12 00:43:42 INFO Vacuumed table table=commit_receipts14232026/11/12 00:43:42 INFO Vacuumed table table=pending_objects14242026/11/12 00:43:42 INFO Vacuumed table table=multipart_uploads14252026/11/12 00:43:42 INFO Vacuumed table table=closures14262026/11/12 00:43:42 INFO Vacuumed table table=objects1427--- PASS: TestCommitRacingPendingCleanupKeepsObjects (8.30s)14282026/11/12 00:43:42 INFO Received uploads request method=POST path=/api/pending_closures1429=== CONT TestCacheConfigHandler1430=== RUN TestCacheConfigHandler/full_config,_no_issuer1431=== PAUSE TestCacheConfigHandler/full_config,_no_issuer1432=== RUN TestCacheConfigHandler/no_cache_url_configured1433=== PAUSE TestCacheConfigHandler/no_cache_url_configured1434=== RUN TestCacheConfigHandler/no_signing_keys1435=== PAUSE TestCacheConfigHandler/no_signing_keys1436=== RUN TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1437=== PAUSE TestCacheConfigHandler/issuer_requested_but_no_OIDC_validator1438=== CONT TestClientSharedPathCommittedMidPush14392026/11/12 00:43:42 INFO Aborted multipart uploads count=0 kept=014402026/11/12 00:43:42 WARN Force mode enabled - objects will be deleted immediately without grace period14412026/11/12 00:43:42 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=014422026/11/12 00:43:42 INFO Vacuumed table table=pending_closures14432026/11/12 00:43:42 INFO Vacuumed table table=commit_receipts14442026/11/12 00:43:42 INFO Vacuumed table table=pending_objects14452026/11/12 00:43:42 INFO Vacuumed table table=multipart_uploads14462026/11/12 00:43:42 INFO Vacuumed table table=closures1447--- PASS: TestPushWaitsForSweepHoldingRow (8.64s)14482026/11/12 00:43:42 INFO Vacuumed table table=objects1449=== CONT TestClientWithDependencies1450--- PASS: TestSweepRechecksPendingAfterLocking (8.15s)1451=== CONT TestClientMultipleUploads14522026/11/12 00:43:42 INFO Received uploads request method=POST path=/api/pending_closures1453--- PASS: TestMultipartUploadAbortedWhenCancelledBeforeRecorded (7.45s)1454=== CONT TestClientIntegration14552026/11/12 00:43:43 INFO Received uploads request method=POST path=/api/pending_closures14562026/11/12 00:43:43 INFO Received cleanup request method=DELETE path=/api/pending_closures14572026/11/12 00:43:43 WARN Failed to abort upload, keeping its closure key=nar/dadb44fdadb44fdadb44fdadb44f000000000000000000000000.nar.zst error="Get \"http://127.0.0.1:1/bucket63/?location=\": dial tcp 127.0.0.1:1: connect: connection refused" code=""14582026/11/12 00:43:43 INFO Aborted multipart uploads count=0 kept=114592026/11/12 00:43:43 INFO Received cleanup request method=DELETE path=/api/pending_closures14602026/11/12 00:43:43 INFO Aborted multipart uploads count=1 kept=01461--- PASS: TestMultipartCleanup (7.57s)1462=== CONT TestClientErrorHandling1463=== RUN TestClientErrorHandling/InvalidStorePath1464=== PAUSE TestClientErrorHandling/InvalidStorePath1465=== RUN TestClientErrorHandling/InvalidAuthToken1466=== PAUSE TestClientErrorHandling/InvalidAuthToken1467=== RUN TestClientErrorHandling/ServerNotAvailable1468=== PAUSE TestClientErrorHandling/ServerNotAvailable1469=== CONT TestService_ReadAuthMiddleware14702026/11/12 00:43:43 INFO Received uploads request method=POST path=/api/pending_closures14712026/11/12 00:43:43 INFO Received uploads request method=POST path=/api/pending_closures14722026/11/12 00:43:43 INFO Received cleanup request method=DELETE path=/api/pending_closures14732026/11/12 00:43:43 INFO Received push request method=POST path=/api/pushes14742026/11/12 00:43:43 INFO Received complete push request method=POST path=/api/pushes/1/complete14752026/11/12 00:43:43 INFO Completed push id=1 objects=4 duration=30.406375ms14762026/11/12 00:43:43 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete14772026/11/12 00:43:43 INFO Completed upload id=2 objects=2 duration=1.564708ms1478--- PASS: TestMetricsCommit (7.82s)1479=== CONT TestService_ReadScope_PublicByDefault14802026/11/12 00:43:44 WARN mTLS auth: subject not in bound subjects subject="CN=reader"14812026/11/12 00:43:44 WARN mTLS auth: subject not in bound subjects subject="CN=reader"1482--- PASS: TestService_NativeMTLS (7.64s)1483=== CONT TestService_RequireScope_OIDC14842026/11/12 00:43:44 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56986/oidc1485--- PASS: TestMetricsInventory (7.62s)1486=== CONT TestService_AuthMiddleware_OIDC14872026/11/12 00:43:44 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:56990/oidc14882026/11/12 00:43:44 INFO lead: acquired remote=192.0.2.1:1234 generation=11489=== NAME TestPinProtectsFromGC1490 client_integration_test.go:791: Pinned store path: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestPinProtectsFromGC279595479/001/store/yliq5l0s9aahp8c9xh240zsgsvnwxf9s-pinned-file.txt1491 client_integration_test.go:792: Unpinned store path: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestPinProtectsFromGC279595479/001/store/phz3555q8nrzh64b1bnyn581vm6ylc35-unpinned-file.txt14922026/11/12 00:43:45 INFO lead: acquired remote=192.0.2.1:1234 generation=114932026/11/12 00:43:45 INFO Received push request method=POST path=/api/pushes14942026/11/12 00:43:45 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)14952026/11/12 00:43:45 INFO Uploading yliq5l0s9aahp8c9xh240zsgsvnwxf9s-pinned-file.txt (128B)14962026/11/12 00:43:45 WARN Failed to register uploaded object key=yliq5l0s9aahp8c9xh240zsgsvnwxf9s.ls error="server returned 404: 404 page not found\n"14972026/11/12 00:43:45 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign14982026/11/12 00:43:45 WARN Failed to register uploaded object key=nar/0qmhsg42pqsz7x6hh02j8gznnac3yi3qpjhrqbwvz88i194jbb5s.nar.zst error="server returned 404: 404 page not found\n"14992026/11/12 00:43:45 INFO Signed narinfos id=1 count=115002026/11/12 00:43:45 INFO Uploading 1 narinfos15012026/11/12 00:43:45 INFO Received complete push request method=POST path=/api/pushes/1/complete15022026/11/12 00:43:45 WARN Failed to register uploaded object key=yliq5l0s9aahp8c9xh240zsgsvnwxf9s.narinfo error="server returned 404: 404 page not found\n"15032026/11/12 00:43:45 INFO Completed push id=1 objects=3 duration=27.68525ms15042026/11/12 00:43:45 INFO Upload complete. (250ms)15052026/11/12 00:43:45 INFO Aborted multipart uploads count=0 kept=015062026/11/12 00:43:45 WARN Force mode enabled - objects will be deleted immediately without grace period15072026/11/12 00:43:45 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=015082026/11/12 00:43:45 INFO Vacuumed table table=pending_closures15092026/11/12 00:43:45 INFO Vacuumed table table=commit_receipts15102026/11/12 00:43:45 INFO Vacuumed table table=pending_objects15112026/11/12 00:43:45 INFO Vacuumed table table=multipart_uploads15122026/11/12 00:43:45 INFO Vacuumed table table=closures15132026/11/12 00:43:45 INFO Vacuumed table table=objects1514--- PASS: TestGCMetrics (7.09s)1515=== CONT TestClientCADerivations15162026/11/12 00:43:45 INFO Received push request method=POST path=/api/pushes15172026/11/12 00:43:45 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)15182026/11/12 00:43:45 INFO Uploading phz3555q8nrzh64b1bnyn581vm6ylc35-unpinned-file.txt (128B)15192026/11/12 00:43:45 INFO Received sign narinfos request method=POST path=/api/pushes/2/sign15202026/11/12 00:43:45 WARN Failed to register uploaded object key=phz3555q8nrzh64b1bnyn581vm6ylc35.ls error="server returned 404: 404 page not found\n"15212026/11/12 00:43:45 WARN Failed to register uploaded object key=nar/0gh0wdnr6gmk4r5sfzzh12isnl14rgmkrwy1ws41hpsrlndclg5h.nar.zst error="server returned 404: 404 page not found\n"15222026/11/12 00:43:45 INFO Signed narinfos id=2 count=115232026/11/12 00:43:45 INFO Uploading 1 narinfos15242026/11/12 00:43:45 INFO Received complete push request method=POST path=/api/pushes/2/complete15252026/11/12 00:43:45 WARN Failed to register uploaded object key=phz3555q8nrzh64b1bnyn581vm6ylc35.narinfo error="server returned 404: 404 page not found\n"15262026/11/12 00:43:45 INFO Completed push id=2 objects=3 duration=1.176958ms15272026/11/12 00:43:45 INFO Upload complete. (205ms)15282026/11/12 00:43:45 INFO Starting cleanup of old closures method=DELETE path=/api/closures15292026/11/12 00:43:45 INFO Garbage collection started15302026/11/12 00:43:45 INFO Aborted multipart uploads count=0 kept=015312026/11/12 00:43:45 WARN Force mode enabled - objects will be deleted immediately without grace period1532--- PASS: TestGCBugBareHashReferences (7.25s)1533=== CONT TestCacheStatsHandler15342026/11/12 00:43:45 ERROR failed to check GC advisory lock error="query pg_locks: failed to connect to `user=_tribuchetbld3 database=niks3`: /nonexistent/.s.PGSQL.5432 (/nonexistent): dial error: dial unix /nonexistent/.s.PGSQL.5432: connect: no such file or directory"1535--- PASS: TestGCAdvisoryLockBlocksConcurrentRun (6.60s)1536=== CONT TestService_AuthMiddleware_MTLSProxyHeader15372026/11/12 00:43:46 INFO lead: released remote=192.0.2.1:1234 generation=115382026/11/12 00:43:46 INFO lead: acquired remote=192.0.2.1:1234 generation=215392026/11/12 00:43:46 INFO lead: given back to the incumbent remote=192.0.2.1:1234 generation=215402026/11/12 00:43:46 INFO lead: acquired remote=192.0.2.1:1234 generation=315412026/11/12 00:43:46 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=015422026/11/12 00:43:46 INFO Vacuumed table table=pending_closures15432026/11/12 00:43:46 INFO Vacuumed table table=commit_receipts15442026/11/12 00:43:46 INFO lead: acquired remote=192.0.2.1:1234 generation=115452026/11/12 00:43:46 INFO Vacuumed table table=pending_objects15462026/11/12 00:43:46 INFO lead: released remote=192.0.2.1:1234 generation=115472026/11/12 00:43:46 INFO Vacuumed table table=multipart_uploads15482026/11/12 00:43:46 INFO Vacuumed table table=closures15492026/11/12 00:43:46 INFO lead: acquired remote=192.0.2.1:1234 generation=215502026/11/12 00:43:46 INFO Vacuumed table table=objects15512026/11/12 00:43:46 INFO lead: acquired remote=192.0.2.1:1234 generation=115522026/11/12 00:43:46 INFO lead: acquired remote=192.0.2.1:1234 generation=115532026/11/12 00:43:46 INFO lead: released remote=192.0.2.1:1234 generation=11554--- PASS: TestLeadIncumbentAnnouncesAtOnce (6.51s)1555=== CONT TestService_AuthMiddleware15562026/11/12 00:43:47 INFO lead: released remote=192.0.2.1:1234 generation=11557--- PASS: TestLeadEndsOnShutdown (7.84s)1558=== CONT TestService_AuthMiddleware_MTLSBoundSubjects15592026/11/12 00:43:47 INFO lead: released remote=192.0.2.1:1234 generation=21560--- PASS: TestLeadElectsOneAndHandsOver (9.19s)1561=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_other_NAR_pushed15622026/11/12 00:43:47 INFO Garbage collection progress phase="" failed_uploads_deleted=0 old_closures_deleted=1 objects_marked=3 objects_deleted=3 objects_failed=015632026/11/12 00:43:47 INFO lead: released remote=192.0.2.1:1234 generation=115642026/11/12 00:43:47 INFO lead: acquired remote=192.0.2.1:1234 generation=215652026/11/12 00:43:47 INFO Received create pin request method=POST path=/api/pins/myapp15662026/11/12 00:43:47 INFO Created/updated pin name=myapp store_path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestPinProtectsFromGC279595479/001/store/yliq5l0s9aahp8c9xh240zsgsvnwxf9s-pinned-file.txt narinfo_key=yliq5l0s9aahp8c9xh240zsgsvnwxf9s.narinfo15672026/11/12 00:43:47 INFO Starting cleanup of old closures method=DELETE path=/api/closures15682026/11/12 00:43:47 INFO Garbage collection started15692026/11/12 00:43:47 INFO Aborted multipart uploads count=0 kept=015702026/11/12 00:43:47 WARN Force mode enabled - objects will be deleted immediately without grace period15712026/11/12 00:43:47 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=015722026/11/12 00:43:47 INFO Vacuumed table table=pending_closures15732026/11/12 00:43:47 INFO Vacuumed table table=commit_receipts15742026/11/12 00:43:47 INFO Vacuumed table table=pending_objects15752026/11/12 00:43:47 INFO Vacuumed table table=multipart_uploads15762026/11/12 00:43:47 INFO Received uploads request method=POST path=/api/pending_closures15772026/11/12 00:43:47 INFO Vacuumed table table=closures15782026/11/12 00:43:47 INFO Vacuumed table table=objects15792026/11/12 00:43:47 INFO Received uploads request method=POST path=/api/pending_closures15802026/11/12 00:43:47 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)15812026/11/12 00:43:47 INFO Uploading p9wmz6727rzkknisribjy7hxkkfn47zk-a (256B)15822026/11/12 00:43:47 INFO Uploading gs77dkwdwwjchpbdnxgzm6ndj71aiq54-shared-dep (136B)15832026/11/12 00:43:48 INFO lead: released remote=192.0.2.1:1234 generation=31584--- PASS: TestLeadUnannouncedStandbyYieldsToIncumbent (10.31s)1585=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost15862026/11/12 00:43:48 WARN Failed to register uploaded object key=p9wmz6727rzkknisribjy7hxkkfn47zk.ls error="server returned 404: 404 page not found\n"15872026/11/12 00:43:48 WARN Failed to register uploaded object key=0n8p41brzfnfas8cngrraarhf5zlca80.ls error="server returned 404: 404 page not found\n"15882026/11/12 00:43:48 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"15892026/11/12 00:43:48 WARN Failed to register uploaded object key=gs77dkwdwwjchpbdnxgzm6ndj71aiq54.ls error="server returned 404: 404 page not found\n"15902026/11/12 00:43:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign15912026/11/12 00:43:48 WARN Failed to register uploaded object key=nar/0qdxwykcdkw3bsm8pd4wn448vdydx0s56215gbdgqas1ajzp7cbw.nar.zst error="server returned 404: 404 page not found\n"15922026/11/12 00:43:48 INFO Signed narinfos id=1 count=215932026/11/12 00:43:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign15942026/11/12 00:43:48 INFO Signed narinfos id=2 count=215952026/11/12 00:43:48 INFO Uploading 4 narinfos15962026/11/12 00:43:48 WARN Failed to register uploaded object key=p9wmz6727rzkknisribjy7hxkkfn47zk.narinfo error="server returned 404: 404 page not found\n"15972026/11/12 00:43:48 WARN Failed to register uploaded object key=gs77dkwdwwjchpbdnxgzm6ndj71aiq54.narinfo error="server returned 404: 404 page not found\n"15982026/11/12 00:43:48 WARN Failed to register uploaded object key=0n8p41brzfnfas8cngrraarhf5zlca80.narinfo error="server returned 404: 404 page not found\n"15992026/11/12 00:43:48 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete16002026/11/12 00:43:48 WARN Failed to register uploaded object key=gs77dkwdwwjchpbdnxgzm6ndj71aiq54.narinfo error="server returned 404: 404 page not found\n"16012026/11/12 00:43:48 INFO Completed upload id=1 objects=6 duration=43.320708ms16022026/11/12 00:43:48 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete16032026/11/12 00:43:48 INFO Completed upload id=2 objects=6 duration=713.75µs16042026/11/12 00:43:48 INFO Upload complete. (532ms)1605=== NAME TestClientFallsBackToClosures1606 client_pushes_test.go:112: Retrieved narinfo from S3:1607 StorePath: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientFallsBackToClosures2734959935/001/store/gs77dkwdwwjchpbdnxgzm6ndj71aiq54-shared-dep1608 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1609 Compression: zstd1610 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821611 NarSize: 1361612 References: 1613 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1614 client_pushes_test.go:112: Retrieved narinfo from S3:1615 StorePath: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientFallsBackToClosures2734959935/001/store/p9wmz6727rzkknisribjy7hxkkfn47zk-a1616 URL: nar/0qdxwykcdkw3bsm8pd4wn448vdydx0s56215gbdgqas1ajzp7cbw.nar.zst1617 Compression: zstd1618 NarHash: sha256:0qdxwykcdkw3bsm8pd4wn448vdydx0s56215gbdgqas1ajzp7cbw1619 NarSize: 2561620 References: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientFallsBackToClosures2734959935/001/store/gs77dkwdwwjchpbdnxgzm6ndj71aiq54-shared-dep1621 CA: text:sha256:0jbasm5g7y6pwx5jb81z45qnvg2c3czvpx0zrhrql6f0pjiy8s501622 client_pushes_test.go:112: Retrieved narinfo from S3:1623 StorePath: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientFallsBackToClosures2734959935/001/store/0n8p41brzfnfas8cngrraarhf5zlca80-b1624 URL: nar/0qdxwykcdkw3bsm8pd4wn448vdydx0s56215gbdgqas1ajzp7cbw.nar.zst1625 Compression: zstd1626 NarHash: sha256:0qdxwykcdkw3bsm8pd4wn448vdydx0s56215gbdgqas1ajzp7cbw1627 NarSize: 2561628 References: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientFallsBackToClosures2734959935/001/store/gs77dkwdwwjchpbdnxgzm6ndj71aiq54-shared-dep1629 CA: text:sha256:0jbasm5g7y6pwx5jb81z45qnvg2c3czvpx0zrhrql6f0pjiy8s501630--- PASS: TestConnectWaitsForAPeerMigration (7.78s)1631=== CONT TestService_verifyS3ReuploadsStaleNarinfo/intact,_other_NAR_pushed1632--- PASS: TestClientFallsBackToClosures (7.37s)1633=== CONT TestService_verifyS3ReuploadsStaleNarinfo/lost,_same_NAR_pushed16342026/11/12 00:43:48 INFO Received push request method=POST path=/api/pushes16352026/11/12 00:43:48 INFO Uploading 2 paths to 127.0.0.1 (1 already cached)16362026/11/12 00:43:48 INFO Uploading jwymy3ppbd50gz39lcnc2lqk4as0qwhd-shared-dep (136B)16372026/11/12 00:43:48 INFO Uploading vdy4spn3j0m568q3g8k0g2l0v4dycn82-a (256B)16382026/11/12 00:43:48 WARN Failed to register uploaded object key=vdy4spn3j0m568q3g8k0g2l0v4dycn82.ls error="server returned 404: 404 page not found\n"16392026/11/12 00:43:48 WARN Failed to register uploaded object key=smhn62zrc6a79akp6lncsjlxiyirp30x.ls error="server returned 404: 404 page not found\n"16402026/11/12 00:43:48 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"16412026/11/12 00:43:48 WARN Failed to register uploaded object key=jwymy3ppbd50gz39lcnc2lqk4as0qwhd.ls error="server returned 404: 404 page not found\n"16422026/11/12 00:43:48 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign16432026/11/12 00:43:48 WARN Failed to register uploaded object key=nar/17ipz2d8sc1gyzxyc335slz89v6cny1dvzyrgkphaygczscwirkj.nar.zst error="server returned 404: 404 page not found\n"16442026/11/12 00:43:48 INFO Signed narinfos id=1 count=316452026/11/12 00:43:48 INFO Uploading 3 narinfos16462026/11/12 00:43:48 INFO lead: released remote=192.0.2.1:1234 generation=216472026/11/12 00:43:48 INFO lead: acquired remote=192.0.2.1:1234 generation=316482026/11/12 00:43:48 INFO lead: released remote=192.0.2.1:1234 generation=31649--- PASS: TestLeadGenerationGrowsWithEveryGrant (8.62s)1650=== CONT TestService_verifyS3ReuploadsStaleNarinfo/first_of_two_lost,_second_pushed16512026/11/12 00:43:48 WARN Failed to register uploaded object key=vdy4spn3j0m568q3g8k0g2l0v4dycn82.narinfo error="server returned 404: 404 page not found\n"16522026/11/12 00:43:48 WARN Failed to register uploaded object key=jwymy3ppbd50gz39lcnc2lqk4as0qwhd.narinfo error="server returned 404: 404 page not found\n"16532026/11/12 00:43:48 INFO Received complete push request method=POST path=/api/pushes/1/complete16542026/11/12 00:43:48 WARN Failed to register uploaded object key=smhn62zrc6a79akp6lncsjlxiyirp30x.narinfo error="server returned 404: 404 page not found\n"1655=== NAME TestOrphanedObjectsGCStressTest1656 orphaned_objects_gc_test.go:422: Created 10 active closures, 5 to-delete closures, 20 orphaned chains16572026/11/12 00:43:48 INFO Completed push id=1 objects=8 duration=83.691583ms16582026/11/12 00:43:48 INFO Upload complete. (472ms)1659=== NAME TestClientPushesUseOnePush1660 client_pushes_test.go:97: Retrieved narinfo from S3:1661 StorePath: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientPushesUseOnePush2834998907/001/store/jwymy3ppbd50gz39lcnc2lqk4as0qwhd-shared-dep1662 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1663 Compression: zstd1664 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821665 NarSize: 1361666 References: 1667 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1668 client_pushes_test.go:97: Retrieved narinfo from S3:1669 StorePath: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientPushesUseOnePush2834998907/001/store/vdy4spn3j0m568q3g8k0g2l0v4dycn82-a1670 URL: nar/17ipz2d8sc1gyzxyc335slz89v6cny1dvzyrgkphaygczscwirkj.nar.zst1671 Compression: zstd1672 NarHash: sha256:17ipz2d8sc1gyzxyc335slz89v6cny1dvzyrgkphaygczscwirkj1673 NarSize: 2561674 References: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientPushesUseOnePush2834998907/001/store/jwymy3ppbd50gz39lcnc2lqk4as0qwhd-shared-dep1675 CA: text:sha256:1is983baqlbz0hyix0h3b1c4swmk6xg02a7k1z7lgqbwxmm8x07a1676 client_pushes_test.go:97: Retrieved narinfo from S3:1677 StorePath: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientPushesUseOnePush2834998907/001/store/smhn62zrc6a79akp6lncsjlxiyirp30x-b1678 URL: nar/17ipz2d8sc1gyzxyc335slz89v6cny1dvzyrgkphaygczscwirkj.nar.zst1679 Compression: zstd1680 NarHash: sha256:17ipz2d8sc1gyzxyc335slz89v6cny1dvzyrgkphaygczscwirkj1681 NarSize: 2561682 References: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientPushesUseOnePush2834998907/001/store/jwymy3ppbd50gz39lcnc2lqk4as0qwhd-shared-dep1683 CA: text:sha256:1is983baqlbz0hyix0h3b1c4swmk6xg02a7k1z7lgqbwxmm8x07a1684--- PASS: TestClientPushesUseOnePush (7.64s)1685=== CONT TestPendingClosureWriteTimeout/empty1686=== CONT TestPendingClosureWriteTimeout/handler_extends_the_server's_deadline1687=== NAME TestOrphanedObjectsGCStressTest1688 orphaned_objects_gc_test.go:440: Marked 210 objects for deletion16892026/11/12 00:43:49 INFO Received push request method=POST path=/api/pushes16902026/11/12 00:43:49 INFO Uploading 2 paths to 127.0.0.1 (0 already cached)16912026/11/12 00:43:49 INFO Uploading bxqpki9f6ys3qzzl78yfmiw092v1y8ff-top (264B)16922026/11/12 00:43:49 INFO Uploading 7a6n30k6p8sb5085z7wc15lvh235am5h-shared-dep (136B)16932026/11/12 00:43:49 WARN Failed to register uploaded object key=7a6n30k6p8sb5085z7wc15lvh235am5h.ls error="server returned 404: 404 page not found\n"16942026/11/12 00:43:49 WARN Failed to register uploaded object key=nar/08nc3jj1ibafzfsprzbbcgshvidp44d7z1vpai5gij09hxy8cib0.nar.zst error="server returned 404: 404 page not found\n"16952026/11/12 00:43:49 WARN Failed to register uploaded object key=bxqpki9f6ys3qzzl78yfmiw092v1y8ff.ls error="server returned 404: 404 page not found\n"1696--- PASS: TestConcurrentCommitsSharingObjectsDoNotDeadlock (7.67s)1697=== CONT TestPendingClosureWriteTimeout/670k_objects16982026/11/12 00:43:49 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign1699=== CONT TestPendingClosureWriteTimeout/400_objects17002026/11/12 00:43:49 WARN Failed to register uploaded object key=nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst error="server returned 404: 404 page not found\n"1701=== CONT TestProxyWriteTimeout/narinfo17022026/11/12 00:43:49 INFO Signed narinfos id=1 count=21703=== CONT TestPresentNotReportedWhileGCDeletesClosure/stale17042026/11/12 00:43:49 INFO Uploading 2 narinfos1705=== NAME TestClientIntegration1706 client_integration_test.go:292: Created store path: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientIntegration4134240804/002/store/7j97dx6fpa81b7wd7zg26lgvidvs9c1h-test-file.txt1707=== NAME TestClientWithDependencies1708 client_integration_test.go:661: Built derivation: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientWithDependencies2004397441/001/store/y27m8skk4imggymqkazbsjzplbyn6kcx-test-script17092026/11/12 00:43:49 INFO Aborted multipart uploads count=1 kept=017102026/11/12 00:43:49 WARN Failed to register uploaded object key=7a6n30k6p8sb5085z7wc15lvh235am5h.narinfo error="server returned 404: 404 page not found\n"17112026/11/12 00:43:49 INFO Received complete push request method=POST path=/api/pushes/1/complete17122026/11/12 00:43:49 WARN Failed to register uploaded object key=bxqpki9f6ys3qzzl78yfmiw092v1y8ff.narinfo error="server returned 404: 404 page not found\n"17132026/11/12 00:43:49 INFO Received cleanup request method=DELETE path=/api/pending_closures17142026/11/12 00:43:49 INFO Completed push id=1 objects=6 duration=29.533375ms17152026/11/12 00:43:49 INFO Upload complete. (551ms)17162026/11/12 00:43:49 INFO Aborted multipart uploads count=1 kept=01717 client_integration_test.go:663: Found 1 dependencies (including self)1718=== NAME TestClientSharedPathCommittedMidPush1719 client_integration_test.go:740: Retrieved narinfo from S3:1720 StorePath: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientSharedPathCommittedMidPush2223505036/001/store/7a6n30k6p8sb5085z7wc15lvh235am5h-shared-dep1721 URL: nar/1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y82.nar.zst1722 Compression: zstd1723--- PASS: TestPendingCleanupUsesOneCutoff (13.99s)1724=== CONT TestProxyWriteTimeout/1_GiB_nar1725=== CONT TestProxyWriteTimeout/unknown_size1726=== CONT TestProxyWriteTimeout/10_GiB_nar1727--- PASS: TestProxyWriteTimeout (0.05s)1728 --- PASS: TestProxyWriteTimeout/narinfo (0.00s)1729 --- PASS: TestProxyWriteTimeout/1_GiB_nar (0.00s)1730 --- PASS: TestProxyWriteTimeout/unknown_size (0.00s)1731 --- PASS: TestProxyWriteTimeout/10_GiB_nar (0.00s)1732=== NAME TestClientSharedPathCommittedMidPush1733 NarHash: sha256:1vnl0g6w4zgpfz06rb321mfd3jqsrs120jcl8dg2dxn014875y821734 NarSize: 1361735 References: 1736 CA: text:sha256:08nxw05k7b0pzzy0f3swvnw4ldxdbclkdnklpidwmrxdhvfrmv7n1737=== CONT TestIsValidUploadKey/narinfo1738=== CONT TestIsValidUploadKey/unknown_type1739=== CONT TestIsValidUploadKey/empty_key1740=== CONT TestIsValidUploadKey/absolute1741=== CONT TestIsValidUploadKey/traversal_nar1742=== CONT TestIsValidUploadKey/traversal1743=== CONT TestIsValidUploadKey/listing_key,_narinfo_type1744=== CONT TestIsValidUploadKey/nar_key,_narinfo_type1745=== CONT TestIsValidUploadKey/narinfo_key,_nar_type1746=== CONT TestIsValidUploadKey/index.html1747=== CONT TestIsValidUploadKey/nix-cache-info1748=== CONT TestIsValidUploadKey/realisation_plus_in_output1749=== CONT TestIsValidUploadKey/realisation1750=== CONT TestIsValidUploadKey/build_log_equals1751=== CONT TestIsValidUploadKey/build_log_question_mark1752=== CONT TestIsValidUploadKey/build_log_plus_in_name1753=== CONT TestIsValidUploadKey/build_log_home-manager_file1754=== CONT TestIsValidUploadKey/build_log1755=== CONT TestIsValidUploadKey/listing1756=== CONT TestIsValidUploadKey/nar_plain1757=== CONT TestIsValidUploadKey/nar_xz1758=== CONT TestIsValidUploadKey/nar_zst1759--- PASS: TestIsValidUploadKey (0.06s)1760 --- PASS: TestIsValidUploadKey/narinfo (0.00s)1761 --- PASS: TestIsValidUploadKey/unknown_type (0.00s)1762 --- PASS: TestIsValidUploadKey/empty_key (0.00s)1763 --- PASS: TestIsValidUploadKey/absolute (0.00s)1764 --- PASS: TestIsValidUploadKey/traversal_nar (0.00s)1765 --- PASS: TestIsValidUploadKey/traversal (0.00s)1766 --- PASS: TestIsValidUploadKey/listing_key,_narinfo_type (0.00s)1767 --- PASS: TestIsValidUploadKey/nar_key,_narinfo_type (0.00s)1768 --- PASS: TestIsValidUploadKey/narinfo_key,_nar_type (0.00s)1769 --- PASS: TestIsValidUploadKey/index.html (0.00s)1770 --- PASS: TestIsValidUploadKey/nix-cache-info (0.00s)1771 --- PASS: TestIsValidUploadKey/realisation_plus_in_output (0.00s)1772 --- PASS: TestIsValidUploadKey/realisation (0.00s)1773 --- PASS: TestIsValidUploadKey/build_log_equals (0.00s)1774 --- PASS: TestIsValidUploadKey/build_log_question_mark (0.00s)1775 --- PASS: TestIsValidUploadKey/build_log_plus_in_name (0.00s)1776 --- PASS: TestIsValidUploadKey/build_log_home-manager_file (0.00s)1777 --- PASS: TestIsValidUploadKey/build_log (0.00s)1778 --- PASS: TestIsValidUploadKey/listing (0.00s)1779 --- PASS: TestIsValidUploadKey/nar_plain (0.00s)1780 --- PASS: TestIsValidUploadKey/nar_xz (0.00s)1781 --- PASS: TestIsValidUploadKey/nar_zst (0.00s)1782=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info17832026/11/12 00:43:49 INFO Received uploads request method=POST path=/1784=== CONT TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key17852026/11/12 00:43:49 INFO Received request for more parts method=POST path=/1786=== CONT TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key17872026/11/12 00:43:49 INFO Received complete multipart upload request method=POST path=/1788=== CONT TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal17892026/11/12 00:43:49 INFO Received uploads request method=POST path=/1790=== NAME TestPinProtectsFromGC1791 client_integration_test.go:900: Pin successfully protected closure from garbage collection1792=== NAME TestClientSharedPathCommittedMidPush1793 client_integration_test.go:740: Retrieved narinfo from S3:1794 StorePath: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientSharedPathCommittedMidPush2223505036/001/store/bxqpki9f6ys3qzzl78yfmiw092v1y8ff-top1795--- PASS: TestUploadHandlersRejectInvalidKeys (0.06s)1796 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_nix-cache-info (0.00s)1797 --- PASS: TestUploadHandlersRejectInvalidKeys/request_more_parts_rejects_non-NAR_key (0.00s)1798 --- PASS: TestUploadHandlersRejectInvalidKeys/complete_multipart_rejects_non-NAR_key (0.00s)1799 --- PASS: TestUploadHandlersRejectInvalidKeys/create_pending_closure_rejects_path_traversal (0.01s)1800=== CONT TestPresentNotReportedWhileGCDeletesClosure/fresh1801=== NAME TestClientSharedPathCommittedMidPush1802 URL: nar/08nc3jj1ibafzfsprzbbcgshvidp44d7z1vpai5gij09hxy8cib0.nar.zst1803 Compression: zstd1804 NarHash: sha256:08nc3jj1ibafzfsprzbbcgshvidp44d7z1vpai5gij09hxy8cib01805 NarSize: 2641806 References: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientSharedPathCommittedMidPush2223505036/001/store/7a6n30k6p8sb5085z7wc15lvh235am5h-shared-dep1807 CA: text:sha256:1ygmfbym9ambwy3qgmaynjfwz1rdkpn8lr0vayccism8734nsa8n18082026/11/12 00:43:49 INFO Received push request method=POST path=/api/pushes1809--- PASS: TestPinProtectsFromGC (12.77s)1810=== CONT TestUploadHandlersRejectOversizedBody/create_pending_closure18112026/11/12 00:43:49 INFO Received uploads request method=POST path=/1812--- PASS: TestClientSharedPathCommittedMidPush (7.59s)1813=== CONT TestUploadHandlersRejectOversizedBody/request_more_parts18142026/11/12 00:43:49 INFO Received request for more parts method=POST path=/18152026/11/12 00:43:49 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)18162026/11/12 00:43:49 INFO Uploading 7j97dx6fpa81b7wd7zg26lgvidvs9c1h-test-file.txt (152B)1817=== CONT TestUploadHandlersRejectOversizedBody/complete_multipart18182026/11/12 00:43:49 INFO Received complete multipart upload request method=POST path=/1819=== NAME TestClientMultipleUploads1820 client_integration_test.go:406: Created store path 0: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientMultipleUploads2934703559/001/store/gjmr9dgsmba3dw0hiy19im3im50qldpm-test-file-0.txt1821=== CONT TestCommitCanBeRepeatedUntilCleanup/push18222026/11/12 00:43:50 WARN Failed to register uploaded object key=7j97dx6fpa81b7wd7zg26lgvidvs9c1h.ls error="server returned 404: 404 page not found\n"18232026/11/12 00:43:50 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18242026/11/12 00:43:50 WARN Failed to register uploaded object key=nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst error="server returned 404: 404 page not found\n"18252026/11/12 00:43:50 INFO Signed narinfos id=1 count=118262026/11/12 00:43:50 INFO Received push request method=POST path=/api/pushes18272026/11/12 00:43:50 INFO Uploading 1 narinfos18282026/11/12 00:43:50 INFO Received complete push request method=POST path=/api/pushes/1/complete18292026/11/12 00:43:50 WARN Failed to register uploaded object key=7j97dx6fpa81b7wd7zg26lgvidvs9c1h.narinfo error="server returned 404: 404 page not found\n"18302026/11/12 00:43:50 INFO Uploading 1 paths to 127.0.0.1 (0 already cached)18312026/11/12 00:43:50 INFO Uploading y27m8skk4imggymqkazbsjzplbyn6kcx-test-script (136B)1832--- PASS: TestService_ReadAuthMiddleware (6.68s)1833=== CONT TestCommitCanBeRepeatedUntilCleanup/pending_closure18342026/11/12 00:43:50 INFO Completed push id=1 objects=3 duration=80.971208ms18352026/11/12 00:43:50 INFO Upload complete. (446ms)18362026/11/12 00:43:50 WARN Failed to register uploaded object key=log/6b8iwzsa5whp7rgwcnq19d40gmlvz7h8-test-script.drv error="server returned 404: 404 page not found\n"18372026/11/12 00:43:50 WARN Failed to register uploaded object key=nar/1s7yghixzpinyqm7g59kyn3sav81pmnsdi5v63m6x4r2vacy907j.nar.zst error="server returned 404: 404 page not found\n"18382026/11/12 00:43:50 WARN Failed to register uploaded object key=y27m8skk4imggymqkazbsjzplbyn6kcx.ls error="server returned 404: 404 page not found\n"18392026/11/12 00:43:50 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign18402026/11/12 00:43:50 INFO Signed narinfos id=1 count=11841=== NAME TestClientMultipleUploads18422026/11/12 00:43:50 INFO Uploading 1 narinfos1843 client_integration_test.go:406: Created store path 1: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientMultipleUploads2934703559/001/store/sxsmmc96wk0ipxz0ing1xl7d7bj7w5kc-test-file-1.txt18442026/11/12 00:43:50 INFO All 1 paths already cached1845=== NAME TestClientIntegration1846 client_integration_test.go:318: Retrieved narinfo from S3:1847 StorePath: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientIntegration4134240804/002/store/7j97dx6fpa81b7wd7zg26lgvidvs9c1h-test-file.txt1848 URL: nar/11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk1.nar.zst1849 Compression: zstd1850 NarHash: sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11851 NarSize: 1521852 References: 1853 CA: fixed:r:sha256:11l1b7hsy9g9pz8i2rcn4n128l6pyr2yf58d49r12hr5caw9dzk11854 client_integration_test.go:319: Retrieved .ls file from S3 (compressed size: 77 bytes)1855 client_integration_test.go:319: Decompressed .ls content (64 bytes):1856 {"version":1,"root":{"type":"regular","size":39,"narOffset":96}}18572026/11/12 00:43:50 INFO Received complete push request method=POST path=/api/pushes/1/complete18582026/11/12 00:43:50 WARN Failed to register uploaded object key=y27m8skk4imggymqkazbsjzplbyn6kcx.narinfo error="server returned 404: 404 page not found\n"18592026/11/12 00:43:50 INFO Completed push id=1 objects=4 duration=100.870958ms18602026/11/12 00:43:50 INFO Upload complete. (565ms)1861=== NAME TestClientMultipleUploads1862 client_integration_test.go:406: Created store path 2: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientMultipleUploads2934703559/001/store/kgbci371qs4cr77biyjsdb9zp26awjfj-test-file-2.txt18632026/11/12 00:43:50 INFO Aborted multipart uploads count=0 kept=018642026/11/12 00:43:50 WARN Force mode enabled - objects will be deleted immediately without grace period1865--- PASS: TestUploadHandlersRejectOversizedBody (0.09s)1866 --- PASS: TestUploadHandlersRejectOversizedBody/request_more_parts (0.04s)1867 --- PASS: TestUploadHandlersRejectOversizedBody/complete_multipart (0.05s)1868 --- PASS: TestUploadHandlersRejectOversizedBody/create_pending_closure (0.60s)1869=== CONT TestPush_RejectsBadRequests/no_roots18702026/11/12 00:43:50 INFO Received push request method=POST path=/api/pushes18712026/11/12 00:43:50 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=01872=== CONT TestPush_RejectsBadRequests/bad_root18732026/11/12 00:43:50 INFO Received push request method=POST path=/api/pushes1874=== CONT TestPush_RejectsBadRequests/root_not_in_objects18752026/11/12 00:43:50 INFO Received push request method=POST path=/api/pushes1876=== CONT TestPush_RejectsBadRequests/no_objects18772026/11/12 00:43:50 INFO Received push request method=POST path=/api/pushes1878--- PASS: TestPush_RejectsBadRequests (4.94s)1879 --- PASS: TestPush_RejectsBadRequests/no_roots (0.00s)1880 --- PASS: TestPush_RejectsBadRequests/bad_root (0.01s)1881 --- PASS: TestPush_RejectsBadRequests/root_not_in_objects (0.01s)1882 --- PASS: TestPush_RejectsBadRequests/no_objects (0.00s)1883=== CONT TestIsValidCachePath/narinfo1884=== CONT TestIsValidCachePath/index.html1885=== CONT TestIsValidCachePath/short_hash1886=== CONT TestIsValidCachePath/wrong_extension1887=== CONT TestIsValidCachePath/leading_slash1888=== CONT TestIsValidCachePath/empty1889=== CONT TestIsValidCachePath/random_path1890=== CONT TestIsValidCachePath/invalid_char_u1891=== CONT TestIsValidCachePath/invalid_char_e1892=== CONT TestIsValidCachePath/traversal_in_middle1893=== CONT TestIsValidCachePath/traversal_parent1894=== CONT TestIsValidCachePath/nar_uncompressed1895=== CONT TestIsValidCachePath/nix-cache-info1896=== CONT TestIsValidCachePath/realisation1897=== CONT TestIsValidCachePath/log1898=== CONT TestIsValidCachePath/ls1899=== CONT TestIsValidCachePath/nar_xz1900=== CONT TestIsValidCachePath/nar_bz21901=== CONT TestIsValidCachePath/nar_zst1902=== CONT TestIsValidCachePath/narinfo_all_nix_base32_chars1903--- PASS: TestIsValidCachePath (0.02s)1904 --- PASS: TestIsValidCachePath/narinfo (0.00s)1905 --- PASS: TestIsValidCachePath/index.html (0.00s)1906 --- PASS: TestIsValidCachePath/short_hash (0.00s)1907 --- PASS: TestIsValidCachePath/wrong_extension (0.00s)1908 --- PASS: TestIsValidCachePath/leading_slash (0.00s)1909 --- PASS: TestIsValidCachePath/empty (0.00s)1910 --- PASS: TestIsValidCachePath/random_path (0.00s)1911 --- PASS: TestIsValidCachePath/invalid_char_u (0.00s)1912 --- PASS: TestIsValidCachePath/invalid_char_e (0.00s)1913 --- PASS: TestIsValidCachePath/traversal_in_middle (0.00s)1914 --- PASS: TestIsValidCachePath/traversal_parent (0.00s)1915 --- PASS: TestIsValidCachePath/nar_uncompressed (0.00s)1916 --- PASS: TestIsValidCachePath/nix-cache-info (0.00s)1917 --- PASS: TestIsValidCachePath/realisation (0.00s)1918 --- PASS: TestIsValidCachePath/log (0.00s)1919 --- PASS: TestIsValidCachePath/ls (0.00s)1920 --- PASS: TestIsValidCachePath/nar_xz (0.00s)1921 --- PASS: TestIsValidCachePath/nar_bz2 (0.00s)1922 --- PASS: TestIsValidCachePath/nar_zst (0.00s)1923 --- PASS: TestIsValidCachePath/narinfo_all_nix_base32_chars (0.00s)1924=== CONT TestForceGCDuringPushOffersSweptObject/before_pending_rows19252026/11/12 00:43:50 INFO Vacuumed table table=pending_closures19262026/11/12 00:43:50 INFO Vacuumed table table=commit_receipts1927--- PASS: TestService_ReadScope_PublicByDefault (6.74s)1928=== CONT TestForceGCDuringPushOffersSweptObject/after_presence_check19292026/11/12 00:43:50 INFO Received push request method=POST path=/api/pushes19302026/11/12 00:43:50 INFO Vacuumed table table=pending_objects19312026/11/12 00:43:50 INFO Vacuumed table table=multipart_uploads19322026/11/12 00:43:50 INFO Uploading 3 paths to 127.0.0.1 (0 already cached)19332026/11/12 00:43:50 INFO Uploading kgbci371qs4cr77biyjsdb9zp26awjfj-test-file-2.txt (160B)19342026/11/12 00:43:50 INFO Uploading gjmr9dgsmba3dw0hiy19im3im50qldpm-test-file-0.txt (160B)19352026/11/12 00:43:50 INFO Uploading sxsmmc96wk0ipxz0ing1xl7d7bj7w5kc-test-file-1.txt (160B)19362026/11/12 00:43:50 INFO Vacuumed table table=closures19372026/11/12 00:43:50 WARN Failed to register uploaded object key=nar/13p9pqi0ga6fx2apcwls9hjnv7nmf740ggpn5zvfzdsp5vz2bbg4.nar.zst error="server returned 404: 404 page not found\n"19382026/11/12 00:43:50 INFO Received push request method=POST path=/api/pushes19392026/11/12 00:43:50 INFO Object in database but missing from S3 key=7j97dx6fpa81b7wd7zg26lgvidvs9c1h.ls19402026/11/12 00:43:50 WARN Found objects in DB but missing from S3, will re-upload count=119412026/11/12 00:43:50 INFO Uploading 0 paths to 127.0.0.1 (1 already cached)19422026/11/12 00:43:50 WARN Failed to register uploaded object key=kgbci371qs4cr77biyjsdb9zp26awjfj.ls error="server returned 404: 404 page not found\n"19432026/11/12 00:43:50 WARN Failed to register uploaded object key=nar/1823g6vx4q5zqvr73m7lkq1wpk40fin7bsc792di430yk7sg4lka.nar.zst error="server returned 404: 404 page not found\n"19442026/11/12 00:43:50 WARN Failed to register uploaded object key=nar/0ahq3izw3h0b0c08p3xdfclh758fwxnp85ccvflasb4vnfsv50ri.nar.zst error="server returned 404: 404 page not found\n"19452026/11/12 00:43:50 INFO Vacuumed table table=objects19462026/11/12 00:43:50 WARN Failed to register uploaded object key=gjmr9dgsmba3dw0hiy19im3im50qldpm.ls error="server returned 404: 404 page not found\n"1947=== NAME TestClientWithDependencies19482026/11/12 00:43:50 INFO Received sign narinfos request method=POST path=/api/pushes/1/sign1949 client_integration_test.go:677: Skipping nix copy test - isolated store (/private/var/tb/3/scratch/cf270c76132c2e86/build/TestClientWithDependencies2004397441/001/store) requires matching store prefix19502026/11/12 00:43:50 WARN Failed to register uploaded object key=sxsmmc96wk0ipxz0ing1xl7d7bj7w5kc.ls error="server returned 404: 404 page not found\n"19512026/11/12 00:43:50 INFO Signed narinfos id=1 count=319522026/11/12 00:43:50 INFO Uploading 3 narinfos19532026/11/12 00:43:50 INFO Received complete push request method=POST path=/api/pushes/2/complete19542026/11/12 00:43:50 WARN Failed to register uploaded object key=7j97dx6fpa81b7wd7zg26lgvidvs9c1h.ls error="server returned 404: 404 page not found\n"19552026/11/12 00:43:50 INFO Completed push id=2 objects=3 duration=1.285041ms19562026/11/12 00:43:50 INFO Upload complete. (206ms)1957=== NAME TestClientIntegration1958 client_integration_test.go:336: Retrieved .ls file from S3 (compressed size: 62 bytes)1959 client_integration_test.go:336: Decompressed .ls content (49 bytes):1960 {"version":1,"root":{"type":"regular","size":39}}19612026/11/12 00:43:50 WARN Failed to register uploaded object key=sxsmmc96wk0ipxz0ing1xl7d7bj7w5kc.narinfo error="server returned 404: 404 page not found\n"19622026/11/12 00:43:50 WARN Failed to register uploaded object key=gjmr9dgsmba3dw0hiy19im3im50qldpm.narinfo error="server returned 404: 404 page not found\n"19632026/11/12 00:43:51 INFO Received complete push request method=POST path=/api/pushes/1/complete19642026/11/12 00:43:51 WARN Failed to register uploaded object key=kgbci371qs4cr77biyjsdb9zp26awjfj.narinfo error="server returned 404: 404 page not found\n"19652026/11/12 00:43:51 INFO Completed push id=1 objects=9 duration=56.77025ms19662026/11/12 00:43:51 INFO Upload complete. (481ms)1967--- PASS: TestClientWithDependencies (8.21s)1968=== CONT TestGCEndsOnShutdown/before_the_run1969=== NAME TestClientMultipleUploads1970 client_integration_test.go:417: Uploaded 3 paths in 580.210833ms1971=== RUN TestService_RequireScope_OIDC/builder_may_write1972=== PAUSE TestService_RequireScope_OIDC/builder_may_write1973=== RUN TestService_RequireScope_OIDC/builder_may_not_admin1974=== PAUSE TestService_RequireScope_OIDC/builder_may_not_admin1975=== RUN TestService_RequireScope_OIDC/ops_may_admin1976=== PAUSE TestService_RequireScope_OIDC/ops_may_admin1977=== RUN TestService_RequireScope_OIDC/ops_may_not_write1978=== PAUSE TestService_RequireScope_OIDC/ops_may_not_write1979=== RUN TestService_RequireScope_OIDC/reader_may_not_write1980=== PAUSE TestService_RequireScope_OIDC/reader_may_not_write1981=== RUN TestService_RequireScope_OIDC/static_token_may_admin1982=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin1983=== RUN TestService_RequireScope_OIDC/static_token_may_write1984=== PAUSE TestService_RequireScope_OIDC/static_token_may_write1985=== RUN TestService_RequireScope_OIDC/reader_may_read1986=== PAUSE TestService_RequireScope_OIDC/reader_may_read1987=== RUN TestService_RequireScope_OIDC/writer_implies_read1988=== PAUSE TestService_RequireScope_OIDC/writer_implies_read1989--- PASS: TestClientMultipleUploads (8.38s)1990=== CONT TestGCEndsOnShutdown/blocked_on_a_row_lock_in_the_mark1991=== RUN TestService_RequireScope_OIDC/anonymous_may_not_read1992=== PAUSE TestService_RequireScope_OIDC/anonymous_may_not_read1993=== RUN TestService_RequireScope_OIDC/reader_may_read_with_netrc1994=== PAUSE TestService_RequireScope_OIDC/reader_may_read_with_netrc1995=== RUN TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1996=== PAUSE TestService_RequireScope_OIDC/static_token_may_admin_with_netrc1997=== RUN TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1998=== PAUSE TestService_RequireScope_OIDC/wrong_netrc_password_may_not_read1999=== RUN TestService_RequireScope_OIDC/empty_netrc_password_may_not_read2000=== PAUSE TestService_RequireScope_OIDC/empty_netrc_password_may_not_read
loading 914 hidden lines…
29152026/11/12 00:44:10 INFO Received push request method=POST path=/29162026/11/12 00:44:10 INFO Aborted multipart uploads count=0 kept=029172026/11/12 00:44:10 INFO Received uploads request method=POST path=/29182026/11/12 00:44:10 INFO Received push request method=POST path=/29192026/11/12 00:44:10 WARN Force mode enabled - objects will be deleted immediately without grace period29202026/11/12 00:44:10 INFO Received push request method=POST path=/29212026/11/12 00:44:10 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029222026/11/12 00:44:10 INFO Received uploads request method=POST path=/29232026/11/12 00:44:10 INFO Vacuumed table table=pending_closures29242026/11/12 00:44:10 INFO Vacuumed table table=commit_receipts29252026/11/12 00:44:10 INFO Vacuumed table table=pending_objects29262026/11/12 00:44:10 INFO Received push request method=POST path=/29272026/11/12 00:44:10 INFO Vacuumed table table=multipart_uploads29282026/11/12 00:44:11 INFO Received push request method=POST path=/29292026/11/12 00:44:11 INFO Received complete upload request method=POST path=/29302026/11/12 00:44:11 INFO Vacuumed table table=closures29312026/11/12 00:44:11 INFO Completed upload id=12 objects=21 duration=6.212542ms29322026/11/12 00:44:11 INFO Vacuumed table table=objects29332026/11/12 00:44:11 INFO Received push request method=POST path=/29342026/11/12 00:44:11 INFO Received uploads request method=POST path=/29352026/11/12 00:44:11 INFO Received complete upload request method=POST path=/29362026/11/12 00:44:11 INFO Completed upload id=17 objects=7 duration=4.522584ms29372026/11/12 00:44:11 INFO Received push request method=POST path=/29382026/11/12 00:44:11 INFO Aborted multipart uploads count=0 kept=029392026/11/12 00:44:11 WARN Force mode enabled - objects will be deleted immediately without grace period29402026/11/12 00:44:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029412026/11/12 00:44:11 INFO Received push request method=POST path=/29422026/11/12 00:44:11 INFO Aborted multipart uploads count=0 kept=029432026/11/12 00:44:11 INFO Received uploads request method=POST path=/29442026/11/12 00:44:11 INFO Aborted multipart uploads count=0 kept=029452026/11/12 00:44:11 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."29462026/11/12 00:44:11 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."29472026/11/12 00:44:11 INFO Vacuumed table table=pending_closures29482026/11/12 00:44:11 WARN Force mode enabled - objects will be deleted immediately without grace period29492026/11/12 00:44:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029502026/11/12 00:44:11 INFO Vacuumed table table=commit_receipts29512026/11/12 00:44:11 INFO Received complete push request method=POST path=/29522026/11/12 00:44:11 INFO Completed push id=14 objects=3 duration=3.732708ms29532026/11/12 00:44:11 INFO Vacuumed table table=pending_closures29542026/11/12 00:44:11 INFO Received complete push request method=POST path=/29552026/11/12 00:44:11 INFO Vacuumed table table=pending_objects29562026/11/12 00:44:11 INFO Completed push id=13 objects=23 duration=4.018417ms29572026/11/12 00:44:11 INFO Vacuumed table table=multipart_uploads29582026/11/12 00:44:11 INFO Received complete push request method=POST path=/29592026/11/12 00:44:11 INFO Vacuumed table table=commit_receipts29602026/11/12 00:44:11 INFO Vacuumed table table=pending_objects29612026/11/12 00:44:11 INFO Vacuumed table table=closures29622026/11/12 00:44:11 INFO Received complete push request method=POST path=/29632026/11/12 00:44:11 INFO Vacuumed table table=multipart_uploads29642026/11/12 00:44:11 INFO Completed push id=1 objects=7 duration=40.488875ms29652026/11/12 00:44:11 INFO Completed push id=5 objects=6 duration=1.383166ms29662026/11/12 00:44:11 INFO Vacuumed table table=objects29672026/11/12 00:44:11 INFO Received uploads request method=POST path=/29682026/11/12 00:44:11 INFO Vacuumed table table=closures29692026/11/12 00:44:11 INFO Aborted multipart uploads count=0 kept=029702026/11/12 00:44:11 INFO Received complete upload request method=POST path=/29712026/11/12 00:44:11 INFO Completed upload id=14 objects=6 duration=2.697541ms29722026/11/12 00:44:11 WARN Force mode enabled - objects will be deleted immediately without grace period29732026/11/12 00:44:11 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029742026/11/12 00:44:11 INFO Vacuumed table table=pending_closures29752026/11/12 00:44:11 INFO Vacuumed table table=commit_receipts29762026/11/12 00:44:11 INFO Received complete push request method=POST path=/29772026/11/12 00:44:11 INFO Vacuumed table table=pending_objects29782026/11/12 00:44:11 INFO Completed push id=17 objects=22 duration=5.735042ms29792026/11/12 00:44:11 INFO Vacuumed table table=multipart_uploads29802026/11/12 00:44:11 INFO Vacuumed table table=objects29812026/11/12 00:44:11 INFO Aborted multipart uploads count=0 kept=029822026/11/12 00:44:11 INFO Received push request method=POST path=/29832026/11/12 00:44:11 INFO Vacuumed table table=closures29842026/11/12 00:44:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=029852026/11/12 00:44:11 INFO Vacuumed table table=objects29862026/11/12 00:44:11 INFO Received uploads request method=POST path=/29872026/11/12 00:44:11 INFO Received complete push request method=POST path=/29882026/11/12 00:44:11 INFO Vacuumed table table=pending_closures29892026/11/12 00:44:11 INFO Completed push id=2 objects=25 duration=1.674625ms29902026/11/12 00:44:11 INFO Vacuumed table table=commit_receipts29912026/11/12 00:44:11 INFO Aborted multipart uploads count=0 kept=029922026/11/12 00:44:11 INFO Received uploads request method=POST path=/29932026/11/12 00:44:11 INFO Received complete push request method=POST path=/29942026/11/12 00:44:11 INFO Vacuumed table table=pending_objects29952026/11/12 00:44:11 INFO Completed push id=6 objects=5 duration=1.740542ms29962026/11/12 00:44:11 INFO Vacuumed table table=multipart_uploads29972026/11/12 00:44:11 INFO Received uploads request method=POST path=/29982026/11/12 00:44:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=029992026/11/12 00:44:11 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."30002026/11/12 00:44:11 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.narinfo error="Access Denied."30012026/11/12 00:44:11 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."30022026/11/12 00:44:11 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."30032026/11/12 00:44:11 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."30042026/11/12 00:44:11 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."30052026/11/12 00:44:11 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."30062026/11/12 00:44:11 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."30072026/11/12 00:44:11 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."30082026/11/12 00:44:11 INFO Vacuumed table table=closures30092026/11/12 00:44:11 INFO Vacuumed table table=pending_closures30102026/11/12 00:44:11 INFO Received push request method=POST path=/30112026/11/12 00:44:11 INFO Received push request method=POST path=/30122026/11/12 00:44:11 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."30132026/11/12 00:44:11 INFO Vacuumed table table=commit_receipts30142026/11/12 00:44:11 INFO Vacuumed table table=objects30152026/11/12 00:44:11 INFO Received uploads request method=POST path=/30162026/11/12 00:44:11 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."30172026/11/12 00:44:11 INFO Received complete upload request method=POST path=/30182026/11/12 00:44:11 INFO Vacuumed table table=pending_objects30192026/11/12 00:44:11 INFO Aborted multipart uploads count=0 kept=030202026/11/12 00:44:11 INFO Received push request method=POST path=/30212026/11/12 00:44:11 INFO Aborted multipart uploads count=0 kept=030222026/11/12 00:44:11 INFO Aborted multipart uploads count=0 kept=030232026/11/12 00:44:11 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."30242026/11/12 00:44:11 INFO Completed upload id=7 objects=3 duration=517.625µs30252026/11/12 00:44:11 INFO Vacuumed table table=multipart_uploads30262026/11/12 00:44:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030272026/11/12 00:44:11 WARN Force mode enabled - objects will be deleted immediately without grace period30282026/11/12 00:44:11 ERROR failed to remove object object=nar/bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.nar.zst error="Access Denied."30292026/11/12 00:44:11 INFO Received complete push request method=POST path=/30302026/11/12 00:44:11 INFO Vacuumed table table=closures30312026/11/12 00:44:11 WARN Force mode enabled - objects will be deleted immediately without grace period30322026/11/12 00:44:11 INFO Received uploads request method=POST path=/30332026/11/12 00:44:11 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."30342026/11/12 00:44:11 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."30352026/11/12 00:44:11 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."30362026/11/12 00:44:11 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."30372026/11/12 00:44:11 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."30382026/11/12 00:44:11 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."30392026/11/12 00:44:11 INFO Vacuumed table table=objects30402026/11/12 00:44:11 INFO Completed push id=1 objects=11 duration=1.062625ms30412026/11/12 00:44:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=4 objects-failed-to-delete=030422026/11/12 00:44:11 INFO Received uploads request method=POST path=/30432026/11/12 00:44:11 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."30442026/11/12 00:44:11 INFO Received uploads request method=POST path=/30452026/11/12 00:44:11 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."30462026/11/12 00:44:11 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."30472026/11/12 00:44:11 INFO Vacuumed table table=pending_closures30482026/11/12 00:44:11 INFO Received complete upload request method=POST path=/30492026/11/12 00:44:11 INFO Vacuumed table table=pending_closures30502026/11/12 00:44:11 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."30512026/11/12 00:44:11 INFO Vacuumed table table=commit_receipts30522026/11/12 00:44:11 INFO Received complete upload request method=POST path=/30532026/11/12 00:44:11 INFO Completed upload id=15 objects=4 duration=1.908333ms30542026/11/12 00:44:11 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."30552026/11/12 00:44:11 INFO Received push request method=POST path=/30562026/11/12 00:44:11 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."30572026/11/12 00:44:11 INFO Completed upload id=4 objects=14 duration=3.461792ms30582026/11/12 00:44:11 INFO Aborted multipart uploads count=0 kept=030592026/11/12 00:44:11 INFO Received push request method=POST path=/30602026/11/12 00:44:11 INFO Received uploads request method=POST path=/30612026/11/12 00:44:11 INFO Vacuumed table table=pending_objects30622026/11/12 00:44:11 INFO Vacuumed table table=commit_receipts30632026/11/12 00:44:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030642026/11/12 00:44:11 INFO Aborted multipart uploads count=0 kept=030652026/11/12 00:44:11 INFO Vacuumed table table=multipart_uploads30662026/11/12 00:44:11 INFO Received uploads request method=POST path=/30672026/11/12 00:44:11 INFO Received push request method=POST path=/30682026/11/12 00:44:11 INFO Received complete upload request method=POST path=/30692026/11/12 00:44:11 WARN Force mode enabled - objects will be deleted immediately without grace period30702026/11/12 00:44:11 INFO Vacuumed table table=closures30712026/11/12 00:44:11 INFO Received complete upload request method=POST path=/30722026/11/12 00:44:11 INFO Received complete push request method=POST path=/30732026/11/12 00:44:11 INFO Received complete upload request method=POST path=/30742026/11/12 00:44:11 INFO Vacuumed table table=pending_objects30752026/11/12 00:44:11 INFO Received uploads request method=POST path=/30762026/11/12 00:44:11 INFO Completed upload id=16 objects=3 duration=705.375µs30772026/11/12 00:44:11 INFO Completed upload id=16 objects=3 duration=522.292µs30782026/11/12 00:44:11 INFO Completed upload id=2 objects=10 duration=602.459µs30792026/11/12 00:44:11 INFO Vacuumed table table=objects30802026/11/12 00:44:11 INFO Completed push id=18 objects=23 duration=950.083µs30812026/11/12 00:44:11 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030822026/11/12 00:44:11 INFO Vacuumed table table=multipart_uploads30832026/11/12 00:44:11 INFO Vacuumed table table=pending_closures30842026/11/12 00:44:11 INFO Received push request method=POST path=/30852026/11/12 00:44:11 INFO Received push request method=POST path=/30862026/11/12 00:44:11 INFO Received push request method=POST path=/30872026/11/12 00:44:11 INFO Aborted multipart uploads count=0 kept=030882026/11/12 00:44:11 INFO Received complete upload request method=POST path=/30892026/11/12 00:44:11 INFO Received uploads request method=POST path=/30902026/11/12 00:44:11 INFO Vacuumed table table=commit_receipts30912026/11/12 00:44:11 INFO Received push request method=POST path=/30922026/11/12 00:44:11 INFO Vacuumed table table=pending_closures30932026/11/12 00:44:11 INFO Received push request method=POST path=/30942026/11/12 00:44:11 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=030952026/11/12 00:44:11 INFO Completed upload id=10 objects=6 duration=2.6405ms30962026/11/12 00:44:12 INFO Vacuumed table table=closures30972026/11/12 00:44:12 INFO Vacuumed table table=commit_receipts30982026/11/12 00:44:12 INFO Aborted multipart uploads count=0 kept=030992026/11/12 00:44:12 WARN Force mode enabled - objects will be deleted immediately without grace period31002026/11/12 00:44:12 INFO Vacuumed table table=pending_closures31012026/11/12 00:44:12 INFO Vacuumed table table=commit_receipts31022026/11/12 00:44:12 INFO Aborted multipart uploads count=0 kept=031032026/11/12 00:44:12 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."31042026/11/12 00:44:12 INFO Vacuumed table table=pending_objects31052026/11/12 00:44:12 INFO Vacuumed table table=pending_objects31062026/11/12 00:44:12 WARN Force mode enabled - objects will be deleted immediately without grace period31072026/11/12 00:44:12 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."31082026/11/12 00:44:12 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."31092026/11/12 00:44:12 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."31102026/11/12 00:44:12 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."31112026/11/12 00:44:12 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."31122026/11/12 00:44:12 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."31132026/11/12 00:44:12 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."31142026/11/12 00:44:12 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."31152026/11/12 00:44:12 INFO Vacuumed table table=multipart_uploads31162026/11/12 00:44:12 INFO Vacuumed table table=objects31172026/11/12 00:44:12 INFO Vacuumed table table=multipart_uploads31182026/11/12 00:44:12 INFO Vacuumed table table=pending_objects31192026/11/12 00:44:12 INFO Vacuumed table table=closures31202026/11/12 00:44:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031212026/11/12 00:44:12 INFO Aborted multipart uploads count=0 kept=031222026/11/12 00:44:12 INFO Vacuumed table table=closures31232026/11/12 00:44:12 INFO Vacuumed table table=multipart_uploads31242026/11/12 00:44:12 INFO Vacuumed table table=objects31252026/11/12 00:44:12 INFO Vacuumed table table=pending_closures31262026/11/12 00:44:12 INFO Vacuumed table table=objects31272026/11/12 00:44:12 WARN Force mode enabled - objects will be deleted immediately without grace period31282026/11/12 00:44:12 INFO Vacuumed table table=closures31292026/11/12 00:44:12 INFO Received complete push request method=POST path=/31302026/11/12 00:44:12 INFO Vacuumed table table=commit_receipts31312026/11/12 00:44:12 INFO Received push request method=POST path=/31322026/11/12 00:44:12 INFO Received push request method=POST path=/31332026/11/12 00:44:12 INFO Vacuumed table table=objects31342026/11/12 00:44:12 INFO Received push request method=POST path=/31352026/11/12 00:44:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=9 objects-failed-to-delete=031362026/11/12 00:44:12 INFO Completed push id=11 objects=9 duration=953.291µs31372026/11/12 00:44:12 INFO Vacuumed table table=pending_objects31382026/11/12 00:44:12 INFO Received push request method=POST path=/31392026/11/12 00:44:12 INFO Aborted multipart uploads count=0 kept=031402026/11/12 00:44:12 INFO Vacuumed table table=multipart_uploads31412026/11/12 00:44:12 INFO Received push request method=POST path=/31422026/11/12 00:44:12 INFO Vacuumed table table=pending_closures31432026/11/12 00:44:12 INFO Received uploads request method=POST path=/31442026/11/12 00:44:12 INFO Aborted multipart uploads count=0 kept=031452026/11/12 00:44:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031462026/11/12 00:44:12 INFO Vacuumed table table=commit_receipts31472026/11/12 00:44:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031482026/11/12 00:44:12 INFO Vacuumed table table=closures31492026/11/12 00:44:12 INFO Vacuumed table table=pending_closures31502026/11/12 00:44:12 INFO Received push request method=POST path=/31512026/11/12 00:44:12 INFO Vacuumed table table=objects31522026/11/12 00:44:12 INFO Vacuumed table table=commit_receipts31532026/11/12 00:44:12 INFO Vacuumed table table=pending_objects31542026/11/12 00:44:12 INFO Vacuumed table table=pending_closures31552026/11/12 00:44:12 INFO Vacuumed table table=commit_receipts31562026/11/12 00:44:12 INFO Vacuumed table table=multipart_uploads31572026/11/12 00:44:12 INFO Vacuumed table table=pending_objects31582026/11/12 00:44:12 INFO Vacuumed table table=closures31592026/11/12 00:44:12 INFO Vacuumed table table=pending_objects31602026/11/12 00:44:12 INFO Vacuumed table table=multipart_uploads31612026/11/12 00:44:12 INFO Aborted multipart uploads count=0 kept=031622026/11/12 00:44:12 INFO Vacuumed table table=multipart_uploads31632026/11/12 00:44:12 INFO Vacuumed table table=closures31642026/11/12 00:44:12 INFO Received uploads request method=POST path=/31652026/11/12 00:44:12 INFO Received complete upload request method=POST path=/31662026/11/12 00:44:12 INFO Vacuumed table table=objects31672026/11/12 00:44:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031682026/11/12 00:44:12 INFO Vacuumed table table=closures31692026/11/12 00:44:12 INFO Vacuumed table table=objects31702026/11/12 00:44:12 INFO Completed upload id=18 objects=4 duration=1.848959ms31712026/11/12 00:44:12 INFO Aborted multipart uploads count=0 kept=031722026/11/12 00:44:12 INFO Vacuumed table table=pending_closures31732026/11/12 00:44:12 INFO Vacuumed table table=objects31742026/11/12 00:44:12 INFO Received push request method=POST path=/31752026/11/12 00:44:12 WARN Force mode enabled - objects will be deleted immediately without grace period31762026/11/12 00:44:12 INFO Vacuumed table table=commit_receipts31772026/11/12 00:44:12 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=031782026/11/12 00:44:12 INFO Vacuumed table table=pending_objects31792026/11/12 00:44:12 INFO Vacuumed table table=pending_closures31802026/11/12 00:44:12 INFO Vacuumed table table=multipart_uploads31812026/11/12 00:44:12 INFO Received uploads request method=POST path=/31822026/11/12 00:44:12 INFO Received complete push request method=POST path=/31832026/11/12 00:44:12 INFO Vacuumed table table=commit_receipts31842026/11/12 00:44:12 INFO Vacuumed table table=closures31852026/11/12 00:44:12 INFO Received complete push request method=POST path=/31862026/11/12 00:44:12 INFO Completed push id=17 objects=6 duration=1.138875ms31872026/11/12 00:44:12 INFO Received complete push request method=POST path=/31882026/11/12 00:44:12 INFO Received push request method=POST path=/31892026/11/12 00:44:12 INFO Vacuumed table table=objects31902026/11/12 00:44:12 INFO Vacuumed table table=pending_objects31912026/11/12 00:44:12 INFO Completed push id=4 objects=4 duration=1.754083ms31922026/11/12 00:44:12 INFO Completed push id=3 objects=5 duration=1.248042ms31932026/11/12 00:44:12 INFO Received push request method=POST path=/31942026/11/12 00:44:12 INFO Received uploads request method=POST path=/31952026/11/12 00:44:12 INFO Received uploads request method=POST path=/31962026/11/12 00:44:12 INFO Vacuumed table table=multipart_uploads31972026/11/12 00:44:12 INFO Received push request method=POST path=/31982026/11/12 00:44:12 INFO Aborted multipart uploads count=0 kept=031992026/11/12 00:44:12 INFO Received push request method=POST path=/32002026/11/12 00:44:12 INFO Vacuumed table table=closures32012026/11/12 00:44:12 INFO Received push request method=POST path=/32022026/11/12 00:44:12 INFO Received complete push request method=POST path=/32032026/11/12 00:44:12 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032042026/11/12 00:44:12 INFO Completed push id=5 objects=8 duration=5.550583ms32052026/11/12 00:44:12 INFO Vacuumed table table=objects32062026/11/12 00:44:12 INFO Vacuumed table table=pending_closures32072026/11/12 00:44:12 INFO Received uploads request method=POST path=/32082026/11/12 00:44:12 INFO Received push request method=POST path=/32092026/11/12 00:44:12 INFO Vacuumed table table=commit_receipts32102026/11/12 00:44:12 INFO Vacuumed table table=pending_objects32112026/11/12 00:44:12 INFO Received uploads request method=POST path=/32122026/11/12 00:44:12 INFO Vacuumed table table=multipart_uploads32132026/11/12 00:44:12 INFO Vacuumed table table=closures32142026/11/12 00:44:12 INFO Vacuumed table table=objects32152026/11/12 00:44:12 INFO Received uploads request method=POST path=/32162026/11/12 00:44:12 INFO Aborted multipart uploads count=0 kept=032172026/11/12 00:44:12 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032182026/11/12 00:44:12 INFO Vacuumed table table=pending_closures32192026/11/12 00:44:12 INFO Received push request method=POST path=/32202026/11/12 00:44:12 INFO Vacuumed table table=commit_receipts32212026/11/12 00:44:12 INFO Vacuumed table table=pending_objects32222026/11/12 00:44:12 INFO Received complete upload request method=POST path=/32232026/11/12 00:44:12 INFO Vacuumed table table=multipart_uploads32242026/11/12 00:44:12 INFO Completed upload id=23 objects=15 duration=2.572541ms32252026/11/12 00:44:12 INFO Vacuumed table table=closures32262026/11/12 00:44:12 INFO Vacuumed table table=objects32272026/11/12 00:44:12 INFO Received complete upload request method=POST path=/32282026/11/12 00:44:12 INFO Received complete push request method=POST path=/32292026/11/12 00:44:12 INFO Received complete push request method=POST path=/32302026/11/12 00:44:12 INFO Completed upload id=7 objects=10 duration=3.12475ms32312026/11/12 00:44:12 INFO Completed push id=6 objects=6 duration=2.553208ms32322026/11/12 00:44:12 INFO Completed push id=4 objects=4 duration=2.171833ms32332026/11/12 00:44:12 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."32342026/11/12 00:44:12 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.narinfo error="Access Denied."32352026/11/12 00:44:12 INFO Received push request method=POST path=/32362026/11/12 00:44:12 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."32372026/11/12 00:44:12 INFO Received uploads request method=POST path=/32382026/11/12 00:44:12 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.narinfo error="Access Denied."32392026/11/12 00:44:12 ERROR failed to remove object object=nar/dddddddddddddddddddddddddddddddddddddddddddddddddddd.nar.zst error="Access Denied."32402026/11/12 00:44:12 INFO Received complete push request method=POST path=/32412026/11/12 00:44:12 INFO Received push request method=POST path=/32422026/11/12 00:44:12 ERROR failed to remove object object=nar/ffffffffffffffffffffffffffffffffffffffffffffffffffff.nar.zst error="Access Denied."32432026/11/12 00:44:12 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."32442026/11/12 00:44:12 INFO Completed push id=4 objects=6 duration=1.160666ms32452026/11/12 00:44:12 INFO Received complete push request method=POST path=/32462026/11/12 00:44:12 ERROR failed to remove object object=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa.ls error="Access Denied."32472026/11/12 00:44:12 ERROR failed to remove object object=nar/cccccccccccccccccccccccccccccccccccccccccccccccccccc.nar.zst error="Access Denied."32482026/11/12 00:44:12 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."32492026/11/12 00:44:12 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."32502026/11/12 00:44:12 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.narinfo error="Access Denied."32512026/11/12 00:44:12 INFO Received uploads request method=POST path=/32522026/11/12 00:44:12 INFO Received complete upload request method=POST path=/32532026/11/12 00:44:12 INFO Received uploads request method=POST path=/32542026/11/12 00:44:12 INFO Completed upload id=19 objects=6 duration=2.023417ms32552026/11/12 00:44:12 INFO Received uploads request method=POST path=/32562026/11/12 00:44:13 INFO Received uploads request method=POST path=/32572026/11/12 00:44:13 INFO Completed push id=1 objects=5 duration=41.906208ms32582026/11/12 00:44:13 INFO Received push request method=POST path=/32592026/11/12 00:44:13 INFO Received push request method=POST path=/32602026/11/12 00:44:13 INFO Received push request method=POST path=/32612026/11/12 00:44:13 INFO Received push request method=POST path=/32622026/11/12 00:44:13 INFO Received push request method=POST path=/32632026/11/12 00:44:13 INFO Received push request method=POST path=/32642026/11/12 00:44:13 INFO Received complete upload request method=POST path=/32652026/11/12 00:44:13 INFO Received push request method=POST path=/32662026/11/12 00:44:13 INFO Received complete upload request method=POST path=/32672026/11/12 00:44:13 INFO Completed upload id=8 objects=3 duration=4.959125ms32682026/11/12 00:44:13 INFO Received uploads request method=POST path=/32692026/11/12 00:44:13 INFO Completed upload id=11 objects=6 duration=2.506208ms32702026/11/12 00:44:13 INFO Received complete upload request method=POST path=/32712026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=032722026/11/12 00:44:13 INFO Completed upload id=22 objects=7 duration=560.792µs32732026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=032742026/11/12 00:44:13 WARN Force mode enabled - objects will be deleted immediately without grace period32752026/11/12 00:44:13 INFO Received uploads request method=POST path=/32762026/11/12 00:44:13 INFO Received push request method=POST path=/32772026/11/12 00:44:13 INFO Received complete push request method=POST path=/32782026/11/12 00:44:13 INFO Received uploads request method=POST path=/32792026/11/12 00:44:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032802026/11/12 00:44:13 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=032812026/11/12 00:44:13 INFO Completed push id=5 objects=3 duration=2.077292ms32822026/11/12 00:44:13 INFO Vacuumed table table=pending_closures32832026/11/12 00:44:13 INFO Vacuumed table table=pending_closures32842026/11/12 00:44:13 INFO Vacuumed table table=commit_receipts32852026/11/12 00:44:13 INFO Vacuumed table table=commit_receipts32862026/11/12 00:44:13 INFO Received push request method=POST path=/32872026/11/12 00:44:13 INFO Received uploads request method=POST path=/32882026/11/12 00:44:13 INFO Vacuumed table table=pending_objects32892026/11/12 00:44:13 INFO Received uploads request method=POST path=/32902026/11/12 00:44:13 INFO Vacuumed table table=pending_objects32912026/11/12 00:44:13 INFO Vacuumed table table=multipart_uploads32922026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=032932026/11/12 00:44:13 INFO Vacuumed table table=multipart_uploads32942026/11/12 00:44:13 INFO Vacuumed table table=closures32952026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=032962026/11/12 00:44:13 INFO Received complete push request method=POST path=/32972026/11/12 00:44:13 INFO Received complete push request method=POST path=/32982026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=032992026/11/12 00:44:13 INFO Received push request method=POST path=/33002026/11/12 00:44:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=033012026/11/12 00:44:13 INFO Vacuumed table table=closures33022026/11/12 00:44:13 INFO Vacuumed table table=objects33032026/11/12 00:44:13 INFO Completed push id=6 objects=6 duration=1.215ms33042026/11/12 00:44:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=4 objects-deleted-after-grace-period=0 objects-failed-to-delete=033052026/11/12 00:44:13 INFO Completed push id=24 objects=9 duration=1.611291ms33062026/11/12 00:44:13 INFO Vacuumed table table=objects33072026/11/12 00:44:13 INFO Vacuumed table table=pending_closures33082026/11/12 00:44:13 INFO Received complete push request method=POST path=/33092026/11/12 00:44:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033102026/11/12 00:44:13 INFO Vacuumed table table=commit_receipts33112026/11/12 00:44:13 INFO Completed push id=7 objects=27 duration=1.647208ms33122026/11/12 00:44:13 INFO Received complete push request method=POST path=/33132026/11/12 00:44:13 INFO Vacuumed table table=pending_objects33142026/11/12 00:44:13 INFO Completed push id=9 objects=5 duration=6.332292ms33152026/11/12 00:44:13 INFO Vacuumed table table=multipart_uploads33162026/11/12 00:44:13 INFO Received push request method=POST path=/33172026/11/12 00:44:13 INFO Received uploads request method=POST path=/33182026/11/12 00:44:13 INFO Received push request method=POST path=/33192026/11/12 00:44:13 INFO Received complete push request method=POST path=/33202026/11/12 00:44:13 INFO Received push request method=POST path=/33212026/11/12 00:44:13 INFO Vacuumed table table=closures33222026/11/12 00:44:13 INFO Vacuumed table table=pending_closures33232026/11/12 00:44:13 INFO Completed push id=2 objects=4 duration=2.620333ms33242026/11/12 00:44:13 INFO Vacuumed table table=pending_closures33252026/11/12 00:44:13 INFO Vacuumed table table=objects33262026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=033272026/11/12 00:44:13 INFO Vacuumed table table=commit_receipts33282026/11/12 00:44:13 INFO Vacuumed table table=commit_receipts33292026/11/12 00:44:13 INFO Vacuumed table table=pending_objects33302026/11/12 00:44:13 INFO Vacuumed table table=multipart_uploads33312026/11/12 00:44:13 WARN Force mode enabled - objects will be deleted immediately without grace period33322026/11/12 00:44:13 INFO Vacuumed table table=closures33332026/11/12 00:44:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033342026/11/12 00:44:13 INFO Received push request method=POST path=/33352026/11/12 00:44:13 INFO Vacuumed table table=pending_closures33362026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=033372026/11/12 00:44:13 INFO Received complete push request method=POST path=/33382026/11/12 00:44:13 INFO Received push request method=POST path=/33392026/11/12 00:44:13 INFO Received uploads request method=POST path=/33402026/11/12 00:44:13 INFO Vacuumed table table=commit_receipts33412026/11/12 00:44:13 WARN Force mode enabled - objects will be deleted immediately without grace period33422026/11/12 00:44:13 INFO Vacuumed table table=pending_objects33432026/11/12 00:44:13 INFO Vacuumed table table=objects33442026/11/12 00:44:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033452026/11/12 00:44:13 INFO Vacuumed table table=pending_closures33462026/11/12 00:44:13 INFO Vacuumed table table=multipart_uploads33472026/11/12 00:44:13 INFO Vacuumed table table=pending_objects33482026/11/12 00:44:13 INFO Vacuumed table table=commit_receipts33492026/11/12 00:44:13 INFO Vacuumed table table=closures33502026/11/12 00:44:13 INFO Received push request method=POST path=/33512026/11/12 00:44:13 INFO Vacuumed table table=multipart_uploads33522026/11/12 00:44:13 INFO Vacuumed table table=pending_objects33532026/11/12 00:44:13 INFO Vacuumed table table=objects33542026/11/12 00:44:13 INFO Received uploads request method=POST path=/33552026/11/12 00:44:13 INFO Received complete upload request method=POST path=/33562026/11/12 00:44:13 INFO Completed push id=1 objects=24 duration=55.086625ms33572026/11/12 00:44:13 INFO Received uploads request method=POST path=/33582026/11/12 00:44:13 INFO Received complete push request method=POST path=/33592026/11/12 00:44:13 INFO Vacuumed table table=multipart_uploads33602026/11/12 00:44:13 INFO Vacuumed table table=closures33612026/11/12 00:44:13 INFO Vacuumed table table=closures33622026/11/12 00:44:13 INFO Completed push id=26 objects=14 duration=2.669458ms33632026/11/12 00:44:13 INFO Completed upload id=10 objects=2 duration=10.015541ms33642026/11/12 00:44:13 INFO Received uploads request method=POST path=/33652026/11/12 00:44:13 INFO Vacuumed table table=objects33662026/11/12 00:44:13 INFO Received push request method=POST path=/33672026/11/12 00:44:13 INFO Received push request method=POST path=/33682026/11/12 00:44:13 INFO Vacuumed table table=objects33692026/11/12 00:44:13 INFO Received complete push request method=POST path=/33702026/11/12 00:44:13 INFO Received push request method=POST path=/33712026/11/12 00:44:13 INFO Completed push id=13 objects=12 duration=2.93125ms33722026/11/12 00:44:13 INFO Received uploads request method=POST path=/33732026/11/12 00:44:13 INFO Received complete upload request method=POST path=/33742026/11/12 00:44:13 INFO Completed upload id=3 objects=9 duration=613.25µs33752026/11/12 00:44:13 INFO Received push request method=POST path=/33762026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=033772026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=033782026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=033792026/11/12 00:44:13 WARN Force mode enabled - objects will be deleted immediately without grace period33802026/11/12 00:44:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=033812026/11/12 00:44:13 INFO Vacuumed table table=pending_closures33822026/11/12 00:44:13 INFO Received complete push request method=POST path=/33832026/11/12 00:44:13 INFO Vacuumed table table=commit_receipts33842026/11/12 00:44:13 INFO Completed push id=10 objects=15 duration=1.706167ms33852026/11/12 00:44:13 INFO Received push request method=POST path=/33862026/11/12 00:44:13 INFO Received uploads request method=POST path=/33872026/11/12 00:44:13 INFO Vacuumed table table=pending_objects33882026/11/12 00:44:13 INFO Received complete upload request method=POST path=/33892026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=033902026/11/12 00:44:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=033912026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=033922026/11/12 00:44:13 INFO Vacuumed table table=multipart_uploads33932026/11/12 00:44:13 INFO Completed upload id=11 objects=7 duration=987.542µs33942026/11/12 00:44:13 INFO Vacuumed table table=closures33952026/11/12 00:44:13 INFO Vacuumed table table=pending_closures33962026/11/12 00:44:13 INFO Vacuumed table table=objects33972026/11/12 00:44:13 INFO Vacuumed table table=commit_receipts33982026/11/12 00:44:13 INFO Vacuumed table table=pending_objects33992026/11/12 00:44:13 INFO Vacuumed table table=multipart_uploads34002026/11/12 00:44:13 INFO Vacuumed table table=closures34012026/11/12 00:44:13 INFO Vacuumed table table=objects34022026/11/12 00:44:13 WARN Force mode enabled - objects will be deleted immediately without grace period34032026/11/12 00:44:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034042026/11/12 00:44:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034052026/11/12 00:44:13 INFO Vacuumed table table=pending_closures34062026/11/12 00:44:13 INFO Vacuumed table table=commit_receipts34072026/11/12 00:44:13 INFO Vacuumed table table=pending_closures34082026/11/12 00:44:13 INFO Vacuumed table table=commit_receipts34092026/11/12 00:44:13 INFO Vacuumed table table=pending_objects34102026/11/12 00:44:13 INFO Vacuumed table table=multipart_uploads34112026/11/12 00:44:13 INFO Vacuumed table table=pending_objects34122026/11/12 00:44:13 INFO Vacuumed table table=closures34132026/11/12 00:44:13 INFO Vacuumed table table=multipart_uploads34142026/11/12 00:44:13 INFO Vacuumed table table=closures34152026/11/12 00:44:13 INFO Vacuumed table table=objects34162026/11/12 00:44:13 INFO Received complete push request method=POST path=/34172026/11/12 00:44:13 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=034182026/11/12 00:44:13 INFO Completed push id=6 objects=26 duration=5.069ms34192026/11/12 00:44:13 INFO Received complete upload request method=POST path=/34202026/11/12 00:44:13 INFO Vacuumed table table=pending_closures34212026/11/12 00:44:13 INFO Vacuumed table table=objects34222026/11/12 00:44:13 INFO Vacuumed table table=commit_receipts34232026/11/12 00:44:13 INFO Vacuumed table table=pending_objects34242026/11/12 00:44:13 INFO Vacuumed table table=multipart_uploads34252026/11/12 00:44:13 INFO Received push request method=POST path=/34262026/11/12 00:44:13 INFO Vacuumed table table=closures34272026/11/12 00:44:13 INFO Completed upload id=1 objects=15 duration=22.372292ms34282026/11/12 00:44:13 INFO Vacuumed table table=objects34292026/11/12 00:44:13 INFO Received uploads request method=POST path=/34302026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=034312026/11/12 00:44:13 INFO Received complete push request method=POST path=/34322026/11/12 00:44:13 INFO Received push request method=POST path=/34332026/11/12 00:44:13 INFO Received push request method=POST path=/34342026/11/12 00:44:13 INFO Completed push id=21 objects=3 duration=1.378291ms34352026/11/12 00:44:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034362026/11/12 00:44:13 INFO Received uploads request method=POST path=/34372026/11/12 00:44:13 INFO Vacuumed table table=pending_closures34382026/11/12 00:44:13 INFO Vacuumed table table=commit_receipts34392026/11/12 00:44:13 INFO Vacuumed table table=pending_objects34402026/11/12 00:44:13 INFO Vacuumed table table=multipart_uploads34412026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=034422026/11/12 00:44:13 INFO Received uploads request method=POST path=/34432026/11/12 00:44:13 INFO Received uploads request method=POST path=/34442026/11/12 00:44:13 INFO Vacuumed table table=closures34452026/11/12 00:44:13 WARN Force mode enabled - objects will be deleted immediately without grace period34462026/11/12 00:44:13 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034472026/11/12 00:44:13 INFO Vacuumed table table=objects34482026/11/12 00:44:13 INFO Vacuumed table table=pending_closures34492026/11/12 00:44:13 INFO Received uploads request method=POST path=/34502026/11/12 00:44:13 INFO Vacuumed table table=commit_receipts34512026/11/12 00:44:13 INFO Vacuumed table table=pending_objects34522026/11/12 00:44:13 INFO Received complete push request method=POST path=/34532026/11/12 00:44:13 INFO Received push request method=POST path=/34542026/11/12 00:44:13 INFO Vacuumed table table=multipart_uploads34552026/11/12 00:44:13 INFO Completed push id=5 objects=6 duration=563.958µs34562026/11/12 00:44:13 INFO Received uploads request method=POST path=/34572026/11/12 00:44:13 INFO Vacuumed table table=closures34582026/11/12 00:44:13 INFO Vacuumed table table=objects34592026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=034602026/11/12 00:44:13 INFO Received push request method=POST path=/34612026/11/12 00:44:13 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."34622026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=034632026/11/12 00:44:13 INFO Received push request method=POST path=/34642026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=034652026/11/12 00:44:13 INFO Received uploads request method=POST path=/34662026/11/12 00:44:13 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."34672026/11/12 00:44:13 INFO Aborted multipart uploads count=0 kept=034682026/11/12 00:44:13 WARN Force mode enabled - objects will be deleted immediately without grace period34692026/11/12 00:44:14 INFO Received push request method=POST path=/34702026/11/12 00:44:14 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."34712026/11/12 00:44:14 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=3 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=034722026/11/12 00:44:14 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."34732026/11/12 00:44:14 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."34742026/11/12 00:44:14 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."34752026/11/12 00:44:14 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."34762026/11/12 00:44:14 INFO Received complete upload request method=POST path=/34772026/11/12 00:44:14 INFO Vacuumed table table=pending_closures34782026/11/12 00:44:14 INFO Completed upload id=17 objects=25 duration=4.66175ms34792026/11/12 00:44:14 WARN Force mode enabled - objects will be deleted immediately without grace period34802026/11/12 00:44:14 INFO Vacuumed table table=commit_receipts34812026/11/12 00:44:14 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=034822026/11/12 00:44:14 INFO Vacuumed table table=pending_objects34832026/11/12 00:44:14 INFO Vacuumed table table=pending_closures34842026/11/12 00:44:14 INFO Vacuumed table table=multipart_uploads34852026/11/12 00:44:14 INFO Vacuumed table table=commit_receipts34862026/11/12 00:44:14 INFO Received uploads request method=POST path=/34872026/11/12 00:44:14 INFO Vacuumed table table=closures34882026/11/12 00:44:14 INFO Vacuumed table table=objects34892026/11/12 00:44:14 INFO Received push request method=POST path=/34902026/11/12 00:44:14 INFO Aborted multipart uploads count=0 kept=034912026/11/12 00:44:14 INFO Received push request method=POST path=/34922026/11/12 00:44:14 INFO Vacuumed table table=pending_objects34932026/11/12 00:44:14 INFO Received push request method=POST path=/34942026/11/12 00:44:14 INFO Vacuumed table table=multipart_uploads34952026/11/12 00:44:14 INFO Received push request method=POST path=/34962026/11/12 00:44:14 INFO Vacuumed table table=closures34972026/11/12 00:44:14 INFO Aborted multipart uploads count=0 kept=034982026/11/12 00:44:14 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."34992026/11/12 00:44:14 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."35002026/11/12 00:44:14 INFO Received push request method=POST path=/35012026/11/12 00:44:14 INFO Vacuumed table table=objects35022026/11/12 00:44:14 WARN Force mode enabled - objects will be deleted immediately without grace period35032026/11/12 00:44:14 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."35042026/11/12 00:44:14 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."35052026/11/12 00:44:14 INFO Received complete upload request method=POST path=/35062026/11/12 00:44:14 INFO Received uploads request method=POST path=/35072026/11/12 00:44:14 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."35082026/11/12 00:44:14 INFO Received uploads request method=POST path=/35092026/11/12 00:44:14 INFO Completed upload id=21 objects=2 duration=1.104208ms35102026/11/12 00:44:14 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."35112026/11/12 00:44:14 INFO Received push request method=POST path=/35122026/11/12 00:44:14 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."35132026/11/12 00:44:14 INFO Received push request method=POST path=/35142026/11/12 00:44:14 INFO Received uploads request method=POST path=/35152026/11/12 00:44:14 INFO Aborted multipart uploads count=0 kept=035162026/11/12 00:44:14 WARN Force mode enabled - objects will be deleted immediately without grace period35172026/11/12 00:44:14 INFO Received uploads request method=POST path=/35182026/11/12 00:44:14 INFO Received push request method=POST path=/35192026/11/12 00:44:14 INFO Received complete push request method=POST path=/35202026/11/12 00:44:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035212026/11/12 00:44:14 INFO Received push request method=POST path=/35222026/11/12 00:44:14 INFO Received complete push request method=POST path=/35232026/11/12 00:44:14 INFO Received uploads request method=POST path=/35242026/11/12 00:44:14 INFO Completed push id=11 objects=8 duration=6.240667ms35252026/11/12 00:44:14 INFO Vacuumed table table=pending_closures35262026/11/12 00:44:14 INFO Completed push id=30 objects=4 duration=4.413583ms35272026/11/12 00:44:14 INFO Vacuumed table table=commit_receipts35282026/11/12 00:44:14 INFO Received uploads request method=POST path=/35292026/11/12 00:44:14 INFO Vacuumed table table=pending_objects35302026/11/12 00:44:14 INFO Vacuumed table table=multipart_uploads35312026/11/12 00:44:14 INFO Received complete upload request method=POST path=/35322026/11/12 00:44:14 INFO Vacuumed table table=closures35332026/11/12 00:44:14 INFO Completed upload id=6 objects=7 duration=3.325334ms35342026/11/12 00:44:14 INFO Aborted multipart uploads count=0 kept=035352026/11/12 00:44:14 INFO Vacuumed table table=objects35362026/11/12 00:44:14 INFO Received uploads request method=POST path=/35372026/11/12 00:44:14 INFO Aborted multipart uploads count=0 kept=035382026/11/12 00:44:14 INFO Received push request method=POST path=/35392026/11/12 00:44:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035402026/11/12 00:44:14 WARN Force mode enabled - objects will be deleted immediately without grace period35412026/11/12 00:44:14 INFO Vacuumed table table=pending_closures35422026/11/12 00:44:14 INFO Vacuumed table table=commit_receipts35432026/11/12 00:44:14 INFO Vacuumed table table=pending_objects35442026/11/12 00:44:14 INFO Received push request method=POST path=/35452026/11/12 00:44:14 INFO Received complete push request method=POST path=/35462026/11/12 00:44:14 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."35472026/11/12 00:44:14 INFO Aborted multipart uploads count=0 kept=035482026/11/12 00:44:14 INFO Vacuumed table table=multipart_uploads35492026/11/12 00:44:14 INFO Received push request method=POST path=/35502026/11/12 00:44:14 INFO Completed push id=27 objects=25 duration=2.494166ms35512026/11/12 00:44:14 INFO Vacuumed table table=closures35522026/11/12 00:44:14 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."35532026/11/12 00:44:14 INFO Vacuumed table table=objects35542026/11/12 00:44:14 INFO Aborted multipart uploads count=0 kept=035552026/11/12 00:44:14 INFO Received push request method=POST path=/35562026/11/12 00:44:14 INFO Received complete push request method=POST path=/35572026/11/12 00:44:14 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=035582026/11/12 00:44:14 INFO Received uploads request method=POST path=/35592026/11/12 00:44:14 INFO Vacuumed table table=pending_closures35602026/11/12 00:44:14 INFO Vacuumed table table=commit_receipts35612026/11/12 00:44:14 INFO Vacuumed table table=pending_objects35622026/11/12 00:44:14 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."35632026/11/12 00:44:14 INFO Received complete push request method=POST path=/35642026/11/12 00:44:14 INFO Vacuumed table table=multipart_uploads35652026/11/12 00:44:14 INFO Received complete push request method=POST path=/35662026/11/12 00:44:14 INFO Received complete upload request method=POST path=/35672026/11/12 00:44:14 INFO Completed push id=32 objects=7 duration=1.032792ms35682026/11/12 00:44:14 INFO Vacuumed table table=closures35692026/11/12 00:44:14 INFO Completed push id=3 objects=8 duration=60.168875ms35702026/11/12 00:44:14 INFO Completed push id=19 objects=6 duration=2.448917ms35712026/11/12 00:44:14 INFO Aborted multipart uploads count=0 kept=035722026/11/12 00:44:14 INFO Completed upload id=5 objects=7 duration=3.297417ms35732026/11/12 00:44:14 INFO Vacuumed table table=objects35742026/11/12 00:44:14 INFO Received complete upload request method=POST path=/35752026/11/12 00:44:14 INFO Received complete push request method=POST path=/35762026/11/12 00:44:14 INFO Completed push id=26 objects=4 duration=883.834µs35772026/11/12 00:44:14 INFO Completed upload id=14 objects=8 duration=2.524709ms35782026/11/12 00:44:14 INFO Received complete push request method=POST path=/35792026/11/12 00:44:14 INFO Received complete upload request method=POST path=/35802026/11/12 00:44:14 INFO Completed push id=24 objects=5 duration=3.705667ms35812026/11/12 00:44:14 INFO Received push request method=POST path=/35822026/11/12 00:44:14 INFO Completed upload id=28 objects=13 duration=1.169667ms35832026/11/12 00:44:14 INFO Aborted multipart uploads count=0 kept=035842026/11/12 00:44:14 INFO Received complete push request method=POST path=/35852026/11/12 00:44:14 WARN Force mode enabled - objects will be deleted immediately without grace period35862026/11/12 00:44:14 INFO Completed push id=4 objects=7 duration=9.64375ms35872026/11/12 00:44:14 INFO Received complete push request method=POST path=/35882026/11/12 00:44:14 INFO Received uploads request method=POST path=/35892026/11/12 00:44:14 INFO Completed push id=29 objects=3 duration=948.709µs35902026/11/12 00:44:14 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."35912026/11/12 00:44:14 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."35922026/11/12 00:44:14 INFO Received uploads request method=POST path=/35932026/11/12 00:44:14 INFO Received complete upload request method=POST path=/35942026/11/12 00:44:14 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."35952026/11/12 00:44:14 INFO Received uploads request method=POST path=/35962026/11/12 00:44:14 INFO Completed upload id=6 objects=9 duration=1.127125ms35972026/11/12 00:44:14 INFO Received complete upload request method=POST path=/35982026/11/12 00:44:14 INFO Aborted multipart uploads count=0 kept=035992026/11/12 00:44:14 INFO Completed upload id=31 objects=2 duration=1.066ms36002026/11/12 00:44:14 WARN Force mode enabled - objects will be deleted immediately without grace period36012026/11/12 00:44:14 INFO Aborted multipart uploads count=0 kept=036022026/11/12 00:44:14 INFO Received complete upload request method=POST path=/36032026/11/12 00:44:14 INFO Received uploads request method=POST path=/36042026/11/12 00:44:14 INFO Completed upload id=10 objects=6 duration=1.552666ms36052026/11/12 00:44:14 WARN Force mode enabled - objects will be deleted immediately without grace period36062026/11/12 00:44:14 ERROR failed to remove object object=dddddddddddddddddddddddddddddddd.ls error="Access Denied."36072026/11/12 00:44:14 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."36082026/11/12 00:44:14 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."36092026/11/12 00:44:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=036102026/11/12 00:44:14 INFO Received complete push request method=POST path=/36112026/11/12 00:44:14 INFO Completed push id=33 objects=6 duration=1.019583ms36122026/11/12 00:44:14 INFO Vacuumed table table=pending_closures36132026/11/12 00:44:14 INFO Received complete push request method=POST path=/36142026/11/12 00:44:14 INFO Received uploads request method=POST path=/36152026/11/12 00:44:14 INFO Vacuumed table table=commit_receipts36162026/11/12 00:44:14 INFO Completed push id=13 objects=6 duration=5.677792ms36172026/11/12 00:44:14 INFO Vacuumed table table=pending_objects36182026/11/12 00:44:14 INFO Vacuumed table table=multipart_uploads36192026/11/12 00:44:14 INFO Received push request method=POST path=/36202026/11/12 00:44:14 INFO Vacuumed table table=closures36212026/11/12 00:44:14 INFO Received push request method=POST path=/36222026/11/12 00:44:14 INFO Aborted multipart uploads count=0 kept=036232026/11/12 00:44:14 INFO Vacuumed table table=objects36242026/11/12 00:44:14 WARN Force mode enabled - objects will be deleted immediately without grace period36252026/11/12 00:44:14 INFO Received complete upload request method=POST path=/36262026/11/12 00:44:14 INFO Received complete push request method=POST path=/36272026/11/12 00:44:14 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=036282026/11/12 00:44:14 INFO Received push request method=POST path=/36292026/11/12 00:44:14 INFO Received push request method=POST path=/36302026/11/12 00:44:14 INFO Completed upload id=3 objects=5 duration=810.917µs36312026/11/12 00:44:14 INFO Aborted multipart uploads count=0 kept=036322026/11/12 00:44:14 INFO Completed push id=7 objects=24 duration=2.919625ms36332026/11/12 00:44:14 INFO Vacuumed table table=pending_closures36342026/11/12 00:44:14 INFO Vacuumed table table=commit_receipts36352026/11/12 00:44:14 INFO Received complete push request method=POST path=/36362026/11/12 00:44:14 WARN Force mode enabled - objects will be deleted immediately without grace period36372026/11/12 00:44:14 INFO Completed push id=8 objects=6 duration=2.614334ms36382026/11/12 00:44:14 INFO Vacuumed table table=pending_objects36392026/11/12 00:44:14 INFO Received uploads request method=POST path=/36402026/11/12 00:44:14 INFO Aborted multipart uploads count=0 kept=036412026/11/12 00:44:14 INFO Vacuumed table table=multipart_uploads36422026/11/12 00:44:14 INFO Vacuumed table table=closures36432026/11/12 00:44:14 INFO Received complete push request method=POST path=/36442026/11/12 00:44:14 INFO Vacuumed table table=objects36452026/11/12 00:44:14 INFO Received uploads request method=POST path=/36462026/11/12 00:44:14 INFO Completed push id=21 objects=26 duration=1.121458ms36472026/11/12 00:44:14 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=036482026/11/12 00:44:15 INFO Vacuumed table table=pending_closures36492026/11/12 00:44:15 INFO Vacuumed table table=commit_receipts36502026/11/12 00:44:15 INFO Vacuumed table table=pending_objects36512026/11/12 00:44:15 ERROR failed to remove object object=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb.ls error="Access Denied."36522026/11/12 00:44:15 INFO Vacuumed table table=multipart_uploads36532026/11/12 00:44:15 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.ls error="Access Denied."36542026/11/12 00:44:15 ERROR failed to remove object object=gggggggggggggggggggggggggggggggg.narinfo error="Access Denied."36552026/11/12 00:44:15 INFO Vacuumed table table=closures36562026/11/12 00:44:15 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."36572026/11/12 00:44:15 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.narinfo error="Access Denied."36582026/11/12 00:44:15 ERROR failed to remove object object=nar/gggggggggggggggggggggggggggggggggggggggggggggggggggg.nar.zst error="Access Denied."36592026/11/12 00:44:15 INFO Vacuumed table table=objects36602026/11/12 00:44:15 ERROR failed to remove object object=nar/hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.nar.zst error="Access Denied."36612026/11/12 00:44:15 ERROR failed to remove object object=realisations/sha256:bbbbbbbb!out.doi error="Access Denied."36622026/11/12 00:44:15 INFO Received uploads request method=POST path=/36632026/11/12 00:44:15 ERROR failed to remove object object=realisations/sha256:dddddddd!out.doi error="Access Denied."36642026/11/12 00:44:15 ERROR failed to remove object object=realisations/sha256:gggggggg!out.doi error="Access Denied."36652026/11/12 00:44:15 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."36662026/11/12 00:44:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=036672026/11/12 00:44:15 INFO Vacuumed table table=pending_closures36682026/11/12 00:44:15 INFO Received uploads request method=POST path=/36692026/11/12 00:44:15 INFO Received complete push request method=POST path=/36702026/11/12 00:44:15 INFO Received push request method=POST path=/36712026/11/12 00:44:15 INFO Completed push id=12 objects=7 duration=1.320125ms36722026/11/12 00:44:15 INFO Received push request method=POST path=/36732026/11/12 00:44:15 INFO Received complete push request method=POST path=/36742026/11/12 00:44:15 INFO Aborted multipart uploads count=0 kept=036752026/11/12 00:44:15 INFO Vacuumed table table=commit_receipts36762026/11/12 00:44:15 INFO Received uploads request method=POST path=/36772026/11/12 00:44:15 INFO Completed push id=8 objects=2 duration=2.073167ms36782026/11/12 00:44:15 INFO Vacuumed table table=pending_objects36792026/11/12 00:44:15 INFO Received uploads request method=POST path=/36802026/11/12 00:44:15 INFO Vacuumed table table=multipart_uploads36812026/11/12 00:44:15 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=6 objects-failed-to-delete=036822026/11/12 00:44:15 INFO Received complete push request method=POST path=/36832026/11/12 00:44:15 INFO Completed push id=13 objects=2 duration=933.625µs36842026/11/12 00:44:15 INFO Received complete upload request method=POST path=/36852026/11/12 00:44:15 INFO Completed upload id=28 objects=17 duration=876.625µs36862026/11/12 00:44:15 INFO Received complete upload request method=POST path=/36872026/11/12 00:44:15 INFO Received complete push request method=POST path=/36882026/11/12 00:44:15 INFO Vacuumed table table=pending_closures36892026/11/12 00:44:15 INFO Completed upload id=27 objects=13 duration=2.148416ms36902026/11/12 00:44:15 INFO Completed push id=2 objects=10 duration=3.413083ms36912026/11/12 00:44:15 INFO Vacuumed table table=closures36922026/11/12 00:44:15 INFO Vacuumed table table=commit_receipts36932026/11/12 00:44:15 INFO Received complete push request method=POST path=/36942026/11/12 00:44:15 INFO Received complete upload request method=POST path=/36952026/11/12 00:44:15 INFO Received complete upload request method=POST path=/36962026/11/12 00:44:15 INFO Completed upload id=19 objects=25 duration=3.536833ms36972026/11/12 00:44:15 INFO Completed upload id=16 objects=17 duration=4.623875ms36982026/11/12 00:44:15 INFO Completed push id=29 objects=2 duration=4.870583ms36992026/11/12 00:44:15 INFO Vacuumed table table=objects37002026/11/12 00:44:15 INFO Aborted multipart uploads count=0 kept=037012026/11/12 00:44:15 INFO Received uploads request method=POST path=/37022026/11/12 00:44:15 INFO Vacuumed table table=pending_objects37032026/11/12 00:44:15 INFO Vacuumed table table=multipart_uploads37042026/11/12 00:44:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037052026/11/12 00:44:15 INFO Vacuumed table table=closures37062026/11/12 00:44:15 INFO Vacuumed table table=objects37072026/11/12 00:44:15 INFO Vacuumed table table=pending_closures37082026/11/12 00:44:15 INFO Aborted multipart uploads count=0 kept=037092026/11/12 00:44:15 INFO Received push request method=POST path=/37102026/11/12 00:44:15 INFO Aborted multipart uploads count=0 kept=037112026/11/12 00:44:15 INFO Vacuumed table table=commit_receipts37122026/11/12 00:44:15 INFO Received complete push request method=POST path=/37132026/11/12 00:44:15 WARN Force mode enabled - objects will be deleted immediately without grace period37142026/11/12 00:44:15 WARN Force mode enabled - objects will be deleted immediately without grace period37152026/11/12 00:44:15 INFO Vacuumed table table=pending_objects37162026/11/12 00:44:15 INFO Received complete push request method=POST path=/37172026/11/12 00:44:15 INFO Completed push id=31 objects=6 duration=1.65175ms37182026/11/12 00:44:15 INFO Vacuumed table table=multipart_uploads37192026/11/12 00:44:15 INFO Received complete push request method=POST path=/37202026/11/12 00:44:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=037212026/11/12 00:44:15 INFO Completed push id=15 objects=24 duration=3.413542ms37222026/11/12 00:44:15 INFO Vacuumed table table=closures37232026/11/12 00:44:15 INFO Vacuumed table table=pending_closures37242026/11/12 00:44:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=037252026/11/12 00:44:15 INFO Completed push id=8 objects=7 duration=1.416666ms37262026/11/12 00:44:15 INFO Vacuumed table table=objects37272026/11/12 00:44:15 INFO Vacuumed table table=commit_receipts37282026/11/12 00:44:15 INFO Vacuumed table table=pending_closures37292026/11/12 00:44:15 INFO Vacuumed table table=pending_objects37302026/11/12 00:44:15 INFO Received uploads request method=POST path=/37312026/11/12 00:44:15 INFO Received uploads request method=POST path=/37322026/11/12 00:44:15 INFO Vacuumed table table=commit_receipts37332026/11/12 00:44:15 INFO Vacuumed table table=pending_objects37342026/11/12 00:44:15 INFO Vacuumed table table=multipart_uploads37352026/11/12 00:44:15 INFO Vacuumed table table=multipart_uploads37362026/11/12 00:44:15 INFO Received complete push request method=POST path=/37372026/11/12 00:44:15 INFO Vacuumed table table=closures37382026/11/12 00:44:15 INFO Received complete push request method=POST path=/37392026/11/12 00:44:15 INFO Vacuumed table table=closures37402026/11/12 00:44:15 INFO Completed push id=32 objects=9 duration=2.838416ms37412026/11/12 00:44:15 INFO Vacuumed table table=objects37422026/11/12 00:44:15 INFO Completed push id=17 objects=6 duration=4.97775ms37432026/11/12 00:44:15 INFO Received uploads request method=POST path=/37442026/11/12 00:44:15 INFO Received uploads request method=POST path=/37452026/11/12 00:44:15 INFO Vacuumed table table=objects37462026/11/12 00:44:15 INFO Received uploads request method=POST path=/37472026/11/12 00:44:15 INFO Received uploads request method=POST path=/37482026/11/12 00:44:15 INFO Received push request method=POST path=/37492026/11/12 00:44:15 INFO Received uploads request method=POST path=/37502026/11/12 00:44:15 INFO Received complete push request method=POST path=/37512026/11/12 00:44:15 INFO Completed push id=13 objects=15 duration=2.185375ms37522026/11/12 00:44:15 INFO Received complete push request method=POST path=/37532026/11/12 00:44:15 INFO Received push request method=POST path=/37542026/11/12 00:44:15 INFO Completed push id=15 objects=7 duration=1.692791ms37552026/11/12 00:44:15 INFO Received uploads request method=POST path=/37562026/11/12 00:44:15 INFO Received push request method=POST path=/37572026/11/12 00:44:15 INFO Aborted multipart uploads count=0 kept=037582026/11/12 00:44:15 INFO Received uploads request method=POST path=/37592026/11/12 00:44:15 INFO Received uploads request method=POST path=/37602026/11/12 00:44:15 INFO Received complete upload request method=POST path=/37612026/11/12 00:44:15 INFO Received uploads request method=POST path=/37622026/11/12 00:44:15 INFO Completed upload id=33 objects=9 duration=691.416µs37632026/11/12 00:44:15 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."37642026/11/12 00:44:15 INFO Received uploads request method=POST path=/37652026/11/12 00:44:15 INFO Received uploads request method=POST path=/37662026/11/12 00:44:15 INFO Received push request method=POST path=/37672026/11/12 00:44:15 INFO Aborted multipart uploads count=0 kept=037682026/11/12 00:44:15 INFO Received complete upload request method=POST path=/37692026/11/12 00:44:15 INFO Received complete upload request method=POST path=/37702026/11/12 00:44:15 INFO Completed upload id=9 objects=16 duration=6.389208ms37712026/11/12 00:44:15 INFO Completed upload id=6 objects=6 duration=6.422208ms37722026/11/12 00:44:15 INFO Aborted multipart uploads count=0 kept=037732026/11/12 00:44:15 INFO Received complete upload request method=POST path=/37742026/11/12 00:44:15 INFO Received uploads request method=POST path=/37752026/11/12 00:44:15 INFO Received uploads request method=POST path=/37762026/11/12 00:44:15 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037772026/11/12 00:44:15 WARN Force mode enabled - objects will be deleted immediately without grace period37782026/11/12 00:44:15 INFO Vacuumed table table=pending_closures37792026/11/12 00:44:15 INFO Vacuumed table table=commit_receipts37802026/11/12 00:44:15 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=037812026/11/12 00:44:15 INFO Received uploads request method=POST path=/37822026/11/12 00:44:15 INFO Vacuumed table table=pending_objects37832026/11/12 00:44:15 INFO Completed upload id=2 objects=26 duration=24.07025ms37842026/11/12 00:44:15 INFO Vacuumed table table=multipart_uploads37852026/11/12 00:44:15 INFO Received push request method=POST path=/37862026/11/12 00:44:15 INFO Vacuumed table table=closures37872026/11/12 00:44:15 INFO Received push request method=POST path=/37882026/11/12 00:44:15 INFO Vacuumed table table=pending_closures37892026/11/12 00:44:16 INFO Received complete upload request method=POST path=/37902026/11/12 00:44:16 INFO Vacuumed table table=objects37912026/11/12 00:44:16 INFO Vacuumed table table=commit_receipts37922026/11/12 00:44:16 INFO Received push request method=POST path=/37932026/11/12 00:44:16 INFO Aborted multipart uploads count=0 kept=037942026/11/12 00:44:16 INFO Received complete upload request method=POST path=/37952026/11/12 00:44:16 INFO Completed upload id=3 objects=24 duration=2.126667ms37962026/11/12 00:44:16 INFO Received push request method=POST path=/37972026/11/12 00:44:16 INFO Vacuumed table table=pending_objects37982026/11/12 00:44:16 WARN Force mode enabled - objects will be deleted immediately without grace period37992026/11/12 00:44:16 INFO Completed upload id=29 objects=4 duration=2.26725ms38002026/11/12 00:44:16 INFO Aborted multipart uploads count=0 kept=038012026/11/12 00:44:16 INFO Vacuumed table table=multipart_uploads38022026/11/12 00:44:16 INFO Received push request method=POST path=/38032026/11/12 00:44:16 WARN Force mode enabled - objects will be deleted immediately without grace period38042026/11/12 00:44:16 INFO Received complete push request method=POST path=/38052026/11/12 00:44:16 INFO Received push request method=POST path=/38062026/11/12 00:44:16 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038072026/11/12 00:44:16 INFO Completed push id=4 objects=7 duration=1.520209ms38082026/11/12 00:44:16 INFO Vacuumed table table=closures38092026/11/12 00:44:16 INFO Vacuumed table table=pending_closures38102026/11/12 00:44:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038112026/11/12 00:44:16 INFO Vacuumed table table=objects38122026/11/12 00:44:16 INFO Vacuumed table table=commit_receipts38132026/11/12 00:44:16 INFO Received push request method=POST path=/38142026/11/12 00:44:16 INFO Received complete upload request method=POST path=/38152026/11/12 00:44:16 INFO Received uploads request method=POST path=/38162026/11/12 00:44:16 INFO Received uploads request method=POST path=/38172026/11/12 00:44:16 INFO Received complete push request method=POST path=/38182026/11/12 00:44:16 INFO Vacuumed table table=pending_closures38192026/11/12 00:44:16 INFO Completed upload id=22 objects=2 duration=1.111708ms38202026/11/12 00:44:16 INFO Received complete push request method=POST path=/38212026/11/12 00:44:16 INFO Completed push id=10 objects=3 duration=1.075416ms38222026/11/12 00:44:16 INFO Vacuumed table table=pending_objects38232026/11/12 00:44:16 INFO Vacuumed table table=commit_receipts38242026/11/12 00:44:16 INFO Completed push id=8 objects=6 duration=3.35725ms38252026/11/12 00:44:16 INFO Vacuumed table table=multipart_uploads38262026/11/12 00:44:16 INFO Vacuumed table table=closures38272026/11/12 00:44:16 INFO Vacuumed table table=pending_objects38282026/11/12 00:44:16 INFO Vacuumed table table=multipart_uploads38292026/11/12 00:44:16 INFO Vacuumed table table=objects38302026/11/12 00:44:16 INFO Vacuumed table table=closures38312026/11/12 00:44:16 INFO Vacuumed table table=objects38322026/11/12 00:44:16 INFO Received uploads request method=POST path=/38332026/11/12 00:44:16 INFO Received push request method=POST path=/38342026/11/12 00:44:16 INFO Received uploads request method=POST path=/38352026/11/12 00:44:16 INFO Aborted multipart uploads count=0 kept=038362026/11/12 00:44:16 INFO Received uploads request method=POST path=/38372026/11/12 00:44:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=038382026/11/12 00:44:16 INFO Received push request method=POST path=/38392026/11/12 00:44:16 INFO Vacuumed table table=pending_closures38402026/11/12 00:44:16 INFO Received uploads request method=POST path=/38412026/11/12 00:44:16 INFO Received complete push request method=POST path=/38422026/11/12 00:44:16 INFO Vacuumed table table=commit_receipts38432026/11/12 00:44:16 INFO Completed push id=20 objects=3 duration=5.757333ms38442026/11/12 00:44:16 INFO Vacuumed table table=pending_objects38452026/11/12 00:44:16 INFO Received push request method=POST path=/38462026/11/12 00:44:16 INFO Vacuumed table table=multipart_uploads38472026/11/12 00:44:16 INFO Vacuumed table table=closures38482026/11/12 00:44:16 INFO Received push request method=POST path=/38492026/11/12 00:44:16 INFO Vacuumed table table=objects38502026/11/12 00:44:16 INFO Received push request method=POST path=/38512026/11/12 00:44:16 INFO Received uploads request method=POST path=/38522026/11/12 00:44:16 INFO Received complete push request method=POST path=/38532026/11/12 00:44:16 INFO Completed push id=16 objects=4 duration=2.0935ms38542026/11/12 00:44:16 INFO Aborted multipart uploads count=0 kept=038552026/11/12 00:44:16 WARN Force mode enabled - objects will be deleted immediately without grace period38562026/11/12 00:44:16 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038572026/11/12 00:44:16 INFO Vacuumed table table=pending_closures38582026/11/12 00:44:16 INFO Received push request method=POST path=/38592026/11/12 00:44:16 INFO Received uploads request method=POST path=/38602026/11/12 00:44:16 INFO Vacuumed table table=commit_receipts38612026/11/12 00:44:16 INFO Received complete push request method=POST path=/38622026/11/12 00:44:16 INFO Vacuumed table table=pending_objects38632026/11/12 00:44:16 INFO Completed push id=11 objects=6 duration=2.041834ms38642026/11/12 00:44:16 INFO Vacuumed table table=multipart_uploads38652026/11/12 00:44:16 INFO Vacuumed table table=closures38662026/11/12 00:44:16 INFO Vacuumed table table=objects38672026/11/12 00:44:16 INFO Received uploads request method=POST path=/38682026/11/12 00:44:16 INFO Aborted multipart uploads count=0 kept=038692026/11/12 00:44:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=038702026/11/12 00:44:16 INFO Vacuumed table table=pending_closures38712026/11/12 00:44:16 INFO Aborted multipart uploads count=0 kept=038722026/11/12 00:44:16 INFO Received uploads request method=POST path=/38732026/11/12 00:44:16 WARN Force mode enabled - objects will be deleted immediately without grace period38742026/11/12 00:44:16 INFO Received complete upload request method=POST path=/38752026/11/12 00:44:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=038762026/11/12 00:44:16 INFO Completed upload id=20 objects=5 duration=1.540625ms38772026/11/12 00:44:16 INFO Vacuumed table table=pending_closures38782026/11/12 00:44:16 INFO Vacuumed table table=commit_receipts38792026/11/12 00:44:16 INFO Vacuumed table table=commit_receipts38802026/11/12 00:44:16 INFO Vacuumed table table=pending_objects38812026/11/12 00:44:16 INFO Vacuumed table table=pending_objects38822026/11/12 00:44:16 INFO Vacuumed table table=multipart_uploads38832026/11/12 00:44:16 INFO Vacuumed table table=multipart_uploads38842026/11/12 00:44:16 INFO Vacuumed table table=closures38852026/11/12 00:44:16 INFO Vacuumed table table=closures38862026/11/12 00:44:16 INFO Vacuumed table table=objects38872026/11/12 00:44:16 INFO Vacuumed table table=objects38882026/11/12 00:44:16 INFO Received complete push request method=POST path=/38892026/11/12 00:44:16 INFO Completed push id=19 objects=8 duration=3.116ms38902026/11/12 00:44:16 INFO Received push request method=POST path=/38912026/11/12 00:44:16 INFO Received complete push request method=POST path=/38922026/11/12 00:44:16 INFO Received complete push request method=POST path=/38932026/11/12 00:44:16 INFO Received push request method=POST path=/38942026/11/12 00:44:16 INFO Completed push id=20 objects=13 duration=1.349959ms38952026/11/12 00:44:16 INFO Received uploads request method=POST path=/38962026/11/12 00:44:16 INFO Received push request method=POST path=/38972026/11/12 00:44:16 INFO Received push request method=POST path=/38982026/11/12 00:44:16 INFO Received push request method=POST path=/38992026/11/12 00:44:16 INFO Received uploads request method=POST path=/39002026/11/12 00:44:16 INFO Completed push id=5 objects=9 duration=10.57575ms39012026/11/12 00:44:16 INFO Received uploads request method=POST path=/39022026/11/12 00:44:16 INFO Received push request method=POST path=/39032026/11/12 00:44:16 INFO Received complete push request method=POST path=/39042026/11/12 00:44:16 INFO Received uploads request method=POST path=/39052026/11/12 00:44:16 INFO Aborted multipart uploads count=0 kept=039062026/11/12 00:44:16 INFO Received push request method=POST path=/39072026/11/12 00:44:16 INFO Completed push id=22 objects=4 duration=1.492084ms39082026/11/12 00:44:16 INFO Received push request method=POST path=/39092026/11/12 00:44:16 INFO Received push request method=POST path=/39102026/11/12 00:44:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039112026/11/12 00:44:16 INFO Received push request method=POST path=/39122026/11/12 00:44:16 INFO Received push request method=POST path=/39132026/11/12 00:44:16 INFO Vacuumed table table=pending_closures39142026/11/12 00:44:16 INFO Aborted multipart uploads count=0 kept=039152026/11/12 00:44:16 INFO Vacuumed table table=commit_receipts39162026/11/12 00:44:16 WARN Force mode enabled - objects will be deleted immediately without grace period39172026/11/12 00:44:16 INFO Vacuumed table table=pending_objects39182026/11/12 00:44:16 INFO Vacuumed table table=multipart_uploads39192026/11/12 00:44:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039202026/11/12 00:44:16 INFO Aborted multipart uploads count=0 kept=039212026/11/12 00:44:16 INFO Received push request method=POST path=/39222026/11/12 00:44:16 INFO Received push request method=POST path=/39232026/11/12 00:44:16 INFO Vacuumed table table=pending_closures39242026/11/12 00:44:16 INFO Received push request method=POST path=/39252026/11/12 00:44:16 INFO Vacuumed table table=closures39262026/11/12 00:44:16 INFO Vacuumed table table=commit_receipts39272026/11/12 00:44:16 INFO Received push request method=POST path=/39282026/11/12 00:44:16 INFO Vacuumed table table=objects39292026/11/12 00:44:16 INFO Vacuumed table table=pending_objects39302026/11/12 00:44:16 INFO Vacuumed table table=multipart_uploads39312026/11/12 00:44:16 INFO Received complete upload request method=POST path=/39322026/11/12 00:44:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=8 objects-deleted-after-grace-period=8 objects-failed-to-delete=039332026/11/12 00:44:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039342026/11/12 00:44:16 INFO Vacuumed table table=closures39352026/11/12 00:44:16 INFO Completed upload id=22 objects=6 duration=3.478334ms39362026/11/12 00:44:16 INFO Vacuumed table table=pending_closures39372026/11/12 00:44:16 INFO Vacuumed table table=pending_closures39382026/11/12 00:44:16 INFO Vacuumed table table=objects39392026/11/12 00:44:16 INFO Vacuumed table table=commit_receipts39402026/11/12 00:44:16 INFO Vacuumed table table=commit_receipts39412026/11/12 00:44:16 INFO Vacuumed table table=pending_objects39422026/11/12 00:44:16 INFO Received uploads request method=POST path=/39432026/11/12 00:44:16 INFO Vacuumed table table=multipart_uploads39442026/11/12 00:44:16 INFO Vacuumed table table=pending_objects39452026/11/12 00:44:16 INFO Aborted multipart uploads count=0 kept=039462026/11/12 00:44:16 INFO Received uploads request method=POST path=/39472026/11/12 00:44:16 INFO Received push request method=POST path=/39482026/11/12 00:44:16 INFO Vacuumed table table=closures39492026/11/12 00:44:16 INFO Aborted multipart uploads count=0 kept=039502026/11/12 00:44:16 INFO Vacuumed table table=multipart_uploads39512026/11/12 00:44:16 INFO Vacuumed table table=closures39522026/11/12 00:44:16 INFO Received complete upload request method=POST path=/39532026/11/12 00:44:16 WARN Force mode enabled - objects will be deleted immediately without grace period39542026/11/12 00:44:16 INFO Received uploads request method=POST path=/39552026/11/12 00:44:16 WARN Force mode enabled - objects will be deleted immediately without grace period39562026/11/12 00:44:16 INFO Completed upload id=23 objects=7 duration=820.791µs39572026/11/12 00:44:16 INFO Vacuumed table table=objects39582026/11/12 00:44:16 INFO Vacuumed table table=objects39592026/11/12 00:44:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039602026/11/12 00:44:16 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039612026/11/12 00:44:16 INFO Received complete upload request method=POST path=/39622026/11/12 00:44:16 INFO Vacuumed table table=pending_closures39632026/11/12 00:44:16 INFO Completed upload id=10 objects=23 duration=5.675917ms39642026/11/12 00:44:16 INFO Vacuumed table table=pending_closures39652026/11/12 00:44:16 INFO Vacuumed table table=commit_receipts39662026/11/12 00:44:16 INFO Vacuumed table table=commit_receipts39672026/11/12 00:44:16 INFO Vacuumed table table=pending_objects39682026/11/12 00:44:16 INFO Vacuumed table table=pending_objects39692026/11/12 00:44:16 INFO Vacuumed table table=multipart_uploads39702026/11/12 00:44:16 INFO Received uploads request method=POST path=/39712026/11/12 00:44:16 INFO Vacuumed table table=multipart_uploads39722026/11/12 00:44:16 INFO Vacuumed table table=closures39732026/11/12 00:44:16 INFO Vacuumed table table=closures39742026/11/12 00:44:16 INFO Vacuumed table table=objects39752026/11/12 00:44:16 INFO Vacuumed table table=objects39762026/11/12 00:44:16 INFO Received push request method=POST path=/39772026/11/12 00:44:16 INFO Received complete push request method=POST path=/39782026/11/12 00:44:16 INFO Received push request method=POST path=/39792026/11/12 00:44:16 INFO Completed push id=24 objects=5 duration=926.416µs39802026/11/12 00:44:16 INFO Received push request method=POST path=/39812026/11/12 00:44:16 INFO Received push request method=POST path=/39822026/11/12 00:44:16 INFO Received uploads request method=POST path=/39832026/11/12 00:44:17 INFO Aborted multipart uploads count=0 kept=039842026/11/12 00:44:17 INFO Received uploads request method=POST path=/39852026/11/12 00:44:17 INFO Received push request method=POST path=/39862026/11/12 00:44:17 WARN Force mode enabled - objects will be deleted immediately without grace period39872026/11/12 00:44:17 INFO Received push request method=POST path=/39882026/11/12 00:44:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=039892026/11/12 00:44:17 INFO Vacuumed table table=pending_closures39902026/11/12 00:44:17 INFO Vacuumed table table=commit_receipts39912026/11/12 00:44:17 INFO Vacuumed table table=pending_objects39922026/11/12 00:44:17 INFO Received complete upload request method=POST path=/39932026/11/12 00:44:17 INFO Vacuumed table table=multipart_uploads39942026/11/12 00:44:17 INFO Completed upload id=25 objects=2 duration=681.917µs39952026/11/12 00:44:17 INFO Vacuumed table table=closures39962026/11/12 00:44:17 INFO Received complete push request method=POST path=/39972026/11/12 00:44:17 INFO Vacuumed table table=objects39982026/11/12 00:44:17 INFO Received uploads request method=POST path=/39992026/11/12 00:44:17 INFO Completed push id=7 objects=7 duration=876.042µs40002026/11/12 00:44:17 INFO Received complete push request method=POST path=/40012026/11/12 00:44:17 INFO Completed push id=7 objects=20 duration=1.847334ms40022026/11/12 00:44:17 INFO Received complete upload request method=POST path=/40032026/11/12 00:44:17 INFO Completed upload id=17 objects=6 duration=1.522291ms40042026/11/12 00:44:17 INFO Received complete push request method=POST path=/40052026/11/12 00:44:17 INFO Completed push id=9 objects=24 duration=952.625µs40062026/11/12 00:44:17 INFO Aborted multipart uploads count=0 kept=040072026/11/12 00:44:17 INFO Received push request method=POST path=/40082026/11/12 00:44:17 WARN Force mode enabled - objects will be deleted immediately without grace period40092026/11/12 00:44:17 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040102026/11/12 00:44:17 INFO Vacuumed table table=pending_closures40112026/11/12 00:44:17 INFO Vacuumed table table=commit_receipts40122026/11/12 00:44:17 INFO Vacuumed table table=pending_objects40132026/11/12 00:44:17 INFO Vacuumed table table=multipart_uploads40142026/11/12 00:44:17 INFO Vacuumed table table=closures40152026/11/12 00:44:17 INFO Vacuumed table table=objects40162026/11/12 00:44:17 INFO Aborted multipart uploads count=0 kept=040172026/11/12 00:44:17 INFO Received push request method=POST path=/40182026/11/12 00:44:17 WARN Force mode enabled - objects will be deleted immediately without grace period40192026/11/12 00:44:17 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040202026/11/12 00:44:17 INFO Vacuumed table table=pending_closures40212026/11/12 00:44:17 INFO Vacuumed table table=commit_receipts40222026/11/12 00:44:17 INFO Vacuumed table table=pending_objects40232026/11/12 00:44:17 INFO Vacuumed table table=multipart_uploads40242026/11/12 00:44:17 INFO Vacuumed table table=closures40252026/11/12 00:44:17 INFO Vacuumed table table=objects40262026/11/12 00:44:17 INFO Received complete upload request method=POST path=/40272026/11/12 00:44:17 INFO Received uploads request method=POST path=/40282026/11/12 00:44:17 INFO Completed upload id=26 objects=20 duration=632.417µs40292026/11/12 00:44:17 INFO Received complete upload request method=POST path=/40302026/11/12 00:44:17 INFO Completed upload id=11 objects=6 duration=1.649542ms40312026/11/12 00:44:17 INFO Received push request method=POST path=/40322026/11/12 00:44:17 INFO Received complete upload request method=POST path=/40332026/11/12 00:44:17 INFO Received uploads request method=POST path=/40342026/11/12 00:44:17 INFO Completed upload id=15 objects=3 duration=1.594125ms40352026/11/12 00:44:17 INFO Received push request method=POST path=/40362026/11/12 00:44:17 INFO Aborted multipart uploads count=0 kept=040372026/11/12 00:44:17 INFO Aborted multipart uploads count=0 kept=040382026/11/12 00:44:17 INFO Received push request method=POST path=/40392026/11/12 00:44:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040402026/11/12 00:44:17 INFO Received uploads request method=POST path=/40412026/11/12 00:44:17 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040422026/11/12 00:44:17 INFO Vacuumed table table=pending_closures40432026/11/12 00:44:17 INFO Vacuumed table table=pending_closures40442026/11/12 00:44:17 INFO Vacuumed table table=commit_receipts40452026/11/12 00:44:17 INFO Vacuumed table table=commit_receipts40462026/11/12 00:44:17 INFO Vacuumed table table=pending_objects40472026/11/12 00:44:17 INFO Vacuumed table table=multipart_uploads40482026/11/12 00:44:17 INFO Vacuumed table table=closures40492026/11/12 00:44:17 INFO Vacuumed table table=pending_objects40502026/11/12 00:44:17 INFO Vacuumed table table=objects40512026/11/12 00:44:17 INFO Vacuumed table table=multipart_uploads40522026/11/12 00:44:17 INFO Received push request method=POST path=/40532026/11/12 00:44:17 INFO Received uploads request method=POST path=/40542026/11/12 00:44:17 INFO Vacuumed table table=closures40552026/11/12 00:44:17 INFO Aborted multipart uploads count=0 kept=040562026/11/12 00:44:17 INFO Received uploads request method=POST path=/40572026/11/12 00:44:17 INFO Received uploads request method=POST path=/40582026/11/12 00:44:17 INFO Vacuumed table table=objects40592026/11/12 00:44:17 WARN Force mode enabled - objects will be deleted immediately without grace period40602026/11/12 00:44:17 INFO Received uploads request method=POST path=/40612026/11/12 00:44:17 INFO Received complete push request method=POST path=/40622026/11/12 00:44:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=040632026/11/12 00:44:17 INFO Vacuumed table table=pending_closures40642026/11/12 00:44:17 INFO Completed push id=28 objects=7 duration=546.5µs40652026/11/12 00:44:17 INFO Vacuumed table table=commit_receipts40662026/11/12 00:44:17 INFO Vacuumed table table=pending_objects40672026/11/12 00:44:17 INFO Received uploads request method=POST path=/40682026/11/12 00:44:17 INFO Vacuumed table table=multipart_uploads40692026/11/12 00:44:17 INFO Received complete upload request method=POST path=/40702026/11/12 00:44:17 INFO Received push request method=POST path=/40712026/11/12 00:44:17 INFO Vacuumed table table=closures40722026/11/12 00:44:17 INFO Completed upload id=10 objects=10 duration=776.333µs40732026/11/12 00:44:17 INFO Vacuumed table table=objects40742026/11/12 00:44:17 INFO Received push request method=POST path=/40752026/11/12 00:44:17 INFO Received uploads request method=POST path=/40762026/11/12 00:44:17 INFO Received uploads request method=POST path=/40772026/11/12 00:44:17 INFO Received push request method=POST path=/40782026/11/12 00:44:17 INFO Received uploads request method=POST path=/40792026/11/12 00:44:17 INFO Received uploads request method=POST path=/40802026/11/12 00:44:17 INFO Received uploads request method=POST path=/40812026/11/12 00:44:17 INFO Received push request method=POST path=/40822026/11/12 00:44:17 INFO Received push request method=POST path=/40832026/11/12 00:44:17 INFO Received uploads request method=POST path=/40842026/11/12 00:44:17 INFO Received push request method=POST path=/40852026/11/12 00:44:17 INFO Received complete push request method=POST path=/40862026/11/12 00:44:17 INFO Completed push id=12 objects=15 duration=1.073958ms40872026/11/12 00:44:17 INFO Received complete push request method=POST path=/40882026/11/12 00:44:17 INFO Received uploads request method=POST path=/40892026/11/12 00:44:17 INFO Received push request method=POST path=/40902026/11/12 00:44:17 INFO Completed push id=5 objects=3 duration=3.447709ms40912026/11/12 00:44:17 INFO Received push request method=POST path=/40922026/11/12 00:44:17 INFO Aborted multipart uploads count=0 kept=040932026/11/12 00:44:17 INFO Received complete upload request method=POST path=/40942026/11/12 00:44:17 INFO Completed upload id=30 objects=6 duration=3.092166ms40952026/11/12 00:44:17 INFO Received complete upload request method=POST path=/40962026/11/12 00:44:17 INFO Completed upload id=23 objects=16 duration=5.949291ms40972026/11/12 00:44:17 WARN Force mode enabled - objects will be deleted immediately without grace period40982026/11/12 00:44:17 INFO Received push request method=POST path=/40992026/11/12 00:44:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041002026/11/12 00:44:17 INFO Vacuumed table table=pending_closures41012026/11/12 00:44:17 INFO Vacuumed table table=commit_receipts41022026/11/12 00:44:17 INFO Vacuumed table table=pending_objects41032026/11/12 00:44:17 INFO Vacuumed table table=multipart_uploads41042026/11/12 00:44:17 INFO Vacuumed table table=closures41052026/11/12 00:44:17 INFO Vacuumed table table=objects41062026/11/12 00:44:17 INFO Received push request method=POST path=/41072026/11/12 00:44:17 INFO Received uploads request method=POST path=/41082026/11/12 00:44:17 INFO Received complete push request method=POST path=/41092026/11/12 00:44:17 INFO Completed push id=11 objects=11 duration=4.174666ms41102026/11/12 00:44:17 INFO Received push request method=POST path=/41112026/11/12 00:44:17 INFO Aborted multipart uploads count=0 kept=041122026/11/12 00:44:17 WARN Force mode enabled - objects will be deleted immediately without grace period41132026/11/12 00:44:17 INFO Received complete push request method=POST path=/41142026/11/12 00:44:17 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041152026/11/12 00:44:17 INFO Received complete upload request method=POST path=/41162026/11/12 00:44:17 INFO Aborted multipart uploads count=0 kept=041172026/11/12 00:44:17 WARN Force mode enabled - objects will be deleted immediately without grace period41182026/11/12 00:44:17 INFO Received uploads request method=POST path=/41192026/11/12 00:44:17 INFO Completed upload id=18 objects=11 duration=5.789ms41202026/11/12 00:44:17 INFO Completed push id=31 objects=5 duration=5.91075ms41212026/11/12 00:44:17 INFO Vacuumed table table=pending_closures41222026/11/12 00:44:17 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041232026/11/12 00:44:17 INFO Vacuumed table table=commit_receipts41242026/11/12 00:44:17 INFO Vacuumed table table=pending_closures41252026/11/12 00:44:17 INFO Vacuumed table table=pending_objects41262026/11/12 00:44:17 INFO Vacuumed table table=commit_receipts41272026/11/12 00:44:17 INFO Vacuumed table table=multipart_uploads41282026/11/12 00:44:17 INFO Vacuumed table table=pending_objects41292026/11/12 00:44:17 INFO Vacuumed table table=multipart_uploads41302026/11/12 00:44:17 INFO Vacuumed table table=closures41312026/11/12 00:44:17 INFO Vacuumed table table=closures41322026/11/12 00:44:17 INFO Vacuumed table table=objects41332026/11/12 00:44:17 INFO Vacuumed table table=objects41342026/11/12 00:44:17 INFO Received push request method=POST path=/41352026/11/12 00:44:17 INFO Received uploads request method=POST path=/41362026/11/12 00:44:17 INFO Received push request method=POST path=/41372026/11/12 00:44:17 INFO Received uploads request method=POST path=/41382026/11/12 00:44:17 INFO Received uploads request method=POST path=/41392026/11/12 00:44:17 INFO Received push request method=POST path=/41402026/11/12 00:44:17 INFO Received uploads request method=POST path=/41412026/11/12 00:44:17 INFO Received push request method=POST path=/41422026/11/12 00:44:17 INFO Received push request method=POST path=/41432026/11/12 00:44:17 INFO Received push request method=POST path=/41442026/11/12 00:44:17 INFO Received uploads request method=POST path=/41452026/11/12 00:44:17 INFO Received uploads request method=POST path=/41462026/11/12 00:44:17 INFO Received complete push request method=POST path=/41472026/11/12 00:44:17 INFO Completed push id=17 objects=22 duration=1.24175ms41482026/11/12 00:44:17 INFO Received complete push request method=POST path=/41492026/11/12 00:44:17 INFO Completed push id=14 objects=3 duration=1.775416ms41502026/11/12 00:44:17 INFO Received complete push request method=POST path=/41512026/11/12 00:44:17 INFO Aborted multipart uploads count=0 kept=041522026/11/12 00:44:17 INFO Completed push id=8 objects=6 duration=1.246917ms41532026/11/12 00:44:17 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041542026/11/12 00:44:17 INFO Received uploads request method=POST path=/41552026/11/12 00:44:17 INFO Vacuumed table table=pending_closures41562026/11/12 00:44:17 INFO Vacuumed table table=commit_receipts41572026/11/12 00:44:17 INFO Vacuumed table table=pending_objects41582026/11/12 00:44:17 INFO Vacuumed table table=multipart_uploads41592026/11/12 00:44:17 INFO Received push request method=POST path=/41602026/11/12 00:44:17 INFO Vacuumed table table=closures41612026/11/12 00:44:17 INFO Received complete upload request method=POST path=/41622026/11/12 00:44:17 INFO Vacuumed table table=objects41632026/11/12 00:44:17 INFO Received push request method=POST path=/41642026/11/12 00:44:17 INFO Received uploads request method=POST path=/41652026/11/12 00:44:17 INFO Completed upload id=15 objects=9 duration=5.030791ms41662026/11/12 00:44:17 INFO Aborted multipart uploads count=0 kept=041672026/11/12 00:44:17 INFO Received complete push request method=POST path=/41682026/11/12 00:44:17 INFO Completed push id=32 objects=26 duration=1.571584ms41692026/11/12 00:44:17 INFO Received push request method=POST path=/41702026/11/12 00:44:17 WARN Force mode enabled - objects will be deleted immediately without grace period41712026/11/12 00:44:17 INFO Received uploads request method=POST path=/41722026/11/12 00:44:17 INFO Received complete push request method=POST path=/41732026/11/12 00:44:17 INFO Received push request method=POST path=/41742026/11/12 00:44:17 INFO Completed push id=31 objects=14 duration=5.688416ms41752026/11/12 00:44:17 INFO Received complete upload request method=POST path=/41762026/11/12 00:44:17 INFO Received push request method=POST path=/41772026/11/12 00:44:17 INFO Received uploads request method=POST path=/41782026/11/12 00:44:17 INFO Completed upload id=33 objects=2 duration=563µs41792026/11/12 00:44:17 INFO Received push request method=POST path=/41802026/11/12 00:44:17 INFO Received uploads request method=POST path=/41812026/11/12 00:44:17 INFO Received complete upload request method=POST path=/41822026/11/12 00:44:17 INFO Completed upload id=34 objects=5 duration=1.963083ms41832026/11/12 00:44:17 INFO Aborted multipart uploads count=0 kept=041842026/11/12 00:44:17 INFO Received complete upload request method=POST path=/41852026/11/12 00:44:17 INFO Received uploads request method=POST path=/41862026/11/12 00:44:17 INFO Completed upload id=33 objects=5 duration=1.507708ms41872026/11/12 00:44:17 WARN Force mode enabled - objects will be deleted immediately without grace period41882026/11/12 00:44:17 INFO Aborted multipart uploads count=0 kept=041892026/11/12 00:44:17 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041902026/11/12 00:44:17 INFO Received push request method=POST path=/41912026/11/12 00:44:17 WARN Force mode enabled - objects will be deleted immediately without grace period41922026/11/12 00:44:17 INFO Vacuumed table table=pending_closures41932026/11/12 00:44:17 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=041942026/11/12 00:44:17 INFO Vacuumed table table=commit_receipts41952026/11/12 00:44:17 INFO Received complete push request method=POST path=/41962026/11/12 00:44:17 INFO Received complete push request method=POST path=/41972026/11/12 00:44:17 INFO Vacuumed table table=pending_closures41982026/11/12 00:44:17 INFO Completed push id=4 objects=6 duration=6.613042ms41992026/11/12 00:44:17 INFO Completed push id=21 objects=17 duration=7.658791ms42002026/11/12 00:44:17 INFO Vacuumed table table=pending_objects42012026/11/12 00:44:17 INFO Vacuumed table table=commit_receipts42022026/11/12 00:44:17 INFO Vacuumed table table=multipart_uploads42032026/11/12 00:44:17 INFO Vacuumed table table=closures42042026/11/12 00:44:17 INFO Received push request method=POST path=/42052026/11/12 00:44:17 INFO Vacuumed table table=objects42062026/11/12 00:44:17 INFO Received push request method=POST path=/42072026/11/12 00:44:17 INFO Received complete upload request method=POST path=/42082026/11/12 00:44:17 INFO Completed upload id=35 objects=11 duration=595.291µs42092026/11/12 00:44:17 INFO Vacuumed table table=pending_objects42102026/11/12 00:44:17 INFO Vacuumed table table=multipart_uploads42112026/11/12 00:44:17 INFO Vacuumed table table=closures42122026/11/12 00:44:17 INFO Vacuumed table table=objects42132026/11/12 00:44:17 INFO Received complete upload request method=POST path=/42142026/11/12 00:44:17 INFO Completed upload id=14 objects=16 duration=3.700208ms42152026/11/12 00:44:17 INFO Received complete push request method=POST path=/42162026/11/12 00:44:17 INFO Received complete upload request method=POST path=/42172026/11/12 00:44:18 INFO Completed upload id=15 objects=6 duration=962.5µs42182026/11/12 00:44:18 INFO Received complete push request method=POST path=/42192026/11/12 00:44:18 INFO Completed push id=19 objects=7 duration=5.10375ms42202026/11/12 00:44:18 INFO Completed push id=19 objects=14 duration=1.280209ms42212026/11/12 00:44:18 INFO Received complete upload request method=POST path=/42222026/11/12 00:44:18 INFO Completed upload id=36 objects=3 duration=1.492333ms42232026/11/12 00:44:18 INFO Received push request method=POST path=/42242026/11/12 00:44:18 INFO Aborted multipart uploads count=0 kept=042252026/11/12 00:44:18 INFO Received push request method=POST path=/42262026/11/12 00:44:18 INFO Received uploads request method=POST path=/42272026/11/12 00:44:18 WARN Force mode enabled - objects will be deleted immediately without grace period42282026/11/12 00:44:18 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042292026/11/12 00:44:18 INFO Vacuumed table table=pending_closures42302026/11/12 00:44:18 INFO Vacuumed table table=commit_receipts42312026/11/12 00:44:18 INFO Aborted multipart uploads count=0 kept=042322026/11/12 00:44:18 WARN Force mode enabled - objects will be deleted immediately without grace period42332026/11/12 00:44:18 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042342026/11/12 00:44:18 INFO Vacuumed table table=pending_objects42352026/11/12 00:44:18 INFO Vacuumed table table=multipart_uploads42362026/11/12 00:44:18 INFO Received push request method=POST path=/42372026/11/12 00:44:18 INFO Vacuumed table table=closures42382026/11/12 00:44:18 INFO Vacuumed table table=pending_closures42392026/11/12 00:44:18 INFO Vacuumed table table=objects42402026/11/12 00:44:18 INFO Received uploads request method=POST path=/42412026/11/12 00:44:18 INFO Vacuumed table table=commit_receipts42422026/11/12 00:44:18 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."42432026/11/12 00:44:18 INFO Vacuumed table table=pending_objects42442026/11/12 00:44:18 INFO Aborted multipart uploads count=0 kept=042452026/11/12 00:44:18 INFO Vacuumed table table=multipart_uploads42462026/11/12 00:44:18 INFO Vacuumed table table=closures42472026/11/12 00:44:18 WARN Force mode enabled - objects will be deleted immediately without grace period42482026/11/12 00:44:18 INFO Vacuumed table table=objects42492026/11/12 00:44:18 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042502026/11/12 00:44:18 INFO Received complete upload request method=POST path=/42512026/11/12 00:44:18 INFO Received push request method=POST path=/42522026/11/12 00:44:18 INFO Received uploads request method=POST path=/42532026/11/12 00:44:18 INFO Vacuumed table table=pending_closures42542026/11/12 00:44:18 INFO Received uploads request method=POST path=/42552026/11/12 00:44:18 INFO Completed upload id=10 objects=17 duration=833.125µs42562026/11/12 00:44:18 INFO Vacuumed table table=commit_receipts42572026/11/12 00:44:18 INFO Vacuumed table table=pending_objects42582026/11/12 00:44:18 INFO Vacuumed table table=multipart_uploads42592026/11/12 00:44:18 INFO Vacuumed table table=closures42602026/11/12 00:44:18 INFO Received complete push request method=POST path=/42612026/11/12 00:44:18 INFO Received complete upload request method=POST path=/42622026/11/12 00:44:18 INFO Vacuumed table table=objects42632026/11/12 00:44:18 INFO Completed push id=16 objects=7 duration=1.7365ms42642026/11/12 00:44:18 INFO Completed upload id=9 objects=2 duration=2.287375ms42652026/11/12 00:44:18 INFO Aborted multipart uploads count=0 kept=042662026/11/12 00:44:18 WARN Force mode enabled - objects will be deleted immediately without grace period42672026/11/12 00:44:18 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=042682026/11/12 00:44:18 INFO Vacuumed table table=pending_closures42692026/11/12 00:44:18 INFO Vacuumed table table=commit_receipts42702026/11/12 00:44:18 INFO Received uploads request method=POST path=/42712026/11/12 00:44:18 INFO Received push request method=POST path=/42722026/11/12 00:44:18 INFO Received complete push request method=POST path=/42732026/11/12 00:44:18 INFO Vacuumed table table=pending_objects42742026/11/12 00:44:18 INFO Completed push id=8 objects=3 duration=5.462958ms42752026/11/12 00:44:18 INFO Vacuumed table table=multipart_uploads42762026/11/12 00:44:18 INFO Received complete upload request method=POST path=/42772026/11/12 00:44:18 INFO Vacuumed table table=closures42782026/11/12 00:44:18 INFO Completed upload id=20 objects=6 duration=1.772375ms42792026/11/12 00:44:18 INFO Vacuumed table table=objects42802026/11/12 00:44:18 INFO Received push request method=POST path=/42812026/11/12 00:44:18 INFO Received complete push request method=POST path=/42822026/11/12 00:44:18 INFO Received push request method=POST path=/42832026/11/12 00:44:18 INFO Completed push id=15 objects=4 duration=892.667µs42842026/11/12 00:44:18 INFO Received complete push request method=POST path=/42852026/11/12 00:44:18 INFO Completed push id=14 objects=6 duration=1.240292ms42862026/11/12 00:44:18 INFO Received uploads request method=POST path=/42872026/11/12 00:44:18 INFO Received push request method=POST path=/42882026/11/12 00:44:18 INFO Received uploads request method=POST path=/42892026/11/12 00:44:18 INFO Received complete push request method=POST path=/42902026/11/12 00:44:18 INFO Received push request method=POST path=/42912026/11/12 00:44:18 INFO Received uploads request method=POST path=/42922026/11/12 00:44:18 INFO Completed push id=17 objects=11 duration=2.536583ms42932026/11/12 00:44:18 INFO Received complete upload request method=POST path=/42942026/11/12 00:44:18 INFO Received complete push request method=POST path=/42952026/11/12 00:44:18 INFO Completed upload id=13 objects=10 duration=5.218333ms42962026/11/12 00:44:18 INFO Received uploads request method=POST path=/42972026/11/12 00:44:18 INFO Completed push id=7 objects=7 duration=2.35725ms42982026/11/12 00:44:18 INFO Received push request method=POST path=/42992026/11/12 00:44:18 INFO Received complete push request method=POST path=/43002026/11/12 00:44:18 INFO Received complete push request method=POST path=/43012026/11/12 00:44:18 INFO Completed push id=16 objects=27 duration=1.239541ms43022026/11/12 00:44:18 INFO Completed push id=23 objects=6 duration=1.124167ms43032026/11/12 00:44:18 INFO Received push request method=POST path=/43042026/11/12 00:44:18 INFO Received complete upload request method=POST path=/43052026/11/12 00:44:18 INFO Received complete push request method=POST path=/43062026/11/12 00:44:18 INFO Received complete upload request method=POST path=/43072026/11/12 00:44:18 INFO Completed upload id=21 objects=25 duration=1.1375ms43082026/11/12 00:44:18 INFO Aborted multipart uploads count=0 kept=043092026/11/12 00:44:18 INFO Completed push id=7 objects=15 duration=1.469167ms43102026/11/12 00:44:18 INFO Completed upload id=12 objects=26 duration=1.635584ms43112026/11/12 00:44:18 INFO Received uploads request method=POST path=/43122026/11/12 00:44:18 INFO Received complete push request method=POST path=/43132026/11/12 00:44:18 INFO Completed push id=12 objects=6 duration=1.214666ms43142026/11/12 00:44:18 WARN Force mode enabled - objects will be deleted immediately without grace period43152026/11/12 00:44:18 INFO Received complete push request method=POST path=/43162026/11/12 00:44:18 INFO Completed push id=3 objects=21 duration=3.375917ms43172026/11/12 00:44:18 INFO Received complete push request method=POST path=/43182026/11/12 00:44:18 INFO Received uploads request method=POST path=/43192026/11/12 00:44:18 INFO Aborted multipart uploads count=0 kept=043202026/11/12 00:44:18 INFO Completed push id=24 objects=6 duration=1.4355ms43212026/11/12 00:44:18 INFO Received push request method=POST path=/43222026/11/12 00:44:18 INFO Received complete upload request method=POST path=/43232026/11/12 00:44:18 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=1 objects-deleted-after-grace-period=0 objects-failed-to-delete=043242026/11/12 00:44:18 INFO Completed upload id=21 objects=2 duration=544.791µs43252026/11/12 00:44:18 INFO Vacuumed table table=pending_closures43262026/11/12 00:44:18 INFO Vacuumed table table=commit_receipts43272026/11/12 00:44:18 INFO Vacuumed table table=pending_objects43282026/11/12 00:44:18 INFO Vacuumed table table=multipart_uploads43292026/11/12 00:44:18 INFO Vacuumed table table=closures43302026/11/12 00:44:18 INFO Received uploads request method=POST path=/43312026/11/12 00:44:18 INFO Received uploads request method=POST path=/43322026/11/12 00:44:18 INFO Vacuumed table table=objects43332026/11/12 00:44:18 INFO Received uploads request method=POST path=/43342026/11/12 00:44:18 INFO Received push request method=POST path=/43352026/11/12 00:44:18 INFO Received push request method=POST path=/43362026/11/12 00:44:18 INFO Received uploads request method=POST path=/43372026/11/12 00:44:18 INFO Received uploads request method=POST path=/43382026/11/12 00:44:18 INFO Received push request method=POST path=/43392026/11/12 00:44:18 INFO Received complete upload request method=POST path=/43402026/11/12 00:44:18 INFO Aborted multipart uploads count=0 kept=043412026/11/12 00:44:18 INFO Received push request method=POST path=/43422026/11/12 00:44:18 INFO Completed upload id=26 objects=6 duration=806.291µs43432026/11/12 00:44:18 WARN Force mode enabled - objects will be deleted immediately without grace period43442026/11/12 00:44:18 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043452026/11/12 00:44:18 INFO Vacuumed table table=pending_closures43462026/11/12 00:44:18 INFO Vacuumed table table=commit_receipts43472026/11/12 00:44:18 INFO Received push request method=POST path=/43482026/11/12 00:44:18 INFO Vacuumed table table=pending_objects43492026/11/12 00:44:18 INFO Received complete push request method=POST path=/43502026/11/12 00:44:18 INFO Received uploads request method=POST path=/43512026/11/12 00:44:18 INFO Vacuumed table table=multipart_uploads43522026/11/12 00:44:18 INFO Vacuumed table table=closures43532026/11/12 00:44:18 INFO Completed push id=16 objects=8 duration=10.067875ms43542026/11/12 00:44:18 INFO Received uploads request method=POST path=/43552026/11/12 00:44:18 INFO Received uploads request method=POST path=/43562026/11/12 00:44:18 INFO Vacuumed table table=objects43572026/11/12 00:44:18 INFO Received uploads request method=POST path=/43582026/11/12 00:44:18 INFO Aborted multipart uploads count=0 kept=043592026/11/12 00:44:18 INFO Received uploads request method=POST path=/43602026/11/12 00:44:18 INFO Received complete push request method=POST path=/43612026/11/12 00:44:18 WARN Force mode enabled - objects will be deleted immediately without grace period43622026/11/12 00:44:18 INFO Received complete upload request method=POST path=/43632026/11/12 00:44:18 INFO Received complete push request method=POST path=/43642026/11/12 00:44:18 INFO Received complete push request method=POST path=/43652026/11/12 00:44:18 INFO Completed push id=25 objects=7 duration=2.24875ms43662026/11/12 00:44:18 INFO Completed upload id=29 objects=5 duration=476.334µs43672026/11/12 00:44:18 INFO Completed push id=14 objects=3 duration=631.084µs43682026/11/12 00:44:18 INFO Completed push id=18 objects=10 duration=684.25µs43692026/11/12 00:44:18 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043702026/11/12 00:44:18 INFO Vacuumed table table=pending_closures43712026/11/12 00:44:18 INFO Vacuumed table table=commit_receipts43722026/11/12 00:44:18 INFO Vacuumed table table=pending_objects43732026/11/12 00:44:18 INFO Vacuumed table table=multipart_uploads43742026/11/12 00:44:18 INFO Received uploads request method=POST path=/43752026/11/12 00:44:18 INFO Received complete push request method=POST path=/43762026/11/12 00:44:18 INFO Vacuumed table table=closures43772026/11/12 00:44:18 INFO Completed push id=18 objects=6 duration=2.782584ms43782026/11/12 00:44:18 INFO Vacuumed table table=objects43792026/11/12 00:44:18 INFO Received uploads request method=POST path=/43802026/11/12 00:44:18 INFO Received complete push request method=POST path=/43812026/11/12 00:44:18 INFO Received complete upload request method=POST path=/43822026/11/12 00:44:18 INFO Received complete push request method=POST path=/43832026/11/12 00:44:18 INFO Completed push id=12 objects=2 duration=2.106292ms43842026/11/12 00:44:18 INFO Completed upload id=23 objects=9 duration=1.006709ms43852026/11/12 00:44:18 INFO Aborted multipart uploads count=0 kept=043862026/11/12 00:44:18 INFO Completed push id=15 objects=5 duration=2.225625ms43872026/11/12 00:44:18 INFO Received push request method=POST path=/43882026/11/12 00:44:18 WARN Force mode enabled - objects will be deleted immediately without grace period43892026/11/12 00:44:18 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=043902026/11/12 00:44:18 INFO Received complete upload request method=POST path=/43912026/11/12 00:44:18 INFO Aborted multipart uploads count=0 kept=043922026/11/12 00:44:18 INFO Vacuumed table table=pending_closures43932026/11/12 00:44:18 INFO Received uploads request method=POST path=/43942026/11/12 00:44:18 INFO Completed upload id=20 objects=4 duration=918.542µs43952026/11/12 00:44:18 INFO Received uploads request method=POST path=/43962026/11/12 00:44:18 WARN Force mode enabled - objects will be deleted immediately without grace period43972026/11/12 00:44:18 INFO Vacuumed table table=commit_receipts43982026/11/12 00:44:19 INFO Vacuumed table table=pending_objects43992026/11/12 00:44:19 INFO Received complete upload request method=POST path=/44002026/11/12 00:44:19 INFO Received complete push request method=POST path=/44012026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=5 old-closures-deleted=2 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044022026/11/12 00:44:19 INFO Completed upload id=12 objects=6 duration=923.167µs44032026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads44042026/11/12 00:44:19 INFO Completed push id=24 objects=24 duration=2.635208ms44052026/11/12 00:44:19 INFO Received push request method=POST path=/44062026/11/12 00:44:19 INFO Vacuumed table table=closures44072026/11/12 00:44:19 INFO Vacuumed table table=objects44082026/11/12 00:44:19 INFO Received complete push request method=POST path=/44092026/11/12 00:44:19 INFO Received complete push request method=POST path=/44102026/11/12 00:44:19 INFO Completed push id=11 objects=6 duration=3.935708ms44112026/11/12 00:44:19 INFO Completed push id=13 objects=22 duration=1.455083ms44122026/11/12 00:44:19 INFO Vacuumed table table=pending_closures44132026/11/12 00:44:19 INFO Received complete push request method=POST path=/44142026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=044152026/11/12 00:44:19 INFO Completed push id=17 objects=3 duration=8.64625ms44162026/11/12 00:44:19 INFO Received push request method=POST path=/44172026/11/12 00:44:19 INFO Received uploads request method=POST path=/44182026/11/12 00:44:19 WARN Force mode enabled - objects will be deleted immediately without grace period44192026/11/12 00:44:19 INFO Received uploads request method=POST path=/44202026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044212026/11/12 00:44:19 INFO Received push request method=POST path=/44222026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts44232026/11/12 00:44:19 INFO Vacuumed table table=pending_closures44242026/11/12 00:44:19 INFO Received complete push request method=POST path=/44252026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts44262026/11/12 00:44:19 INFO Vacuumed table table=pending_objects44272026/11/12 00:44:19 INFO Completed push id=21 objects=6 duration=3.379791ms44282026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=044292026/11/12 00:44:19 INFO Vacuumed table table=pending_objects44302026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads44312026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=11 objects-failed-to-delete=044322026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044332026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads44342026/11/12 00:44:19 INFO Vacuumed table table=closures44352026/11/12 00:44:19 INFO Vacuumed table table=pending_closures44362026/11/12 00:44:19 INFO Vacuumed table table=pending_closures44372026/11/12 00:44:19 INFO Vacuumed table table=objects44382026/11/12 00:44:19 INFO Vacuumed table table=closures44392026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts44402026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts44412026/11/12 00:44:19 INFO Received complete upload request method=POST path=/44422026/11/12 00:44:19 INFO Vacuumed table table=objects44432026/11/12 00:44:19 INFO Received uploads request method=POST path=/44442026/11/12 00:44:19 INFO Received complete push request method=POST path=/44452026/11/12 00:44:19 INFO Vacuumed table table=pending_objects44462026/11/12 00:44:19 INFO Completed upload id=22 objects=7 duration=3.255375ms44472026/11/12 00:44:19 INFO Received push request method=POST path=/44482026/11/12 00:44:19 INFO Vacuumed table table=pending_objects44492026/11/12 00:44:19 INFO Received complete push request method=POST path=/44502026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads44512026/11/12 00:44:19 INFO Completed push id=23 objects=5 duration=905.292µs44522026/11/12 00:44:19 INFO Completed push id=18 objects=6 duration=931.208µs44532026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads44542026/11/12 00:44:19 INFO Vacuumed table table=closures44552026/11/12 00:44:19 INFO Vacuumed table table=closures44562026/11/12 00:44:19 INFO Received uploads request method=POST path=/44572026/11/12 00:44:19 INFO Vacuumed table table=objects44582026/11/12 00:44:19 INFO Vacuumed table table=objects44592026/11/12 00:44:19 INFO Received uploads request method=POST path=/44602026/11/12 00:44:19 INFO Received complete upload request method=POST path=/44612026/11/12 00:44:19 INFO Completed upload id=26 objects=24 duration=589.584µs44622026/11/12 00:44:19 INFO Received complete push request method=POST path=/44632026/11/12 00:44:19 INFO Completed push id=16 objects=5 duration=728.208µs44642026/11/12 00:44:19 INFO Received complete push request method=POST path=/44652026/11/12 00:44:19 INFO Completed push id=24 objects=4 duration=618.792µs44662026/11/12 00:44:19 INFO Received push request method=POST path=/44672026/11/12 00:44:19 INFO Received uploads request method=POST path=/44682026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=044692026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=044702026/11/12 00:44:19 INFO Vacuumed table table=pending_closures44712026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts44722026/11/12 00:44:19 INFO Vacuumed table table=pending_objects44732026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads44742026/11/12 00:44:19 INFO Received push request method=POST path=/44752026/11/12 00:44:19 INFO Vacuumed table table=closures44762026/11/12 00:44:19 INFO Vacuumed table table=objects44772026/11/12 00:44:19 INFO Received uploads request method=POST path=/44782026/11/12 00:44:19 INFO Received push request method=POST path=/44792026/11/12 00:44:19 INFO Received complete upload request method=POST path=/44802026/11/12 00:44:19 INFO Received complete upload request method=POST path=/44812026/11/12 00:44:19 INFO Completed upload id=24 objects=3 duration=543.292µs44822026/11/12 00:44:19 INFO Completed upload id=31 objects=3 duration=1.474709ms44832026/11/12 00:44:19 INFO Received complete push request method=POST path=/44842026/11/12 00:44:19 INFO Received complete push request method=POST path=/44852026/11/12 00:44:19 INFO Completed push id=25 objects=9 duration=695.416µs44862026/11/12 00:44:19 INFO Completed push id=16 objects=26 duration=801.5µs44872026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=044882026/11/12 00:44:19 INFO Received complete push request method=POST path=/44892026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=044902026/11/12 00:44:19 INFO Completed push id=25 objects=4 duration=5.811916ms44912026/11/12 00:44:19 INFO Received uploads request method=POST path=/44922026/11/12 00:44:19 INFO Vacuumed table table=pending_closures44932026/11/12 00:44:19 INFO Received complete upload request method=POST path=/44942026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts44952026/11/12 00:44:19 INFO Completed upload id=20 objects=3 duration=3.89025ms44962026/11/12 00:44:19 INFO Vacuumed table table=pending_objects44972026/11/12 00:44:19 INFO Received complete upload request method=POST path=/44982026/11/12 00:44:19 INFO Received complete upload request method=POST path=/44992026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads45002026/11/12 00:44:19 INFO Completed upload id=22 objects=2 duration=1.289208ms45012026/11/12 00:44:19 INFO Completed upload id=26 objects=3 duration=1.294083ms45022026/11/12 00:44:19 INFO Vacuumed table table=closures45032026/11/12 00:44:19 INFO Vacuumed table table=objects45042026/11/12 00:44:19 INFO Received push request method=POST path=/45052026/11/12 00:44:19 INFO Received complete push request method=POST path=/45062026/11/12 00:44:19 INFO Received push request method=POST path=/45072026/11/12 00:44:19 INFO Completed push id=19 objects=14 duration=2.125ms45082026/11/12 00:44:19 INFO Received complete push request method=POST path=/45092026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=045102026/11/12 00:44:19 INFO Completed push id=14 objects=4 duration=8.741542ms45112026/11/12 00:44:19 WARN Force mode enabled - objects will be deleted immediately without grace period45122026/11/12 00:44:19 INFO Received push request method=POST path=/45132026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045142026/11/12 00:44:19 INFO Vacuumed table table=pending_closures45152026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=045162026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts45172026/11/12 00:44:19 INFO Received uploads request method=POST path=/45182026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=045192026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=045202026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=045212026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=045222026/11/12 00:44:19 INFO Vacuumed table table=pending_objects45232026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045242026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads45252026/11/12 00:44:19 INFO Received push request method=POST path=/45262026/11/12 00:44:19 WARN Force mode enabled - objects will be deleted immediately without grace period45272026/11/12 00:44:19 WARN Force mode enabled - objects will be deleted immediately without grace period45282026/11/12 00:44:19 WARN Force mode enabled - objects will be deleted immediately without grace period45292026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045302026/11/12 00:44:19 INFO Vacuumed table table=pending_closures45312026/11/12 00:44:19 INFO Vacuumed table table=closures45322026/11/12 00:44:19 INFO Vacuumed table table=pending_closures45332026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts45342026/11/12 00:44:19 INFO Vacuumed table table=objects45352026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts45362026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045372026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=4 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045382026/11/12 00:44:19 INFO Vacuumed table table=pending_closures45392026/11/12 00:44:19 INFO Vacuumed table table=pending_objects45402026/11/12 00:44:19 INFO Vacuumed table table=pending_objects45412026/11/12 00:44:19 INFO Vacuumed table table=pending_closures45422026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts45432026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts45442026/11/12 00:44:19 INFO Vacuumed table table=pending_objects45452026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads45462026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads45472026/11/12 00:44:19 INFO Received push request method=POST path=/45482026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads45492026/11/12 00:44:19 INFO Vacuumed table table=pending_objects45502026/11/12 00:44:19 INFO Vacuumed table table=closures45512026/11/12 00:44:19 INFO Vacuumed table table=closures45522026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=045532026/11/12 00:44:19 INFO Received complete upload request method=POST path=/45542026/11/12 00:44:19 INFO Received push request method=POST path=/45552026/11/12 00:44:19 INFO Vacuumed table table=objects45562026/11/12 00:44:19 INFO Vacuumed table table=objects45572026/11/12 00:44:19 INFO Vacuumed table table=closures45582026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads45592026/11/12 00:44:19 INFO Completed upload id=27 objects=7 duration=1.031583ms45602026/11/12 00:44:19 INFO Vacuumed table table=objects45612026/11/12 00:44:19 WARN Force mode enabled - objects will be deleted immediately without grace period45622026/11/12 00:44:19 INFO Vacuumed table table=closures45632026/11/12 00:44:19 INFO Received complete push request method=POST path=/45642026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045652026/11/12 00:44:19 INFO Vacuumed table table=objects45662026/11/12 00:44:19 INFO Vacuumed table table=pending_closures45672026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts45682026/11/12 00:44:19 INFO Completed push id=27 objects=6 duration=5.023292ms45692026/11/12 00:44:19 INFO Vacuumed table table=pending_objects45702026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads45712026/11/12 00:44:19 INFO Vacuumed table table=closures45722026/11/12 00:44:19 INFO Received uploads request method=POST path=/45732026/11/12 00:44:19 INFO Vacuumed table table=objects45742026/11/12 00:44:19 INFO Received complete upload request method=POST path=/45752026/11/12 00:44:19 INFO Completed upload id=27 objects=26 duration=1.754459ms45762026/11/12 00:44:19 INFO Received uploads request method=POST path=/45772026/11/12 00:44:19 INFO Received complete upload request method=POST path=/45782026/11/12 00:44:19 INFO Completed upload id=28 objects=3 duration=1.022625ms45792026/11/12 00:44:19 INFO Received complete upload request method=POST path=/45802026/11/12 00:44:19 INFO Completed upload id=18 objects=2 duration=1.528375ms45812026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=045822026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=045832026/11/12 00:44:19 INFO Received complete upload request method=POST path=/45842026/11/12 00:44:19 INFO Vacuumed table table=pending_closures45852026/11/12 00:44:19 INFO Completed upload id=24 objects=7 duration=640.375µs45862026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts45872026/11/12 00:44:19 INFO Received complete push request method=POST path=/45882026/11/12 00:44:19 INFO Received complete push request method=POST path=/45892026/11/12 00:44:19 INFO Completed push id=28 objects=6 duration=392.375µs45902026/11/12 00:44:19 INFO Completed push id=22 objects=3 duration=2.044333ms45912026/11/12 00:44:19 INFO Vacuumed table table=pending_objects45922026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads45932026/11/12 00:44:19 INFO Vacuumed table table=closures45942026/11/12 00:44:19 INFO Vacuumed table table=objects45952026/11/12 00:44:19 INFO Received uploads request method=POST path=/45962026/11/12 00:44:19 INFO Received uploads request method=POST path=/45972026/11/12 00:44:19 INFO Received complete upload request method=POST path=/45982026/11/12 00:44:19 INFO Received push request method=POST path=/45992026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=046002026/11/12 00:44:19 INFO Completed upload id=15 objects=2 duration=762.5µs46012026/11/12 00:44:19 INFO Received push request method=POST path=/46022026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046032026/11/12 00:44:19 INFO Vacuumed table table=pending_closures46042026/11/12 00:44:19 INFO Received complete push request method=POST path=/46052026/11/12 00:44:19 INFO Completed push id=29 objects=6 duration=1.814458ms46062026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts46072026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=046082026/11/12 00:44:19 INFO Received push request method=POST path=/46092026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=046102026/11/12 00:44:19 INFO Vacuumed table table=pending_objects46112026/11/12 00:44:19 INFO Received uploads request method=POST path=/46122026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads46132026/11/12 00:44:19 WARN Force mode enabled - objects will be deleted immediately without grace period46142026/11/12 00:44:19 WARN Force mode enabled - objects will be deleted immediately without grace period46152026/11/12 00:44:19 INFO Vacuumed table table=closures46162026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046172026/11/12 00:44:19 INFO Received complete push request method=POST path=/46182026/11/12 00:44:19 INFO Vacuumed table table=objects46192026/11/12 00:44:19 INFO Vacuumed table table=pending_closures46202026/11/12 00:44:19 INFO Completed push id=29 objects=23 duration=2.402833ms46212026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts46222026/11/12 00:44:19 INFO Vacuumed table table=pending_objects46232026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=046242026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads46252026/11/12 00:44:19 INFO Received complete push request method=POST path=/46262026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046272026/11/12 00:44:19 INFO Received uploads request method=POST path=/46282026/11/12 00:44:19 INFO Received complete push request method=POST path=/46292026/11/12 00:44:19 INFO Vacuumed table table=closures46302026/11/12 00:44:19 INFO Completed push id=14 objects=3 duration=2.101667ms46312026/11/12 00:44:19 INFO Vacuumed table table=pending_closures46322026/11/12 00:44:19 INFO Received complete upload request method=POST path=/46332026/11/12 00:44:19 INFO Completed push id=30 objects=6 duration=1.464917ms46342026/11/12 00:44:19 INFO Vacuumed table table=objects46352026/11/12 00:44:19 INFO Received uploads request method=POST path=/46362026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts46372026/11/12 00:44:19 INFO Completed upload id=23 objects=7 duration=3.505125ms46382026/11/12 00:44:19 INFO Vacuumed table table=pending_objects46392026/11/12 00:44:19 INFO Received push request method=POST path=/46402026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads46412026/11/12 00:44:19 INFO Received complete push request method=POST path=/46422026/11/12 00:44:19 INFO Vacuumed table table=closures46432026/11/12 00:44:19 INFO Completed push id=26 objects=23 duration=1.382833ms46442026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=046452026/11/12 00:44:19 INFO Vacuumed table table=objects46462026/11/12 00:44:19 INFO Received complete upload request method=POST path=/46472026/11/12 00:44:19 INFO Completed upload id=26 objects=5 duration=2.921041ms46482026/11/12 00:44:19 WARN Force mode enabled - objects will be deleted immediately without grace period46492026/11/12 00:44:19 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046502026/11/12 00:44:19 INFO Vacuumed table table=pending_closures46512026/11/12 00:44:19 INFO Vacuumed table table=commit_receipts46522026/11/12 00:44:19 INFO Received uploads request method=POST path=/46532026/11/12 00:44:19 INFO Vacuumed table table=pending_objects46542026/11/12 00:44:19 INFO Vacuumed table table=multipart_uploads46552026/11/12 00:44:19 INFO Vacuumed table table=closures46562026/11/12 00:44:19 INFO Received complete upload request method=POST path=/46572026/11/12 00:44:19 INFO Received complete push request method=POST path=/46582026/11/12 00:44:19 INFO Aborted multipart uploads count=0 kept=046592026/11/12 00:44:19 INFO Vacuumed table table=objects46602026/11/12 00:44:19 INFO Received complete push request method=POST path=/46612026/11/12 00:44:19 INFO Completed upload id=13 objects=7 duration=3.73525ms46622026/11/12 00:44:19 INFO Received uploads request method=POST path=/46632026/11/12 00:44:20 INFO Completed push id=27 objects=3 duration=2.15475ms46642026/11/12 00:44:20 INFO Received uploads request method=POST path=/46652026/11/12 00:44:20 INFO Completed push id=21 objects=9 duration=1.298291ms46662026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=046672026/11/12 00:44:20 INFO Received complete push request method=POST path=/46682026/11/12 00:44:20 INFO Vacuumed table table=pending_closures46692026/11/12 00:44:20 INFO Completed push id=33 objects=3 duration=2.961833ms46702026/11/12 00:44:20 INFO Vacuumed table table=commit_receipts46712026/11/12 00:44:20 INFO Vacuumed table table=pending_objects46722026/11/12 00:44:20 INFO Vacuumed table table=multipart_uploads46732026/11/12 00:44:20 INFO Vacuumed table table=closures46742026/11/12 00:44:20 INFO Vacuumed table table=objects46752026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=3 old-closures-deleted=1 objects-marked-for-deletion=3 objects-deleted-after-grace-period=3 objects-failed-to-delete=046762026/11/12 00:44:20 INFO Received complete upload request method=POST path=/46772026/11/12 00:44:20 INFO Received complete upload request method=POST path=/46782026/11/12 00:44:20 INFO Received complete upload request method=POST path=/46792026/11/12 00:44:20 INFO Received complete upload request method=POST path=/46802026/11/12 00:44:20 INFO Vacuumed table table=pending_closures46812026/11/12 00:44:20 INFO Completed upload id=20 objects=4 duration=2.787375ms46822026/11/12 00:44:20 INFO Completed upload id=26 objects=8 duration=2.625667ms46832026/11/12 00:44:20 INFO Completed upload id=23 objects=18 duration=2.8165ms46842026/11/12 00:44:20 INFO Completed upload id=28 objects=5 duration=2.745208ms46852026/11/12 00:44:20 INFO Vacuumed table table=commit_receipts46862026/11/12 00:44:20 INFO Received complete push request method=POST path=/46872026/11/12 00:44:20 INFO Completed push id=34 objects=6 duration=2.097542ms46882026/11/12 00:44:20 INFO Vacuumed table table=pending_objects46892026/11/12 00:44:20 INFO Vacuumed table table=multipart_uploads46902026/11/12 00:44:20 INFO Vacuumed table table=closures46912026/11/12 00:44:20 INFO Received complete upload request method=POST path=/46922026/11/12 00:44:20 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."46932026/11/12 00:44:20 INFO Completed upload id=30 objects=8 duration=3.777833ms46942026/11/12 00:44:20 INFO Vacuumed table table=objects46952026/11/12 00:44:20 INFO Received uploads request method=POST path=/46962026/11/12 00:44:20 INFO Received complete upload request method=POST path=/46972026/11/12 00:44:20 INFO Completed upload id=23 objects=7 duration=4.399791ms46982026/11/12 00:44:20 INFO Received complete push request method=POST path=/46992026/11/12 00:44:20 INFO Received complete push request method=POST path=/47002026/11/12 00:44:20 INFO Completed push id=29 objects=17 duration=3.187291ms47012026/11/12 00:44:20 INFO Completed push id=28 objects=14 duration=5.779958ms47022026/11/12 00:44:20 INFO Received complete push request method=POST path=/47032026/11/12 00:44:20 INFO Completed push id=23 objects=28 duration=6.812583ms47042026/11/12 00:44:20 INFO Received uploads request method=POST path=/47052026/11/12 00:44:20 INFO Received complete push request method=POST path=/47062026/11/12 00:44:20 INFO Completed push id=35 objects=8 duration=1.826209ms47072026/11/12 00:44:20 INFO Received push request method=POST path=/47082026/11/12 00:44:20 INFO Received uploads request method=POST path=/47092026/11/12 00:44:20 INFO Received complete push request method=POST path=/47102026/11/12 00:44:20 INFO Completed push id=27 objects=15 duration=757.416µs47112026/11/12 00:44:20 INFO Received push request method=POST path=/47122026/11/12 00:44:20 INFO Received push request method=POST path=/47132026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=047142026/11/12 00:44:20 WARN Force mode enabled - objects will be deleted immediately without grace period47152026/11/12 00:44:20 INFO Received push request method=POST path=/47162026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047172026/11/12 00:44:20 INFO Received push request method=POST path=/47182026/11/12 00:44:20 INFO Vacuumed table table=pending_closures47192026/11/12 00:44:20 INFO Vacuumed table table=commit_receipts47202026/11/12 00:44:20 INFO Received uploads request method=POST path=/47212026/11/12 00:44:20 INFO Vacuumed table table=pending_objects47222026/11/12 00:44:20 INFO Vacuumed table table=multipart_uploads47232026/11/12 00:44:20 INFO Received complete upload request method=POST path=/47242026/11/12 00:44:20 INFO Completed upload id=31 objects=6 duration=397.458µs47252026/11/12 00:44:20 INFO Vacuumed table table=closures47262026/11/12 00:44:20 INFO Received complete upload request method=POST path=/47272026/11/12 00:44:20 INFO Received complete upload request method=POST path=/47282026/11/12 00:44:20 INFO Received complete upload request method=POST path=/47292026/11/12 00:44:20 INFO Received uploads request method=POST path=/47302026/11/12 00:44:20 INFO Vacuumed table table=objects47312026/11/12 00:44:20 INFO Completed upload id=27 objects=8 duration=1.346708ms47322026/11/12 00:44:20 INFO Completed upload id=22 objects=22 duration=651.25µs47332026/11/12 00:44:20 INFO Completed upload id=36 objects=9 duration=608.625µs47342026/11/12 00:44:20 INFO Received uploads request method=POST path=/47352026/11/12 00:44:20 INFO Received uploads request method=POST path=/47362026/11/12 00:44:20 INFO Received complete upload request method=POST path=/47372026/11/12 00:44:20 INFO Completed upload id=32 objects=4 duration=978.542µs47382026/11/12 00:44:20 INFO Received complete push request method=POST path=/47392026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=047402026/11/12 00:44:20 INFO Received push request method=POST path=/47412026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=047422026/11/12 00:44:20 INFO Completed push id=16 objects=16 duration=1.701ms47432026/11/12 00:44:20 WARN Force mode enabled - objects will be deleted immediately without grace period47442026/11/12 00:44:20 WARN Force mode enabled - objects will be deleted immediately without grace period47452026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047462026/11/12 00:44:20 INFO Received push request method=POST path=/47472026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=047482026/11/12 00:44:20 INFO Vacuumed table table=pending_closures47492026/11/12 00:44:20 INFO Vacuumed table table=commit_receipts47502026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047512026/11/12 00:44:20 INFO Vacuumed table table=pending_objects47522026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047532026/11/12 00:44:20 INFO Vacuumed table table=pending_closures47542026/11/12 00:44:20 INFO Vacuumed table table=multipart_uploads47552026/11/12 00:44:20 INFO Vacuumed table table=commit_receipts47562026/11/12 00:44:20 INFO Vacuumed table table=pending_closures47572026/11/12 00:44:20 INFO Vacuumed table table=commit_receipts47582026/11/12 00:44:20 INFO Vacuumed table table=closures47592026/11/12 00:44:20 INFO Vacuumed table table=pending_objects47602026/11/12 00:44:20 INFO Vacuumed table table=pending_objects47612026/11/12 00:44:20 INFO Vacuumed table table=objects47622026/11/12 00:44:20 INFO Vacuumed table table=multipart_uploads47632026/11/12 00:44:20 INFO Vacuumed table table=multipart_uploads47642026/11/12 00:44:20 INFO Vacuumed table table=closures47652026/11/12 00:44:20 INFO Vacuumed table table=closures47662026/11/12 00:44:20 INFO Vacuumed table table=objects47672026/11/12 00:44:20 INFO Vacuumed table table=objects47682026/11/12 00:44:20 INFO Received push request method=POST path=/47692026/11/12 00:44:20 INFO Received complete push request method=POST path=/47702026/11/12 00:44:20 INFO Completed push id=26 objects=5 duration=938.625µs47712026/11/12 00:44:20 INFO Received push request method=POST path=/47722026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=047732026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=047742026/11/12 00:44:20 INFO Received uploads request method=POST path=/47752026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=047762026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047772026/11/12 00:44:20 ERROR failed to remove object object=hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh.ls error="Access Denied."47782026/11/12 00:44:20 ERROR failed to remove object object=realisations/sha256:cccccccc!out.doi error="Access Denied."47792026/11/12 00:44:20 INFO Vacuumed table table=pending_closures47802026/11/12 00:44:20 INFO Received complete upload request method=POST path=/47812026/11/12 00:44:20 INFO Received push request method=POST path=/47822026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=047832026/11/12 00:44:20 INFO Vacuumed table table=commit_receipts47842026/11/12 00:44:20 INFO Completed upload id=24 objects=14 duration=850.667µs47852026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=047862026/11/12 00:44:20 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."47872026/11/12 00:44:20 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."47882026/11/12 00:44:20 INFO Received complete upload request method=POST path=/47892026/11/12 00:44:20 INFO Vacuumed table table=pending_objects47902026/11/12 00:44:20 INFO Vacuumed table table=pending_closures47912026/11/12 00:44:20 INFO Received complete push request method=POST path=/47922026/11/12 00:44:20 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."47932026/11/12 00:44:20 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."47942026/11/12 00:44:20 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."47952026/11/12 00:44:20 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."47962026/11/12 00:44:20 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."47972026/11/12 00:44:20 INFO Vacuumed table table=multipart_uploads47982026/11/12 00:44:20 INFO Vacuumed table table=commit_receipts47992026/11/12 00:44:20 INFO Completed push id=19 objects=3 duration=676.958µs48002026/11/12 00:44:20 INFO Completed upload id=21 objects=9 duration=2.261167ms48012026/11/12 00:44:20 INFO Vacuumed table table=closures48022026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=048032026/11/12 00:44:20 INFO Vacuumed table table=pending_objects48042026/11/12 00:44:20 INFO Received uploads request method=POST path=/48052026/11/12 00:44:20 INFO Vacuumed table table=multipart_uploads48062026/11/12 00:44:20 WARN Force mode enabled - objects will be deleted immediately without grace period48072026/11/12 00:44:20 INFO Vacuumed table table=closures48082026/11/12 00:44:20 INFO Vacuumed table table=objects48092026/11/12 00:44:20 INFO Vacuumed table table=objects48102026/11/12 00:44:20 INFO Received complete push request method=POST path=/48112026/11/12 00:44:20 INFO Received uploads request method=POST path=/48122026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=3 objects-failed-to-delete=048132026/11/12 00:44:20 INFO Completed push id=25 objects=10 duration=2.093916ms48142026/11/12 00:44:20 INFO Received complete upload request method=POST path=/48152026/11/12 00:44:20 INFO Vacuumed table table=pending_closures48162026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=048172026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=048182026/11/12 00:44:20 INFO Received uploads request method=POST path=/48192026/11/12 00:44:20 INFO Completed upload id=33 objects=10 duration=506.25µs48202026/11/12 00:44:20 INFO Vacuumed table table=commit_receipts48212026/11/12 00:44:20 INFO Received push request method=POST path=/48222026/11/12 00:44:20 WARN Force mode enabled - objects will be deleted immediately without grace period48232026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048242026/11/12 00:44:20 INFO Vacuumed table table=pending_objects48252026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048262026/11/12 00:44:20 INFO Vacuumed table table=pending_closures48272026/11/12 00:44:20 INFO Vacuumed table table=pending_closures48282026/11/12 00:44:20 INFO Vacuumed table table=multipart_uploads48292026/11/12 00:44:20 INFO Vacuumed table table=commit_receipts48302026/11/12 00:44:20 INFO Vacuumed table table=commit_receipts48312026/11/12 00:44:20 INFO Vacuumed table table=closures48322026/11/12 00:44:20 INFO Vacuumed table table=pending_objects48332026/11/12 00:44:20 INFO Vacuumed table table=pending_objects48342026/11/12 00:44:20 INFO Vacuumed table table=multipart_uploads48352026/11/12 00:44:20 INFO Vacuumed table table=objects48362026/11/12 00:44:20 INFO Vacuumed table table=multipart_uploads48372026/11/12 00:44:20 INFO Vacuumed table table=closures48382026/11/12 00:44:20 INFO Vacuumed table table=closures48392026/11/12 00:44:20 INFO Vacuumed table table=objects48402026/11/12 00:44:20 INFO Vacuumed table table=objects48412026/11/12 00:44:20 INFO Received complete upload request method=POST path=/48422026/11/12 00:44:20 INFO Received complete upload request method=POST path=/48432026/11/12 00:44:20 INFO Completed upload id=29 objects=5 duration=1.198709ms48442026/11/12 00:44:20 INFO Completed upload id=13 objects=8 duration=6.248833ms48452026/11/12 00:44:20 INFO Received complete upload request method=POST path=/48462026/11/12 00:44:20 INFO Completed upload id=19 objects=2 duration=2.737958ms48472026/11/12 00:44:20 INFO Received complete upload request method=POST path=/48482026/11/12 00:44:20 INFO Received complete upload request method=POST path=/48492026/11/12 00:44:20 INFO Completed upload id=21 objects=7 duration=1.8595ms48502026/11/12 00:44:20 INFO Completed upload id=25 objects=16 duration=12.883334ms48512026/11/12 00:44:20 INFO Received uploads request method=POST path=/48522026/11/12 00:44:20 INFO Received complete push request method=POST path=/48532026/11/12 00:44:20 INFO Completed push id=28 objects=19 duration=528.292µs48542026/11/12 00:44:20 INFO Received complete upload request method=POST path=/48552026/11/12 00:44:20 INFO Received uploads request method=POST path=/48562026/11/12 00:44:20 INFO Received uploads request method=POST path=/48572026/11/12 00:44:20 INFO Completed upload id=30 objects=2 duration=1.468666ms48582026/11/12 00:44:20 INFO Received uploads request method=POST path=/48592026/11/12 00:44:20 INFO Received uploads request method=POST path=/48602026/11/12 00:44:20 INFO Received complete push request method=POST path=/48612026/11/12 00:44:20 INFO Completed push id=33 objects=7 duration=930.625µs48622026/11/12 00:44:20 INFO Received complete upload request method=POST path=/48632026/11/12 00:44:20 INFO Completed upload id=25 objects=4 duration=1.79375ms48642026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=048652026/11/12 00:44:20 INFO Received complete upload request method=POST path=/48662026/11/12 00:44:20 WARN Force mode enabled - objects will be deleted immediately without grace period48672026/11/12 00:44:20 INFO Completed upload id=24 objects=18 duration=511.416µs48682026/11/12 00:44:20 INFO Received complete push request method=POST path=/48692026/11/12 00:44:20 INFO Received push request method=POST path=/48702026/11/12 00:44:20 INFO Completed push id=31 objects=20 duration=795.5µs48712026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048722026/11/12 00:44:20 INFO Received push request method=POST path=/48732026/11/12 00:44:20 INFO Vacuumed table table=pending_closures48742026/11/12 00:44:20 INFO Received complete upload request method=POST path=/48752026/11/12 00:44:20 INFO Vacuumed table table=commit_receipts48762026/11/12 00:44:20 INFO Completed upload id=26 objects=3 duration=816.208µs48772026/11/12 00:44:20 INFO Vacuumed table table=pending_objects48782026/11/12 00:44:20 INFO Vacuumed table table=multipart_uploads48792026/11/12 00:44:20 INFO Vacuumed table table=closures48802026/11/12 00:44:20 INFO Received complete push request method=POST path=/48812026/11/12 00:44:20 INFO Vacuumed table table=objects48822026/11/12 00:44:20 INFO Received push request method=POST path=/48832026/11/12 00:44:20 INFO Completed push id=25 objects=2 duration=1.069791ms48842026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=048852026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=048862026/11/12 00:44:20 WARN Force mode enabled - objects will be deleted immediately without grace period48872026/11/12 00:44:20 WARN Force mode enabled - objects will be deleted immediately without grace period48882026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=048892026/11/12 00:44:20 INFO Vacuumed table table=pending_closures48902026/11/12 00:44:20 INFO Received complete upload request method=POST path=/48912026/11/12 00:44:20 INFO Completed upload id=32 objects=5 duration=750.75µs48922026/11/12 00:44:20 INFO Vacuumed table table=commit_receipts48932026/11/12 00:44:20 INFO Vacuumed table table=pending_objects48942026/11/12 00:44:20 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."48952026/11/12 00:44:20 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.ls error="Access Denied."48962026/11/12 00:44:20 INFO Received complete upload request method=POST path=/48972026/11/12 00:44:20 INFO Vacuumed table table=multipart_uploads48982026/11/12 00:44:20 INFO Received push request method=POST path=/48992026/11/12 00:44:20 INFO Received complete push request method=POST path=/49002026/11/12 00:44:20 INFO Received uploads request method=POST path=/49012026/11/12 00:44:20 ERROR failed to remove object object=iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.narinfo error="Access Denied."49022026/11/12 00:44:20 INFO Vacuumed table table=closures49032026/11/12 00:44:20 INFO Completed upload id=34 objects=23 duration=891.666µs49042026/11/12 00:44:20 INFO Received complete upload request method=POST path=/49052026/11/12 00:44:20 ERROR failed to remove object object=nar/iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii.nar.zst error="Access Denied."49062026/11/12 00:44:20 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."49072026/11/12 00:44:20 ERROR failed to remove object object=realisations/sha256:hhhhhhhh!out.doi error="Access Denied."49082026/11/12 00:44:20 ERROR failed to remove object object=realisations/sha256:iiiiiiii!out.doi error="Access Denied."49092026/11/12 00:44:20 INFO Completed push id=27 objects=4 duration=653.958µs49102026/11/12 00:44:20 INFO Vacuumed table table=objects49112026/11/12 00:44:20 INFO Completed upload id=33 objects=3 duration=1.216916ms49122026/11/12 00:44:20 INFO Received complete push request method=POST path=/49132026/11/12 00:44:20 INFO Received push request method=POST path=/49142026/11/12 00:44:20 INFO Completed push id=20 objects=8 duration=841.667µs49152026/11/12 00:44:20 INFO Received complete push request method=POST path=/49162026/11/12 00:44:20 INFO Received uploads request method=POST path=/49172026/11/12 00:44:20 INFO Completed push id=29 objects=10 duration=900.875µs49182026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=049192026/11/12 00:44:20 WARN Force mode enabled - objects will be deleted immediately without grace period49202026/11/12 00:44:20 INFO Received uploads request method=POST path=/49212026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049222026/11/12 00:44:20 INFO Received complete upload request method=POST path=/49232026/11/12 00:44:20 INFO Completed upload id=21 objects=3 duration=1.116209ms49242026/11/12 00:44:20 INFO Vacuumed table table=pending_closures49252026/11/12 00:44:20 INFO Received complete upload request method=POST path=/49262026/11/12 00:44:20 INFO Aborted multipart uploads count=0 kept=049272026/11/12 00:44:20 INFO Received push request method=POST path=/49282026/11/12 00:44:20 INFO Vacuumed table table=commit_receipts49292026/11/12 00:44:20 INFO Completed upload id=27 objects=4 duration=609.375µs49302026/11/12 00:44:20 INFO Received uploads request method=POST path=/49312026/11/12 00:44:20 WARN Force mode enabled - objects will be deleted immediately without grace period49322026/11/12 00:44:20 INFO Received complete push request method=POST path=/49332026/11/12 00:44:20 INFO Vacuumed table table=pending_objects49342026/11/12 00:44:20 INFO Completed push id=35 objects=6 duration=2.669708ms49352026/11/12 00:44:20 INFO Received complete push request method=POST path=/49362026/11/12 00:44:20 INFO Vacuumed table table=multipart_uploads49372026/11/12 00:44:20 INFO Received complete upload request method=POST path=/49382026/11/12 00:44:20 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=7 objects-failed-to-delete=049392026/11/12 00:44:21 INFO Completed push id=13 objects=11 duration=1.228209ms49402026/11/12 00:44:21 INFO Vacuumed table table=closures49412026/11/12 00:44:21 INFO Completed upload id=22 objects=5 duration=1.547083ms49422026/11/12 00:44:21 INFO Received complete push request method=POST path=/49432026/11/12 00:44:21 INFO Vacuumed table table=objects49442026/11/12 00:44:21 INFO Completed push id=27 objects=11 duration=1.805667ms49452026/11/12 00:44:21 INFO Aborted multipart uploads count=0 kept=049462026/11/12 00:44:21 INFO Vacuumed table table=pending_closures49472026/11/12 00:44:21 INFO Received complete upload request method=POST path=/49482026/11/12 00:44:21 INFO Received uploads request method=POST path=/49492026/11/12 00:44:21 INFO Received complete push request method=POST path=/49502026/11/12 00:44:21 WARN Force mode enabled - objects will be deleted immediately without grace period49512026/11/12 00:44:21 INFO Vacuumed table table=commit_receipts49522026/11/12 00:44:21 INFO Completed push id=28 objects=3 duration=2.402333ms49532026/11/12 00:44:21 INFO Completed upload id=30 objects=6 duration=2.776375ms49542026/11/12 00:44:21 INFO Vacuumed table table=pending_objects49552026/11/12 00:44:21 INFO Vacuumed table table=multipart_uploads49562026/11/12 00:44:21 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=049572026/11/12 00:44:21 INFO Vacuumed table table=closures49582026/11/12 00:44:21 INFO Received complete upload request method=POST path=/49592026/11/12 00:44:21 INFO Completed upload id=29 objects=26 duration=878.708µs49602026/11/12 00:44:21 INFO Vacuumed table table=objects49612026/11/12 00:44:21 INFO Aborted multipart uploads count=0 kept=049622026/11/12 00:44:21 INFO Vacuumed table table=pending_closures49632026/11/12 00:44:21 INFO Received complete push request method=POST path=/49642026/11/12 00:44:21 INFO Received complete upload request method=POST path=/49652026/11/12 00:44:21 WARN Force mode enabled - objects will be deleted immediately without grace period49662026/11/12 00:44:21 INFO Vacuumed table table=commit_receipts49672026/11/12 00:44:21 INFO Completed upload id=31 objects=5 duration=6.585625ms49682026/11/12 00:44:21 INFO Completed push id=29 objects=15 duration=6.619584ms49692026/11/12 00:44:21 INFO Received complete push request method=POST path=/49702026/11/12 00:44:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049712026/11/12 00:44:21 INFO Completed push id=28 objects=26 duration=2.963583ms49722026/11/12 00:44:21 INFO Vacuumed table table=pending_closures49732026/11/12 00:44:21 INFO Vacuumed table table=pending_objects49742026/11/12 00:44:21 INFO Received complete push request method=POST path=/49752026/11/12 00:44:21 INFO Vacuumed table table=multipart_uploads49762026/11/12 00:44:21 INFO Vacuumed table table=commit_receipts49772026/11/12 00:44:21 INFO Completed push id=30 objects=8 duration=913.25µs49782026/11/12 00:44:21 INFO Vacuumed table table=closures49792026/11/12 00:44:21 INFO Received complete push request method=POST path=/49802026/11/12 00:44:21 INFO Vacuumed table table=objects49812026/11/12 00:44:21 INFO Completed push id=30 objects=6 duration=1.235292ms49822026/11/12 00:44:21 INFO Received complete upload request method=POST path=/49832026/11/12 00:44:21 INFO Vacuumed table table=pending_objects49842026/11/12 00:44:21 INFO Received complete push request method=POST path=/49852026/11/12 00:44:21 INFO Vacuumed table table=multipart_uploads49862026/11/12 00:44:21 INFO Completed upload id=34 objects=3 duration=900.791µs49872026/11/12 00:44:21 INFO Aborted multipart uploads count=0 kept=049882026/11/12 00:44:21 INFO Completed push id=30 objects=3 duration=637.542µs49892026/11/12 00:44:21 INFO Vacuumed table table=closures49902026/11/12 00:44:21 INFO Vacuumed table table=objects49912026/11/12 00:44:21 INFO Received complete upload request method=POST path=/49922026/11/12 00:44:21 WARN Force mode enabled - objects will be deleted immediately without grace period49932026/11/12 00:44:21 INFO Received complete upload request method=POST path=/49942026/11/12 00:44:21 INFO Completed upload id=31 objects=5 duration=1.047709ms49952026/11/12 00:44:21 INFO Completed upload id=31 objects=3 duration=1.063416ms49962026/11/12 00:44:21 INFO Received complete push request method=POST path=/49972026/11/12 00:44:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=049982026/11/12 00:44:21 INFO Completed push id=5 objects=28 duration=1.85925ms49992026/11/12 00:44:21 INFO Vacuumed table table=pending_closures50002026/11/12 00:44:21 INFO Vacuumed table table=commit_receipts50012026/11/12 00:44:21 INFO Received complete push request method=POST path=/50022026/11/12 00:44:21 INFO Vacuumed table table=pending_objects50032026/11/12 00:44:21 INFO Aborted multipart uploads count=0 kept=050042026/11/12 00:44:21 INFO Vacuumed table table=multipart_uploads50052026/11/12 00:44:21 INFO Completed push id=32 objects=19 duration=3.4775ms50062026/11/12 00:44:21 INFO Vacuumed table table=closures50072026/11/12 00:44:21 INFO Vacuumed table table=objects50082026/11/12 00:44:21 WARN Force mode enabled - objects will be deleted immediately without grace period50092026/11/12 00:44:21 INFO Received complete push request method=POST path=/50102026/11/12 00:44:21 INFO Aborted multipart uploads count=0 kept=050112026/11/12 00:44:21 INFO Received complete push request method=POST path=/50122026/11/12 00:44:21 INFO Completed push id=32 objects=3 duration=1.3775ms50132026/11/12 00:44:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050142026/11/12 00:44:21 INFO Completed push id=10 objects=2 duration=2.571166ms50152026/11/12 00:44:21 WARN Force mode enabled - objects will be deleted immediately without grace period50162026/11/12 00:44:21 INFO Received uploads request method=POST path=/50172026/11/12 00:44:21 INFO Aborted multipart uploads count=0 kept=050182026/11/12 00:44:21 INFO Vacuumed table table=pending_closures50192026/11/12 00:44:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050202026/11/12 00:44:21 INFO Vacuumed table table=commit_receipts50212026/11/12 00:44:21 INFO Received complete upload request method=POST path=/50222026/11/12 00:44:21 WARN Force mode enabled - objects will be deleted immediately without grace period50232026/11/12 00:44:21 INFO Received push request method=POST path=/50242026/11/12 00:44:21 INFO Vacuumed table table=pending_closures50252026/11/12 00:44:21 INFO Vacuumed table table=pending_objects50262026/11/12 00:44:21 INFO Vacuumed table table=commit_receipts50272026/11/12 00:44:21 INFO Vacuumed table table=multipart_uploads50282026/11/12 00:44:21 INFO Completed upload id=13 objects=14 duration=3.339667ms50292026/11/12 00:44:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050302026/11/12 00:44:21 INFO Vacuumed table table=closures50312026/11/12 00:44:21 INFO Vacuumed table table=objects50322026/11/12 00:44:21 INFO Vacuumed table table=pending_objects50332026/11/12 00:44:21 INFO Aborted multipart uploads count=0 kept=050342026/11/12 00:44:21 INFO Vacuumed table table=multipart_uploads50352026/11/12 00:44:21 INFO Vacuumed table table=pending_closures50362026/11/12 00:44:21 INFO Vacuumed table table=closures50372026/11/12 00:44:21 INFO Vacuumed table table=commit_receipts50382026/11/12 00:44:21 INFO Vacuumed table table=objects50392026/11/12 00:44:21 WARN Force mode enabled - objects will be deleted immediately without grace period50402026/11/12 00:44:21 INFO Vacuumed table table=pending_objects50412026/11/12 00:44:21 INFO Garbage collection completed failed-uploads-deleted=2 old-closures-deleted=1 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050422026/11/12 00:44:21 INFO Received complete push request method=POST path=/50432026/11/12 00:44:21 INFO Vacuumed table table=multipart_uploads50442026/11/12 00:44:21 INFO Vacuumed table table=closures50452026/11/12 00:44:21 INFO Completed push id=22 objects=3 duration=6.298708ms50462026/11/12 00:44:21 INFO Vacuumed table table=pending_closures50472026/11/12 00:44:21 INFO Vacuumed table table=objects50482026/11/12 00:44:21 INFO Aborted multipart uploads count=0 kept=050492026/11/12 00:44:21 INFO Vacuumed table table=commit_receipts50502026/11/12 00:44:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050512026/11/12 00:44:21 INFO Received uploads request method=POST path=/50522026/11/12 00:44:21 INFO Vacuumed table table=pending_closures50532026/11/12 00:44:21 INFO Vacuumed table table=pending_objects50542026/11/12 00:44:21 INFO Vacuumed table table=multipart_uploads50552026/11/12 00:44:21 INFO Vacuumed table table=commit_receipts50562026/11/12 00:44:21 INFO Received complete push request method=POST path=/50572026/11/12 00:44:21 INFO Vacuumed table table=closures50582026/11/12 00:44:21 INFO Vacuumed table table=pending_objects50592026/11/12 00:44:21 INFO Received complete upload request method=POST path=/50602026/11/12 00:44:21 INFO Completed push id=16 objects=14 duration=2.398125ms50612026/11/12 00:44:21 INFO Vacuumed table table=multipart_uploads50622026/11/12 00:44:21 INFO Vacuumed table table=objects50632026/11/12 00:44:21 INFO Completed upload id=17 objects=5 duration=1.212083ms50642026/11/12 00:44:21 INFO Vacuumed table table=closures50652026/11/12 00:44:21 INFO Vacuumed table table=objects50662026/11/12 00:44:21 INFO Aborted multipart uploads count=0 kept=050672026/11/12 00:44:21 WARN Force mode enabled - objects will be deleted immediately without grace period50682026/11/12 00:44:21 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050692026/11/12 00:44:21 INFO Vacuumed table table=pending_closures50702026/11/12 00:44:21 INFO Vacuumed table table=commit_receipts50712026/11/12 00:44:21 INFO Received push request method=POST path=/50722026/11/12 00:44:21 INFO Vacuumed table table=pending_objects50732026/11/12 00:44:21 INFO Vacuumed table table=multipart_uploads50742026/11/12 00:44:21 INFO Received complete push request method=POST path=/50752026/11/12 00:44:21 INFO Vacuumed table table=closures50762026/11/12 00:44:21 INFO Completed push id=9 objects=9 duration=5.605625ms50772026/11/12 00:44:21 INFO Received complete push request method=POST path=/50782026/11/12 00:44:21 INFO Vacuumed table table=objects50792026/11/12 00:44:21 INFO Completed push id=24 objects=3 duration=6.686333ms50802026/11/12 00:44:21 INFO Received complete push request method=POST path=/50812026/11/12 00:44:21 INFO Completed push id=19 objects=3 duration=1.406666ms50822026/11/12 00:44:21 INFO Received complete push request method=POST path=/50832026/11/12 00:44:21 INFO Completed push id=31 objects=6 duration=6.250292ms50842026/11/12 00:44:21 INFO Received push request method=POST path=/50852026/11/12 00:44:21 INFO Aborted multipart uploads count=0 kept=050862026/11/12 00:44:21 INFO Received complete upload request method=POST path=/50872026/11/12 00:44:21 INFO Completed upload id=21 objects=8 duration=1.119292ms50882026/11/12 00:44:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=050892026/11/12 00:44:21 INFO Vacuumed table table=pending_closures50902026/11/12 00:44:21 INFO Vacuumed table table=commit_receipts50912026/11/12 00:44:21 INFO Received uploads request method=POST path=/50922026/11/12 00:44:21 INFO Vacuumed table table=pending_objects50932026/11/12 00:44:21 INFO Vacuumed table table=multipart_uploads50942026/11/12 00:44:21 INFO Vacuumed table table=closures50952026/11/12 00:44:21 INFO Vacuumed table table=objects50962026/11/12 00:44:21 INFO Received complete push request method=POST path=/50972026/11/12 00:44:21 INFO Completed push id=22 objects=6 duration=1.069459ms50982026/11/12 00:44:21 INFO Aborted multipart uploads count=0 kept=050992026/11/12 00:44:21 INFO Received push request method=POST path=/51002026/11/12 00:44:21 WARN Force mode enabled - objects will be deleted immediately without grace period51012026/11/12 00:44:21 INFO Received complete push request method=POST path=/51022026/11/12 00:44:21 INFO Completed push id=24 objects=8 duration=885.666µs51032026/11/12 00:44:21 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051042026/11/12 00:44:21 INFO Received uploads request method=POST path=/51052026/11/12 00:44:21 INFO Vacuumed table table=pending_closures51062026/11/12 00:44:21 INFO Received complete upload request method=POST path=/51072026/11/12 00:44:21 INFO Vacuumed table table=commit_receipts51082026/11/12 00:44:21 INFO Completed upload id=25 objects=9 duration=575.625µs51092026/11/12 00:44:21 INFO Vacuumed table table=pending_objects51102026/11/12 00:44:21 INFO Vacuumed table table=multipart_uploads51112026/11/12 00:44:21 INFO Received complete push request method=POST path=/51122026/11/12 00:44:21 INFO Vacuumed table table=closures51132026/11/12 00:44:21 INFO Completed push id=20 objects=4 duration=674.167µs51142026/11/12 00:44:21 INFO Vacuumed table table=objects51152026/11/12 00:44:21 INFO Received complete upload request method=POST path=/51162026/11/12 00:44:21 INFO Completed upload id=27 objects=3 duration=808.041µs51172026/11/12 00:44:21 INFO Received complete upload request method=POST path=/51182026/11/12 00:44:21 INFO Completed upload id=25 objects=6 duration=1.71625ms51192026/11/12 00:44:21 INFO Received complete upload request method=POST path=/51202026/11/12 00:44:21 INFO Completed upload id=28 objects=6 duration=460.292µs51212026/11/12 00:44:21 INFO Received complete push request method=POST path=/51222026/11/12 00:44:21 INFO Completed push id=30 objects=4 duration=1.095667ms51232026/11/12 00:44:21 INFO Received push request method=POST path=/51242026/11/12 00:44:21 INFO Aborted multipart uploads count=0 kept=051252026/11/12 00:44:21 INFO Received complete push request method=POST path=/51262026/11/12 00:44:21 INFO Completed push id=31 objects=7 duration=1.399875ms51272026/11/12 00:44:21 INFO Garbage collection completed failed-uploads-deleted=1 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=2 objects-failed-to-delete=051282026/11/12 00:44:21 INFO Vacuumed table table=pending_closures51292026/11/12 00:44:21 INFO Received uploads request method=POST path=/51302026/11/12 00:44:21 INFO Vacuumed table table=commit_receipts51312026/11/12 00:44:21 INFO Received complete upload request method=POST path=/51322026/11/12 00:44:21 INFO Completed upload id=32 objects=5 duration=934.125µs51332026/11/12 00:44:21 INFO Vacuumed table table=pending_objects51342026/11/12 00:44:21 INFO Vacuumed table table=multipart_uploads51352026/11/12 00:44:21 INFO Vacuumed table table=closures51362026/11/12 00:44:21 INFO Received complete upload request method=POST path=/51372026/11/12 00:44:21 INFO Vacuumed table table=objects51382026/11/12 00:44:21 INFO Received complete push request method=POST path=/51392026/11/12 00:44:21 INFO Completed upload id=22 objects=8 duration=1.6155ms51402026/11/12 00:44:21 INFO Completed push id=11 objects=9 duration=1.170708ms51412026/11/12 00:44:21 INFO Received complete push request method=POST path=/51422026/11/12 00:44:21 INFO Completed push id=23 objects=5 duration=916µs51432026/11/12 00:44:21 INFO Received uploads request method=POST path=/51442026/11/12 00:44:21 INFO Received complete push request method=POST path=/51452026/11/12 00:44:21 INFO Received complete upload request method=POST path=/51462026/11/12 00:44:21 INFO Completed push id=15 objects=24 duration=5.692583ms51472026/11/12 00:44:21 INFO Completed upload id=24 objects=8 duration=1.022834ms51482026/11/12 00:44:21 INFO Received complete upload request method=POST path=/51492026/11/12 00:44:21 INFO Completed upload id=26 objects=2 duration=1.176333ms51502026/11/12 00:44:21 INFO Received complete upload request method=POST path=/51512026/11/12 00:44:21 INFO Completed upload id=28 objects=5 duration=415.709µs51522026/11/12 00:44:21 INFO Received complete upload request method=POST path=/51532026/11/12 00:44:21 INFO Completed upload id=19 objects=2 duration=851.916µs51542026/11/12 00:44:21 INFO Aborted multipart uploads count=0 kept=051552026/11/12 00:44:21 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051562026/11/12 00:44:21 INFO Vacuumed table table=pending_closures51572026/11/12 00:44:21 INFO Vacuumed table table=commit_receipts51582026/11/12 00:44:21 INFO Vacuumed table table=pending_objects51592026/11/12 00:44:21 INFO Vacuumed table table=multipart_uploads51602026/11/12 00:44:21 INFO Vacuumed table table=closures51612026/11/12 00:44:21 INFO Vacuumed table table=objects51622026/11/12 00:44:21 INFO Received complete upload request method=POST path=/51632026/11/12 00:44:21 INFO Completed upload id=24 objects=7 duration=747.792µs51642026/11/12 00:44:21 INFO Received push request method=POST path=/51652026/11/12 00:44:21 INFO Received complete upload request method=POST path=/51662026/11/12 00:44:21 INFO Received complete push request method=POST path=/51672026/11/12 00:44:21 INFO Completed upload id=30 objects=5 duration=574.375µs51682026/11/12 00:44:21 INFO Completed push id=26 objects=3 duration=545.416µs51692026/11/12 00:44:21 INFO Received complete upload request method=POST path=/51702026/11/12 00:44:21 INFO Completed upload id=28 objects=8 duration=826.833µs51712026/11/12 00:44:21 INFO Received complete push request method=POST path=/51722026/11/12 00:44:21 INFO Completed push id=31 objects=3 duration=787.125µs51732026/11/12 00:44:21 INFO Received complete upload request method=POST path=/51742026/11/12 00:44:22 INFO Completed upload id=29 objects=10 duration=965.666µs51752026/11/12 00:44:22 INFO Received complete upload request method=POST path=/51762026/11/12 00:44:22 INFO Aborted multipart uploads count=0 kept=051772026/11/12 00:44:22 INFO Completed upload id=34 objects=24 duration=4.234125ms51782026/11/12 00:44:22 INFO Received complete push request method=POST path=/51792026/11/12 00:44:22 INFO Completed push id=1 objects=9 duration=870.542µs51802026/11/12 00:44:22 ERROR failed to remove object object=cccccccccccccccccccccccccccccccc.ls error="Access Denied."51812026/11/12 00:44:22 ERROR failed to remove object object=ffffffffffffffffffffffffffffffff.ls error="Access Denied."51822026/11/12 00:44:22 ERROR failed to remove object object=realisations/sha256:ffffffff!out.doi error="Access Denied."51832026/11/12 00:44:22 INFO Received complete upload request method=POST path=/51842026/11/12 00:44:22 INFO Completed upload id=30 objects=20 duration=818.667µs51852026/11/12 00:44:22 INFO Aborted multipart uploads count=0 kept=051862026/11/12 00:44:22 WARN Force mode enabled - objects will be deleted immediately without grace period51872026/11/12 00:44:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=051882026/11/12 00:44:22 INFO Vacuumed table table=pending_closures51892026/11/12 00:44:22 INFO Vacuumed table table=commit_receipts51902026/11/12 00:44:22 INFO Vacuumed table table=pending_objects51912026/11/12 00:44:22 INFO Received push request method=POST path=/51922026/11/12 00:44:22 INFO Vacuumed table table=multipart_uploads51932026/11/12 00:44:22 INFO Received complete push request method=POST path=/51942026/11/12 00:44:22 INFO Vacuumed table table=closures51952026/11/12 00:44:22 INFO Completed push id=31 objects=11 duration=861.75µs51962026/11/12 00:44:22 INFO Vacuumed table table=objects51972026/11/12 00:44:22 INFO Received complete push request method=POST path=/51982026/11/12 00:44:22 INFO Completed push id=32 objects=6 duration=684.791µs51992026/11/12 00:44:22 INFO Received complete upload request method=POST path=/52002026/11/12 00:44:22 INFO Completed upload id=32 objects=2 duration=637.417µs52012026/11/12 00:44:22 INFO Received complete upload request method=POST path=/52022026/11/12 00:44:22 INFO Completed upload id=33 objects=7 duration=563.167µs52032026/11/12 00:44:22 INFO Received push request method=POST path=/52042026/11/12 00:44:22 INFO Received complete push request method=POST path=/52052026/11/12 00:44:22 INFO Completed push id=33 objects=3 duration=1.714125ms52062026/11/12 00:44:22 INFO Received complete upload request method=POST path=/52072026/11/12 00:44:22 INFO Completed upload id=34 objects=3 duration=924.125µs52082026/11/12 00:44:22 INFO Received complete upload request method=POST path=/52092026/11/12 00:44:22 INFO Received complete push request method=POST path=/52102026/11/12 00:44:22 INFO Completed upload id=34 objects=3 duration=527.708µs52112026/11/12 00:44:22 INFO Completed push id=28 objects=11 duration=2.546083ms52122026/11/12 00:44:22 INFO Received complete push request method=POST path=/52132026/11/12 00:44:22 INFO Completed push id=35 objects=11 duration=709.958µs52142026/11/12 00:44:22 INFO Aborted multipart uploads count=0 kept=052152026/11/12 00:44:22 WARN Force mode enabled - objects will be deleted immediately without grace period52162026/11/12 00:44:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052172026/11/12 00:44:22 INFO Aborted multipart uploads count=0 kept=052182026/11/12 00:44:22 INFO Vacuumed table table=pending_closures52192026/11/12 00:44:22 INFO Vacuumed table table=commit_receipts52202026/11/12 00:44:22 WARN Force mode enabled - objects will be deleted immediately without grace period52212026/11/12 00:44:22 INFO Vacuumed table table=pending_objects52222026/11/12 00:44:22 INFO Vacuumed table table=multipart_uploads52232026/11/12 00:44:22 INFO Garbage collection completed failed-uploads-deleted=6 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052242026/11/12 00:44:22 INFO Vacuumed table table=closures52252026/11/12 00:44:22 INFO Vacuumed table table=objects52262026/11/12 00:44:22 INFO Vacuumed table table=pending_closures52272026/11/12 00:44:22 INFO Vacuumed table table=commit_receipts52282026/11/12 00:44:22 INFO Vacuumed table table=pending_objects52292026/11/12 00:44:22 INFO Vacuumed table table=multipart_uploads52302026/11/12 00:44:22 INFO Vacuumed table table=closures52312026/11/12 00:44:22 INFO Vacuumed table table=objects52322026/11/12 00:44:22 INFO Received complete upload request method=POST path=/52332026/11/12 00:44:22 INFO Completed upload id=2 objects=7 duration=661.583µs52342026/11/12 00:44:22 INFO Received complete push request method=POST path=/52352026/11/12 00:44:22 INFO Completed push id=29 objects=24 duration=1.11775ms52362026/11/12 00:44:22 INFO Aborted multipart uploads count=0 kept=052372026/11/12 00:44:22 WARN Force mode enabled - objects will be deleted immediately without grace period52382026/11/12 00:44:22 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=1 objects-failed-to-delete=052392026/11/12 00:44:22 INFO Vacuumed table table=pending_closures52402026/11/12 00:44:22 INFO Vacuumed table table=commit_receipts52412026/11/12 00:44:22 INFO Vacuumed table table=pending_objects52422026/11/12 00:44:22 INFO Vacuumed table table=multipart_uploads52432026/11/12 00:44:22 INFO Vacuumed table table=closures52442026/11/12 00:44:22 INFO Vacuumed table table=objects52452026/11/12 00:44:22 INFO Received complete upload request method=POST path=/52462026/11/12 00:44:22 INFO Completed upload id=8 objects=6 duration=775.625µs52472026/11/12 00:44:22 INFO Received complete upload request method=POST path=/52482026/11/12 00:44:22 INFO Completed upload id=12 objects=9 duration=510.583µs52492026/11/12 00:44:22 INFO Received complete upload request method=POST path=/52502026/11/12 00:44:22 INFO Completed upload id=18 objects=22 duration=785.667µs52512026/11/12 00:44:22 INFO Received complete upload request method=POST path=/52522026/11/12 00:44:22 INFO Completed upload id=21 objects=8 duration=620.708µs52532026/11/12 00:44:22 INFO Received complete upload request method=POST path=/52542026/11/12 00:44:22 INFO Completed upload id=22 objects=7 duration=587.458µs52552026/11/12 00:44:23 INFO Received complete push request method=POST path=/52562026/11/12 00:44:23 INFO Completed push id=24 objects=5 duration=602.417µs52572026/11/12 00:44:23 INFO Received complete upload request method=POST path=/52582026/11/12 00:44:23 INFO Completed upload id=25 objects=8 duration=627.417µs52592026/11/12 00:44:23 INFO Received complete push request method=POST path=/52602026/11/12 00:44:23 INFO Completed push id=26 objects=4 duration=611.792µs52612026/11/12 00:44:23 INFO Received complete push request method=POST path=/52622026/11/12 00:44:23 INFO Completed push id=30 objects=2 duration=654.125µs52632026/11/12 00:44:23 INFO Received complete push request method=POST path=/52642026/11/12 00:44:23 INFO Completed push id=32 objects=7 duration=681.958µs52652026/11/12 00:44:23 INFO Received complete upload request method=POST path=/52662026/11/12 00:44:23 INFO Completed upload id=33 objects=6 duration=644.875µs52672026/11/12 00:44:23 INFO Received complete upload request method=POST path=/52682026/11/12 00:44:23 INFO Completed upload id=34 objects=3 duration=560.667µs52692026/11/12 00:44:23 INFO Aborted multipart uploads count=0 kept=052702026/11/12 00:44:23 WARN Force mode enabled - objects will be deleted immediately without grace period52712026/11/12 00:44:23 INFO Garbage collection completed failed-uploads-deleted=0 old-closures-deleted=0 objects-marked-for-deletion=0 objects-deleted-after-grace-period=0 objects-failed-to-delete=052722026/11/12 00:44:23 INFO Vacuumed table table=pending_closures52732026/11/12 00:44:23 INFO Vacuumed table table=commit_receipts52742026/11/12 00:44:23 INFO Vacuumed table table=pending_objects52752026/11/12 00:44:23 INFO Vacuumed table table=multipart_uploads52762026/11/12 00:44:23 INFO Vacuumed table table=closures52772026/11/12 00:44:23 INFO Vacuumed table table=objects5278--- PASS: FuzzPushGC (0.00s)5279 --- PASS: FuzzPushGC/seed#14 (9.82s)5280 --- PASS: FuzzPushGC/seed#0 (10.85s)5281 --- PASS: FuzzPushGC/seed#11 (12.93s)5282 --- PASS: FuzzPushGC/seed#1 (15.06s)5283 --- PASS: FuzzPushGC/seed#4 (15.14s)5284 --- PASS: FuzzPushGC/seed#6 (15.77s)5285 --- PASS: FuzzPushGC/seed#12 (15.87s)5286 --- PASS: FuzzPushGC/seed#13 (15.96s)5287 --- PASS: FuzzPushGC/seed#15 (15.97s)5288 --- PASS: FuzzPushGC/seed#7 (16.00s)5289 --- PASS: FuzzPushGC/seed#3 (16.03s)5290 --- PASS: FuzzPushGC/seed#9 (16.87s)5291 --- PASS: FuzzPushGC/seed#5 (17.03s)5292 --- PASS: FuzzPushGC/seed#8 (17.05s)5293 --- PASS: FuzzPushGC/seed#2 (17.15s)5294 --- PASS: FuzzPushGC/seed#10 (17.91s)5295PASS5296{"timestamp":"2026-11-12T00:44:23.194732Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:57116","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(6)"}5297{"timestamp":"2026-11-12T00:44:23.194733Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:56786","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(11)"}5298{"timestamp":"2026-11-12T00:44:23.194732Z","level":"ERROR","message":"HTTP transport failed","event":"http_transport_failed","component":"server","subsystem":"transport","peer_addr":"127.0.0.1:57215","error_kind":"io_error","error":"Cancelled","result":"transport_error","target":"rustfs::server::http","filename":"rustfs/src/server/http.rs","line_number":2561,"threadName":"rustfs-worker","threadId":"ThreadId(3)"}52992026-11-12 00:44:23.416 UTC [40635] LOG: received smart shutdown request53002026-11-12 00:44:23.417 UTC [40635] LOG: background worker "logical replication launcher" (PID 40646) exited with exit code 153012026-11-12 00:44:25.028 UTC [40640] LOG: checkpoint complete: wrote 3341 buffers (20.4%), wrote 4 SLRU buffers; 0 WAL file(s) added, 0 removed, 33 recycled; write=30.690 s, sync=1.474 s, total=32.334 s; sync files=43472, longest=0.008 s, average=0.001 s; distance=533227 kB, estimate=533227 kB; lsn=0/2867A6D0, redo lsn=0/22047BE053022026-11-12 00:44:25.031 UTC [40640] LOG: shutting down53032026-11-12 00:44:25.031 UTC [40640] LOG: checkpoint starting: shutdown immediate53042026-11-12 00:44:25.794 UTC [40640] LOG: checkpoint complete: wrote 9271 buffers (56.6%), wrote 2 SLRU buffers; 0 WAL file(s) added, 0 removed, 6 recycled; write=0.632 s, sync=0.112 s, total=0.764 s; sync files=3142, longest=0.003 s, average=0.001 s; distance=104650 kB, estimate=490369 kB; lsn=0/2867A748, redo lsn=0/2867A74853052026-11-12 00:44:25.808 UTC [40635] LOG: database system is shut down5306Running OIDC tests...5307=== RUN TestAudienceForIssuer5308=== PAUSE TestAudienceForIssuer5309=== RUN TestProviderClientGivesUpOnSilentIssuer5310=== PAUSE TestProviderClientGivesUpOnSilentIssuer5311=== RUN TestGlobMatch5312=== PAUSE TestGlobMatch5313=== RUN TestValidateToken_ValidToken5314=== PAUSE TestValidateToken_ValidToken5315=== RUN TestValidateToken_WrongAudience5316=== PAUSE TestValidateToken_WrongAudience5317=== RUN TestValidateToken_Expired5318=== PAUSE TestValidateToken_Expired5319=== RUN TestValidateToken_BoundClaimsMismatch5320=== PAUSE TestValidateToken_BoundClaimsMismatch5321=== RUN TestValidateToken_BoundSubjectMismatch5322=== PAUSE TestValidateToken_BoundSubjectMismatch5323=== RUN TestValidateToken_MultipleProviders5324=== PAUSE TestValidateToken_MultipleProviders5325=== RUN TestValidateToken_UnknownIssuerFetchesNoKeys5326=== PAUSE TestValidateToken_UnknownIssuerFetchesNoKeys5327=== RUN TestValidateToken_GoogleSchemelessIssuer5328=== PAUSE TestValidateToken_GoogleSchemelessIssuer5329=== RUN TestValidateToken_KubernetesServiceAccount5330=== PAUSE TestValidateToken_KubernetesServiceAccount5331=== RUN TestNewValidator_KubernetesRequiresCA5332=== PAUSE TestNewValidator_KubernetesRequiresCA5333=== RUN TestValidateToken_KubernetesIssuerFromOwnToken5334=== PAUSE TestValidateToken_KubernetesIssuerFromOwnToken5335=== RUN TestPins_ReservedForMatchingRule5336=== PAUSE TestPins_ReservedForMatchingRule5337=== RUN TestPins_TopLevelShorthand5338=== PAUSE TestPins_TopLevelShorthand5339=== RUN TestPins_ConfigValidation5340=== PAUSE TestPins_ConfigValidation5341=== RUN TestScopes_LegacyProviderDefaultsToWrite5342=== PAUSE TestScopes_LegacyProviderDefaultsToWrite5343=== RUN TestScopes_Rules5344=== PAUSE TestScopes_Rules5345=== RUN TestScopes_ConfigValidation5346=== PAUSE TestScopes_ConfigValidation5347=== CONT TestAudienceForIssuer5348--- PASS: TestAudienceForIssuer (0.00s)5349=== CONT TestValidateToken_BoundSubjectMismatch5350=== CONT TestValidateToken_KubernetesServiceAccount5351=== CONT TestValidateToken_BoundClaimsMismatch5352=== CONT TestValidateToken_Expired5353=== CONT TestValidateToken_WrongAudience5354=== CONT TestValidateToken_ValidToken5355=== CONT TestGlobMatch5356=== RUN TestGlobMatch/foo_foo5357=== PAUSE TestGlobMatch/foo_foo5358=== RUN TestGlobMatch/foo_bar5359=== PAUSE TestGlobMatch/foo_bar5360=== RUN TestGlobMatch/*_5361=== PAUSE TestGlobMatch/*_5362=== RUN TestGlobMatch/*_anything5363=== PAUSE TestGlobMatch/*_anything5364=== RUN TestGlobMatch/foo*_foo5365=== PAUSE TestGlobMatch/foo*_foo5366=== RUN TestGlobMatch/foo*_foobar5367=== PAUSE TestGlobMatch/foo*_foobar5368=== RUN TestGlobMatch/foo*_bar5369=== PAUSE TestGlobMatch/foo*_bar5370=== RUN TestGlobMatch/*bar_bar5371=== PAUSE TestGlobMatch/*bar_bar5372=== RUN TestGlobMatch/*bar_foobar5373=== PAUSE TestGlobMatch/*bar_foobar5374=== RUN TestGlobMatch/*bar_foo5375=== CONT TestProviderClientGivesUpOnSilentIssuer5376=== CONT TestValidateToken_KubernetesIssuerFromOwnToken5377--- PASS: TestProviderClientGivesUpOnSilentIssuer (0.00s)5378=== CONT TestValidateToken_UnknownIssuerFetchesNoKeys5379=== CONT TestValidateToken_GoogleSchemelessIssuer53802026/11/12 00:44:30 INFO OIDC provider initialized name=known issuer=https://known.example5381=== CONT TestValidateToken_MultipleProviders5382=== CONT TestNewValidator_KubernetesRequiresCA5383=== CONT TestScopes_LegacyProviderDefaultsToWrite5384=== CONT TestScopes_ConfigValidation5385--- PASS: TestScopes_ConfigValidation (0.00s)5386=== CONT TestScopes_Rules5387=== CONT TestPins_ConfigValidation5388--- PASS: TestPins_ConfigValidation (0.00s)5389=== CONT TestPins_TopLevelShorthand5390=== CONT TestPins_ReservedForMatchingRule5391=== PAUSE TestGlobMatch/*bar_foo5392=== RUN TestGlobMatch/foo*bar_foobar5393=== PAUSE TestGlobMatch/foo*bar_foobar5394=== RUN TestGlobMatch/foo*bar_foo123bar5395=== PAUSE TestGlobMatch/foo*bar_foo123bar5396=== RUN TestGlobMatch/foo*bar_foobarbaz5397=== PAUSE TestGlobMatch/foo*bar_foobarbaz5398=== RUN TestGlobMatch/*/*_foo/bar5399=== PAUSE TestGlobMatch/*/*_foo/bar5400=== RUN TestGlobMatch/*/*_foo5401=== PAUSE TestGlobMatch/*/*_foo5402=== RUN TestGlobMatch/refs/heads/*_refs/heads/main5403=== PAUSE TestGlobMatch/refs/heads/*_refs/heads/main5404=== RUN TestGlobMatch/refs/heads/*_refs/tags/v1.05405=== PAUSE TestGlobMatch/refs/heads/*_refs/tags/v1.05406=== RUN TestGlobMatch/refs/*/main_refs/heads/main5407=== PAUSE TestGlobMatch/refs/*/main_refs/heads/main5408=== RUN TestGlobMatch/fo?_foo5409=== PAUSE TestGlobMatch/fo?_foo5410=== RUN TestGlobMatch/fo?_fo5411=== PAUSE TestGlobMatch/fo?_fo5412=== RUN TestGlobMatch/fo?_fooo5413=== PAUSE TestGlobMatch/fo?_fooo5414=== RUN TestGlobMatch/?oo_foo5415=== PAUSE TestGlobMatch/?oo_foo5416=== RUN TestGlobMatch/?oo_boo5417=== PAUSE TestGlobMatch/?oo_boo5418=== RUN TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5419=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5420=== RUN TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5421=== PAUSE TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5422=== CONT TestGlobMatch/foo_foo5423=== CONT TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main5424=== CONT TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main5425=== CONT TestGlobMatch/?oo_boo5426=== CONT TestGlobMatch/?oo_foo5427=== CONT TestGlobMatch/fo?_fooo5428=== CONT TestGlobMatch/fo?_fo5429=== CONT TestGlobMatch/fo?_foo5430=== CONT TestGlobMatch/refs/*/main_refs/heads/main5431=== CONT TestGlobMatch/refs/heads/*_refs/tags/v1.05432=== CONT TestGlobMatch/refs/heads/*_refs/heads/main5433=== CONT TestGlobMatch/*/*_foo5434=== CONT TestGlobMatch/*/*_foo/bar5435=== CONT TestGlobMatch/foo*bar_foobarbaz5436=== CONT TestGlobMatch/foo*bar_foo123bar5437=== CONT TestGlobMatch/foo*bar_foobar5438=== CONT TestGlobMatch/*bar_foo5439=== CONT TestGlobMatch/*bar_foobar5440=== CONT TestGlobMatch/*bar_bar5441=== CONT TestGlobMatch/foo*_bar5442=== CONT TestGlobMatch/foo*_foobar5443=== CONT TestGlobMatch/foo*_foo5444=== CONT TestGlobMatch/*_anything5445=== CONT TestGlobMatch/*_5446=== CONT TestGlobMatch/foo_bar5447--- PASS: TestGlobMatch (0.05s)5448 --- PASS: TestGlobMatch/foo_foo (0.00s)5449 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:otherorg/myrepo:ref:refs/heads/main (0.00s)5450 --- PASS: TestGlobMatch/repo:myorg/*:*_repo:myorg/myrepo:ref:refs/heads/main (0.00s)5451 --- PASS: TestGlobMatch/?oo_boo (0.00s)5452 --- PASS: TestGlobMatch/?oo_foo (0.00s)5453 --- PASS: TestGlobMatch/fo?_fooo (0.00s)5454 --- PASS: TestGlobMatch/fo?_fo (0.00s)5455 --- PASS: TestGlobMatch/fo?_foo (0.00s)5456 --- PASS: TestGlobMatch/refs/*/main_refs/heads/main (0.00s)5457 --- PASS: TestGlobMatch/refs/heads/*_refs/tags/v1.0 (0.00s)5458 --- PASS: TestGlobMatch/refs/heads/*_refs/heads/main (0.00s)5459 --- PASS: TestGlobMatch/*/*_foo (0.00s)5460 --- PASS: TestGlobMatch/*/*_foo/bar (0.00s)5461 --- PASS: TestGlobMatch/foo*bar_foobarbaz (0.00s)5462 --- PASS: TestGlobMatch/foo*bar_foo123bar (0.00s)5463 --- PASS: TestGlobMatch/foo*bar_foobar (0.00s)5464 --- PASS: TestGlobMatch/*bar_foo (0.00s)5465 --- PASS: TestGlobMatch/*bar_foobar (0.00s)5466 --- PASS: TestGlobMatch/*bar_bar (0.00s)5467 --- PASS: TestGlobMatch/foo*_bar (0.00s)5468 --- PASS: TestGlobMatch/foo*_foobar (0.00s)5469 --- PASS: TestGlobMatch/foo*_foo (0.00s)5470 --- PASS: TestGlobMatch/*_anything (0.00s)5471 --- PASS: TestGlobMatch/*_ (0.00s)5472 --- PASS: TestGlobMatch/foo_bar (0.00s)54732026/11/12 00:44:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58583/oidc5474--- PASS: TestValidateToken_BoundClaimsMismatch (0.10s)54752026/11/12 00:44:30 INFO OIDC provider initialized name=google issuer=https://accounts.google.com5476--- PASS: TestValidateToken_GoogleSchemelessIssuer (0.17s)54772026/11/12 00:44:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58588/oidc54782026/11/12 00:44:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58589/oidc5479--- PASS: TestPins_TopLevelShorthand (0.19s)5480--- PASS: TestValidateToken_BoundSubjectMismatch (0.22s)54812026/11/12 00:44:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58593/oidc5482--- PASS: TestValidateToken_Expired (0.26s)54832026/11/12 00:44:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58598/oidc54842026/11/12 00:44:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58600/oidc5485--- PASS: TestValidateToken_ValidToken (0.28s)54862026/11/12 00:44:30 INFO OIDC provider initialized name=kubernetes issuer=https://127.0.0.1:585955487--- PASS: TestPins_ReservedForMatchingRule (0.25s)5488--- PASS: TestValidateToken_KubernetesServiceAccount (0.29s)54892026/11/12 00:44:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58602/oidc54902026/11/12 00:44:30 INFO OIDC provider initialized name=kubernetes issuer=https://oidc.eks.invalid/id/ABC1235491--- PASS: TestScopes_LegacyProviderDefaultsToWrite (0.29s)5492--- PASS: TestValidateToken_UnknownIssuerFetchesNoKeys (0.30s)5493--- PASS: TestValidateToken_KubernetesIssuerFromOwnToken (0.33s)54942026/11/12 00:44:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58608/oidc5495--- PASS: TestValidateToken_WrongAudience (0.37s)54962026/11/12 00:44:30 INFO OIDC provider initialized name=provider1 issuer=http://127.0.0.1:58596/oidc54972026/11/12 00:44:30 INFO OIDC provider initialized name=provider2 issuer=http://127.0.0.1:58610/oidc5498--- PASS: TestValidateToken_MultipleProviders (0.39s)54992026/11/12 00:44:30 http: TLS handshake error from 127.0.0.1:58614: remote error: tls: bad certificate5500--- PASS: TestNewValidator_KubernetesRequiresCA (0.43s)55012026/11/12 00:44:30 INFO OIDC provider initialized name=test issuer=http://127.0.0.1:58615/oidc5502--- PASS: TestScopes_Rules (0.48s)5503PASS5504Running hook tests...5505=== RUN TestSendPathsEmpty5506=== PAUSE TestSendPathsEmpty5507=== RUN TestQueueEnqueueAndFetch5508=== PAUSE TestQueueEnqueueAndFetch5509=== RUN TestQueueDeduplication5510=== PAUSE TestQueueDeduplication5511=== RUN TestQueueRemove5512=== PAUSE TestQueueRemove5513=== RUN TestQueueFetchBatchLimit5514=== PAUSE TestQueueFetchBatchLimit5515=== RUN TestQueueRetryMovesToBack5516=== PAUSE TestQueueRetryMovesToBack5517=== RUN TestQueueFetchRemoveLifecycle5518=== PAUSE TestQueueFetchRemoveLifecycle5519=== RUN TestQueueConcurrentWriters5520=== PAUSE TestQueueConcurrentWriters5521=== RUN TestQueueRemoveLargeClosure5522=== PAUSE TestQueueRemoveLargeClosure5523=== RUN TestServerClientIntegration5524--- PASS: TestServerClientIntegration (0.00s)5525=== RUN TestServerQueueError55262026/11/12 00:44:31 ERROR Failed to queue paths error="permission denied" count=15527--- PASS: TestServerQueueError (0.00s)5528=== RUN TestGetListenerSocketActivation5529 server_test.go:147: systemd socket activation does not exist on darwin5530--- SKIP: TestGetListenerSocketActivation (0.00s)5531=== RUN TestServerStalledClientDoesNotBlockShutdown5532=== PAUSE TestServerStalledClientDoesNotBlockShutdown5533=== RUN TestServerBacksOffOnAcceptErrors5534=== PAUSE TestServerBacksOffOnAcceptErrors5535=== RUN TestDrainIsolatesPoisonPath5536=== PAUSE TestDrainIsolatesPoisonPath5537=== RUN TestRunNotBlockedByPoisonHead5538=== PAUSE TestRunNotBlockedByPoisonHead5539=== RUN TestDrainGivesUpWhenServerDown5540=== PAUSE TestDrainGivesUpWhenServerDown5541=== RUN TestFailedPathPrunedByLaterClosure5542=== PAUSE TestFailedPathPrunedByLaterClosure5543=== RUN TestWorkerUploadsAndRemoves5544=== PAUSE TestWorkerUploadsAndRemoves5545=== RUN TestWorkerSkipsGCdPaths5546=== PAUSE TestWorkerSkipsGCdPaths5547=== RUN TestWorkerPrunesClosureDeps5548=== PAUSE TestWorkerPrunesClosureDeps5549=== RUN TestDrainTimeout5550=== PAUSE TestDrainTimeout5551=== RUN TestWorkerRemoveFailureIsNotProgress5552=== PAUSE TestWorkerRemoveFailureIsNotProgress5553=== RUN TestWorkerKeepsPathItCannotStat5554=== PAUSE TestWorkerKeepsPathItCannotStat5555=== CONT TestSendPathsEmpty5556--- PASS: TestSendPathsEmpty (0.00s)5557=== CONT TestWorkerKeepsPathItCannotStat5558=== CONT TestServerBacksOffOnAcceptErrors55592000/01/01 00:00:00 ERROR Accept failed error="too many open files"55602000/01/01 00:00:00 ERROR Accept failed error="too many open files"55612000/01/01 00:00:00 ERROR Accept failed error="too many open files"55622000/01/01 00:00:00 ERROR Accept failed error="too many open files"55632000/01/01 00:00:00 ERROR Accept failed error="too many open files"55642000/01/01 00:00:00 ERROR Accept failed error="too many open files"55652000/01/01 00:00:00 ERROR Accept failed error="too many open files"55662000/01/01 00:00:00 ERROR Accept failed error="too many open files"5567=== CONT TestQueueRetryMovesToBack5568--- PASS: TestServerBacksOffOnAcceptErrors (0.00s)5569=== CONT TestServerStalledClientDoesNotBlockShutdown5570=== CONT TestQueueRemoveLargeClosure5571=== CONT TestQueueConcurrentWriters5572=== CONT TestQueueFetchRemoveLifecycle5573=== CONT TestWorkerUploadsAndRemoves5574=== CONT TestWorkerRemoveFailureIsNotProgress5575=== CONT TestDrainTimeout5576=== CONT TestWorkerPrunesClosureDeps5577=== CONT TestWorkerSkipsGCdPaths5578=== CONT TestDrainGivesUpWhenServerDown5579=== CONT TestFailedPathPrunedByLaterClosure5580=== CONT TestQueueRemove5581=== CONT TestQueueFetchBatchLimit5582=== CONT TestQueueDeduplication5583=== CONT TestRunNotBlockedByPoisonHead5584=== CONT TestQueueEnqueueAndFetch5585=== CONT TestDrainIsolatesPoisonPath55862000/01/01 00:00:00 INFO Uploading batch count=255872000/01/01 00:00:00 ERROR Upload failed error="context deadline exceeded" count=255882000/01/01 00:00:00 ERROR Drain finished with paths left in queue remaining=455892026/11/12 00:44:31 INFO Upload queue status pending=255902026/11/12 00:44:31 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestWorkerSkipsGCdPaths1824577752/002/nonexistent55912026/11/12 00:44:31 INFO Upload queue status pending=255922026/11/12 00:44:31 WARN Cannot stat store path, will retry later path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestWorkerKeepsPathItCannotStat194553206/002/locked/aaa error="stat /private/var/tb/3/scratch/cf270c76132c2e86/build/TestWorkerKeepsPathItCannotStat194553206/002/locked/aaa: permission denied"55932026/11/12 00:44:31 INFO Uploading batch count=155942026/11/12 00:44:31 INFO Uploading batch count=155952026/11/12 00:44:31 INFO Uploading batch count=155962026/11/12 00:44:31 ERROR Upload failed error="upload failed" count=155972026/11/12 00:44:31 INFO Uploading batch count=15598--- PASS: TestQueueRetryMovesToBack (0.01s)5599--- PASS: TestQueueFetchBatchLimit (0.01s)56002026/11/12 00:44:31 WARN Cannot stat store path, will retry later path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestWorkerKeepsPathItCannotStat194553206/002/locked/aaa error="stat /private/var/tb/3/scratch/cf270c76132c2e86/build/TestWorkerKeepsPathItCannotStat194553206/002/locked/aaa: permission denied"5601--- PASS: TestQueueRemove (0.01s)56022026/11/12 00:44:31 INFO Uploading batch count=156032026/11/12 00:44:31 WARN Cannot stat store path, will retry later path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestWorkerKeepsPathItCannotStat194553206/002/locked/aaa error="stat /private/var/tb/3/scratch/cf270c76132c2e86/build/TestWorkerKeepsPathItCannotStat194553206/002/locked/aaa: permission denied"5604--- PASS: TestQueueFetchRemoveLifecycle (0.01s)56052026/11/12 00:44:31 INFO Uploading batch count=456062026/11/12 00:44:31 ERROR Upload failed error="upload failed" count=456072026/11/12 00:44:31 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestDrainIsolatesPoisonPath812165069/002/bbb56082026/11/12 00:44:31 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestWorkerRemoveFailureIsNotProgress2747766358/002/nonexistent5609--- PASS: TestQueueDeduplication (0.01s)56102026/11/12 00:44:31 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=156112026/11/12 00:44:31 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestWorkerRemoveFailureIsNotProgress2747766358/002/nonexistent56122026/11/12 00:44:31 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=156132026/11/12 00:44:31 WARN Store path no longer exists (garbage collected?), removing from queue path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestWorkerRemoveFailureIsNotProgress2747766358/002/nonexistent56142026/11/12 00:44:31 INFO Uploading batch count=256152026/11/12 00:44:31 ERROR Upload failed error="upload failed" count=256162026/11/12 00:44:31 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestDrainGivesUpWhenServerDown1830410486/002/a56172026/11/12 00:44:31 INFO Upload queue status pending=256182026/11/12 00:44:31 INFO Uploading batch count=256192026/11/12 00:44:31 INFO Upload queue status pending=356202026/11/12 00:44:31 ERROR Failed to remove paths from queue error="deleting paths: attempt to write a readonly database (8)" count=156212026/11/12 00:44:31 ERROR Drain finished with paths left in queue remaining=15622--- PASS: TestDrainTimeout (0.02s)5623--- PASS: TestFailedPathPrunedByLaterClosure (0.01s)56242026/11/12 00:44:31 INFO Uploading batch count=156252026/11/12 00:44:31 ERROR Upload failed error="upload failed" count=156262026/11/12 00:44:31 ERROR Drain finished with paths left in queue remaining=156272026/11/12 00:44:31 INFO Uploading batch count=156282026/11/12 00:44:31 ERROR Upload failed error="upload failed" count=156292026/11/12 00:44:31 INFO Uploading batch count=156302026/11/12 00:44:31 ERROR Upload failed error="upload failed" count=156312026/11/12 00:44:31 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestDrainGivesUpWhenServerDown1830410486/002/b5632--- PASS: TestQueueEnqueueAndFetch (0.01s)5633--- PASS: TestWorkerRemoveFailureIsNotProgress (0.02s)56342026/11/12 00:44:31 INFO Uploading batch count=156352026/11/12 00:44:31 ERROR Upload failed error="upload failed" count=156362026/11/12 00:44:31 ERROR Drain finished with paths left in queue remaining=156372026/11/12 00:44:31 INFO Uploading batch count=256382026/11/12 00:44:31 ERROR Upload failed error="upload failed" count=256392026/11/12 00:44:31 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestDrainGivesUpWhenServerDown1830410486/002/c56402026/11/12 00:44:31 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestDrainGivesUpWhenServerDown1830410486/002/d5641--- PASS: TestWorkerKeepsPathItCannotStat (0.02s)56422026/11/12 00:44:31 INFO Uploading batch count=256432026/11/12 00:44:31 ERROR Upload failed error="upload failed" count=256442026/11/12 00:44:31 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestDrainGivesUpWhenServerDown1830410486/002/e56452026/11/12 00:44:31 ERROR Upload failed, will retry later error="upload failed" path=/private/var/tb/3/scratch/cf270c76132c2e86/build/TestDrainGivesUpWhenServerDown1830410486/002/f56462026/11/12 00:44:31 ERROR Drain finished with paths left in queue remaining=105647--- PASS: TestDrainIsolatesPoisonPath (0.01s)5648--- PASS: TestDrainGivesUpWhenServerDown (0.02s)5649--- PASS: TestWorkerSkipsGCdPaths (0.03s)5650--- PASS: TestWorkerPrunesClosureDeps (0.03s)5651--- PASS: TestWorkerUploadsAndRemoves (0.04s)5652--- PASS: TestQueueConcurrentWriters (0.05s)5653--- PASS: TestQueueRemoveLargeClosure (0.06s)56542026/11/12 00:44:31 ERROR Failed to decode request error="read unix /private/var/tb/3/scratch/cf270c76132c2e86/build/hook2219646748/test.sock->: i/o timeout"56552026/11/12 00:44:31 ERROR Failed to write response error="write unix /private/var/tb/3/scratch/cf270c76132c2e86/build/hook2219646748/test.sock->: i/o timeout"5656--- PASS: TestServerStalledClientDoesNotBlockShutdown (0.20s)56572026/11/12 00:44:32 INFO Uploading batch count=156582026/11/12 00:44:32 INFO Uploading batch count=156592026/11/12 00:44:32 INFO Uploading batch count=156602026/11/12 00:44:32 ERROR Upload failed error="upload failed" count=156612026/11/12 00:44:32 INFO Uploading batch count=156622026/11/12 00:44:32 ERROR Upload failed error="upload failed" count=156632026/11/12 00:44:32 INFO Uploading batch count=156642026/11/12 00:44:32 ERROR Upload failed error="upload failed" count=156652026/11/12 00:44:32 INFO Uploading batch count=156662026/11/12 00:44:32 ERROR Upload failed error="upload failed" count=156672026/11/12 00:44:32 ERROR Drain finished with paths left in queue remaining=15668--- PASS: TestRunNotBlockedByPoisonHead (1.04s)5669PASS5670Running signing tests...5671=== RUN TestExternalSigner5672=== PAUSE TestExternalSigner5673=== RUN TestExternalSignerWrongPublicKey5674=== PAUSE TestExternalSignerWrongPublicKey5675=== RUN TestExternalSignerErrors5676=== PAUSE TestExternalSignerErrors5677=== RUN TestExternalSignerRestartsAfterCrash5678=== PAUSE TestExternalSignerRestartsAfterCrash5679=== RUN TestExternalSignerProgramErrorKeepsProcess5680=== PAUSE TestExternalSignerProgramErrorKeepsProcess5681=== RUN TestExternalSignerClose5682=== PAUSE TestExternalSignerClose5683=== RUN TestExternalSignerRejectsInvalidPublicKey5684=== PAUSE TestExternalSignerRejectsInvalidPublicKey5685=== RUN TestExternalSignerCancellation5686=== PAUSE TestExternalSignerCancellation5687=== RUN TestGenerateFingerprint5688=== PAUSE TestGenerateFingerprint5689=== RUN TestParseSigningKey5690=== PAUSE TestParseSigningKey5691=== RUN TestSign5692=== PAUSE TestSign5693=== RUN TestSignBatch5694=== PAUSE TestSignBatch5695=== RUN TestNilKeyPublicKey5696=== PAUSE TestNilKeyPublicKey5697=== RUN TestSignErrors5698=== PAUSE TestSignErrors5699=== CONT TestExternalSigner5700=== CONT TestExternalSignerCancellation5701=== CONT TestSignBatch5702=== CONT TestExternalSignerProgramErrorKeepsProcess5703=== CONT TestSignErrors5704=== CONT TestExternalSignerErrors5705=== RUN TestExternalSignerErrors/wrong_signatures_shape5706=== PAUSE TestExternalSignerErrors/wrong_signatures_shape5707=== RUN TestExternalSignerErrors/wrong_signature_length5708=== PAUSE TestExternalSignerErrors/wrong_signature_length5709=== RUN TestExternalSignerErrors/exit_on_request5710=== PAUSE TestExternalSignerErrors/exit_on_request5711=== RUN TestExternalSignerErrors/null5712=== PAUSE TestExternalSignerErrors/null5713=== RUN TestExternalSignerErrors/missing_signatures5714=== CONT TestExternalSignerWrongPublicKey5715=== CONT TestNilKeyPublicKey5716--- PASS: TestNilKeyPublicKey (0.00s)5717=== CONT TestParseSigningKey5718=== RUN TestParseSigningKey/valid_32-byte_key5719=== PAUSE TestParseSigningKey/valid_32-byte_key5720=== RUN TestParseSigningKey/valid_32-byte_key_with_different_name5721=== PAUSE TestParseSigningKey/valid_32-byte_key_with_different_name5722=== RUN TestParseSigningKey/no_colon5723=== PAUSE TestParseSigningKey/no_colon5724=== RUN TestParseSigningKey/empty_name5725=== CONT TestGenerateFingerprint5726=== RUN TestGenerateFingerprint/basic_with_references5727=== PAUSE TestGenerateFingerprint/basic_with_references5728=== RUN TestGenerateFingerprint/no_references5729=== PAUSE TestGenerateFingerprint/no_references5730=== RUN TestGenerateFingerprint/unsorted_references_get_sorted5731=== PAUSE TestGenerateFingerprint/unsorted_references_get_sorted5732=== RUN TestGenerateFingerprint/invalid_nar_hash_prefix5733=== PAUSE TestGenerateFingerprint/invalid_nar_hash_prefix5734=== RUN TestGenerateFingerprint/invalid_nar_hash_length5735=== PAUSE TestGenerateFingerprint/invalid_nar_hash_length5736=== RUN TestGenerateFingerprint/invalid_store_path_prefix5737=== PAUSE TestGenerateFingerprint/invalid_store_path_prefix5738=== RUN TestGenerateFingerprint/invalid_reference_prefix5739=== PAUSE TestGenerateFingerprint/invalid_reference_prefix5740=== PAUSE TestParseSigningKey/empty_name5741=== RUN TestParseSigningKey/invalid_base645742=== CONT TestExternalSignerRestartsAfterCrash5743=== PAUSE TestParseSigningKey/invalid_base645744=== RUN TestParseSigningKey/wrong_length5745=== PAUSE TestParseSigningKey/wrong_length5746=== CONT TestParseSigningKey/valid_32-byte_key5747=== CONT TestParseSigningKey/wrong_length5748=== CONT TestParseSigningKey/invalid_base645749=== CONT TestParseSigningKey/empty_name5750=== CONT TestParseSigningKey/no_colon5751=== CONT TestParseSigningKey/valid_32-byte_key_with_different_name5752=== CONT TestExternalSignerRejectsInvalidPublicKey5753=== RUN TestExternalSignerRejectsInvalidPublicKey/#005754=== PAUSE TestExternalSignerRejectsInvalidPublicKey/#005755=== RUN TestExternalSignerRejectsInvalidPublicKey/no-colon5756=== PAUSE TestExternalSignerRejectsInvalidPublicKey/no-colon5757=== RUN TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5758=== PAUSE TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5759=== CONT TestExternalSignerClose5760=== CONT TestSign5761=== PAUSE TestExternalSignerErrors/missing_signatures5762=== RUN TestExternalSignerErrors/invalid_JSON5763=== PAUSE TestExternalSignerErrors/invalid_JSON5764=== RUN TestExternalSignerErrors/wrong_shape5765=== PAUSE TestExternalSignerErrors/wrong_shape5766=== RUN TestExternalSignerErrors/wrong_id5767=== PAUSE TestExternalSignerErrors/wrong_id5768=== RUN TestExternalSignerErrors/null_signatures5769=== PAUSE TestExternalSignerErrors/null_signatures5770=== RUN TestExternalSignerErrors/program_error5771=== PAUSE TestExternalSignerErrors/program_error5772=== RUN TestExternalSignerErrors/too_few_signatures5773=== PAUSE TestExternalSignerErrors/too_few_signatures5774=== RUN TestExternalSignerErrors/unrequested_key5775=== PAUSE TestExternalSignerErrors/unrequested_key5776=== CONT TestGenerateFingerprint/basic_with_references5777=== CONT TestGenerateFingerprint/invalid_reference_prefix5778=== CONT TestGenerateFingerprint/invalid_store_path_prefix5779=== CONT TestGenerateFingerprint/invalid_nar_hash_length5780=== CONT TestGenerateFingerprint/invalid_nar_hash_prefix5781=== CONT TestGenerateFingerprint/unsorted_references_get_sorted5782=== CONT TestGenerateFingerprint/no_references5783--- PASS: TestGenerateFingerprint (0.00s)5784 --- PASS: TestGenerateFingerprint/basic_with_references (0.00s)5785 --- PASS: TestGenerateFingerprint/invalid_reference_prefix (0.00s)5786 --- PASS: TestGenerateFingerprint/invalid_store_path_prefix (0.00s)5787 --- PASS: TestGenerateFingerprint/invalid_nar_hash_length (0.00s)5788 --- PASS: TestGenerateFingerprint/invalid_nar_hash_prefix (0.00s)5789 --- PASS: TestGenerateFingerprint/unsorted_references_get_sorted (0.00s)5790 --- PASS: TestGenerateFingerprint/no_references (0.00s)5791=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5792=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5793=== RUN TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5794=== RUN TestExternalSignerErrors/missing_signature_name5795=== PAUSE TestExternalSignerErrors/missing_signature_name5796=== RUN TestExternalSignerErrors/wrong_signature_name5797=== PAUSE TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5798=== CONT TestExternalSignerRejectsInvalidPublicKey/#005799=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:invalid!5800=== CONT TestExternalSignerRejectsInvalidPublicKey/test-key:YQ==5801=== CONT TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=5802=== CONT TestExternalSignerRejectsInvalidPublicKey/no-colon5803=== PAUSE TestExternalSignerErrors/wrong_signature_name5804--- PASS: TestExternalSignerRejectsInvalidPublicKey (0.00s)5805 --- PASS: TestExternalSignerRejectsInvalidPublicKey/#00 (0.00s)5806 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:invalid! (0.00s)5807 --- PASS: TestExternalSignerRejectsInvalidPublicKey/test-key:YQ== (0.00s)5808 --- PASS: TestExternalSignerRejectsInvalidPublicKey/:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA= (0.00s)5809 --- PASS: TestExternalSignerRejectsInvalidPublicKey/no-colon (0.00s)5810=== RUN TestExternalSignerErrors/invalid_signature_base645811=== PAUSE TestExternalSignerErrors/invalid_signature_base645812=== RUN TestExternalSignerErrors/exit_at_startup5813=== PAUSE TestExternalSignerErrors/exit_at_startup5814=== CONT TestExternalSignerErrors/wrong_signatures_shape5815=== CONT TestExternalSignerErrors/null_signatures5816=== CONT TestExternalSignerErrors/wrong_id5817=== CONT TestExternalSignerErrors/null5818=== CONT TestExternalSignerErrors/too_few_signatures5819=== CONT TestExternalSignerErrors/wrong_shape5820=== CONT TestExternalSignerErrors/exit_at_startup5821=== RUN TestSignErrors/nil_key5822=== PAUSE TestSignErrors/nil_key5823=== RUN TestSignErrors/nil_narinfo5824=== PAUSE TestSignErrors/nil_narinfo5825=== RUN TestSignErrors/invalid_narinfo5826=== PAUSE TestSignErrors/invalid_narinfo5827--- PASS: TestSignBatch (0.00s)5828=== CONT TestExternalSignerErrors/wrong_signature_name5829=== CONT TestExternalSignerErrors/missing_signature_name5830=== CONT TestExternalSignerErrors/unrequested_key5831=== CONT TestExternalSignerErrors/invalid_JSON5832=== CONT TestExternalSignerErrors/program_error5833=== CONT TestExternalSignerErrors/exit_on_request5834=== CONT TestExternalSignerErrors/wrong_signature_length5835=== CONT TestExternalSignerErrors/missing_signatures5836=== CONT TestExternalSignerErrors/invalid_signature_base645837=== RUN TestSignErrors/canceled5838--- PASS: TestParseSigningKey (0.00s)5839 --- PASS: TestParseSigningKey/wrong_length (0.00s)5840 --- PASS: TestParseSigningKey/invalid_base64 (0.00s)5841 --- PASS: TestParseSigningKey/empty_name (0.00s)5842 --- PASS: TestParseSigningKey/no_colon (0.00s)5843 --- PASS: TestParseSigningKey/valid_32-byte_key (0.00s)5844 --- PASS: TestParseSigningKey/valid_32-byte_key_with_different_name (0.00s)5845=== PAUSE TestSignErrors/canceled5846--- PASS: TestSign (0.00s)5847=== CONT TestSignErrors/nil_key5848=== CONT TestSignErrors/canceled5849=== CONT TestSignErrors/nil_narinfo5850=== CONT TestSignErrors/invalid_narinfo5851--- PASS: TestSignErrors (0.00s)5852 --- PASS: TestSignErrors/nil_key (0.00s)5853 --- PASS: TestSignErrors/canceled (0.00s)5854 --- PASS: TestSignErrors/nil_narinfo (0.00s)5855 --- PASS: TestSignErrors/invalid_narinfo (0.00s)5856--- PASS: TestExternalSigner (0.31s)5857--- PASS: TestExternalSignerRestartsAfterCrash (1.88s)5858/private/var/tb/3/scratch/cf270c76132c2e86/build/TestExternalSignerRestartsAfterCrash4106240708/001/sign program: line 7: /private/var/tb/3/scratch/cf270c76132c2e86/build/TestExternalSignerRestartsAfterCrash4106240708/001/sign program.starts: No such file or directory5859--- PASS: TestExternalSignerCancellation (2.07s)5860--- PASS: TestExternalSignerClose (2.63s)5861--- PASS: TestExternalSignerProgramErrorKeepsProcess (3.02s)5862--- PASS: TestExternalSignerWrongPublicKey (3.41s)5863--- PASS: TestExternalSignerErrors (0.00s)5864 --- PASS: TestExternalSignerErrors/too_few_signatures (0.49s)5865 --- PASS: TestExternalSignerErrors/unrequested_key (0.69s)5866 --- PASS: TestExternalSignerErrors/null (0.88s)5867 --- PASS: TestExternalSignerErrors/exit_on_request (1.07s)5868 --- PASS: TestExternalSignerErrors/wrong_signatures_shape (1.27s)5869 --- PASS: TestExternalSignerErrors/missing_signature_name (1.46s)5870 --- PASS: TestExternalSignerErrors/invalid_JSON (1.66s)5871 --- PASS: TestExternalSignerErrors/wrong_shape (2.24s)5872 --- PASS: TestExternalSignerErrors/null_signatures (2.44s)5873 --- PASS: TestExternalSignerErrors/exit_at_startup (2.83s)5874 --- PASS: TestExternalSignerErrors/program_error (3.21s)5875 --- PASS: TestExternalSignerErrors/invalid_signature_base64 (3.61s)5876 --- PASS: TestExternalSignerErrors/wrong_id (3.81s)5877 --- PASS: TestExternalSignerErrors/wrong_signature_name (4.00s)5878 --- PASS: TestExternalSignerErrors/wrong_signature_length (4.21s)5879 --- PASS: TestExternalSignerErrors/missing_signatures (4.40s)5880PASS5881Running OpenBao signer tests...5882=== RUN TestSign5883=== PAUSE TestSign5884=== RUN TestSignErrors5885=== PAUSE TestSignErrors5886=== RUN TestServe5887=== PAUSE TestServe5888=== RUN TestServeRejectsGarbage5889=== PAUSE TestServeRejectsGarbage5890=== CONT TestSign5891=== CONT TestSignErrors5892=== RUN TestSignErrors/invalid_json5893=== PAUSE TestSignErrors/invalid_json5894=== CONT TestServeRejectsGarbage5895=== RUN TestSignErrors/no_prefix5896=== PAUSE TestSignErrors/no_prefix5897=== RUN TestSignErrors/invalid_base645898=== PAUSE TestSignErrors/invalid_base645899=== RUN TestSignErrors/http_error5900=== PAUSE TestSignErrors/http_error5901=== CONT TestSignErrors/invalid_json5902=== CONT TestSignErrors/http_error5903=== CONT TestSignErrors/invalid_base645904=== CONT TestServe5905=== CONT TestSignErrors/no_prefix5906--- PASS: TestServeRejectsGarbage (0.00s)5907--- PASS: TestServe (0.00s)5908--- PASS: TestSign (0.00s)5909--- PASS: TestSignErrors (0.00s)5910 --- PASS: TestSignErrors/invalid_json (0.00s)5911 --- PASS: TestSignErrors/no_prefix (0.00s)5912 --- PASS: TestSignErrors/http_error (0.00s)5913 --- PASS: TestSignErrors/invalid_base64 (0.00s)5914PASS